5.4 Metaclasses

Any object, even a class object, has a type. In Python, types and classes are also first-class objects. The type of a class object is also known as the class's metaclass.^[1] An object's behavior is determined largely by the type of the object. This also holds for classes: a class's behavior is determined largely by the class's metaclass. Metaclasses are an advanced subject, and you may want to skip the rest of this chapter on first reading. However, fully grasping metaclasses can help you obtain a deeper understanding of Python, and sometimes it can even be useful to define your own custom metaclasses.

^[1] Strictly speaking, the type of a class C could be said to be the metaclass only of instances of C, rather than of C itself, but this exceedingly subtle terminological distinction is rarely, if ever, observed in practice.

The distinction between classic and new-style classes relies on the fact that each class's behavior is determined by its metaclass. In other words, the reason classic classes behave differently from new-style classes is that classic and new-style classes are object of different types (metaclasses):

class Classic: pass
class Newstyle(object): pass
print type(Classic)                  # prints: <type 'class'>
print type(Newstyle)                 # prints: <type 'type'>

The type of Classic is object types.ClassType from standard module types, while the type of Newstyle is built-in object type. type is also the metaclass of all Python built-in types, including itself (i.e., print type(type) also prints <type 'type'>).

5.4.1 How Python Determines a Class's Metaclass

To execute a class statement, Python first collects the base classes into a tuple t (an empty one, if there are no base classes) and executes the class body in a temporary dictionary d. Then, Python determines the metaclass M to use for the new class object C created by the class statement.

When '_ _metaclass_ _' is a key in d, M is d['_ _metaclass_ _']. Thus, you can explicitly control class C's metaclass by binding the attribute _ _metaclass_ _ in C's class body. Otherwise, when t is non-empty (i.e., when C has one or more base classes), M is type(t[0]), the metaclass of C's first base class. This is why inheriting from object indicates that C is a new-style class. Since type(object) is type, a class C that inherits from object (or some other built-in type) gets the same metaclass as object (i.e., type(C), C's metaclass, is also type) Thus, being a new-style class is synonymous with having type as the metaclass.

When C has no base classes, but the current module has a global variable named _ _metaclass_ _, M is the value of that global variable. This lets you make classes without base classes default to new-style classes, rather than classic classes, throughout a module. Just place the following statement toward the start of the module body:

_ _metaclass_ = type

Failing all of these, in Python 2.2 and 2.3, M defaults to types.ClassType. This last default of defaults clause is why classes without base classes are classic classes by default, when _ _metaclass_ _ is not bound in the class body or as a global variable of the module.

5.4.2 How a Metaclass Creates a Class

Having determined M, Python calls M with three arguments: the class name (a string), the tuple of base classes t, and the dictionary d. The call returns the class object C, which Python then binds to the class name, completing the execution of the class statement. Note that this is in fact an instantiation of type M, so the call to M executes M._ _init_ _(C,namestring,t,d), where C is the return value of M._ _new_ _(M,namestring,t,d), just as in any other similar instantiation of a new-style class (or built-in type).

After class object C is created, the relationship between class C and its type (type(C), normally M) is the same as that between any object and its type. For example, when you call class C (to create an instance of C), M._ _call_ _ executes, with class object C as the first actual argument.

Note the benefit of the new-style approach described in Section 5.2.4.4 earlier in this chapter. Calling C to instantiate it must execute the metaclass's M._ _call_ _, whether or not C has a per-instance attribute (method) _ _call_ _ (i.e., independently of whether instances of C are or aren't callable). This requirement is simply incompatible with the classic object model, where per-instance methods override per-class ones�even for implicitly called special methods. The new-style approach avoids having to make the relationship between a class and its metaclass an ad hoc special case. Avoiding ad hoc special cases is a key to Python's power: Python has few, simple, general rules, and applies them consistently.

5.4.2.1 Defining and using your own metaclasses

It's easy to define metaclasses in Python 2.2 and later, by inheriting from type and overriding some methods. You can also perform most of these tasks with _ _new_ _, _ _init_ _, _ _getattribute_ _, and so on, without involving metaclasses. However, a custom metaclass can be faster, since special processing is done only at class creation time, which is a rare operation. A custom metaclass also lets you define a whole category of classes in a framework that magically acquires whatever interesting behavior you've coded, quite independently of what special methods the classes may choose to define. Moreover, some behavior of class objects can be customized only in metaclasses. The following example shows how to use a metaclass to change the string format of class objects:

class MyMeta(type):
    def _ _str_ _(cls): return "Beautiful class '%s'"%cls._ _name_ _
class MyClass:
    _ _metaclass_ _ = MyMeta
x = MyClass(  )
print type(x)

Strictly speaking, classes that instantiate your own custom metaclass are neither classic nor new-style: the semantics of classes and of their instances is entirely defined by their metaclass. In practice, your custom metaclasses will almost invariably subclass built-in type. Therefore, the semantics of the classes that instantiate them are best thought of as secondary variations with respect to the semantics of new-style classes.

5.4.2.2 A substantial custom metaclass example

Suppose that, programming in Python, we miss C's struct type: an object that is just a bunch of data attributes with fixed names. Python lets us easily define an appropriate Bunch class, apart from the fixed names:

class Bunch(object):
    def _ _init_ _(self, **fields): self._ _dict_ _ = fields
p = Bunch(x=2.3, y=4.5)
print p                     # prints: <_ _main_ _.Bunch object at 0x00AE8B10>

However, a custom metaclass lets us exploit the fact that the attribute names are fixed at class creation time. The code shown in Example 5-1 defines a metaclass, metaMetaBunch, and a class, MetaBunch, that let us write code like the following:

class Point(MetaBunch):
    """ A point has x and y coordinates, defaulting to 0.0, and a color,
        defaulting to 'gray' -- and nothing more, except what Python and
        the metaclass conspire to add, such as _ _init_ _ and _ _repr_ _
    """
    x = 0.0
    y = 0.0
    color = 'gray'
# example uses of class Point
q = Point(  )
print q                     # prints: Point(  )
p = Point(x=1.2, y=3.4)
print p                     # prints: Point(y=3.399999999, x=1.2)

In this code, the print statements print readable string representations of our Point instances. Point instances are also quite memory-lean, and their performance is basically the same as for instances of the simple class Bunch in the previous example (no extra overhead due to special methods getting called implicitly). Note that Example 5-1 is quite substantial, and following all its details requires understanding aspects of Python covered later in this book, such as strings (Chapter 9) and module warnings (Chapter 17).

Example 5-1. The metaMetaBunch metaclass

import warnings
class metaMetaBunch(type):
    """
    metaclass for new and improved "Bunch": implicitly defines _ _slots_ _,
   _ _init_ _ and _ _repr_ _ from variables bound in class scope.
    A class statement for an instance of metaMetaBunch (i.e., for a class
    whose metaclass is metaMetaBunch) must define only class-scope data
    attributes (and possibly special methods, but NOT _ _init_ _ and 
    _ _repr_ _!).  metaMetaBunch removes the data attributes from class
    scope, snuggles them instead as items in a class-scope dict named
    _ _dflts_ _, and puts in the class a _ _slots_ _ with those attributes'
    names, an _ _init_ _ that takes as optional keyword arguments each of
    them (using the values in _ _dflts_ _ as defaults for missing ones), and
    a _ _repr_ _ that shows the repr of each attribute that differs from its
    default value (the output of _ _repr_ _ can be passed to _ _eval_ _ to 
    make an equal instance, as per the usual convention in the matter, if
    each of the non-default-valued attributes respects the convention too)
    """
    def _ _new_ _(cls, classname, bases, classdict):
        """ Everything needs to be done in _ _new_ _, since type._ _new_ _ is
            where _ _slots_ _ are taken into account.
        """
        # define as local functions the _ _init_ _ and _ _repr_ _ that we'll
        # use in the new class
        def _ _init_ _(self, **kw):
            """ Simplistic _ _init_ _: first set all attributes to default
                values, then override those explicitly passed in kw.
            """
            for k in self._ _dflts_ _: setattr(self, k, self._ _dflts_ _[k])
            for k in kw: setattr(self, k, kw[k])
        def _ _repr_ _(self):
            """ Clever _ _repr_ _: show only attributes that differ from the
                respective default values, for compactness.
            """
            rep = ['%s=%r' % (k, getattr(self, k)) for k in self._ _dflts_ _
                    if getattr(self, k) != self._ _dflts_ _[k]
                  ]
            return '%s(%s)' % (classname, ', '.join(rep))
        # build the newdict that we'll use as class-dict for the new class
        newdict = { '_ _slots_ _':[  ], '_ _dflts_ _':{  },
            '_ _init_ _':_ _init_ _, '_ _repr_ _':_ _repr_ _, }
        for k in classdict:
            if k.startswith('_ _') and k.endswith('_ _'):
                # special methods: copy to newdict, warn about conflicts
                if k in newdict:
                    warnings.warn("Can't set attr %r in bunch-class %r"
                        % (k, classname))
                else:
                    newdict[k] = classdict[k]
            else:
                # class variables, store name in _ _slots_ _, and name and
                # value as an item in _ _dflts_ _
                newdict['_ _slots_ _'].append(k)
                newdict['_ _dflts_ _'][k] = classdict[k]
        # finally delegate the rest of the work to type._ _new_ _
        return type._ _new_ _(cls, classname, bases, newdict)
class MetaBunch(object):
    """ For convenience: inheriting from MetaBunch can be used to get
        the new metaclass (same as defining _ _metaclass_ _ yourself).
    """
    _ _metaclass_ _ = metaMetaBunch