Bitten by the python

Note: This entry has been restored from old archives.

I play with the snake from time to time, Python having now almost entirely supplanted Perl as my hak-n-slash language. But I’m certainly still learning as I go. Today I’ve been trying to work out what’s gone wrong with something I’m doing. I thought I’d done something strange with inheritance, not understanding some case where inheritance causes data to become static (in the sense of static members in C++.) What was really the problem is that I didn’t realise that default parameters to members are kept and reused, and if you go and use a parameter you can end up with state being held on to where you don’t (well, I didn’t) expect it.

I’ve hit this at least once before I think, it is vaguely familiar. Anyway, it’s a bit of a tricky gotcha as far as I’m concerned. Intuitive? I wouldn’t say so. Here’s the example:

#!/usr/bin/python

class Parent:
    def __init__(self, stuff = []):
        self.stuff = stuff
    def doIt(self):
        print " ".join(self.stuff)
    def addStuff(self, stuff):
        self.stuff.append(stuff)

class Child(Parent):
    def __init__(self, stuff = []):
        Parent.__init__(self)
    def doIt(self):
        self.addStuff("foo")
        Parent.doIt(self)

c1 = Child()
c1.doIt()
c2 = Child()
c2.doIt()
c3 = Child()
c3.doIt()

Execute this and we see:

foo
foo foo
foo foo foo

What? Why is it accumulating “foo”s? The answer is in the “stuff = []” argument to __init__. What is going on, as far as I can garner from the documentation, is that that default argument (a list) is being instantiated once and then kept for future use. What’s more, I’m assigning self.stuff to this, which is kept as a reference and doesn’t create a copy. So I have ended up with a static value for self.stuff, well, within the functionality of this code – it isn’t entirely congruous to a static member.

How to fix it? I don’t know the definitive approach, but here’s a couple of ways. To achieve complete deep copying of the argument whether it be default or passed in use copy.deepcopy:

#!/usr/bin/python
import copy

class Parent:
    def __init__(self, stuff = []):
        self.stuff = copy.deepcopy(stuff)
...

Alternatively, you could use copy.copy for a shallow copy and I guess that might be analogous to this:

#!/usr/bin/python

class Parent:
    def __init__(self, stuff = []):
        self.stuff = []
        self.stuff.extend(stuff)
...

I’m sure there are great uses for this behaviour in Python. What are they? Something more fundamental than “neat tricks” involving incremental/changing default state? Am I the only one to think this somewhat of a gotcha?

Now that I’ve worked out what was wrong I can retrospectively build the right Google magic to get straight to the answer.

No time to read up on more casual chatter about it though. The documented formula is something like:

#!/usr/bin/python

class Parent:
    def __init__(self, stuff = None):
        if stuff is None:
            stuff = []
        self.stuff = stuff
...

3 thoughts on “Bitten by the python”

  1. pylint complains about that particular insanity… It’s also in pretty much every list of python gotchas in existence. Of course now what you need to do is use the behviour as much as possible, poor future code readers/maintainers…

    python is a nice language which a whole stack of crazy landmines like that. The landmines make sense when you think about them and how python works, in this case def is a statement so when it’s executed the reference to [] gets assigned to the var and is shared forever more…

    My least favourite:

    l = []
    for i in range(5):
        l.append(lambda x: x*i)
    print l[2](10)

    Call me crazy, but I’d really like the output to be 20. But no such luck…

    The quick fix I’ve resorted to in the past:

        l.append(lambda x,i=i: x*i)

    which is fugly…

  2. pylint, hrm, really should pay more attention to such tools.

    It produces, for me, a curiosity, demonstrated by this silly code:

    ————————–
    #!/usr/bin/python
    “””
    Attempt to create ‘/tmp/blah’
    “””
    try:
        myfile = file(“/tmp/blah”, “w”)
        myfile.close()
    except IOError, e:
        print “Error: “, e
    ————————–

    I have to wonder why it says:

    C: 7: Invalid name “myfile” (should match (([A-Z_][A-Z1-9_]*)|(.*))$)

    If I do an s/myfile/MYFILE/ it becomes quite happy!

    I also wonder what is so bad about “except Exception, e:” – seems reasonable enough at the top level of a script, although it may be nicer to be more precise deeper within classes/modules.

  3. It’s a lint tool, it reports a bunch of crap that the author thinks is bad style. Some of those things are what I would consider fundamental python feature usage.

    You can turn off individual warnings and C0103 is the first on the list.

    You have defined a module level name with that code snippet and the folks who wrote pylint use uppercase for those. Pylint is really designed for modules not standalone scripts.

    It’s not something I use often, it is something I give a whirl when python does something I don’t expect, of course it doesn’t seem to catch anything anymore, I guess I’ve stumbled across most of those gotchas already 🙂

Comments are closed.