Python Black Magic

by jbail • 16 years ago

19 24 16 years ago

Estragon• 16 years ago

I learned these kinds of tricks when I started using python 12 years ago, but I have basically never had a need for them. Perhaps my needs are simple, though.

jjames• 16 years ago

Same here. Like many things in python, the unconventional hack is made just inconvenient enough to persuade most people to rethink their strategy.

jbail3• 16 years ago

Agreed. I can't think of a time when you'd need to use it either (and I wrote the post). But, coming from working in Java for the past few years, it seems pretty cool that you can do it.

tsuraan• 16 years ago

You also can do it in java: http://www.fordevs.com/2008/11/reflection-is-mechanism-by-wh... . As a python programmer who never uses "private" variables (I use a single underscore to indicate fields that shouldn't be used), I'm a bit torn as to which private variable abuse method is uglier :)

sophacles2• 16 years ago

They are nice to know when debugging -- looking at dir(obj) can lead to things like "where the heck is _foo__attr defined?" if you don't know the __x -> _classname__x mangling.

mccutchen• 16 years ago

I learned this trick when I first picked up Python, too. I came from Java and, naturally, started writing Python code in the style I had learned to write Java: Heavily object-oriented, with getters and setters everywhere.

Eventually, though, I stopped trying to program in Java in Python. Once you relax and stop trying to hide your precious instance variables from all of the potentially pernicious or incompetent programmers who might be using your code (usually, in my case, just me), you no longer need tricks like this. And you realize that you never really needed them to begin with.

jbail3• 16 years ago

I'm not advocating using getters/setters in Python. Just pointing out that you can access "private" members. I don't typically use getters and setters in JavaScript either.

Java, on the other hand, kind of forces you into using getters and setters because the POJO is so ingrained into the ecosystem. That said, if you write Android apps in Java, you should avoid getters and setters because they aren't really necessary and they create extra overhead.

koenigdavidmj2• 16 years ago

And you can turn your member access into called code when you need it:

    class Foo(object):
        @property
        def bar(self):
            print "Running getter"
            return self._bar
        
        @bar.setter
        def bar(self, value):
            print "Running setter"
            self._bar = value

lukeqsee• 16 years ago

Forgive my incompetence, but I have never gotten the point of private methods/variables. Could someone please explain why they exist?

Edit: I know it's slightly off-topic.

ianp• 16 years ago

Private members are generally just to protect your from yourself. If you have an integer that should /always/ be 1-10, it's easier to use a private variable and a setter that ensures the variable is always valid, than to hope all your code follows the rules. (Then also if the rule changes, e.g. to 5-100, you just need to change one function).

There are some other uses and examples, but this is the simplest one i can think of off the top of my head.

sam191• 16 years ago

I may not be the most qualified person to answer this, but I always thought private methods and variables were for internal use inside of a class while the public methods were what one would actually call with their objects. For example your object had a method foo() which would be better defined as smaller methods which are combined together. Those smaller functions serve no other purpose and therefore do not need to be called by the object directly.

sophacles2• 16 years ago

One of the points not discussed in the other replies yet, is that public methods and variables are the interface, the well defined, relatively static way of using a class. The private members are internal implementation details, and may change frequently -- not to mention completely change in style, functionality and so on.

The idea then, is that larger teams can be divided into sub-teams, and work in parallel because the interfaces between components don't change, even if the internals change drastically. Essentially it allows components to be somewhat independent. How this works in practice tho...

epo• 16 years ago

Private methods/variables etc are those which are not publicly available. Typically internal state for the object.

Ideally, or at least for purity, all variables should be private and only accessible via get/set methods, thus encapsulation is enforced.

rryyan• 16 years ago

Access specifiers (like "private") are language features used to implement the OOP notion of encapsulation: http://en.wikipedia.org/wiki/Encapsulation_(object-oriented_.... Encapsulation is generally cited as one of the defining features of object oriented programming.

silentbicycle• 16 years ago

Which is weird, because it's actually a defining feature of modular programming.

Public/private access are used to specify an interface to arbitrary functionality. Other modules can call a public interface, but all private internals can be removed / modified freely.

ToastOpt• 16 years ago

Arguably, it's a matter of isolation from 'irrelevant' changes. In a function, one wishes to keep the ability to redefine their local variables (perhaps, to implement a faster/smaller-space algorithm) without otherwise affecting the overall program -- when they call other code, that other code should avoid accessing the caller's variables, lest the caller be stuck with precisely those variables and that implementation. If this isolation exists, I have more freedom to vary the implementation.

Likewise, private members of a class are implementation details I may wish to change later -- but I would like some assurance that other components will be isolated from this change, so as not to break.

Edit-add: oh, I of course forgot to mention -- for those not familiar, python permits both :-)

meese_• 16 years ago

This is not exactly "black magic", it's fairly well-known and explained directly in the official documentation: http://docs.python.org/tutorial/classes.html#private-variabl....

jbail3• 16 years ago

I think developers coming to Python from other languages (like I am from a heavy Java background) find the auto-creation of "shadow variables" pretty interesting. To someone used to Java, being able to access a private member via its shadow variable is a lot like black magic.

Discovering this for the first time in Python seemed pretty cool to me. I really hope this feeling of magic in working with Python never leaves me. I want to remain amazed.

piramida• 16 years ago

Coming from C++, this is very similar to what all C++ compilers do internally (i.e. name mangling). Python just opens a possibility to access that "mangled" names directly, which should never be necessary unless you are doing some metaprogramming and need full access to class internals.

Jach• 16 years ago

You can access private data in C++ too quite trivially, especially since there's no compile-time encapsulation...

Guido van Rossum on the lack of private members: "We're all consenting adults here."

(And a discussion: http://mail.python.org/pipermail/tutor/2003-October/025932.h... )

didip• 16 years ago

While we are on the black magic thread, are there other meta-programming tricks that people find useful/clever?

I'll share one: import inspect. It's handy to use during debugging.

zefhous• 16 years ago

Also, Ruby lets you call private/protected methods with Object#send.

Python Black Magic

Related Stories