The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Data hiding in Python

I've been teaching myself Python over the last few months, and there are a lot of things I really like about it, particularly the functional-style stuff you can do. I really *want* to like it whole-heartedly, but I just can't seem to get "into the spirit" of the philosophy that the language doesn't need to restrict the programmer. I come from a background of mainly C/C++, Java and Haskell, and in Python I often really wish I could, for example, make a member variable private/protected or make something const. I've been using the underscore convention for private member variables, and all caps for constants, but it still doesn't feel solid.

Has anyone else felt this way about Python (or a similar language) initially, and overcome it? Can anyone give me any clues on how I can get out of this mindset? As I said, I really want to get to a point where I am happy and comfortable programming in Python, but at the moment, I start to lose confidence in my programs once they get past a few hundred lines because it just feels so "shaky" with no data hiding.

The argument I've seen a few times for why these kinds of options aren't available in Python goes something like this: "Having things like private/protected in your language doesn't force the programmer into good design anyway; you can always just make all your member variables public if you want. So we'll just leave it up to you to make a convention of what's public and what's private and stick to it."

I don't really follow this argument. It's true that having the choice between public and private doesn't force you to choose a good design, but if you do choose good design, it forces you to obey it; in the words above, it doesn't force you to "make a [useful] convention of what's public and what's private", but it does force you to "stick to it". If I make something private, it's so that the compiler will yell at me if I try to use it from outside, because if that happens I will be going against my design, which is something I want to know so that I can fix it. In other words, it's purely to aid in the development process. If, immediately before something I've written in C++ were being deployed, someone went through the code and relabelled all the private things public, made no other changes, built it and shipped it, I wouldn't really mind; I'd know that the code was consistent with the design because otherwise I wouldn't have been able to finish writing it. This doesn't imply, however, that I would have been happy to have everything labelled public *during* the development process. (I wouldn't.) But that seems to be what the argument above assumes.

Am I misunderstanding the argument? Or misunderstanding something more general about Python? I kind of hope I am!
Tim H
Tuesday, December 12, 2006
Using two leading underscores (see e.g. [ ]) for private variables -- they'll be about as inaccessible as they are in C++; that is, it's not impossible, but it does take some additional conscious effort.

Regarding const etc. (and actually, public/private as well), Python gives you the tool to enforce much than that -- by defining a value as property, you can say it is read-only, write-only, increase-only, positive-only, or whatever else you might want it to be.

Experience with Python (which now dates 15 years or so -- lesser but not negligible when compared to C++ or Java) has shown that by and large, "private" and "const" are rarely as helpful as they are believed to be in other languages. The double-underscore private variables in python were actually implemented to solve a namespace collision problem (which was widely accepted as a real problem) rather than access control (which wasn't). In fact, Early Python versions _did_ have access control in the language, but this was dropped for lack of use (I know it was there in 0.9.9 which was the first verison I used, and I think it wasn't there in 1.4 -- not sure when exactly it was dropped).

That said, you can have everything you you're missing from Java/C++/Haskell at a library level: Definitions as concise as those in Haskell will also give you automatic update and verification dependencies, and a GUI -- read up on [ ].

If you insist on this kind of design, you have the tools. However, wide ranging experience in Python has shown that by and large, this is not the case. If you work in Python enough, you'll probably reach the same conclusion even if you insist on straightjacketing yourself with something like traits.
Ori Berger
Tuesday, December 12, 2006
"Experience with Python (which now dates 15 years or so -- lesser but not negligible when compared to C++ or Java) has shown that by and large, "private" and "const" are rarely as helpful as they are believed to be in other languages"

I sympathize with Tim's confusion and I agree with your answer. I've always preferred "bondage and discipline" languages. I liked Java. God help me, I even liked Pascal. I was sure that the static typing and strict enforcement of access level were essential for sound engineering. Three years ago I started working in Python. I liked it, I was productive, and the mistakes I did make had nothing to do with typing or access level. I was very surprised. I see in retrospect that compilers rarely had to rebuke me for trying to violate access levels. I could keep that straight without help from the compiler. However, I could have sworn that I used to get lots of type mismatch errors when I wrote in C/C++ or Java.

Check out some of Bruce Eckel's observations on Python at He was surprised about Python too, and comes to the conclusion that rigorous unit testing is more powerful then strict compilers.
Charles E. Grant
Tuesday, December 12, 2006

If that's what you want, you're using the wrong language.
Uncle Istvan
Tuesday, December 12, 2006
This is mostly a psychological issue.  You're protecting stuff from whom exactly?  Yourself?  Evil programmers?
James Send private email
Tuesday, December 12, 2006
Yeah, the only problem I've ever had w/Python's flexibility is accidentally mistyping a member variable name & having it automatically created for me :-) But you can solve that by just using an editor that has auto-complete like Wingware (cheap & good, I highly recommend it). If you want discipline & bondage that bad, book an appointment with Mistress Helga & don't take it out on your poor software :-)
Johnny the Geek
Tuesday, December 12, 2006
> "This is mostly a psychological issue.  You're protecting stuff from whom exactly?  Yourself?  Evil programmers?"

I realize this is a "Python" related thread HOWEVER it seems like just yesterday (figuratively, not literally) the Ruby on Rails crowd ran into just such a "psychological issue" regarding using instance variables in the framework versus instance variable the programmer created and issued a warning* - essentially the very thing this thread is about and your comment directly addresses.  I'm not sure that dismissing it trivially as a non-issue is so wise in the long run.

Of course therein lies the rub - if you take off safety features in a language to relax it from tough and rigid to soft and malleable one has to expect old problems to go go away while a new crop of problems emerge.  It is these trade-offs that one needs to be objective about in order to decide if the right tool is being used for whatever the current task is at hand...In my humble opinion.

*That warning being, my understanding anyhow, is that the rails camp had to issue a public statement regarding a position on this matter which was:  do not access instance variables you did not create in a class, mixins, modules, etc.  Granted this seems like a no-brainer to avoid coupling directly to implementation of an interface but Python/Ruby/etc seem to make programming more accessible to people that don't know better.  I believe one can still find documentation that needs to be adjusted as a result of this.  It didn't end up being so trivial for some rails users.
Chris Send private email
Tuesday, December 12, 2006
@chris: ha ha ha!  That's hilarious!  Can you provide a link to the "announcement"?

Kids, this is the reason why the so called "scripting languages" are best for writing your crash-and-burn, get-stuff-done-quickly scripts.  For writing an SDK or other public API, we adults prefer to use grown up languages that give us these abilities and save us from having to tell people "please don't mess with our implementation details".

hee hee hee, best laugh I've had all week.
monkEy fuEl
Wednesday, December 13, 2006

First item on the list of "Things You Should Not..."

Other than mailing lists, unless this has been addressed and I am unaware*, given the nature of Rails and the community around it - there seems to be a lack of centralized knowledge base that is all encompassing on topics like this.  My understanding is that the site I have linked is one of a handful that are legitimate for information of this nature though.

*Entirely possible and most likely probable
Chris Send private email
Wednesday, December 13, 2006
A java/C++/ HASKELL background?

Wow.  Interesting.

Where did you do Haskell?
Wednesday, December 13, 2006
You can always implement it too. See

as an example. I can't comment on the usefulness of it but there are other examples out there too if you're really worried about it.
Alex Le Dain Send private email
Wednesday, December 13, 2006
monkEy fuEl, no one likes a language bigot.  You completely kill your argument by saying things like "we adults prefer to use grown up languages."  Yes, you certainly should use a Screwdriver instead of a Hammer when presented with a screw, however that doesn't make the Hammer patently inferior to the Screwdriver*.

If you are doing something that truly requires strict access requirements then Python is not the language you should be using.  If you simply think you need access restrictions because that's the way you've always done it, then you need to look a little deeper into your motivations.  Encapsulation for the sake of encapsulation is hardly a noble goal.  In almost every language there are ways of circumventing access restrictions (Reflection in .Net is a great example) so in the end it's simply like locking your front door*, it simply helps keep the honest people honest.

Whatever you do don't try and bend the language to your will as it will simply end in bloodshed and a brittle product.

* Have I met my metaphor quota yet?
Bondo Send private email
Friday, December 15, 2006
Yes, that's the whole point of static typing, encapsulation, access controls, and so on -- while they can often be worked round if you really want to, their PURPOSE is not to place you in a straitjacket, but rather to provide a simple means whereby you can indicate to the computer what your intention is, and it can tell you if you then write code which is not consistent with your stated intention.

They're not God's answer to all bugs, and nor are they curses sent by Satan to torment mankind.  They're just tools that many people find useful to help them write correct code.

(I find it somewhat amusing that converts to dynamically typed languages and converts to static functional languages alike preach the same kinds of incredible benefits and unprecedented productivity, all the while failing to notice that the rest of the world is rolling its eyes and getting on with C++, C#, and Java...)
Monday, December 18, 2006

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz