The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Anyone have experience mixing Python/C?

I am working on a project with a back-end that's close to the metal & really needs to be written in C.  I've been trying to get away from C++ for years, & was thinking about trying Python for the GUI.  Then I started looking at all the pieces that need to be integrated: (1) Make the C code into a Python extension, not trivial (2) Integrate the WxPython widgets (3) may need several different versions of 1 and 2 to support various incarnations of the Python interpreter that users may have installed? (4) port the whole mess to Windows (I like to develop in Linux)...  At this point I'm wondering if I'm better off just sticking to Borland C++ Builder for the whole thing?  Will the added productivity of Python get wiped out 100-fold by the deployment mess of trying to integrate all these pieces?  (Especially since I'm very experienced with C++)  At least C++ Builder has everything I need integrated in one package & supports Windows & Linux...

I bought into the whole Java "productivity" thing & tried it out on a project & was later sorry I did...  Is this deja vu all over again?  Anyone have experience with this?

Everybody says they hate C++ and it's on its way to the junkpile of tech history, but I'm beginning to wonder...
John Foxx
Sunday, March 20, 2005
Yeah, it's a pain to integrate it all and then a nightmare to keep synchronized.

Here's what you do:

1. Write a great templatized hash library.
2. Write a great reg exp library.

And then use a restrained and minimalist style of C++.

To simulate Python's format sensitivity, run your code through a prettifier every time you save.
Rich Rogers
Sunday, March 20, 2005
>>was thinking about trying Python for the GUI

Since it's originally built for text-based apps, Python is hardly the best solution to build GUI apps, unless your needs are quite basic, in which case it shouldn't be much work to write it in C or C++ using tools like wxWidgets, QT, or GTK. Indeed, Borland C++ Builder seems the best solution for what you'd like to do.
Sunday, March 20, 2005
> (1) Make the C code into a Python extension, not trivial
To make it trivial use either SWIG, PyBoost or PyRex
(I use SWIG)

> (2) Integrate the WxPython widgets
Just use wxPython, it'll *very* easy. And has greate documentation/demo and a very helpful community.

> (3) may need several different versions of 1 and 2 to support various incarnations of the Python interpreter that users may have installed?
Nope. Using py2exe, cxFreeze ... the users will not need to install Python on their machine.

> Will the added productivity of Python get wiped out 100-fold by the deployment mess of trying to integrate all these pieces?
I've done several such projects here at work. One you get the initial configuration to work (SWIG + py2exe for me), everything else is super easy.
Miki Tebeka Send private email
Sunday, March 20, 2005
Great approach if you want hello world to be a 20MB distributable.
Rich Rogers
Sunday, March 20, 2005
Hey Rich, who told you about my secret killer app?!
Sunday, March 20, 2005
I had a bit of experience with this a couple of years ago. I'd written a C++ program using wxWidgets, targeting Windows (it uses DirectX), and was going to make it with a C++ back end and Python for the GUI. I decided to embed the python interpreter into my program.

In the end I left it as straight C++, which was a shame but I found it didn't for me *quite* work out, for reasons that were mainly my fault.

What went well:

1. Python integration -- easy, I seem to recall, within about an hour it was all set up and I could run python strings from within my program;

2. wxPython integration -- less easy, but this should be fixed as I reported my findings to the wxPython author, who's pretty helpful, and the problem was clear anyway once I'd done some debugging. (something like: you need to call an init function (normally called by the wxwidgets framework) again once you've set up wxpython -- otherwise python code can't use the widgets properly)

3. Python/C++ glue -- I only needed functions and global builtins (ints and strings), so I wrote it all by hand. It wasn't a bottleneck.

4. python -- if you're designing your dialogs without a dialog editor it's really handy to be able to do this more rapidly. (I hacked the GUI scripts to work outside my app, for double-quick testing, which was very handy.) And it's easier in Python to create a much more helpful UI, in less time, and with fewer bugs.

5. wxwidgets -- python can talk to widgets created in C++, and vice versa; you can add python handlers to windows created in C++. (OK, so you'd expect this to work, but it's still pretty cool.)

What didn't work:

1. wxPython doesn't clean up after itself, so you get a ton of unfreed memory at the end. This makes it hard to track your own problems. I fixed some missing Py_DECREF calls in swig, which helped a bit, but there was still a good 2 or 3 pages of debug output at the end. Very tiresome.

2. python's C code is hard to debug; the VC++ debugger is really at the wrong level of abstraction. And the python datatypes are somewhat debugger-hostile. So if you create something in C++ which later causes problems in some python code, it's a bit tiresome to track down the problem.

3. Script reloading. This was my fault; I didn't set this up properly, partly due to laziness and partly due to the existing structure of the program, and so to try out a change to the main driver script I had to... erm... rerun the program. Really time-consuming if you've got some error in the python code, which is common if (like me) you're used to being able to rely on a compiler to catch your typos! (The GUI scripts I made reload automatically when the appropriate menu item was selected, which worked *much* better; I recommend structuring your program so all the scripts can be loaded like this, it would have helped me quite a bit.)

4. I couldn't work out how to get an interactive python prompt running in my program, for post-mortem debugging, so I had to do all the debugging of python code using print statements. This works fine for some things, but poorly for others.

I'm fairly handy with C++, so continuing to write the entire program in C++ didn't bother me much, and the VC++ debugger works very much better with pure C++ than a mixture of C++ and a scripting language. And after the entire code had been up on bricks for a while, and my TODO list was filling up with reimplementations of what I'd already written once, I started to feel like I wasn't really gaining anything. (see joel's rewrite essay :)

So with a heavy heart I scrapped the change. It's a shame, because as I was writing it it became very clear that less and less of the code actually needed to be C++, and in fact the majority of it could be written in Python...

Were I starting again, I'd be considering writing the program as a python extension. Then (I suspect) by running PythonWin under the debugger, you would get the best of both worlds -- with the debugger running, the same environment for debugging C++ as you'd have anyway, and a better environment for python debugging.

Unfortunately this program is a spare-time project (and 30,000 lines long) so as it stands this would be far too much work :(

So, there you go, for whatever it's worth...
Sunday, March 20, 2005
"Great approach if you want hello world to be a 20MB distributable."

This is false. The distributable is ~12mb uncompressed, and ~5mb in a compressed installer.
Sunday, March 20, 2005
Thanks to everybody for the great replies, it really helps a lot to talk to people who have been there & hear first-hand about what land mines they stepped on.  To Rich Rogers: I agree from personal experience, a restrained & minimalist style is the secret to success in C++.  Declare as much as you can on the stack, avoid gratuitous use of new and delete.  Bjarne Stroustrup points out that the C++ standard library is a lot more extensive these days, e.g. with a string class, and a vector class that nicely replaces arrays & does bounds-checking and memory management for you.  By using the existing tools, you can reduce the number of explicit new/delete statements to just a handful, or maybe none at all.

For this project, it's beginning to sound like the bang-to-buck ratio of Python would be quite low, since the GUI part is trivial & all the heavy lifting is in the back-end (which, as I have said, is close to the metal & necessarily coded in C).  Maybe Python would be a better choice for an app which is 90% GUI with at most a few low-level functions here & there in C.

A side note on this "productivity" thing--the needle on my BS meter is being driven into the red zone.  Just because language B can implement something in 50% fewer lines than language A does not mean that B is 50% more productive than A, because banging out lines of code is the smallest part of what a SW engineer does.  Some of the other tasks include deployment & fielding complaints from customers who can't get the thing to run right on their machine.  Someone who writes computer books for a living, though, can go ahead & plant the victory flag once they get their stuff running on their own machine, & these folks don't have to deal w/the messy realities of deploying & supporting real software.  When you experience what a time-suck that can be, you begin to see the virtue of releasing software as a monolithic, homogeneous block w/no external dependencies.

But then again, I'm full of s*** 50% of the time, and I leave it as an exercise to the reader to figure out which 50% :-)  So I welcome all comments, & thanks again everyone.
John Foxx
Sunday, March 20, 2005
"Declare as much as you can on the stack"

Amen to that, Brother Foxx!
Rich Rogers
Sunday, March 20, 2005
While I'm not averse to Python (I personally think it's pretty cool, along with Ruby & Haskell), or language mixing, for me it ultimately comes down to debugging.  And debugging in two or more languages simultaneously could be the showstopper (at least for me).

It's the same reason why I won't use little-known or "community supported" third party components in my apps -- even though they're written in the same language I'm using -- unless such components have a history of being well known and well supported.

If the language you know best is C++, you're going to much more adept at finding and fixing bugs in C++, and also refactoring your code, even though you *might* (but not necessarily) have to use twice as many lines of code.

That shouldn't stop you from using Python in projects of smaller scope where the stakes aren't as high as those for your "crown jewel" project.  It might be in a few years that you do choose Python over C++, and you do rewrite your app as a Python/C hybrid, but in the meantime, you really don't lose anything (except having to do more typing) by not doing that.

Anyway, I think the newer languages like Python (and Ruby, Haskell, etc.) are the future of app development, but the future isn't quite as standardized, time-tested, or as FAST as "the past" is.  At least in terms of overall years of service...  (and that's not a criticism of Python, just an observation!)...

Anyway, wishing you luck,

Peter Sherman Send private email
Monday, March 21, 2005
> When you experience what a time-suck that can be, you begin to see the virtue of releasing software as a monolithic, homogeneous block w/no external dependencies.

Right. That's why you should be cautious with that Python + C + GUI set if you need to have something install easily on very different computers. There's just to many things that can go wrong.
Monday, March 21, 2005

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz