lunglet.net

After employing various debugging techniques (mostly some RTFM and printf debugging), I was able to get C++ threads to call back into Python code successfully. In pyactivemq, this allows you to register a MessageListener on a Session and have the thread associated with the Session call into Python whenever a message arrives.

All the info I needed to solve the problem was contained in PEP 311: Simplified Global Interpreter Lock Acquisition for Extensions and Thread State and the Global Interpreter Lock. My main mistake was that I didn’t call PyEval_InitThreads() in my module’s initialization code, which seems to turn attempts to acquire the GIL into a no-op, which meant that I had multiple threads calling into the interpreter at the same time, with the associated “interesting” results.

After searching far and wide (actually, after Googling for the right phrase and clicking on the 13th result), I found a blog entry that deals with the issues you can run into when allocating and deallocating memory across module boundaries on Windows. Read it at The Old New Thing: Allocating and freeing memory across module boundaries.

I have sent pyactivemq 0.0.2 out into the world. pyactivemq is a wrapper for ActiveMQ message broker. pyactivemq uses Boost.Python to wrap the ActiveMQ C++ library.

Update: Since making this release, I’ve done a few more hours of work on pyactivemq. I’ve almost gotten multiple consumers with message listeners in separate sessions to receive messages, i.e. multiple threads calling back into Python code almost works. I need to stick a lock or grab onto the GIL somewhere, but I’m not exactly sure where yet.

I was recently writing some tests for a Python extension module that uses Boost.Python. To keep things agile (i.e. keep compile times down) I have a few #ifdefs in my code that allows me to disable various parts of the module when I’m only interested in other parts. This greatly reduces the time of edit-compile-test cycle when your module starts getting really large. I wanted to skip certain tests in my test suite depending on whether a part of the module was available. Here’s what I came up with:

import unittest

class _test_foo:
    def setUp(self): pass
    def test_baz(self):
        self.assert_(True)
    def tearDown(self): pass

try:
    from extmodule import foo
    class test_foo(_test_foo, unittest.TestCase):
        pass
except ImportError:
    print 'skipping foo tests'

if __name__ == '__main__':
    unittest.main()

I played with JRuby and Rails this weekend.

I’m trying to create a standalone JAR that contains Jetty and a Rails application so that you can fire up a server using java -jar railsapp.jar, like you can do with Hudson. This involves taking the existing work that allows you to bundle your Rails application into a WAR (which can be deployed to a server) a bit further. I haven’t quite figured out how to configure Jetty to do this yet. It seems Jetty’s WebAppContext wants to be configured with a directory that contains the web application. I’ve asked on the Jetty mailing list for some tips.

I choose redMine at random for my experiments. I couldn’t get redMine to create its database with Apache Derby, due to some issue with DATEs and TIMESTAMPs. Getting this to work would be great, since then you have a server-in-JAR that is completely standalone.

However, everything worked fine with MySQL over JDBC. Running redMine under JRuby using WEBrick worked great (after a minor workaround for JRUBY-964), but I had an issue with it failing to start when deployed as a WAR, due to some problem with the IConv.open method. I’ll investigate this a bit more.

Along the way, I submitted two JRuby bugs (JRUBY-954, JRUBY-964), commented on JRUBY-925 and produced a simple test case for JRUBY-956, which is a bug the Mingle folks ran into. I also did some debugging of JRUBY-822.

Along the way, this Rails Cheat Sheet came in handy, since I haven’t looked at Rails since 2005.

If I remember correctly, the The Pragmatic Programmer suggests that you learn a new language every now and then (every year?). Judging from a bunch of blogs I’ve been reading recently, you might want to look at one of the following:

• Erlang
• Haskell
• Squeak (Smalltalk)
• Scala
• Ruby

Along these lines I recently discovered two very interesting blogs you might want to keep an eye on:

• HREF Considered Harmful by Avi Bryant (I think of Seaside fame)
• murphee’s Rant

UISpec4J is library for testing Swing-based Java applications, built on top of JUnit. I’m not Swing’s biggest fan, but this looks pretty useful.

Now, on to why I don’t like Swing. Let’s take a look at a Open File dialog box in a a Swing application (NetBeans 6.0M9):

vs an SWT application (Eclipse 3.2.2):

Because SWT uses the native control, it works exactly like my other Windows applications. I can type something like %HOME%\foo\bar into the File name field and it will open the corresponding directory. Swing doesn’t understand %HOME%. Sure, it could be fixed to understand it, but that’s not the only issue. Another thing that pains me is that interaction with Swing using the keyboard doesn’t feel like other Windows applications. Here I’m thinking about using TAB to skip between elements, typing the first few letters of an item to jump to it or navigating with the arrow keys.

If NetBeans didn’t have such great support for Ruby and Rails, I wouldn’t have any major Swing applications on my machine. (By the way, how about a NetBeans download that only includes the core and the Ruby plugin? That would rock.)

Sonatype are working on a free book about Maven, Maven: The Definitive Guide.

Sun has open sourced the Java Dynamic Management Kit as OpenDMK. If you haven’t experienced the beauty that is JConsole (which uses JMX to retrieve all kinds of useful bits from your JVM), be sure to fire it up sometime. Now imagine JConsole on steroids, built right into your application. This can really take management of a large Java system (spread across many physical machines) to a new level. Disclaimer: I haven’t used this library yet, so I can only hope that it’s as great as they say it is.

GCC 4.2.0 has been released (after some delays). The most interesting feature for me is that OpenMP is now supported for the C, C++ and Fortran compilers. OpenMP is already supported by the Intel and Microsoft Visual Studio 2005 compilers (but not the Express editions of the latter). OpenMP allows you to make use of all the spankin’ new cores on your brand new CPU, without requiring any thread programming on your part. See also Wikipedia: OpenMP and the OpenMP tutorial.

I’m still a little unsure about what happens when other threaded code calls into OpenMP threaded code (is it even safe to do this?). I’ll see if I can find out.

According to Ulrich Drepper in The Growing Importance of Parallel Programming, the OpenMP bits have already been backported to GCC 4.1 in Fedora Core 6 and RHEL 5. This is why I love the Fedora guys.

By the way, you can get recent versions of MATLAB (everything from R2006a and up) to make use of multiple cores through OpenMP by setting the OMP_NUM_THREADS environment variable to some number greater than 1 before running MATLAB. You’ll see multiple cores being used for operations like matrix-matrix multiplication.

Also look out for the new -fstrict-overflow option in GCC. I haven’t looked at this in detail yet, but it’s been mentioned on the numpy-discussion mailing list as something to take notice of.

Update: See also Linux.com: New GCC 4.2.0 — boon to developers, bore to distros and Google Video: GCC: Current Topics and Future Directions.