Steve Frécinaux

Hacking PyGObject in Prague

.. gallery: pictures/*.jpg

It is amazing how fast time flies when you are busy. I was about to finally write down the last words of this article about the Gnome Python Hackfest from Prague, but I’m realizing now that one month ago, we were going out for our last beerful evening in Prague!

We had a really wonderful time there with the pygobject team, and despite we probably didn’t have enough daylight to visit most of the nice city of Prague, we got acquainted to the local culture of huge glasses of light beer and simple and nice (but very caloric) cooking. Check out the pictures on the right of this page!

Bringing more predictability into pygobject

My week at the hackfest started out by fixing what I’d call a “predictability bug”, in the sense that what happened before was logical according to code and history, but quite weird and unexpected when you don’t know it.

Historically, pygobject with static bindings didn’t register a new GType when you were subclassing an existing, bound GType, unless either you asked for this registration explicitely or some conditions where fulfilled for auto-registration to kick in. When the type gets registered, the eventual method overriding happened and everything was just fine. If you didn’t register your new class as a GType, then the methods were not overriden.

But then, GObject introspection made an appearance into pygobject. The way the dynamic python bindings for introspected types were created was very different, and, for very understandable reasons, the subclassing support hooked into it didn’t work exactly the same way as it did with the static bindings. One of the consequencies was that the method overriding happened sooner than before, and weird bugs started to pop here and there when subclasses didn’t trigger auto-registration.

This is quite easy to understand: since the method overriding happened directly when the python class was created (without giving a chance to the programmer to call gobject.type_register() himself), methods could be overriden on the wrong GType. In other words, if you used the following bit of code:

class MyWindow(Gtk.Window):
    def do_show(self):
        print "Shown!"
        super(MyWindow, self).do_show()

You would think that you just created a new GtkWindow subclass which would print something in the console when you show it. But you guessed wrong. Once that bit of code had been run, all the windows of your application would actually start to display such messages in the console, because since we didn’t register a new GType for this class, the new implementation of the show() virtual method would have been hooked up on the actual GtkWindow type. Weird isn’t it?

The fix for this issue was to make the pygobject behaviour much more predictable. From now on it would automatically register a new GType as soon as you create a subclass in Python, even if you don’t override anything of don’t touch the properties. Since your own application window is not the same as a bare GtkWindow, it makes sense to be able to tell them appart even at the C level… The only remaining special casing in this area of the pygobject code is to avoid creating GTypes for overrides, as it would quite obviously defeat the purpose of overrides, ie providing an alternative python class with refinements for an introspected type.

Are we leaking references?

The rest of my hackfest week revolved around fixing bug 639597 from EoG. Apparently, when using libpeas, at least one reference of the Eye of Gnome main window was leaked because when the window is closed, it wouldn’t get finalized.

This was the occasion for me to discover how to use refdbg and gdb to track down missing unrefs. I finally put my finger on an issue related to GtkWindow. This class is a strange beast: it is a widget (so, it’s a GInitiallyUnowned), but a toplevel one (as such, it doesn’t have a floating reference when its constructor returns). This reference can be understood as a reference owned by the windowing system, and will get released when using gtk_widget_destroy().

To understand this issue, we first need to know and understand that the objects we use from python are actually proxies: PyObject subclasses which forward python calls to the actual GObject instance. There is one proxy for each object instance, and it needs to own a reference to the object to ensure this object will continue to live while the proxy itself lives. But the issue with GInitiallyUnowned subclasses is the floating reference they have, which is helpful when you are programming in C but need to be converted into a full reference when you are managing references automatically. This can be done using g_object_ref_sink(), whose doc says:

If the object is floating, then this call “assumes ownership” of the floating reference, converting it to a normal while leaving the reference count unchanged. If the object is not floating, then this call adds a new normal reference increasing the reference count by one.

By reading the excerpt above, you can easily understand that if you end up calling this function more than once, then your refcount will increase and you’ll leak references. And you can not just say “do this if the reference is floating”, because you need to handle the GtkWindow case.

And the problem is that it can end up being called several times, because during an object’s lifetime, several wrappers can be created, because they get discarded when they get out of scope, and get recreated when an object is returned by some introspected method.

So the fix here was to ensure this function was only called once. The fix is actually different in the master and stable branch due to ABI stability issues with static bindings.

I also came across a python reference count issue and changed pygobject to always use pygobject_ref_sink() to take ownership on an object when gobject introspection’s meaning for a floating reference got altered from “one reference” to “zero reference”. Again, the fix is not the same in master and in the stable branch due to the guaranteed ABI stability for static bindings in the stable line.

I also wrote a bunch of tests to check the resulting reference counts in various scenarios. This is an advice I can give to all the contributors of bindings: Add checks for the expected reference counts in your test suite!

There were other people, too

Of course I wasn’t the only one working at the Hackfest, but I must say my personal favourite improvement from there was Laszlo’s efforts to make pygobject start much faster than before. With several enhancements in the pythonic side of the introspected API with overrides, it wasn’t long before Gtk would take a long time before starting up properly.

Laszlo basically managed to reduce the import time of the Gtk 3 module from 2.5 seconds to 0.4 seconds, “just” by sanitizing the logics in the way python methods were hooked up into the python classes.

At the same time, Martin Pitt, which has been a wonderful roommate, was enhancing the GVariant and GDBus APIs, John Palmieri was working on his overhaul of the method invocation code, Tomeu Vizoso was fixing Cairo, Simon van der Linden was giving a shot at rewriting the import logic in C, while Ignacio Casal, Simon Schampijer and Sebastian Pölsterl were porting pygtk code to the new introspected bindings. At the same time, remotely, Pavel was providing us a huge patch fixing a lot of Gtk+ annotations.

After the Hackfest

Since the Hackfest, there were a bunch of releases in the stable pygobject-2.28 branch, giving GI to the masses. It contains most of the fixes and improvements which were done by the team at the Hackfest. Only a very few are still waiting in the master branch, because those break the support for the static bindings. This is fine, because the plan is to break the ABI and remove most of the legacy cruft remaining in pygobject in the master branch, and make a clean start for Python 3.

But most importantly, this Hackfest had a notable effect of involving quite a lot of people into coding on pygobject, bringing back pygobject from a quasi-unmaintained state to a very active state.

For that and for all the drinks we had, I think both the Gnome Foundation and Collabora should be thanked. And, of course, everyone involved in pygobject and in Gnome ;-)

Sponsored by Gnome Foundation

On my side, I thought that Hackfest would be a good way to get back on track coding on Gnome, but unfortunately I’ve been very busy lastly… and I still have a few half-baked patches from the Hackfest to finish and submit. Time…