Armin's personal Blog

I have left Openismus at the End of June as I will be doing a summer internship at Duke during the next three months for which I have received a DAAD scholarship. I will be heading to the States on July 13, and I'm not going to attend GCDS therefore (even though it wouldn't overlap, I don't feel comfortable with such few time inbetween. Plus I would miss even more University stuff here at Karlsruhe).

At Duke, I will be working with the HEP Neutrino group (supervised by Kate Scholberg) by taking part in simulation studies for next-generation Water Cherenkov neutrino detectors, with special regard to Supernova neutrinos. The goal is to choose detector design parameters so that we gain a maximum of information from the detector, for example about Neutrino oscillation. This will be the first time that I will actually take part in modern research, and actually be applying (some of) the stuff I have learned at University. After having finished the internship, I also hope to have gained insights into whether I want to write my Diploma thesis (starting in a bit more than a year, if everything goes well) in the field of theoretical or experimental (Particle) Physics.

I would like to thank everyone at Openismus. Earning money for working on Open Source Software is a great thing. This helped me a lot to afford my studies. At the end of october, there is a scholarship holder meeting in Berlin. I hope I can visit the Openismus Office then.

A small Openismus task I recently carried out was to investigate whether there is a generic timeline widget for GTK+ out there, for example to show photos associated with the day they were taken. The main feature this is supposed to have is that the items it contains are grouped by time periods, such as day, week or month, and that it should be possible to view multiple periods at once, or to allow easy browsing between them.

The closest thing I found is Gnome Zeitgeist, an application which takes up Federico's Journal Idea. Gnome Zeitgeist is still under heavy development (I also had a short look at it yesterday, and when I updated the branch today it looked totally different), but the timeline widget looks promising already. It shows recently used files ordered by the day they have been used. By default, it shows three days, but when filtering by tags, it can also show more (scrolling horizontally if necessary), omitting days with no entries at all. This is implemented using multiple GtkTreeViews, therefore I guess it would be easy to generalize the widget to show arbitrary data by allowing to pack own cell renderers into it.

Then, there is Nemo, a program which aims at making file management easier by showing all documents (not only recently used ones) in a calendar type of view. There are views for a day, a week, a month or a year. However, when there are more items than there is space in a calendar cell, then it simply shows "+ 309 more". Gnome Zeitgeist allows scrolling instead, which is more handy when for example previewing photos.

Other projects I had a look at include Gnome-Shell (which shows recently used icons ordered, but not grouped), Wizbit (which has a timeline widget for the revision graph, but Wizbit has slightly different usecases), Paperbox and tracker-search-tool (both of which don't group items by time periods).

The Gnome Zeitgeist widget looks really promising, though. Although it is not a stand-alone widget, I don't think it would be too much work to separate it and make it more generic.

Are there more solutions to this kind of problem in the GTK+ world? Any application I did not have a look at, although I should in this context? If so, please tell me in the comments.

Over the last few months I implemented SQLite support in Glom for Openismus. When creating a new document, Glom now allows to save the data in a SQLite database instead of a postgresql database. This is more light-weight, and there are generally fewer things that can go wrong as everything is performed within the same process. However, there are some obvious limitations: As SQLite does not support authentication, Glom documents using an SQLite database can be opened without access restriction either. Also, the Glom session is not announced on the network since the SQLite database can not be accessed from remote anyway.

When implementing this, I tried to get rid of most of the postgresql-specific code which was in quite a few places throughout Glom, and putting it into a single place, and using libgda instead of hand-written SQL when possible. This should make it relatively easy to add more database backends in the future, if people need any. There is only some code dealing with users and privileges that may be postgres-specific and which I didn't touch, simply because it is not used with SQLite anyway.

One of the interesting parts was to map the different glom data types to SQLite data types. The switch to libgda-4 helped with this, since libgda-4 allows storing dates and timestamps in SQLite databases, by converting them to standardized strings. However, SQLite does not support a "numeric" type, but only integer and real. Currently, Glom maps its numeric type always to an SQLite real, which worked surprisingly well so far, though I'm still not 100% happy with it, mostly due to possible floating point inaccuracies.

Another problem was changing a table, meaning adding, modifying or removing columns, or changing the primary key column. As SQLite only supports adding non-primary key columns, but not changing existing columns or removing columns, Glom recreates the whole table in that case and moves the data from the old table into the new one. The tricky part of this was to get all possible type conversions right. For example, when changing the type from Image to something else, then the conversion failed with a "Database table is locked" error only if at least one row contained an actual (non-NULL) image. It took me some time to find out that SQLite does not allow dropping a table when a recordset from a SELECT query is still in use, and that Glom was indeed still holding such a recordset to display records from the current table. The problem went away when I released that recordset before changing the field type, though I wonder why there was no problem when there were no images in that column, or for other field type changes not involving images.

This work also yielded some bugs in the upcoming libgda-4, but Vivien has been very responsive so that they have been fixed very quickly. Kudos!

The SQLite version feels much faster and more responsive than the postgresql one when creating a new database or populating the list view. I wonder whether this can be explained with the overhead that comes with postgresql (talking to another process via TCP/IP) or there is a bottleneck elsewhere, which, if fixed, would make (postgresql) glom more enjoyable to use.

The current tarball version, 1.9.1, already supports SQLite, but still has some problems with changing field types which have been fixed in SVN. A 1.9.2 will hopefully be released soon. The next stable version of Glom, 1.10, will also contain SQLite support.

Sorry, no screenshots this time, since there wouldn't be anything to see that would be different from a postgresql-based Glom, except maybe an option to create a SQLite database when creating a new Glom document.

Look at this screenshot of the recently released Gobby 0.4.90. See that nice yellow arrow in the toolbar? Yes, this means we finally have Undo/Redo support. It was a really good feeling having closed bug #39

Gobby 0.4.90 is the first release in the 0.5 series, and 99% of the code have been rewritten from scratch. It is not yet 0.5.0 since it does not yet contain all the features that previous versions had (for example, it does not yet support self-hosting or password protection). However, removing the integrated chat was a design decision, since there are so many other possibilities for communication, including VoIP. As it supports removing documents, only use it with people you trust enough not to delete all your data.

Gobby 0.4.90 does no longer use obby or net6 as backend libraries doing the hard work, but libinfinity, implementing the Infinote protocol. libinfinity is not yet stable API and ABI wise, and probably will not be for still some time. libinfinity contains a dedicated server, called infinoted that Gobby 0.4.90 can connect to.

You can safely install the new version since it is parallel-installable to previous Gobby versions. But please consider everything as very experimental. Things may crash or otherwise behave badly. Report bugs if they do. We set up a playground server on dalaran.0x539.de if you want to test the new version without having to install server yourself.

I was curious where most time in libinfinity, Gobby's infinote implementation, is spent. Not because it was too slow or something, but just because I wanted to know. So I used valgrind's callgrind tool to gain some profile data while running libinfinity's main test for the concurrncy control algorithm. When I displayed the result in kcachegrind (there is still no GNOME equivalent, no?) everything looked as I would have expected, until I noticed this line:

This means that 14.5% of the overall program time is spent in g_type_instance_get_private(). This is because libinfinity uses g_type_class_add_private for every class to store its members into, so it is easy to extend functionality later without breaking ABI. However, this also means that for every function that needs to access a member, g_type_instance_get_private() is called. I didn't expect such a huge impact, though.

To fix this, I added a simple gpointer to the public instance struct, and let it point to the private field in the instance_init function. So this means one call to g_type_instance_get_private() per instance, instead of roughly one per call to a function operating on an instance of the class in question. After doing this for the five classes suffering most from this, g_type_instance_get_private does no longer show up anywhere near relevant in kcachegrind.

I'd like it if improving performance was always as easy as this.

With the release of gtkmm 2.14 I created new installers for Microsoft Windows; the last ones were only for 2.10. We also wrote down instructions on how to use the installers, and how to recreate them (in case anybody feels the need...). They now contain everything needed for gtkmm to run on Windows, including GTK+ itself (from the GTK+ all-in-one bundle).

I used the NSIS installer script from Cédric Gustin, the previous gtkmm Windows maintainer. It is now in gtkmm SVN. Comparing with InnoSetup (which I used for Glom's Windows installer) I must say that InnoSetup scripts are easier to write and maintain (especially for a few common tasks such as creating StartMenu icons or creating an uninstaller). The gtkmm NSIS script has around 1700 lines while the Glom InnoSetup script has only around 200. I know that the gtkmm installer is more complex, but that's still a huge difference.

Gregory Haynes is working on Kobby featuring collaborative text editing in your favourite KDE text editor as a GSoC project. He started to wrap the (still unreleased!) libinfinity (my infinote implementation) API for C++ for this task. I'm really excited to see other collaborative text editors coming up that work together with Gobby, since this is exactly one of the reasons why I took the burden to write the library in GObject-based C.

Speaking of which; I didn't blog on it for a long time. We have moved to git in the meantime. I'm currently working on porting Gobby to use libinfinity instead of obby, heading towards a first usable release, so that the hard work pays off. There are still some things in libinfinity that need to be fixed (performance issues, a few known bugs and way more testing) which are probably easier to fix when a usable Gobby version using it exists. I hope to have these done in a few months, though the most difficult part is always to get the little details right, which is probably why I'm not good at meeting (self-set) deadlines.

I updated the installer. It now includes jpeg62.dll and python25.dll which were missing (thanks to the reporters on my initial post). I also included the glom and gda python modules. If I do understand things right, then installing Python and pygtk makes Python support in Glom automatically work with this. I tried this out by reinstalling Python into a different directory, which worked.

Again, if there are still problems with it, please tell me.

Here it is, Glom's installer for Windows. Thanks to the opinions on my last blog entry. I set up an all-in-one installer, so Glom should run out of the box after installing (please tell me if not). It includes GTK+ from SVN with my patch from bug #506062 (hint, hint) so that recent files filtering works.

Since most people seem to use NSIS for Windows installers (comment on this blog entry, gtk-win.sourceforge.net) I also had a look at it. However, the scripting language reminds me more of assembly than anything else, and I think these days are over. So I used InnoSetup which has Pascal scripting for the rare cases you need it.

The only thing that is not going to work is python scripts for buttons and calculated fields. Ideally, one would just install Python and PyGtk, and having Glom use that installation automatically when present. This shouldn't be too hard, it just needs to tell python where to find the own gda and glom python modules. I am going to tackle this after my physics exam on Thursday.

With added support for self-hosting, Glom for Windows now supports all the major features that the Linux version does (apart from Service Publishing, which is probably way more effort since Avahi is not available on Windows). It's time to think about how to package it, and what dependencies to include, especially whether to ship with GTK+ or not.

All the major GTK+-using projects such as Inkscape, Pidgin and Wireshark include GTK+. Even the GIMP does so since version 2.4. This actually generates a lot of duplication, which is why we (or, rather, I) did decide not to ship it with Gobby. Of course we are getting complaints such as "Hey, this doesn't work because libgtk-win32-2-0-0.dll was not found!!1" from time to time, but if people are not able to read the instructions right above the download link, well, then I'm sorry. Do others think the added duplication is worth that the user needs to install GTK+ manually once? Or, maybe, is this just because there is no "official" GTK+ installer?

Then, there is python, and I have not so much experience here. Glom links statically against libpython, so it should even run without python being installed. However, buttons and calculated fields won't work then, unless we ship with at least pyGTK. Perhaps the best thing is to state that for button scripts and calculated fields to work, python and pyGTK have to be installed. We just have to make sure that Python finds the glom and pygda modules then, even if Python was installed after Glom (so it had no chance to install them into the standard location for such modules), but probably Python has some API for this.

Last but not least, we have gtkmm which also has an installer that is hosted on ftp.gnome.org even. I think we should either ship both GTK+ and gtkmm, or none of them.

If you have any thoughts on this, recommendations and suggestions are welcome.

There are several ways to fix the recent chooser showing no recent files in Glom on Windows, but they are all sort of hacks except one, namely make the recent chooser guess the mime type of an URI on Windows as it already does on Unix. Currently, it always falls back to application/octet-stream and this is why the filter on application/x-glom that glom uses does not show up any files. I looked around a bit in the registry, and it seems that all known file extensions are in HKEY_CLASSES_ROOT/.$ext, and some of them have a "Content Type" key that looks like the mime type we are after. So all we need to do is to register the .glom type there (this can also be used to associate a nice icon with it) and to make the GTK+ recent chooser use that "Content Type" key for the extension of the URI it gets as default mime type.

To implement this, I need to be able to build GTK+ (and glib which GTK+ from SVN trunk depends on) on Windows. The easy method I already used for glom itself was to copy tarballs created by "make dist" to the windows machine so I don't have to create the configure script there, not requiring the autotools stuff. However, this gets horribly annoying when the repositories are updated frequently. It would be much nicer to have a working copy directly on Windows, and to (re)create the configure script there when necessary. In theory, this should work since mingw provides all necessary tools. In practise, I already stumbled upon several ugly problems in the past. I gave it another try, though, and I ended up with a freshly built glib from SVN at the end of the day. Below are the exact steps that got me this far.

I used the environment I set up as described here as a starting point. It is probably equally well possible to start with a freshly set up MSYS+MingW, though.

• First, install termcap, libtool, automake and autoconf from mingw.org into C:\msys (without the usr/ prefix that is in the tarball, such that files end up as /bin/autoconf etc.). Copy /bin/automake to /bin/automake-1.10 and /bin/aclocal to /bin/aclocal-1.10. The -1.10 versions are just 0-byte stubs, but glib's autogen.sh calls them explicitely.
• Then, rename the /bin/perl wrapper script and install the mingw-provided perl package. This is still 5.6, but it is enough for what we need, and the problems with paths in arguments seem to go away with this. Perhaps it would be enough to install ActivePerl directly so that /bin/perl.exe is the ActivePerl one.
• Add gtk-doc stubs to your system. Most GNOME libraries (including glib and GTK) require gtk-doc, but I could not find any success story of getting it to work on Windows. I used the instructions by Hans Breuer:

  cd gtk-doc
  cp gtk-doc.m4 /gtk/share/aclocal
  cp gtkdocize /gtk/bin
  mkdir -p /gtk/share/gtk-doc/data/
  cp gtk-doc.make /gtk/share/gtk-doc/data

  You need a gtk-doc tarball from ftp.gnome.org for this.
• Next, load svn-win32-1.4.5.zip (or a more recent version) from subversion.tigris.org, unpack the zip to C:\svn and add /c/svn/bin to the PATH in your ~/.profile. After a relogin or exec bash -l you should be able to use the SVN client in msys. Try getting glib via svn co http://svn.gnome.org/svn/glib/trunk glib.
• The most hacky thing I had to do was changing the /bin/aclocal file. Add

  $file =~ s/^C:\/msys\//\/usr\//;

  around line 680, right before

  $map_traced_defs{$arg1} = $file
    if ($macro eq 'AC_DEFUN'
        || $macro eq 'AC_DEFUN_ONCE'
        || $macro eq 'AU_DEFUN');
  

  Note that you need to adjust C:\/msys\/ if you installed msys elsewhere. Also remember to copy it to /bin/aclocal-1.10 (By the way, NTFS is capable of symlinks somehow, isn't it)? This is required so that aclocal converts paths starting with C:/msys/share/[...] to /usr/share/[...]. Both refer to the same file on disk, but since aclocal does a simple string comparison, it does not recognize this and always creates an empty aclocal.m4, causing autoconf to fail due to not finding any macros. Better solutions to this are very much appreciated.
• Last, replace AM_CONFIG_HEADER by AC_CONFIG_HEADER in glib's configure.in. autoheader complains about the first. Quick googling seems to suggest that AM_CONFIG_HEADER is deprecated by AC_CONFIG_HEADER, but I am not sure. If that is really the case we should probably change glib.
• Run ./autogen.sh and watch the magic.

So far for today, I am going to try GTK+ and glom tomorrow. Perhaps I even get to what I originally wanted to do, making the GTK+ recent chooser guess mime types on Windows. If you try these steps out and run into further problems, I would be glad to hear about it. I am especially looking forward to John Stowers' work on jhbuild on Windows. Building GNOME libraries and applications directly from SVN via jhbuild on Windows sounds really cool.

After porting Glom to maemo, the next target was Windows. Glom's dependencies are already ported to Windows, so getting it to work wasn't too hard, though some hacks were still necessary to get things going (see the Windows build instructions). Currently, only the client-only mode of Glom runs on Windows, but I hope the full version follows soon.

I installed Windows XP within a virtual machine on my laptop to do the port. The good thing is that this doesn't force me to leave my usual Linux environment, especially when I am not at home where I still have a desktop Linux machine. However, on the other hand compilation is slowed down pretty much, becoming really significant when compiling rather big C++ projects such as gtkmm and glom, probably due to memory constraints.

As always, pictures say more than a thousand words, so enjoy. The third screenshot shows a weird problem when scrolling directly after connecting to the database. The bug disappears when minimizing and re-maximizing the window. The screenshots are truncated to the left because the Glom Window requires more space than the 800x600 assigned to the virtual machine for the Small Business Example (which the screenshots are from). I could also use 1024x768 (the laptop has 1280x800), but I like to have a terminal or other windows next to the VM, and I don't like scrollbars in it either. Other drawbacks the current version suffers from and which I am going to tackle within the next days are listed in the Glom Wiki.

I recently ported glom to the maemo platform for Nokia's internet tablets for Openismus. The major work of this was compiling glom and its dependencies in scratchbox, i.e. make it build and run without using some convencience API in glibmm and friends to save code size. This leads to a lot of #ifdef statements uglifying the code (especially because you cannot use exceptions for the code to compile with the -fno-exceptions flag), but well, that's the price you have to pay if you want to use C++ on maemo (you can at least save the #ifdefs if you are writing a maemo-only application, though). It's still much more convenient than plain C. Note that the maemo version of glom is a client only version, i.e. it does not support self-hosting and developer mode.

The work has already been committed to glom trunk, though I keep making changes in the GLOM_CLIENTONLY branch to be able to make debian packages for sardine extras that are based on glom 1.6. With friendly help from Johannes Schmid and Philipp Kern I managed to build debian packages for glom and its dependencies that are already in sardine extras. Feel free to contact me if there are problems with them, since these are the first ones I ever made, and I am not too familiar with debian based distributions either, because I am not using one.

These screenshots show how the whole thing currently looks like. I made use of the hildonmm APIs where appropriate to get a more native maemo look-and-feel, though there is still some work left in that regard. At some places, glom measures size of text to determine the width and/or height of its widgets, using gtk_widget_create_pango_layout() and pango_layout_get_pixel_size(). However, this seems to result in the size of the text for the normal GTK+ theme, not the one with the bigger font on maemo, and this renders glom's guesses about widget sizes rather unusable. I fixed the most obvious uses (like the height of the tree rows in the glom list view) by hardcoding the size, but of course that is not a permanent solution. I am thankful for any hints how to get a correctly setup PangoLayout to do text measuring on maemo.

The automatic column sizing of GtkTreeView also seems not too useful for glom on maemo (see second screenshot). We probably should rather allow the user to scroll the list view horizontally, and introduce a minimum width for the columns.