Armin's personal Blog - Openismus

Timeline widgets for GTK+

nospam@example.com (Armin Burgmeier) — Wed, 25 Mar 2009 16:56:00 +0100

A small Openismus task I recently carried out was to investigate whether there is a generic timeline widget for GTK+ out there, for example to show photos associated with the day they were taken. The main feature this is supposed to have is that the items it contains are grouped by time periods, such as day, week or month, and that it should be possible to view multiple periods at once, or to allow easy browsing between them.

The closest thing I found is Gnome Zeitgeist, an application which takes up Federico's Journal Idea. Gnome Zeitgeist is still under heavy development (I also had a short look at it yesterday, and when I updated the branch today it looked totally different), but the timeline widget looks promising already. It shows recently used files ordered by the day they have been used. By default, it shows three days, but when filtering by tags, it can also show more (scrolling horizontally if necessary), omitting days with no entries at all. This is implemented using multiple GtkTreeViews, therefore I guess it would be easy to generalize the widget to show arbitrary data by allowing to pack own cell renderers into it.

Then, there is Nemo, a program which aims at making file management easier by showing all documents (not only recently used ones) in a calendar type of view. There are views for a day, a week, a month or a year. However, when there are more items than there is space in a calendar cell, then it simply shows "+ 309 more". Gnome Zeitgeist allows scrolling instead, which is more handy when for example previewing photos.

Other projects I had a look at include Gnome-Shell (which shows recently used icons ordered, but not grouped), Wizbit (which has a timeline widget for the revision graph, but Wizbit has slightly different usecases), Paperbox and tracker-search-tool (both of which don't group items by time periods).

The Gnome Zeitgeist widget looks really promising, though. Although it is not a stand-alone widget, I don't think it would be too much work to separate it and make it more generic.

Are there more solutions to this kind of problem in the GTK+ world? Any application I did not have a look at, although I should in this context? If so, please tell me in the comments.

SQLite support in Glom

nospam@example.com (Armin Burgmeier) — Wed, 25 Feb 2009 22:50:00 +0100

Over the last few months I implemented SQLite support in Glom for Openismus. When creating a new document, Glom now allows to save the data in a SQLite database instead of a postgresql database. This is more light-weight, and there are generally fewer things that can go wrong as everything is performed within the same process. However, there are some obvious limitations: As SQLite does not support authentication, Glom documents using an SQLite database can be opened without access restriction either. Also, the Glom session is not announced on the network since the SQLite database can not be accessed from remote anyway.

When implementing this, I tried to get rid of most of the postgresql-specific code which was in quite a few places throughout Glom, and putting it into a single place, and using libgda instead of hand-written SQL when possible. This should make it relatively easy to add more database backends in the future, if people need any. There is only some code dealing with users and privileges that may be postgres-specific and which I didn't touch, simply because it is not used with SQLite anyway.

One of the interesting parts was to map the different glom data types to SQLite data types. The switch to libgda-4 helped with this, since libgda-4 allows storing dates and timestamps in SQLite databases, by converting them to standardized strings. However, SQLite does not support a "numeric" type, but only integer and real. Currently, Glom maps its numeric type always to an SQLite real, which worked surprisingly well so far, though I'm still not 100% happy with it, mostly due to possible floating point inaccuracies.

Another problem was changing a table, meaning adding, modifying or removing columns, or changing the primary key column. As SQLite only supports adding non-primary key columns, but not changing existing columns or removing columns, Glom recreates the whole table in that case and moves the data from the old table into the new one. The tricky part of this was to get all possible type conversions right. For example, when changing the type from Image to something else, then the conversion failed with a "Database table is locked" error only if at least one row contained an actual (non-NULL) image. It took me some time to find out that SQLite does not allow dropping a table when a recordset from a SELECT query is still in use, and that Glom was indeed still holding such a recordset to display records from the current table. The problem went away when I released that recordset before changing the field type, though I wonder why there was no problem when there were no images in that column, or for other field type changes not involving images.

This work also yielded some bugs in the upcoming libgda-4, but Vivien has been very responsive so that they have been fixed very quickly. Kudos!

The SQLite version feels much faster and more responsive than the postgresql one when creating a new database or populating the list view. I wonder whether this can be explained with the overhead that comes with postgresql (talking to another process via TCP/IP) or there is a bottleneck elsewhere, which, if fixed, would make (postgresql) glom more enjoyable to use.

The current tarball version, 1.9.1, already supports SQLite, but still has some problems with changing field types which have been fixed in SVN. A 1.9.2 will hopefully be released soon. The next stable version of Glom, 1.10, will also contain SQLite support.

Sorry, no screenshots this time, since there wouldn't be anything to see that would be different from a postgresql-based Glom, except maybe an option to create a SQLite database when creating a new Glom document.

Glom Windows Installer 2

nospam@example.com (Armin Burgmeier) — Sat, 09 Feb 2008 14:29:12 +0100

I updated the installer. It now includes jpeg62.dll and python25.dll which were missing (thanks to the reporters on my initial post). I also included the glom and gda python modules. If I do understand things right, then installing Python and pygtk makes Python support in Glom automatically work with this. I tried this out by reinstalling Python into a different directory, which worked.

Again, if there are still problems with it, please tell me.

Glom Windows Installer

nospam@example.com (Armin Burgmeier) — Mon, 04 Feb 2008 21:58:00 +0100

Here it is, Glom's installer for Windows. Thanks to the opinions on my last blog entry. I set up an all-in-one installer, so Glom should run out of the box after installing (please tell me if not). It includes GTK+ from SVN with my patch from bug #506062 (hint, hint) so that recent files filtering works.

Since most people seem to use NSIS for Windows installers (comment on this blog entry, gtk-win.sourceforge.net) I also had a look at it. However, the scripting language reminds me more of assembly than anything else, and I think these days are over. So I used InnoSetup which has Pascal scripting for the rare cases you need it.

The only thing that is not going to work is python scripts for buttons and calculated fields. Ideally, one would just install Python and PyGtk, and having Glom use that installation automatically when present. This shouldn't be too hard, it just needs to tell python where to find the own gda and glom python modules. I am going to tackle this after my physics exam on Thursday.

Glom packaging on Windows

nospam@example.com (Armin Burgmeier) — Sun, 27 Jan 2008 01:41:00 +0100

With added support for self-hosting, Glom for Windows now supports all the major features that the Linux version does (apart from Service Publishing, which is probably way more effort since Avahi is not available on Windows). It's time to think about how to package it, and what dependencies to include, especially whether to ship with GTK+ or not.

All the major GTK+-using projects such as Inkscape, Pidgin and Wireshark include GTK+. Even the GIMP does so since version 2.4. This actually generates a lot of duplication, which is why we (or, rather, I) did decide not to ship it with Gobby. Of course we are getting complaints such as "Hey, this doesn't work because libgtk-win32-2-0-0.dll was not found!!1" from time to time, but if people are not able to read the instructions right above the download link, well, then I'm sorry. Do others think the added duplication is worth that the user needs to install GTK+ manually once? Or, maybe, is this just because there is no "official" GTK+ installer?

Then, there is python, and I have not so much experience here. Glom links statically against libpython, so it should even run without python being installed. However, buttons and calculated fields won't work then, unless we ship with at least pyGTK. Perhaps the best thing is to state that for button scripts and calculated fields to work, python and pyGTK have to be installed. We just have to make sure that Python finds the glom and pygda modules then, even if Python was installed after Glom (so it had no chance to install them into the standard location for such modules), but probably Python has some API for this.

Last but not least, we have gtkmm which also has an installer that is hosted on ftp.gnome.org even. I think we should either ship both GTK+ and gtkmm, or none of them.

If you have any thoughts on this, recommendations and suggestions are welcome.

autotools on Windows

nospam@example.com (Armin Burgmeier) — Wed, 19 Dec 2007 00:53:00 +0100

There are several ways to fix the recent chooser showing no recent files in Glom on Windows, but they are all sort of hacks except one, namely make the recent chooser guess the mime type of an URI on Windows as it already does on Unix. Currently, it always falls back to application/octet-stream and this is why the filter on application/x-glom that glom uses does not show up any files. I looked around a bit in the registry, and it seems that all known file extensions are in HKEY_CLASSES_ROOT/.$ext, and some of them have a "Content Type" key that looks like the mime type we are after. So all we need to do is to register the .glom type there (this can also be used to associate a nice icon with it) and to make the GTK+ recent chooser use that "Content Type" key for the extension of the URI it gets as default mime type.

To implement this, I need to be able to build GTK+ (and glib which GTK+ from SVN trunk depends on) on Windows. The easy method I already used for glom itself was to copy tarballs created by "make dist" to the windows machine so I don't have to create the configure script there, not requiring the autotools stuff. However, this gets horribly annoying when the repositories are updated frequently. It would be much nicer to have a working copy directly on Windows, and to (re)create the configure script there when necessary. In theory, this should work since mingw provides all necessary tools. In practise, I already stumbled upon several ugly problems in the past. I gave it another try, though, and I ended up with a freshly built glib from SVN at the end of the day. Below are the exact steps that got me this far.

I used the environment I set up as described here as a starting point. It is probably equally well possible to start with a freshly set up MSYS+MingW, though.

First, install termcap, libtool, automake and autoconf from mingw.org into C:\msys (without the usr/ prefix that is in the tarball, such that files end up as /bin/autoconf etc.). Copy /bin/automake to /bin/automake-1.10 and /bin/aclocal to /bin/aclocal-1.10. The -1.10 versions are just 0-byte stubs, but glib's autogen.sh calls them explicitely.
Then, rename the /bin/perl wrapper script and install the mingw-provided perl package. This is still 5.6, but it is enough for what we need, and the problems with paths in arguments seem to go away with this. Perhaps it would be enough to install ActivePerl directly so that /bin/perl.exe is the ActivePerl one.
Add gtk-doc stubs to your system. Most GNOME libraries (including glib and GTK) require gtk-doc, but I could not find any success story of getting it to work on Windows. I used the instructions by Hans Breuer:
```
cd gtk-doc
cp gtk-doc.m4 /gtk/share/aclocal
cp gtkdocize /gtk/bin
mkdir -p /gtk/share/gtk-doc/data/
cp gtk-doc.make /gtk/share/gtk-doc/data
```
You need a gtk-doc tarball from ftp.gnome.org for this.
Next, load svn-win32-1.4.5.zip (or a more recent version) from subversion.tigris.org, unpack the zip to C:\svn and add /c/svn/bin to the PATH in your ~/.profile. After a relogin or exec bash -l you should be able to use the SVN client in msys. Try getting glib via svn co http://svn.gnome.org/svn/glib/trunk glib.
The most hacky thing I had to do was changing the /bin/aclocal file. Add
```
$file =~ s/^C:\/msys\//\/usr\//;
```
around line 680, right before
```
$map_traced_defs{$arg1} = $file
  if ($macro eq 'AC_DEFUN'
      || $macro eq 'AC_DEFUN_ONCE'
      || $macro eq 'AU_DEFUN');
```
Note that you need to adjust C:\/msys\/ if you installed msys elsewhere. Also remember to copy it to /bin/aclocal-1.10 (By the way, NTFS is capable of symlinks somehow, isn't it)? This is required so that aclocal converts paths starting with C:/msys/share/[...] to /usr/share/[...]. Both refer to the same file on disk, but since aclocal does a simple string comparison, it does not recognize this and always creates an empty aclocal.m4, causing autoconf to fail due to not finding any macros. Better solutions to this are very much appreciated.
Last, replace AM_CONFIG_HEADER by AC_CONFIG_HEADER in glib's configure.in. autoheader complains about the first. Quick googling seems to suggest that AM_CONFIG_HEADER is deprecated by AC_CONFIG_HEADER, but I am not sure. If that is really the case we should probably change glib.
Run ./autogen.sh and watch the magic.

So far for today, I am going to try GTK+ and glom tomorrow. Perhaps I even get to what I originally wanted to do, making the GTK+ recent chooser guess mime types on Windows. If you try these steps out and run into further problems, I would be glad to hear about it. I am especially looking forward to John Stowers' work on jhbuild on Windows. Building GNOME libraries and applications directly from SVN via jhbuild on Windows sounds really cool.

Glom on Windows

nospam@example.com (Armin Burgmeier) — Wed, 05 Dec 2007 00:07:00 +0100

After porting Glom to maemo, the next target was Windows. Glom's dependencies are already ported to Windows, so getting it to work wasn't too hard, though some hacks were still necessary to get things going (see the Windows build instructions). Currently, only the client-only mode of Glom runs on Windows, but I hope the full version follows soon.

I installed Windows XP within a virtual machine on my laptop to do the port. The good thing is that this doesn't force me to leave my usual Linux environment, especially when I am not at home where I still have a desktop Linux machine. However, on the other hand compilation is slowed down pretty much, becoming really significant when compiling rather big C++ projects such as gtkmm and glom, probably due to memory constraints.

As always, pictures say more than a thousand words, so enjoy. The third screenshot shows a weird problem when scrolling directly after connecting to the database. The bug disappears when minimizing and re-maximizing the window. The screenshots are truncated to the left because the Glom Window requires more space than the 800x600 assigned to the virtual machine for the Small Business Example (which the screenshots are from). I could also use 1024x768 (the laptop has 1280x800), but I like to have a terminal or other windows next to the VM, and I don't like scrollbars in it either. Other drawbacks the current version suffers from and which I am going to tackle within the next days are listed in the Glom Wiki.

Glom on maemo

nospam@example.com (Armin Burgmeier) — Thu, 18 Oct 2007 22:55:00 +0200

I recently ported glom to the maemo platform for Nokia's internet tablets for Openismus. The major work of this was compiling glom and its dependencies in scratchbox, i.e. make it build and run without using some convencience API in glibmm and friends to save code size. This leads to a lot of #ifdef statements uglifying the code (especially because you cannot use exceptions for the code to compile with the -fno-exceptions flag), but well, that's the price you have to pay if you want to use C++ on maemo (you can at least save the #ifdefs if you are writing a maemo-only application, though). It's still much more convenient than plain C. Note that the maemo version of glom is a client only version, i.e. it does not support self-hosting and developer mode.

The work has already been committed to glom trunk, though I keep making changes in the GLOM_CLIENTONLY branch to be able to make debian packages for sardine extras that are based on glom 1.6. With friendly help from Johannes Schmid and Philipp Kern I managed to build debian packages for glom and its dependencies that are already in sardine extras. Feel free to contact me if there are problems with them, since these are the first ones I ever made, and I am not too familiar with debian based distributions either, because I am not using one.

These screenshots show how the whole thing currently looks like. I made use of the hildonmm APIs where appropriate to get a more native maemo look-and-feel, though there is still some work left in that regard. At some places, glom measures size of text to determine the width and/or height of its widgets, using gtk_widget_create_pango_layout() and pango_layout_get_pixel_size(). However, this seems to result in the size of the text for the normal GTK+ theme, not the one with the bigger font on maemo, and this renders glom's guesses about widget sizes rather unusable. I fixed the most obvious uses (like the height of the tree rows in the glom list view) by hardcoding the size, but of course that is not a permanent solution. I am thankful for any hints how to get a correctly setup PangoLayout to do text measuring on maemo.

The automatic column sizing of GtkTreeView also seems not too useful for glom on maemo (see second screenshot). We probably should rather allow the user to scroll the list view horizontally, and introduce a minimum width for the columns.

Speeding up gtkmm

nospam@example.com (Armin Burgmeier) — Fri, 16 Mar 2007 16:52:00 +0100

According to test test application, gtkmm needs significantly more time when default signal handlers and virtual functions are enabled. What gtkmm basically does is creating a new class inheriting from the corresponding class in GTK and overriding all vfuncs and default signal handlers. In the overridden method, it gets the C++ wrapper object for the C object (if any) and casts it to the actual type (e.g. Gtk::Window). As an optimization it checks already whether the C++ object is a custom one that derives from the gtkmm one. If not, the vfunc could not have been overridden anyway and we just call the parent's (GtkWindow) vfunc implementation. Otherwise, a C++ virtual function is called that classes deriving from Gtk::Window might override. This way, GObject vfuncs can be overridden in the C++ code.

For Openismus, I tried to find out what makes this procedure slow. To do so I used callgrind to collect some profiling data from the test program mentioned above. Then, I installed Qt and kdelibs just to be able to use kcachegrind to visualize that data. Wouldn't someone like to write a kcachegrind-like application for GNOME?

I had a look at Container_Class::forall_vfunc because this was the one called most times (64 025 calls, to be accurate). kcachegrind shows that 3.10% of the total time was spent in forall_vfunc (actually, its not time but CPU cycles or something, but as a physicist I assume they are proportional...). Out of these 3.10%, 1.48% are spent in dynamic_cast<>, another 0.99% in ObjectBase::_get_current_wrapper(). The latter call looks up the C++ object from the C object, and the dynamic_cast casts it from ObjectBase to the actual type. Both calls are at the very beginning of the function. Also of notice is that 2.50% of the overall time is in dynamic_cast<>. This means that just the forall_vfunc issues more than the half of all dynamic_casts in the whole program, not counting all the other vfuncs.

What I tried out now is to do the dynamic_cast only if the C++ object is derived from the gtkmm class. In the test application, this is only the case for ExampleWindow but not for all the other widgets that are created. If the object is not derived, the actual C++ type is not needed because we just call the parent vfunc anyway. The result looks like this:

The time spent in forall_vfunc is now nearly only the half as before. It does no more call to dynamic_cast. This is because all containers on which the forall vfunc was called are not overridden by a C++ class. Other vfuncs, as expose_event, still showed a small amount of dynamic_cast calls. These were the expose events the Window received because of being derived by ExampleWindow. There is a bug open about this.

The next thing I had a look at was the ObjectBase::_get_current_wrapper() call. kcachegrind shows a total amount of 1.70% out of which 0.14% are spent in the function itself (and not in functions being called by get_current_wrapper() in turn). However, the function does nothing more than calling g_object_get_qdata on the given GObject to get the C++ wrapper. Therefore, it seems that a significant amount of time is lost due to the overhead of a function call here because the function is often called (106 674 times), so I tried inlining it:

Any calls to get_current_wrapper() disappeared now. Instead, g_object_get_qdata() shows up directly in forall_vfunc. A good way to verify that the compiler actually inlined the function call. Again, there is a performance gain, but way smaller.

So long, but what about actual execution times? When running the test application, there was no noticeable speedup. However, when running under valgrind (meaning about 10-30 times slower), it finished one or two seconds earlier. To conclude, for most users the changes will probably not be visible, but for large applications and/or less powerful machines gtkmm applications might run a bit more smoothly now.

My GNOME Account is ready

nospam@example.com (Armin Burgmeier) — Tue, 16 Jan 2007 22:41:00 +0100

Today I got a GNOME developer account which I requested December 1st. I guess this took longer than usual, probably also due to the GNOME SVN migration which happend on December 29th. Also thanks to Murray Cumming who bugged them multiple times. This means I can directly check in stuff into GNOME SVN (which will mostly be related to Openismus). The first commit I did was adding libgnomedbmm to the gnomemm SVN module.

libgnomedbmm is a C++ wrapper for libgnomedb which I have been working on in the last weeks for Openismus. Wrapping the C API in a GObject-based project (like libgnomedb) is normally pretty easy because most code is auto-generated by gmmproc included in glibmm. However, this was the first time that libgnomedb (and the underlaying library that provides access to the database, libgda) have been seriously wrapped for a higher level language, so some things had (and have still) to be corrected in the C libraries to make them easily wrappable: Signals and Properties should be declared with the correct type, not just G_TYPE_POINTER if it is GObject-derived or a boxed type. Also, the _new-functions of each type should just be wrappers around g_object_new with some construction properties set. This also allows subclassing them (note that the subclass does not call your _new function) and subclassing is actually what the C++ wrapper does to allow overriding vfuncs and default signal handlers.

In other News, I also got a MSDNAA account today (which I requested two to three months ago). This allows me to get some Microsoft software for free (as in beer) via the university. What do I actually want with Microsoft software, I hear you ask. Currently, I use EverNote as a virtual notepad. Playing around with ink notes and my graphics tablet (a Wacom PenPartner2) makes fun and is nearly as productive as with a regular pen. There is a similar Application from Microsoft called OneNote which I would like to compare to EverNote. Actually, I would be happy to have a free Evernote-like application for linux, but that's another story. Using MSDNAA I am also able to make gobby binaries using Microsoft Visual Studio and compare the result (code size, performance) to what gcc produces. Finally, I can get a free look into Windows Vista as soon as it comes out (If I want to blame it, I should have had a look at it at least). And if it fixes some annoyances of XP, I could also keep it.

Working for Openismus

nospam@example.com (Armin Burgmeier) — Wed, 29 Nov 2006 23:01:22 +0100

Since yesterday, I am part-time employee of Openismus GmbH. Openismus provides custom software development using open source software. I am working five to ten hours a week which fits nice into my time management, especially because I can write some code during idle time at the university and on the way from/to Karlsruhe. I am very happy to earn money for writing (open source) software since this is one of the few things that actually make fun and one is paid for.

My first task was to update libgdamm, the C++ bindings to libgda so they build again with the newest version (from CVS HEAD) from libgda. It was pretty straight-forward since gmmproc complains when the signature of a function has changed or a function has been removed (of course after having recreated the method definition file). There was only one thing that was a bit strange: The C header gda-dict.h defines some macros that call functions from another header called gda-dict-reg-aggregates.h. However, it did not include that file and, which was even more surprising, the function declarations in gda-dict-reg-aggregates.h did not have a G_BEGIN_DECLS/G_END_DECLS pair around them. This means when I include that file myself, I get linker errors because the extern "C" stuff is missing.

Yes, I am aware that this is bleeding-edge software and it also might as well be a problem with my understanding of those files. If it would not be that late, I would look for and poke the libgda mailing list about this, but I am horribly tired by now since yesterday, I already slept only four hours... Anyway, a first patch is available that ignores this aggregate stuff (examples build fine without it, so it cannot be that important) and makes libgdamm compile again.