Armin's personal Blog

There are several ways to fix the recent chooser showing no recent files in Glom on Windows, but they are all sort of hacks except one, namely make the recent chooser guess the mime type of an URI on Windows as it already does on Unix. Currently, it always falls back to application/octet-stream and this is why the filter on application/x-glom that glom uses does not show up any files. I looked around a bit in the registry, and it seems that all known file extensions are in HKEY_CLASSES_ROOT/.$ext, and some of them have a "Content Type" key that looks like the mime type we are after. So all we need to do is to register the .glom type there (this can also be used to associate a nice icon with it) and to make the GTK+ recent chooser use that "Content Type" key for the extension of the URI it gets as default mime type.

To implement this, I need to be able to build GTK+ (and glib which GTK+ from SVN trunk depends on) on Windows. The easy method I already used for glom itself was to copy tarballs created by "make dist" to the windows machine so I don't have to create the configure script there, not requiring the autotools stuff. However, this gets horribly annoying when the repositories are updated frequently. It would be much nicer to have a working copy directly on Windows, and to (re)create the configure script there when necessary. In theory, this should work since mingw provides all necessary tools. In practise, I already stumbled upon several ugly problems in the past. I gave it another try, though, and I ended up with a freshly built glib from SVN at the end of the day. Below are the exact steps that got me this far.

I used the environment I set up as described here as a starting point. It is probably equally well possible to start with a freshly set up MSYS+MingW, though.

• First, install termcap, libtool, automake and autoconf from mingw.org into C:\msys (without the usr/ prefix that is in the tarball, such that files end up as /bin/autoconf etc.). Copy /bin/automake to /bin/automake-1.10 and /bin/aclocal to /bin/aclocal-1.10. The -1.10 versions are just 0-byte stubs, but glib's autogen.sh calls them explicitely.
• Then, rename the /bin/perl wrapper script and install the mingw-provided perl package. This is still 5.6, but it is enough for what we need, and the problems with paths in arguments seem to go away with this. Perhaps it would be enough to install ActivePerl directly so that /bin/perl.exe is the ActivePerl one.
• Add gtk-doc stubs to your system. Most GNOME libraries (including glib and GTK) require gtk-doc, but I could not find any success story of getting it to work on Windows. I used the instructions by Hans Breuer:

  cd gtk-doc
  cp gtk-doc.m4 /gtk/share/aclocal
  cp gtkdocize /gtk/bin
  mkdir -p /gtk/share/gtk-doc/data/
  cp gtk-doc.make /gtk/share/gtk-doc/data

  You need a gtk-doc tarball from ftp.gnome.org for this.
• Next, load svn-win32-1.4.5.zip (or a more recent version) from subversion.tigris.org, unpack the zip to C:\svn and add /c/svn/bin to the PATH in your ~/.profile. After a relogin or exec bash -l you should be able to use the SVN client in msys. Try getting glib via svn co http://svn.gnome.org/svn/glib/trunk glib.
• The most hacky thing I had to do was changing the /bin/aclocal file. Add

  $file =~ s/^C:\/msys\//\/usr\//;

  around line 680, right before

  $map_traced_defs{$arg1} = $file
    if ($macro eq 'AC_DEFUN'
        || $macro eq 'AC_DEFUN_ONCE'
        || $macro eq 'AU_DEFUN');
  

  Note that you need to adjust C:\/msys\/ if you installed msys elsewhere. Also remember to copy it to /bin/aclocal-1.10 (By the way, NTFS is capable of symlinks somehow, isn't it)? This is required so that aclocal converts paths starting with C:/msys/share/[...] to /usr/share/[...]. Both refer to the same file on disk, but since aclocal does a simple string comparison, it does not recognize this and always creates an empty aclocal.m4, causing autoconf to fail due to not finding any macros. Better solutions to this are very much appreciated.
• Last, replace AM_CONFIG_HEADER by AC_CONFIG_HEADER in glib's configure.in. autoheader complains about the first. Quick googling seems to suggest that AM_CONFIG_HEADER is deprecated by AC_CONFIG_HEADER, but I am not sure. If that is really the case we should probably change glib.
• Run ./autogen.sh and watch the magic.

So far for today, I am going to try GTK+ and glom tomorrow. Perhaps I even get to what I originally wanted to do, making the GTK+ recent chooser guess mime types on Windows. If you try these steps out and run into further problems, I would be glad to hear about it. I am especially looking forward to John Stowers' work on jhbuild on Windows. Building GNOME libraries and applications directly from SVN via jhbuild on Windows sounds really cool.

After porting Glom to maemo, the next target was Windows. Glom's dependencies are already ported to Windows, so getting it to work wasn't too hard, though some hacks were still necessary to get things going (see the Windows build instructions). Currently, only the client-only mode of Glom runs on Windows, but I hope the full version follows soon.

I installed Windows XP within a virtual machine on my laptop to do the port. The good thing is that this doesn't force me to leave my usual Linux environment, especially when I am not at home where I still have a desktop Linux machine. However, on the other hand compilation is slowed down pretty much, becoming really significant when compiling rather big C++ projects such as gtkmm and glom, probably due to memory constraints.

As always, pictures say more than a thousand words, so enjoy. The third screenshot shows a weird problem when scrolling directly after connecting to the database. The bug disappears when minimizing and re-maximizing the window. The screenshots are truncated to the left because the Glom Window requires more space than the 800x600 assigned to the virtual machine for the Small Business Example (which the screenshots are from). I could also use 1024x768 (the laptop has 1280x800), but I like to have a terminal or other windows next to the VM, and I don't like scrollbars in it either. Other drawbacks the current version suffers from and which I am going to tackle within the next days are listed in the Glom Wiki.

I recently ported glom to the maemo platform for Nokia's internet tablets for Openismus. The major work of this was compiling glom and its dependencies in scratchbox, i.e. make it build and run without using some convencience API in glibmm and friends to save code size. This leads to a lot of #ifdef statements uglifying the code (especially because you cannot use exceptions for the code to compile with the -fno-exceptions flag), but well, that's the price you have to pay if you want to use C++ on maemo (you can at least save the #ifdefs if you are writing a maemo-only application, though). It's still much more convenient than plain C. Note that the maemo version of glom is a client only version, i.e. it does not support self-hosting and developer mode.

The work has already been committed to glom trunk, though I keep making changes in the GLOM_CLIENTONLY branch to be able to make debian packages for sardine extras that are based on glom 1.6. With friendly help from Johannes Schmid and Philipp Kern I managed to build debian packages for glom and its dependencies that are already in sardine extras. Feel free to contact me if there are problems with them, since these are the first ones I ever made, and I am not too familiar with debian based distributions either, because I am not using one.

These screenshots show how the whole thing currently looks like. I made use of the hildonmm APIs where appropriate to get a more native maemo look-and-feel, though there is still some work left in that regard. At some places, glom measures size of text to determine the width and/or height of its widgets, using gtk_widget_create_pango_layout() and pango_layout_get_pixel_size(). However, this seems to result in the size of the text for the normal GTK+ theme, not the one with the bigger font on maemo, and this renders glom's guesses about widget sizes rather unusable. I fixed the most obvious uses (like the height of the tree rows in the glom list view) by hardcoding the size, but of course that is not a permanent solution. I am thankful for any hints how to get a correctly setup PangoLayout to do text measuring on maemo.

The automatic column sizing of GtkTreeView also seems not too useful for glom on maemo (see second screenshot). We probably should rather allow the user to scroll the list view horizontally, and introduce a minimum width for the columns.

Many people using gobby complain that it does not support Undo, and some do not even use it therefore. Of course, they are right. It is not only convenient but may also save you lots of data in case you accidentally delete it. However, it is not easy to implement it in a collaborative editor because between your own do and undo operation pairs others might have altered the document. It is thus not reverted to a previous state as with a single user editor. We got suggestions to add some sort of "primitive" Undo like just undoing the last operation, no matter from whom it was. I didn't like that, though, because I prefer to either do things correctly or not at all.

Fortunately, other people also thought about how to achieve collaborative Undo. I read some papers [1] describing an algorithm (adOPTed) that not only achieves convergence, meaning that two users don't end up with a different document after having applied some operations concurrently (which the obby algorithm (called Jupiter) obviously also achieves), but also allows Undo/Redo (which Jupiter does not when more than two users are involved). This is however not the one-and-only solution. There are other approaches to collaborative Undo (or resolving collisions in general), like the AbiCollab one. adOPTed has some nice properties though:

• It has enough information to not only undo your own but also any other's operation. Actually, when a user wants to Undo something, it just says "I am doing an Undo now" and the others calculate the required operation. This especially allows to easily restore user colors when a deletion is being undone (without allowing arbitrary insertion of text colored with another user's color). It could also be used to undo vandalism issued by another user.
• You can always undo and redo your operations, independent of what others have done in between. However, it might happen that an undo has no effect (for example, if you type something, another one deletes it and you press undo then). I wonder by the way what a user expects in that case: Imagine you are typing two letters, and another user deletes the second one. When you press Undo now, would you expect nothing to happen (because another one already "manually" undid your latest operation) or would you expect that the first letter also disappears (because of exactly the same reason)?
• It doesn't need a server to make some final decisions or to keep things in order. You just have to make the other participants receive your requests somehow. Also, there are no conflict situations or something requiring additional round-trips.

However, as always, everything has its price:

• My implementation requires already a lot of CPU power in my simple test cases to calculate the required transformations, especially when there are lots of operations between a do/undo pair.
• As every user calculates the undo operation by itself, every user needs to keep track of every request made by any user. This also requires to send the whole request log to a joining user. Of course, this can be limited to the latest N operations, but it is still a considerable amount of data (especially when many users are working together) and restricts the number of operations people can undo.

My proof of concept featuring Undo and Redo is here. However, you probably won't understand the interesting part of that code without having read the papers referenced above (which you should do as well if you are interested in more details, by the way). There a still a couple of issues before that can be used in a real-world environment, though. I think the performance problems mentioned above can be fixed by caching transformed requests, so that it does not have to compute the same requests all over again. Another problem that needs to be solved is a dynamic amount of users. My test code only works for a fixed number of users. The difficulty is to find out when it is safe to forget about operations from a user that has left the editing session, to save memory and bandwidth.

I hope to get that done really soon now so that it can be used in Infinote and perhaps even Gobby (I have not yet decided whether I want to implement that in Gobby because it is still a considerable amount of work).

I am off to Ireland at the 23th of July and am visiting some friends near Osnabrück right after that, but when I am back home (presumably at 12th of August) I really hope to get something usable (read: a gedit plugin) before my third semester at university begins in October. Unfortunately, I cannot make it to GUADEC this year because I have to be at university that week, but I really look forward to finally get there next year.

[1] Achieving Convergence with Operational Transformation in Distributed Groupware Systems (by Abdessamad Imine et al.)
An Integrating, Transformation-Oriented Approach to Concurrency Control and Undo in Group Editors (by Matthias Ressel, Rul Gunzenhäuser and Doris Nitsche-Ruhland)
Reducing the problems of group undo (by Matthias Ressel and Rul Gunzenhäuser)

Today, I read the GtkSourceview 1.90 release announcement on gedit-list. After having finished my math excerises much earlier then expected I decided to hack GtkSourceview2 support into Gobby. I committed it a few minutes ago into SVN, after having received some advice from Yevgen Muntyan to get it to work (You have to associate a style theme by hand to get highlighting, which will hopefully be changed in the future so that a default theme is used if none is given).

GtkSourceView2 basically comes with a new syntax highlighting engine that allows embedding languages into others (such as JavaScript into HTML, as shown on the screenshot). It also has theming support and mapping between filename patterns and language definitions, which Gobby had to implement before (and it still does since GtkSourceView1 is still supported).

Gobby development has not stalled. We released 0.4.3 with native Avahi support a few weeks ago. However, it is not planned to add any great new features in favor of the sequel I already announced half a year ago. Unfortunately, it does not make as much progress as one could expect it to do, and especially not as much as I would like it to do. My second semester at university began three weeks ago, so I also don't think I get too much done in the following two to three months, either.

The successor of Gobby is going to be called Infinote. It will offer some (long-requested) features the current Gobby architecture does not support, such as different document types, XMPP-based messaging and showing remote cursors/selections. However, the fundamental change is that I want to provide a kind of framework rather than an editor, and to get people to implement Infinote support into their editors so that ideally you can just continue to use your favorite editor for collaborative work. For this to work, I need to properly document the API and network protocol which is also one of the points Gobby is rather weak in currently.

A propos protocol: More and more collaborative editing efforts pop up lately, like Pisara Editor, AbiCollab (which I should really try out in the near feature...) seems to be mature and a GUADEC session description mentions some gedit/collab (over which I did not find out any further details, though). Then, there are of course ACE and MateEdit which I am already aware of. All of these (including Gobby) use different protocols, and are therefore unable to interoperate with each other. Most seem to agree that XMPP is a good base, but there is still the application-specific stuff on top of it that needs to be compatible. It would be great if there was some project like freedesktop.org to agree on a standard for collaborative (text) editing.

The current Infinote code is available in a public SVN at svn://svn.0x539.de/inifinote for those who want to have a look at it. However, most of the ideas do only exist in my head by now and are not written down somewhere in there.

Do you remember back in school when you learned stuff you certainly would not need anymore in your future life? I thought the same thing when we programmed things in Delphi (those things actually were quite interesting, though). After Phil poked me once again yesterday, I finally decided to fix long-standing bug #145. So, guess what, the setup system we use (InnoSetup) is not only written in Delphi but also allows custom Pascal Code to be executed during setup. So this is what I came up with, and it would certainly have taken me some more hours if I did not know Delphi before.

I wonder however whether I will have to use Java for some real-world application one day. I would be nearly as good prepared as I was for the Delphi task.

I got home from FOSTEL 2007 a few hours ago. I was there to represent Gobby and to spread the word about Openismus a bit.

I met a couple of interesting people there like Philip van Hoof, the author of the Tinymail framework, Robert McQueen from the Telepathy project and Jochen Topf who pointed me to the OpenStreetMap project (and who is also living in Karlsruhe, by the way). There was also a lot of VoIP and Telephony stuff I do not know much about and I did not understand that much, but it was nevertheless pretty interesting and fun. I really enjoyed Sean Egan's talk on XMPP, too.

Things I want to keep in mind for the next conference:

• Do not pronounce Gobby in a way so that people might understand Kopete.
• When doing a five minute talk, only prepare for about four minutes, because it will eventually take longer. Apropos lightning talk, the slides of my Gobby Status talk are here. OpenOffice messed the headers up when exporting to PDF, but I don't want to fiddle around with that at the moment. Also expect a blog entry about that topic in the near future.

Last but not least, I want to thank Dave Neary (and probably OpenWengo) for providing me free accommodation and for the dinner Wednesday night. Perhaps we meet again next year.

If you are stumbling over this while reading Planet Gnome, then I have been added to the latter. Thanks Jeff!

For those who do not know me yet: I am a 20-year old German guy, developer of Gobby, and doing various GNOME-related things as a part-time Openismus employee. Currently, I am working on porting Glom to the new libgda 3.0 API.

According to test test application, gtkmm needs significantly more time when default signal handlers and virtual functions are enabled. What gtkmm basically does is creating a new class inheriting from the corresponding class in GTK and overriding all vfuncs and default signal handlers. In the overridden method, it gets the C++ wrapper object for the C object (if any) and casts it to the actual type (e.g. Gtk::Window). As an optimization it checks already whether the C++ object is a custom one that derives from the gtkmm one. If not, the vfunc could not have been overridden anyway and we just call the parent's (GtkWindow) vfunc implementation. Otherwise, a C++ virtual function is called that classes deriving from Gtk::Window might override. This way, GObject vfuncs can be overridden in the C++ code.

For Openismus, I tried to find out what makes this procedure slow. To do so I used callgrind to collect some profiling data from the test program mentioned above. Then, I installed Qt and kdelibs just to be able to use kcachegrind to visualize that data. Wouldn't someone like to write a kcachegrind-like application for GNOME?

I had a look at Container_Class::forall_vfunc because this was the one called most times (64 025 calls, to be accurate). kcachegrind shows that 3.10% of the total time was spent in forall_vfunc (actually, its not time but CPU cycles or something, but as a physicist I assume they are proportional...). Out of these 3.10%, 1.48% are spent in dynamic_cast<>, another 0.99% in ObjectBase::_get_current_wrapper(). The latter call looks up the C++ object from the C object, and the dynamic_cast casts it from ObjectBase to the actual type. Both calls are at the very beginning of the function. Also of notice is that 2.50% of the overall time is in dynamic_cast<>. This means that just the forall_vfunc issues more than the half of all dynamic_casts in the whole program, not counting all the other vfuncs.

What I tried out now is to do the dynamic_cast only if the C++ object is derived from the gtkmm class. In the test application, this is only the case for ExampleWindow but not for all the other widgets that are created. If the object is not derived, the actual C++ type is not needed because we just call the parent vfunc anyway. The result looks like this:

The time spent in forall_vfunc is now nearly only the half as before. It does no more call to dynamic_cast. This is because all containers on which the forall vfunc was called are not overridden by a C++ class. Other vfuncs, as expose_event, still showed a small amount of dynamic_cast calls. These were the expose events the Window received because of being derived by ExampleWindow. There is a bug open about this.

The next thing I had a look at was the ObjectBase::_get_current_wrapper() call. kcachegrind shows a total amount of 1.70% out of which 0.14% are spent in the function itself (and not in functions being called by get_current_wrapper() in turn). However, the function does nothing more than calling g_object_get_qdata on the given GObject to get the C++ wrapper. Therefore, it seems that a significant amount of time is lost due to the overhead of a function call here because the function is often called (106 674 times), so I tried inlining it:

Any calls to get_current_wrapper() disappeared now. Instead, g_object_get_qdata() shows up directly in forall_vfunc. A good way to verify that the compiler actually inlined the function call. Again, there is a performance gain, but way smaller.

So long, but what about actual execution times? When running the test application, there was no noticeable speedup. However, when running under valgrind (meaning about 10-30 times slower), it finished one or two seconds earlier. To conclude, for most users the changes will probably not be visible, but for large applications and/or less powerful machines gtkmm applications might run a bit more smoothly now.

We released Gobby 0.4.2 today. It does not support that much new features, but fixes some annoyances people have complained about: The urgency hint (the window flashing when a user wrote something in the chat) is configurable, documents can be subscribed to by double-clicking them in the user list and the editor tries to keep the line you are currently editing at the same position within the viewport if others add or remove lines elsewhere.

Sobby, the dedicated server, supports initial loading of regular documents instead of serialized obby sessions. Just pass them on the command line as with sessions, sobby will detect whether its a session, and if it is not, it will open it as a document. Sobby now also allows autosaving individual documents into a directory, via the --autosave-directory command line option or the autosave_directory entry in the configuration file, respectively.

Grab them on the Downloads page on our wiki.

Furthermore, I will attend this year's FOSTEL on April 4-5 to represent Gobby there (Thanks to Openismus for making this possible). So if you ever wanted to talk to one of the Gobby developers, be sure to meet me there.

I installed qemu a few days ago on my Windows laptop to test how fast it is and whether it is able to emulate an X server so that it is actually usable. The result is surprisingly positive. I installed Gentoo Linux on it, so it did hardly anything else than compiling a packet after the other since I set it up. I think this is the first time, the dual core processor in the notebook is of real use: While qemu only emulates one processor for the virtual machine (although it could emulate both, I believe), the other one remains for the rest of the system and so my other applications run without any noticeable delay.

I always wanted to be able to do some gnome development when I am not at home, but I do not want to install Linux on a separate partition because the hard disk capacity is "only" 60GB and I had some space problems on Windows on the last laptop. It also annoys me that I have to reboot every time I want to switch from Windows to Linux or vice versa. On the other side, I do not want to remove Windows from the machine either (well, I probably want, but it is not an option) because there are some applications like Evernote that exist only on Windows and since Gobby is officially supported on Windows anyway, I should be able to provide Windows builds and perform some testing.

To summarize, the qemu approach seems promising and I look forward to work with both operating systems without having to reboot all the time. Currently, the Gentoo is still compiling Gnome (36 out of 215 packets done).

In december, I bought a wacom graphics tablet (a PenPartner2) for my notebook to be able to note stuff right into the computer as I would note it on a sheet of paper. Corresponding software preconditioned, this not only makes fun but is also easier to modify lateron and can easily be sent over the internet. As the notebook runs Windows, I just had to install the shipped driver and everything worked out of the box.

Saturday, a second tablet for my linux desktop computer arrived. Today, I tried to set it up. Nearly two hours later, it almost works. After having installed the linuxwacom package, compiled Wacom tablet support into the kernel and hacked some file in /etc/udev/rules.d for great udev magic (As adviced on the gentoo wiki), nothing happend. There was no file in /dev/input as there should be. It took me some time until I realized that I was running kernel 2.6.18 with which linking a /dev/input-file to the wacom device was broken. So I upgraded the kernel, and voilà, there was a file, and catting it produced some output when I moved the stylus on the tablet.

After having configured the appropriate input devices in xorg.conf, I was able to move the pointer with the stylus. The only drawback was that the pointer always was in the upper left quarter of the screen. The X log file told me some coordinates it seems to use to map the tablet area to the screen area. I divided them by two and set the BottomX respectively BottomY parameters using the xsetwacom utility from linuxwacom. These values can also be set in xorg.conf, in the section of every input device, so this is what I have done:

Note however that these are most probably depend on the size of the tablet, so don't just blindly copy them of you do not own an Intuos3 A6 Wide.

Now at least the absolute positioning seems to work correctly. Gimp still does not support pressure sensitivity (which it did on Windows out of the box, by the way) and keeps telling me there were not any extended input devices in the preferences. It also does not recognize the eraser. Anyway, I am going to have a look at this in the next few days.

I already mentioned notetaking software to be used with the tablet. While there is EverNote for Windows (even its commercial and non-free, blahblah), I could not find an alternative for Linux that actually satisfies my needs. The most promising application I tried out was xournal, but I miss the great selector and cutter tools from evernote where I either cross or surround something I have written to select respectively delete it. I also prefer the "endless sheet of paper" approach to the multiple pages which xournal provides. Using a computer for notetaking, I can just get around those limitations like "Oh, I have only one line left but need to write two further lines". However, I must admit that printing in EverNote is pretty broken due to that reason: It truncates notes in the middle of letters or a figure to split them up into pages which it needs to actually print.

I also had a quick look at Jarnal (which is written in Java which should be reason enough not to use it...). The collaboration feature sounds like a great idea, but in Jarnal it is limited to only one participant to actively edit the document which takes away a lot of the greatness. Anyway, semester break begins in about a week (if you do not see the computer science exam I still have to write in the end of February), so perhaps I am motivated enough to get something done in this direction.

Last night was the first time I slept in my room in Karlsruhe. Saturday, my parents, my brother and another friend helped me to carry and install the furniture which went quite well and quick, so I had some time left to learn for the theoretical physics exam I wrote yesterday. People here are nice and internet and television work, so there is nothing I could complain about. But as you know a picture tells more than thousand words, so enjoy.

I also wrote experimental physics on Friday last week, but most problems were taken from old exams I already have seen before. Actually, I expected more from an elite university, but well, the good thing is that I most likely passed that one.

Today I signed a contract about renting a room in Karlsruhe. It is in Baumeisterstraße 8a, room number 35 and 19m² in size. The whole building is a kind of flat share, so I am going to share the kitchen, toilets and stuff with other students. The building itself is pretty old (with high rooms) that was restored last year and it is located near the center somewhere between the university and the station which fits quite well since these are the institutions I (currently) visit most in Karlsruhe. I am already going to move in in the first week of February. I hope that I gain some more time during the week as soon as I do not have to commute anymore from Offenburg to Karlsruhe every day. I will post some photos when I am there.