Tag: Qt

Application-hosted and compositor-hosted Maliit

Posted on7 Comments

The standard way of deploying Maliit is to have a single maliit-server instance (per user session), hosting the actual input method (virtual keyboard, handwriting). Applications then communicate with the server (and by extension, the IM) through an IPC.

This allows for a single instance of Maliit to serve all applications, which is memory efficient and robust. A crash in a Maliit IM plugin cannot take down the application and risk loss of significant user data. The disadvantage is the increased system complexity (a separate server process needs to be running at all times*) and requiring compositing of the application and input method windows. The latter can be quite challenging to do in a well-performing way on low-powered mobile/embedded devices. See Jans blogpost for how we handled that on the Nokia N9.

* By default we make use of DBus autostarting, of course.

Application-hosted Maliit

To make Maliit more suitable for systems where only a single application runs (embedded) or compositing performance is not good enough, we now also allow Maliit to be “application-hosted”: the Maliit server and input method plugins lives in the application process, not a separate server process. Enabling this feature has been a long running task of mine: All the code in input-context and server was made transport independent, a direct transport (no IPC) was introduced, and setting up the server for a given configuration (X11, QPA, app-hosted) was simplified. Other motivations for this work include being able to run the server and IM plugin easily for automated end-to-end system or acceptance testing, or just to easily start the server with a given IM plugin loaded for quick manual testing during development (see Michaels merge request).

An example application exists as part of the Maliit SDK that demonstrates the feature: maliit-exampleapp-embedded

Maliit running in application-hosted mode: The Maliit Server and input method plugin is embedded in the application instead of running in a standalone server process.

This works by having a special input-context “MaliitDirect” which instead of connecting to the server over DBus, creates the server and a direct connection. As when running standalone the server will instantiate and manage the necessary input method plug-ins.

Because the IM does not have its own window in this configuration, the application is responsible for retrieving the IM widget from the server, and re-parenting it into the appropriate place in the widget hierarchy. For all other purposes the application uses the same interface as if the IM was hosted remotely, making sure the abstraction is not broken and that one can easily use the application with Maliit deployed in different configuration.

This feature currently works with Qt4 applications, and is in Maliit since the latest release (0.90.0). One issue is that with the current input method API, the plugin assumes a fullscreen window; overlays extending the base area of the IM will be clipped and size needs to be overridden. This is something we are fixing in the new improved API.

Compositor-hosted Maliit

Another approach to make rendering perform better is to host the input method in the process responsible for the compositing. This also reduces the number or processes involved in rendering/compositing, and the associated overhead. This could be a X11 compositing window manager (like KWin or mcompositor), but a more realistic use-case is a Wayland compositor (for instance based on QtCompositor).

The API allows the consumer to inject an class instance for the configuration dependent logic, allowing to integrate the Maliit server with the logic in the rest of the compositor. Applications will use the normal “Maliit” inputcontext and communicate to the server through an IPC like DBus.

After the work with application-hosted Maliit, this feature was completed by making the server and connection libraries available as public API. The API is available in the latest  Maliit release (0.90.0), but is considered unstable until Maliit hits the 1.0 mark.

 

 

 

 

Maliit on Windows: Basic build working

Posted on4 Comments

Enabling third-party developers of input methods is one of the primary goals in the Maliit project. In an attempt to improve this story I spent some time on getting Maliit to work on Windows.

Since we use Qt there were few changes needed to the code, but since we use qmake, quite many to the build system. One of the bigger changes was making glib-dbus and qdbus  optional, which is also useful for Maliit on embedded systems.

With the Windows build fixes merge requests for maliit-framework and for maliit-plugins, one can now build Maliit on Windows and run the provided example applications. This feature is currently being reviewed and should be in the next Maliit release.

Thanks to the standalone viewer application for Maliit Keyboard this allows one to develop new features, theming and language layouts for it on Windows.

Sadly loading an input method plugin in the maliit server crashes for an unknown reason. With my limited Windows software development experience I was not able to solve this within the couple of days I had available. This is necessary for application-hosted Maliit to work and to enable general development of Maliit input method plugins (not just maliit-keyboard). Help would be much appreciated, even just someone checking  if it is reproducible on another Windows system.

Also left on the todo-list due to lack of time is to set up a Windows build slave for the Maliit buildbot, to test that the build continues to work on Windows and to produce executables.

Combining PySide and PyGObject introspection bindings

Posted onLeave a comment

Some while back I added basic GObject Introspection support to GEGL and GEGL-GTK master a while back. This will* allow application developers to write their Gegl + Gtk based applications in any language supported by GObject Introspection, like Python, Vala and Javascript. For GeglQt, the Qt integration library for using Gegl in Qt based applications, it was natural to use PySide to provide Python bindings for it. The initial setup was quick and easy, thanks to the binding tutorial, but there was one challenge.

The current widgets provided by GeglQt are for displaying the output of a node in the GEGL graph. Therefore they have methods with the following signature to hook up it up:

From gegl-qt/nodeviewwidget.h
GeglNode *inputNode() const;
void setInputNode(GeglNode *node);

GeglNode is a GObject (from the C based glib) subclass, and without help the bindings generator (Shiboken) does not know what to do with it so the method cannot be bound. PySide could have been used to also generate bindings for Gegl itself, but what we actually want to do is to make use of the existing PyGObject based bindings.

Marcelo Lira on #pyside let me know that this should be possible by adding some annotations to the typesystem.xml file, and implementing a Shiboken::Converter<T>. It is indeed possible, and for the above type looks something like this:

From typesystem_gegl-qt.xml
<primitive-type name="GeglNodePtr">
      <conversion-rule file="geglnode_conversions.h"/>
      <include file-name="pygobject.h" location="global"/>
</primitive-type>


From geglnode_conversions.h
namespace Shiboken {
template<>
struct Converter<GeglNodePtr>
{
    static inline bool checkType(PyObject* pyObj)
    {
        return GEGL_IS_NODE(((PyGObject *)pyObj)->obj);
    }

    static inline bool isConvertible(PyObject* pyObj)
    {
        return GEGL_IS_NODE(((PyGObject *)pyObj)->obj);
    }

    static inline PyObject* toPython(void* cppObj)
    {
        return pygobject_new(G_OBJECT((cppObj)));
    }

    static inline PyObject* toPython(const GeglNodePtr geglNode)
    {
        return pygobject_new(G_OBJECT(geglNode));
    }

    static inline GeglNodePtr toCpp(PyObject* pyObj)
    {
        return GEGL_NODE(((PyGObject *)pyObj)->obj);
    }
};
}

The PyGObject C API and the GObject type system is here being used to implement what Shiboken needs. The attentive reader will note that GeglNodePtr is used and not GeglNode*. This is a simple “typedef GeglNode * GeglNodePtr“, which looks to be neccesary with current PySide (1.0.6) to avoid it being confused by the pointer. Hopefully that is fixable and won’t be necessary in the future.

With this solved, I committed the initial Python support to GeglQt master yesterday. It contains a trivial Python example showing the usage. Some build cleanups, binding generator tweaks and testing remains to be done, but expect Python support to be a prominent feature for GeglQt 0.1.0

 

* There are still a lot of GObject Introspection annotations missing in Gegl. See the tracking bug. Help wanted!

Making GEGL easier to use in graphical applications

Posted on2 Comments

So, in the last couple of months I’ve been working a bit on GEGL. Some of the work has already been covered by LWN, so I guess it is time that I blog about it…

GEGL is a generic image processing library which is used by applications like GIMP, (and in the future maybe MyPaint and DarkTable). It provides applications with a graph based image processing backend that can do non-destructive processing of high-bitdepth images, among other things.

One of the problems that I think has been limiting adaptation of GEGL has been the entry barrier to starting to use it in a graphical application. While GEGL provides the image processing backend, it did not provide good and easy ways of displaying the output on screen. Now it does!

GTK+, Clutter and Qt integration libraries

Some code for integrating GEGL in GTK+ based applications has existed in the GEGL tree for a long time, but it was not well maintained and there was no public API. After brushing up the code to use Cairo for rendering and to support both GTK+ 2 and 3, it was split out to a separate library and repository: gegl-gtk. This library now provides a GtkWidget for displaying the output of a node in the GEGL graph, with basic support for scaling and translations. Any change in the GEGL graph will be reflected in the view widget. This makes it trivial for applications using a GTK+ based user interface to get started using GEGL, see for instance the provided examples in C or in Python.

The same functionality is provided for Clutter based user interfaces by gegl-clutter in form of a ClutterActor. This code was previously available as clutter-gegl, but has now been renamed and moved to be a part of the GEGL project, and is maintained by Øyvind Kolsås. Example code in C.

Last but not least, gegl-qt was created to serve the needs of applications using Qt based user interfaces. The different widget systems (QWidget-, QGraphicsWidget- and QML-based) are all supported. In addition to the features currently available in the GTK+ and Clutter versions, the Qt view widgets also support auto-scaling and auto-centering. Python bindings via PySide is planned, but blocking on a PySide issue at the moment.

A pretty boring screenshot showing two QWidget based examples (code: 1, 2) for transformations:

Artwork: “Wanted“, speedpainting by David Revoy

The first stable release of gegl-qt and gegl-gtk will hopefully be available soon. The list of tasks can be found in the README files.

Display operations

In GEGL, image processing is described as a graph of operations. “gegl:display” and “gegl:gtk-display” operations existed in the gegl tree, and by attaching one of these to a node in the graph one could display the output of the graph at the given node in a window . Such display operations are useful for applications that just want to show the output of a graph without having to use a GUI library directly.

The problem was that both of these operations were optional, so applications could not rely on this functionality to be present. This is solved by letting the “gegl:display” operation be a meta-operation, which uses other operations as a handler to actually display the output. Such display handler operations are now provided by gegl (optional, using SDL), gegl-gtk (using GTK+) and gegl-qt (using Qt). In addition a fallback operation that will export a PNG file and launch an external application to display it will be provided in GEGL.

More to GEGL stuff to come soon, hopefully.

Playing with Qt Quick; Imago

Posted on2 Comments

In learning Qt, I of course needed to get some experience with the newest and shiniest bits of Qt: Qt Quick.

One of the things I’ve done with that is to make a simple application, an image viewer called Imago. The code can be found at Gitorious. It is a bit different from typical Qt Quick applications as it is not a mobile application but rather targets the desktop. Thus I’m using standard desktop UI concepts like drop-down menues, and ordinary file-chosers.

Imago - single image view

Imago - single image view

Imago - "traditional" view

Imago - "traditional" view

Imago - grid view

Imago - grid view

The lists can be “flicked” through using the pointer, or using the scroll wheel. Moving between the views that show multiple images and the single-image view has a small animation.

Implementation

In a Qt Quick application two languages and runtimes are involved; C++ and the declarative language that forms the basis of Qt Quick, QML. Qt supports bi-directional communication between the two using a number of metods.
In my case I chose to make a QObject class that  maintains all of the central state in the application. This includes things like what folder is currenty open, the images in that folder, which image is currently focused/selected. All these attributes are implemented as QObject properties, and are notifyiable (meaning they have an attached signal that you guarantee to emit each time the value changes). Exporting this object to QML then allows these properties to be used in property bindings, or for one to connect to the notification signals.
So in Imago QML is basically used as a “stupid” UI layer for the image view. Other things, like window title and drop-down menues are handed by the C++ side, but by using the signals and properties of the same object.

Writing a desktop application means that one cannot assume a fixed window size like people tend to do for mobile device applications. With setting the resize property of the QDeclarativeView (the widget that displays the QML stuff) to QDeclarativeView::SizeRootObjectToView and using anchor layouts throughout this is possible, but it took me a good while to get working properly. Mostly because the concepts are new, and most (all?) the examples in the documentation seem to neglect this aspect.

The grid and traditional view are actually implemented in different QML files (though the delegate for the images in the list is shared). I did this to test how having several UIs would be like, and for a simple application like this it works out just fine.

Where to now?

Imago right now is very simple,  and could benefit from some added features and some designer love. I also just found a bug in the traditional view that needs to be fixed. I have documented these things in the README file in case someone is interested. As I have moved my focus over to other projects, I probably wont be taking this application much further, at least not at this time.

Image preview support for OpenRaster in Qt working

Posted on2 Comments

While learning Qt here at Openismus I’ve written a basic, working plug-in for Qt that adds support for the OpenRaster file format*. Here is my Qt-based test application demoing this functionality by showing some awesome multi-layered abstract test art made by yours truly using Krita:

I asked for real art, but I did not get any

I asked for real art but as you can see, I did not get any.

The level of features supported is such that you will be able to preview OpenRaster documents created with applications like MyPaint, Drawpile, Nathive and GIMP, with the limitation that it will have a white background for transparent areas. The code can be found in qopenraster repository on gitorious (no tarballs), and the README file documents how to install as well as things that remain to be done.

The plug-in is basically a thin wrapper around libora, the OpenRaster reference library. libora takes care of parsing the OpenRaster document, reading out the layer data and rendering it into a single buffer. The rendering ability was added by me as part of this work, in addition to some other minor stuff. The Qt plug-in does RGBA to ARGB conversion and provides the QImageIOPlugin interface expected by Qt.

Doing this has also exposed several limitations and not-so-nice things in libora that should/needs to be improved. I’ve updated libora’s README file to reflect this.

*Assuming the Qt application actually uses QImage in a straight-forward way. The KDE image viewer Gwenview does not seem to use QImage directly, so you will not automatically get support there by installing the plug-in :((. I fear that other KDE applications might be the same, though I was not able to test Digikam. If anyone has a suggestion for a Qt based image viewer that works sanely in this area, don’t hesitate to leave a comment.

PS: I have an almost-working gdk-pixbuf module as well, will push that to gitorious soon.

Back from Qt Dev Days, first Qt projects

Posted onLeave a comment

I’ve actually been back close to a week now, but never mind that…

In the per-conference day with training sessions I attended the Qt Essentials track, which was more or less as expected. Glad I read a full Qt book beforehand, it would have been challenging to keep up with the shear amount of information without it.
The keynotes I attended on the second day were not particularly exciting: no major announcements nor insights were given. The technical talks on the other hand were filled with goodies. The talks by Jens Bache-Wiig and Roberto Raggi on Qt Quick were especially good.*

The talks definitely made me want to try Qt Quick for doing user interfaces for small-form factor devices, especially because it allows for very rapid prototyping and iterations when developing. The current lack of widgets and traditional layouts probably limits its usefulness for typical desktop application with more complex user interfaces though. There is nothing that helps you achieve a native look and feel either, but the Qt Components project is aiming to bridge those gaps.
I also suspect that the declarative and dynamic nature of QML poses several new challenges for developers, especially for those that are mostly used to traditional Qt programming with C++. I’m especially concerned that there was no way to visualize or do static checking on the property-bindings that are so central in QML. Very curious as to how that plays out in practice.

*I’m told the talks will be online after the Qt Developer Days event in San Fransisco is over.

Qt projects you said?

Going forward I’ll be doing some projects with Qt, in the same way I have done with GTK. My first project has already started: implementing viewer-class OpenRaster support for Qt. This means that applications using Qt and QImage will soon be able to display fully-rendered OpenRaster images!
Development of the Qt integration happens in the repository on gitorious, and the libora modifications currently lives in my personal clone. It will be pushed to mainline as soon as I have more-or-less settled on the API, and done a basic implementation. Using libora for all the OpenRaster specific stuff is being a bit more painful than expected, but it is the right thing to do as it means that other consumers benefits as well. Like a potential GdkPixbuf plugin or applications not using Qt or GTK. I’ll write more once it reaches a useful state.

After that is done I will probably do something with more UI, like a proper application. Hopefully I will get to toss Qt Quick into the mix as well. I’ve got an idea that I think would be a nice fit, so we’ll see.

Qt Developer Days 2010 and more

Posted onLeave a comment

Next week I’m going to Qt Developer Days in Munich where I will be attending Qt training and technical talks. And almost just as importantly, meet and talk to people who do related things to what we do at Openismus.  I suspect the overall style and feel of this conference will be quite different from past free and open source software events I’ve been to (like GUADEC and Libre Graphics Meeting). More business-y, perhaps even enterprise-y? None the less,  looking forward to it.

In related news, I’m also involved as part of the local team for Desktop Summit 2011, along with several of my co-workers. The initial announcements have just been made public; the conference will be held in Berlin August 6th-12th, at Humboldt University. Mark your calendars 😉 See for instance the story by the GNOME Foundation, the official website or the original announcement for more information.

Hopefully I will also be going to the Meego conference in Dublin in November. Fingers crossed!