Latest posts of series qt5-custom-build

This is part of a series of posts on compiling a custom version of Qt5 in order to develop for both amd64 and a Raspberry Pi.

The problem

While testing the cross-compiler, we noticed that the designer library was not being built.

The designer library is needed to build designer plugins, which allow loading, dynamically at runtime, .ui interface files that use custom widgets.

The error the customer got at runtime is: QFormBuilder was unable to create a custom widget of the class '…'; defaulting to base class 'QWidget'.

The library with the custom widget implementation was correctly linked, and indeed the same custom widget was used by the application in other parts of its interface not loaded via .ui files.

It turns out that it is not sufficient, and to load custom widgets automatically, QUiLoader wants to read their metadata from plugin libraries containing objects that implement the QDesignerCustomWidgetInterface interface.

Sadly, building such a library requires using QT += designer, and the designer library, that was not being built by Qt5's build system. This looks very much like a Qt5 bug.

A work around would be to subclass QUiLoader extending createWidget to teach it how to create the custom widgets we need. Unfortunately, the customer has many custom widgets.

The investigation

To find out why designer was not being built, I added -d to the qmake invocation at the end of qtbase/configure, and trawled through the 3.1G build output.

The needle in the haystack seems to be here:

DEBUG 1: /home/build/armhf/qt-everywhere-src-5.15.0/qttools/src/designer/src/src.pro:18: SUBDIRS := uiplugin uitools lib components designer plugins
DEBUG 1: /home/build/armhf/qt-everywhere-src-5.15.0/qttools/src/designer/src/src.pro:23: calling qtNomakeTools(lib components designer plugins)

As far as I can understand, qtNomakeTools seems to be intended to disable building those components if QT_BUILD_PARTS doesn't contain tools. For cross-building, QT_BUILD_PARTS is libs examples, so designer does not get built.

However, designer contains the library part needed for QDesignerCustomWidgetInterface and that really needs to be built. I assume that part should really be built as part of libs, not tools.

The fixes/workarounds

I tried removing designer from the qtNomakeTools invocation at qttools/src/designer/src/src.pro:23, to see if qttools/src/designer/src/designer/ would get built.

It did get built, but then build failed with designer/src/designer and designer/src/uitools both claiming the designer plugin.

I tried editing qttools/src/designer/src/uitools/uitools.pro not to claim the designer plugin when tools is not a build part.

I added the tweaks to the Qt5 build system as debian/patches.

2 hours of build time later...

make check is broken:

make[6]: Leaving directory '/home/build/armhf/qt-everywhere-src-5.15.0/qttools/src/designer/src/uitools'
make[5]: *** No rule to make target 'sub-components-check', needed by 'sub-designer-check'.  Stop.

But since make check doesn't do anything in this build, we can simply override dh_auto_test to skip that step.

Finally, this patch builds a new executable, of an architecture that makes dh_shlibdeps struggle:

dpkg-shlibdeps: error: cannot find library libQt5DesignerComponentssystem.so.5 needed by debian/qtbase5system-armhf-dev/opt/qt5system-armhf/bin/designer (ELF format: 'elf32-little' abi: '0101002800000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libQt5Designersystem.so.5 needed by debian/qtbase5system-armhf-dev/opt/qt5system-armhf/bin/designer (ELF format: 'elf32-little' abi: '0101002800000000'; RPATH: '')

And we can just skip running dh_shlibdeps on the designer executable.

The result is in the qt5custom git repository.

This is part of a series of posts on compiling a custom version of Qt5 in order to develop for both amd64 and a Raspberry Pi.

These are instructions for building Qt Creator with the custom Qt, so that it can load and use Designer plugins built with it.

Sadly, because of the requirement of being able to load Designer plugins, and because of the requirement of being able to compile custom widgets using the custom Qt and use them in the Designer, we need to also rebuild Qt Creator.

The resulting packaging is at https://github.com/Truelite/qt5custom.

Set up sources

Open the source tarball, and add the Qt Creator packaging:

tar axf qt-creator-enterprise-src-4.12.2.tar.xz
cp -a debian-qtcreator qt-creator-enterprise-src-4.12.2/debian
ln -s qt-creator-enterprise-src-4.12.2.tar.xz qt-creator-enterprise-src_4.12.2.orig.tar.xz

If needed, install the Qt license:

cp qt-license.txt ~/.qt-license

Install build dependencies

You can use apt build-dep to install dependencies manually:

cd qt-creator-enterprise-src-4.12.2
apt build-dep .

Alternatively, you can create an installable .deb metapackage that depends on the build dependencies:

apt install devscripts
mk-build-deps debian-qtcreator/control
apt -f install qt-creator-enterprise-src-build-deps_4.12.2-1_all.deb

Package build

The package is built by debian/rules base on the excellent work done by Debian Qt5 maintainers.

After installing the build dependencies, you can build like this:

cd qt-creator-enterprise-src-4.12.2
debuild -us -uc -rfakeroot

In debian/rules you can configure NUMJOBS with the number of available CPUs in the machine, to have parallel builds.

debian/rules automatically picks qt5custom as the Qt version to use for the build.

NOTE: Qt Creator 4.12.2 will NOT build if qtbase5custom-armhf-dev is installed. One needs to make sure to have qtbase5custom-dev installed, but NOT qtbase5custom-armhf-dev. Despite quite a bit of investigation, I have been unable to understand why, if both are installed, Qt Creator's build chooses the wrong one, and fails the build.

Build output

Building sources generates 4 packages:

  • qtcreator-qt5custom: the program
  • qtcreator-qt5custom-data: program data
  • qtcreator-qt5custom-doc: documentation
  • qtcreator-qt5custom-dbgsym: debugging symbols

Using the custom Qt Creator Enterprise

The packages are built with qt5custom and install their content in /opt/qt5custom.

The packages are coinstallable with the version of Qt Creator packaged in Debian.

The custom Qt Creator executable is installed in /opt/qt5custom/bin/qtcreator, which is not in $PATH by default. To run it, you can explitly use /opt/qt5custom/bin/qtcreator. qtcreator ran without an explicit path, runs the standard Debian version.

Installing Designer plugins

Designer plugings can be compiled with qt5custom and installed in /opt/qt5custom/plugins/designer/.

Cross-building with Qt Creator

Once the cross-build Qt5 packages are installed, one should see it appear in the Qt Creator kit configuration, where it can be selected and used normally.

If one sets device type to "Generic Linux Device", chooses a compiler for "arm 32bit" and sets Qt Version to "qt5custom-armhf", one can smoothly cross-compile and execute and debug the built program directly on the device.

This is part of a series of posts on compiling a custom version of Qt5 in order to develop for both amd64 and a Raspberry Pi.

I split building Qt5 for armhf development in two parts: one cross-build environment to be installed on amd64 develpment systems, and a runtime part to be installed on the target armhf hardware.

Building a Qt5 cross-building environment builds a mix of armhf and amd64 binares: the amd64 tools to use for cross-building, like moc, qmake, plugins for Qt Creator, and so on; armhf headers and libraries to use at cross-build time; armhf shared libraries to use at runtime.

The procedure I came up with builds a devel package for amd64 development machines, which contains everything, and a second package that extracts from it only what is needed at runtime.

The cross-build environment is coinstallable both with the version of Qt distributed with Debian, and with the amd64 custom Qt development package.

The current build is sadly using -skip qtwebengine, because I have had no success so far getting QtWebEngine to compile as part of a cross-build Qt setup (the last road bump I can't overcome is nss and nspr not being coinstallable on amd64 and armhf, while both seem to be needed for it).

The resulting packaging is at https://github.com/Truelite/qt5custom.

Set up sources

Open the source tarball, and add the amd64 packaging:

tar axf qt-everywhere-src-5.15.0.tar.xz
cp -a debian-cross qt-everywhere-src-5.15.0/debian

If needed, install the Qt license:

cp qt-license.txt ~/.qt-license

If debugging information are not needed in armhf development, remove --no-strip from ./configure invocation in the rules file, to build significantly smaller .deb packages.

Install build dependencies

Install cross-compilers:

dpkg --add-architecture armhf
apt install crossbuild-essential-armhf

You can use apt build-dep to install dependencies manually:

cd qt-everywhere-src-5.15.0
apt build-dep .
apt -a armhf build-dep .

Alternatively, you can create installable .deb metapackages that depends on the build dependencies:

apt install devscripts
mk-build-deps --host-arch amd64 debian-cross/control
mk-build-deps --host-arch armhf debian-cross/control
apt -f install qt-everywhere-cross-build-deps_5.15.0-1_amd64.deb qt-everywhere-cross-cross-build-deps_5.15.0-1_armhf.deb

Note that there are two sets of dependencies: one of amd64 packages, and one of armhf packages.

Building the cross-build environment

After installing the build dependencies, you can build like this:

cd qt-everywhere-src-5.15.0
fakeroot debian/rules binary

In debian/rules you can configure NUMJOBS with the number of available CPUs in the machine, to have parallel builds.

This will build a package with the cross-build development environment for amd64, called qtbase5custom-armhf-dev

Building the runtime environment

To generate the runtime package for armhf, one needs to have the cross-build package (qtbase5custom-armhf-dev) installed in the system together with its build dependencies.

At that point, the armhf runtime package can be built using the debian-armhf directory without further sources:

apt install crossbuild-essential-armhf debhelper qtbase5custom-armhf-dev*_amd64.deb qt-everywhere-src-cross-build-deps*_armhf.deb
mkdir runtime
cp -a debian-armhf runtime/debian
cd runtime
dpkg-buildpackage -a armhf

Building the runtime environment generates:

  • a libqt5custom package for armhf, installable on the target devices, containing the runtime Qt libraries and depending on the packages that they need to run;
  • a libqt5custom-dbgym package for armhf with debugging symbols, to use for debugging on the target hardware.

If, while generating the cross-build environment, --no-strip was removed, the libqtcustom-dbgsym package with debugging symbols will not be generated.

Using the cross-build environment

These install their content in /opt, and are coninstallable with the version of Qt distributed in Debian, and with the custom Qt packages for amd64.

One needs to be careful not to create programs that link, either directly or indirectly, with more than one of these coinstalled Qt, because the in memory layout of objects could be different and incompatible, causing unexpected results.

Selecting which Qt version to use: qtchooser

These Qt custom packages integrate with qtchooser to select the version of Qt to use at compile time.

qtchooser --list-versions lists available versions. One can choose what to use by exporting QT_SELECT:

# apt install qtchooser qt5-qmake qt5-default
$ qtchooser --list-versions
4
5
qt4-x86_64-linux-gnu
qt4
qt5-x86_64-linux-gnu
qt5
qt5custom-x86_64-linux-gnu
qt5custom
qt5custom-armhf-x86_64-linux-gnu
qt5custom-armhf

$ qmake --version
QMake version 3.1
Using Qt version 5.11.3 in /usr/lib/x86_64-linux-gnu

$ export QT_SELECT=qt5custom-armhf
$ qmake --version
QMake version 3.1
Using Qt version 5.15.0 in /opt/qt5custom-armhf/lib

Cross-building software using custom Qt

One just needs to export QT_SELECT=qt5custom-armhf in the environment, then proceed to build normally:

export QT_SELECT=qt5custom-armhf
fakeroot ./debian/rules clean binary

Or:

export QT_SELECT=qt5custom-armhf
qmake file.pro

If switching from one Qt to another, it is possible that the makefiles created by one qmake are not working well with the other. In that case, one can just remove them and regenerate them.

The build result is ready to be copied into, and run in, the target armhf device.

This is part of a series of posts on compiling a custom version of Qt5 in order to develop for both amd64 and a Raspberry Pi.

A customer needs a procedure for a custom build of Qt5 5.15, the last LTS release of Qt 5.

They develop for industrial systems that are managed by an amd64 industrial computer. This computer is accessed either through an attached panel touch screen, or through touch screens driven by Raspberry Pi clients connected via an internal ethernet network.

The control interfaces use mostly a full screen Qt5 application. The customer relies heavily on Qt5, has a full Enterprise license, and needs to stay on top of the most recent releases, to make use of new features or bug fixes that have made it upstream since the last Debian stable was released.

This is a list of requirements for this job:

  • Build .deb packages of the custom builds of Qt5, so they can be integrated with the existing provisioning infrastructure
  • Easily repackage hopefully at least new Qt minor versions
  • Custom builds should be coinstallable with the standard Debian Qt5 packages, to be able to use existing Qt-based Debian packages without rebuilding them
  • One needs to be able to develop custom widgets, and use them in the Form Editor in Qt Creator
  • One needs to be able to load custom widgets via .ui files at runtime
  • One needs to develop amd64 Qt5 applications
  • One needs to develop armhf Qt5 applications
  • One needs to develop armhf Qt5 applications from Qt Creator, with the nice feature it has to cross-compile them on the fast amd64 development machine, and run and debug them directly on a network-connected device
  • One needs to package the resulting amd64 or armhf binaries in .deb format, so they can be integrated with the existing provisioning infrastructure.

To make things easier, .deb packages are for internal use only and do not need to be compliant with Debian policy.

I estimate a difficulty level of: "Bring the One Ring to Mount Doom and remember to get milk on the way back".

The journey begins.

The resulting packaging is at https://github.com/Truelite/qt5custom.

This is part of a series of posts on compiling a custom version of Qt5 in order to develop for both amd64 and a Raspberry Pi.

First step, build Qt5 5.15 packages for amd64.

To prevent conflicting with Debian Qt packages, we'll install everything in /opt.

We can install qtchooser configuration files to allow developers to easily switch between Debian's standard Qt version or the custom version, at will.

The resulting packaging is at https://github.com/Truelite/qt5custom.

Set up sources

Open the source tarball, and add the amd64 packaging:

tar axf qt-everywhere-src-5.15.0.tar.xz
cp -a debian-amd64 qt-everywhere-src-5.15.0/debian

If needed, install the Qt license:

cp qt-license.txt ~/.qt-license

Install build dependencies

You can use apt build-dep to install dependencies manually:

cd qt-everywhere-src-5.15.0
apt build-dep .

Alternatively, you can create an installable .deb metapackage that depends on the build dependencies:

apt install devscripts
mk-build-deps debian-amd64/control
apt -f install qt-everywhere-src-build-deps_5.15.0-1_amd64.deb

Package build

The package is built by debian/rules base on the excellent work done by Debian Qt5 maintainers

After installing the build dependencies, you can build like this:

cd qt-everywhere-src-5.15.0
fakeroot debian/rules binary

In debian/rules you can configure NUMJOBS with the number of available CPUs in the machine, to have parallel builds.

Build output

Building sources generates 4 packages:

  • libqt5custom: the runtime environment
  • libqt5custom-dbgsym: debugging symbols for the runtime environment
  • qtbase5custom-dev: the build environment
  • qtbase5custom-dev-dbgsym: debugging symbols for the build environment

qtbase5custom-dev and libqt5custom are needed for development; only libqt5custom is needed to run build programs.

Using custom Qt for amd64

These Qt custom packages install their content in /opt, and are coninstallable with the version of Qt distributed in Debian.

One needs to be careful not to create programs that link, either directly or indirectly, with both the Debian Qt and the custom Qt, because the in memory layout of objects could be different and incompatible, causing unexpected results.

Selecting which Qt version to use: qtchooser

These Qt custom packages integrate with qtchooser to select the version of Qt to use at compile time.

qtchooser --list-versions lists available versions. One can choose what to use by exporting QT_SELECT:

# apt install qtchooser qt5-qmake qt5-default
$ qtchooser --list-versions
4
5
qt4-x86_64-linux-gnu
qt4
qt5-x86_64-linux-gnu
qt5
qt5custom-x86_64-linux-gnu
qt5custom

$ qmake --version
QMake version 3.1
Using Qt version 5.11.3 in /usr/lib/x86_64-linux-gnu

$ export QT_SELECT=qt5custom
$ qmake --version
QMake version 3.1
Using Qt version 5.15.0 in /opt/qt5custom/lib

Building software using custom Qt

One just needs to export QT_SELECT=qt5custom in the environment, then proceed to build normally:

export QT_SELECT=qt5custom
fakeroot ./debian/rules clean binary

Or:

export QT_SELECT=qt5custom
qmake file.pro

If switching from one Qt to another, it is possible that the makefiles created by one qmake are not working well with the other. In that case, one can just remove them and regenerate them.