Latest posts for tag qt-cross

Qt Creator cross-platform development in Stretch: chroot automation

I wrote a tool to automate the creation and maintenance of Qt cross-build environments built from packages in Debian Stretch.

It allows to:

  • Do cross-architecture development with Qt Creator, including remote debugging on the target architecture
  • Compile using native compilers and cross-compilers, to avoid running the compilers inside an emulator making the build slower
  • Leverage all of Debian as development environment, using existing Debian packages for cross-build-dependencies

Getting started

# Creates an armhf environment under the current directory
$ sudo cbqt ./armhf --create --verbose
2017-11-30 14:09:23,715 INFO armhf: Creating /home/enrico/lavori/truelite/system/cross/armhf
…
2017-11-30 14:14:49,887 INFO armhf: Configuring cross-build environment

# Get information about an existing chroot.
# Note: the output is machine-parsable yaml
$ cbqt ./armhf --info
name: armhf
path: ./armhf
arch: armhf
arch_triplet: arm-linux-gnueabihf
exists: true
configured: true
issues: []

# Create a qmake wrapper for this environment
$ sudo ./cbqt ./armhf --qmake -o /usr/local/bin/qmake-armhf

# Install the build-depends you need
# Note: :arch is added automatically to package names if no arch is explicitly
#       specified
$ sudo ./cbqt ./armhf --install libqt5svg5-dev libmosquittopp-dev qtwebengine5-dev

Building packages

To build a package, use the qmake wrapper generated with cbqt --qmake instead of the normal qmake:

$ qmake-armhf -makefile
$ make
arm-linux-gnueabihf-g++ … -I../armhf/usr/include/arm-linux-gnueabihf/… -I../armhf/usr/lib/arm-linux-gnueabihf/qt5/mkspecs/arm-linux-gnueabihf -o browser.o browser.cpp
…
/home/enrico/…/armhf/usr/bin/moc …
…
arm-linux-gnueabihf-g++ … -Wl,-rpath-link,/home/enrico/…/armhf/lib/arm-linux-gnueabihf -Wl,-rpath-link,/home/enrico/…/armhf/usr/lib/arm-linux-gnueabihf -Wl,-rpath-link,/home/enrico/…/armhf/usr/lib/ -o program browser.o … -L/home/enrico/…/armhf/usr/lib/arm-linux-gnueabihf …

Building in Qt Creator

Configure a new Kit in Qt Creator:

  1. Tools/Options, then Build & Run, then Kits, then Add
  2. Name: armhf (or anything you like)
  3. Qt version: choose the one autodetected from /usr/local/bin/qmake-armhf
  4. In Compilers, add a GCC C compiler with path arm-linux-gnueabihf-gcc, and a GCC C++ compiler with path arm-linux-gnueabihf-g++
  5. Choose the newly created compilers in the kit
  6. Dismiss the dialog with "OK": the new kit is ready

Now you can choose the default kit to build and run locally, and the armhf kit for remote cross-development.

Where to get it

Here!

Credits

This has been done as part of my work with Truelite.

pdo debian eng qt-cross

Qt Creator cross-platform development in Stretch: chroots

My next step in creating a cross-platform Qt development environment is trying to set it up on a chroot and make it usable from Qt Creator, so that both buil-dependencies and cross-build-dependencies can be available even when they are not coinstallable.

Building a chroot

I built a chroot to host the armhf build-dependencies:

$ sudo cdebootstrap stretch arm-linux-gnueabihf
$ sudo systemd-nspawn -D arm-linux-gnueabihf
# dpkg --add-architecture armhf
# apt update
# apt install qtbase5-dev qtbase5-dev-tools qtbase5-dev:armhf

The cross-compilers need to be outside the chroot. I tried to use cross-compilers installed inside the chroot while building outside the chroot, and they fail to link at runtime, since they expect their shared libraries to be in /usr.

I put this qt.conf in the chroot:

[Paths]
Prefix=/home/enrico/…/arm-linux-gnueabihf/usr
ArchData=lib/arm-linux-gnueabihf/qt5
Binaries=lib/qt5/bin
Data=share/qt5
Documentation=share/qt5/doc
Examples=lib/arm-linux-gnueabihf/qt5/examples
Headers=include/arm-linux-gnueabihf/qt5
HostBinaries=bin
HostData=lib/arm-linux-gnueabihf/qt5
HostLibraries=lib/arm-linux-gnueabihf
Imports=lib/arm-linux-gnueabihf/qt5/imports
Libraries=lib/arm-linux-gnueabihf
LibraryExecutables=lib/arm-linux-gnueabihf/qt5/libexec
Plugins=lib/arm-linux-gnueabihf/qt5/plugins
Qml2Imports=lib/arm-linux-gnueabihf/qt5/qml
Settings=/etc/xdg
Translations=share/qt5/translations
TargetSpec=arm-linux-gnueabihf

I added the custom mkspecs to the chroot:

$ cd arm-linux-gnueabihf/usr/lib/arm-linux-gnueabihf/qt5/mkspecs/
$ sudo cp -r linux-g++ arm-linux-gnueabihf
$ sudo vi arm-linux-gnueabihf/qmake.conf

This is the content of usr/lib/arm-linux-gnueabihf/qt5/mkspecs/arm-linux-gnueabihf/qmake.conf:

#
# qmake configuration for arm-linux-gnueabihf
#

MAKEFILE_GENERATOR      = UNIX
CONFIG                 += incremental
QMAKE_INCREMENTAL_STYLE = sublib

include(../common/linux.conf)
include(../common/gcc-base-unix.conf)
include(../common/g++-unix.conf)

QMAKE_RPATHLINKDIR += /home/enrico/…/arm-linux-gnueabihf/lib/arm-linux-gnueabihf
QMAKE_RPATHLINKDIR += /home/enrico/…/arm-linux-gnueabihf/usr/lib/arm-linux-gnueabihf
QMAKE_RPATHLINKDIR += /home/enrico/…/arm-linux-gnueabihf/usr/lib/

QMAKE_COMPILER          = arm-linux-gnueabihf-gcc

QMAKE_CC                = arm-linux-gnueabihf-gcc

QMAKE_LINK_C            = $$QMAKE_CC
QMAKE_LINK_C_SHLIB      = $$QMAKE_CC

QMAKE_CXX               = arm-linux-gnueabihf-g++

QMAKE_LINK              = $$QMAKE_CXX
QMAKE_LINK_SHLIB        = $$QMAKE_CXX

load(qt_config)

Note QMAKE_RPATHLINKDIR, which was not present in the previous post: since linking needs to happen against libraries in /home/enrico/…/arm-linux-gnueabihf/…, we need to tell the linker not to go and search in /.

The extra QMAKE_RPATHLINKDIR pointing to usr/lib is a workaround for libsrtp0 installing files outside multiarch directories (#765173):

# dpkg -L libsrtp0 | grep usr/lib
/usr/lib
/usr/lib/libsrtp.so.0.0
/usr/lib/libsrtp.so.0

(libsrtp0 is a dependency of libqt5webenginecore5)

I changed the wrapper /usr/local/bin/qmake-arm-linux-gnueabihf to point to the chroot:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
#!/usr/bin/python3

import sys, os

QT_CONFIG = "/home/enrico/…/arm-linux-gnueabihf/qt.conf"

argv0 = os.path.join(os.path.dirname(sys.argv[0]), "qmake")

if len(sys.argv) == 1:
    os.execv("/usr/bin/qmake", [argv0] + sys.argv[1:])
else:
    os.execv("/usr/bin/qmake", [argv0] + sys.argv[1:] + ["-qtconf", QT_CONFIG])

And it all works. Even the test application that requires QtWebEngine builds and links fine.

Credits

This has been done as part of my work with Truelite.

pdo debian eng qt-cross

Qt Creator cross-platform development in Stretch: consolidation

Time to consolidate the exploration done yesterday.

qmake -qtconf

Looking at the work done by the Qt/KDE team I found that there is no need to rebuild qmake, as it can be configured by using the -qtconf option.

The work recently done on debhelper provides a starting point that I can build on.

This qtconf makes qmake use the right paths, including running rcc, moc and uic from /usr/bin/, removing the need for the dirty hack:

[Paths]
Prefix=/usr
ArchData=lib/arm-linux-gnueabihf/qt5
Binaries=lib/qt5/bin
Data=share/qt5
Documentation=share/qt5/doc
Examples=lib/arm-linux-gnueabihf/qt5/examples
Headers=include/arm-linux-gnueabihf/qt5
HostBinaries=bin
HostData=lib/arm-linux-gnueabihf/qt5
HostLibraries=lib/arm-linux-gnueabihf
Imports=lib/arm-linux-gnueabihf/qt5/imports
Libraries=lib/arm-linux-gnueabihf
LibraryExecutables=lib/arm-linux-gnueabihf/qt5/libexec
Plugins=lib/arm-linux-gnueabihf/qt5/plugins
Qml2Imports=lib/arm-linux-gnueabihf/qt5/qml
Settings=/etc/xdg
Translations=share/qt5/translations

It still needs overriding the compiler settings, to make it use a compiler different than g++:

qmake -makefile -qtconf /tmp/qmake.conf QMAKE_CC=arm-linux-gnueabihf-gcc QMAKE_CXX=arm-linux-gnueabihf-g++ QMAKE_LINK=arm-linux-gnueabihf-g++

But then -m64 creeps in and breaks the cross-build (this has been fixed since qtbase 5.9.1+dfsg-12. but my target is stretch):

$ make
arm-linux-gnueabihf-g++ -c -m64 -pipe -std=c++11 -O2 -Wall -W -D_REENTRANT -fPIC -DQT_DEPRECATED_WARNINGS -DQT_NO_DEBUG -DQT_SVG_LIB -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -I. -isystem /usr/include/arm-linux-gnueabihf/qt5 -isystem /usr/include/arm-linux-gnueabihf/qt5/QtSvg -isystem /usr/include/arm-linux-gnueabihf/qt5/QtWidgets -isystem /usr/include/arm-linux-gnueabihf/qt5/QtGui -isystem /usr/include/arm-linux-gnueabihf/qt5/QtCore -I. -I/usr/lib/arm-linux-gnueabihf/qt5/mkspecs/linux-g++-64 -o main.o main.cpp
arm-linux-gnueabihf-g++: error: unrecognized command line option ‘-m64’
Makefile:401: recipe for target 'main.o' failed
make: *** [main.o] Error 1

Where does it come from? I found that I could run qmake -query:

$ qmake -query -qtconf /tmp/qmake.conf
QT_SYSROOT:
QT_INSTALL_PREFIX:/usr
QT_INSTALL_ARCHDATA:/usr/lib/arm-linux-gnueabihf/qt5
QT_INSTALL_DATA:/usr/share/qt5
QT_INSTALL_DOCS:/usr/share/qt5/doc
QT_INSTALL_HEADERS:/usr/include/arm-linux-gnueabihf/qt5
QT_INSTALL_LIBS:/usr/lib/arm-linux-gnueabihf
QT_INSTALL_LIBEXECS:/usr/lib/arm-linux-gnueabihf/qt5/libexec
QT_INSTALL_BINS:/usr/lib/qt5/bin
QT_INSTALL_TESTS:/usr/tests
QT_INSTALL_PLUGINS:/usr/lib/arm-linux-gnueabihf/qt5/plugins
QT_INSTALL_IMPORTS:/usr/lib/arm-linux-gnueabihf/qt5/imports
QT_INSTALL_QML:/usr/lib/arm-linux-gnueabihf/qt5/qml
QT_INSTALL_TRANSLATIONS:/usr/share/qt5/translations
QT_INSTALL_CONFIGURATION:/etc/xdg
QT_INSTALL_EXAMPLES:/usr/lib/arm-linux-gnueabihf/qt5/examples
QT_INSTALL_DEMOS:/usr/lib/arm-linux-gnueabihf/qt5/examples
QT_HOST_PREFIX:/usr
QT_HOST_DATA:/usr/lib/arm-linux-gnueabihf/qt5
QT_HOST_BINS:/usr/bin
QT_HOST_LIBS:/usr/lib/arm-linux-gnueabihf
QMAKE_SPEC:linux-g++-64
QMAKE_XSPEC:linux-g++-64
QMAKE_VERSION:3.0
QT_VERSION:5.7.1

I suppose that QMAKE_XSPEC should be linux-g++ instead of linux-g++-64

Poking through Qt's sources I found that I could also tweak TargetSpec, to change it to linux-g++ from its default of linux-g++-64:

[Paths]
Prefix=/usr
ArchData=lib/arm-linux-gnueabihf/qt5
Binaries=lib/qt5/bin
Data=share/qt5
Documentation=share/qt5/doc
Examples=lib/arm-linux-gnueabihf/qt5/examples
Headers=include/arm-linux-gnueabihf/qt5
HostBinaries=bin
HostData=lib/arm-linux-gnueabihf/qt5
HostLibraries=lib/arm-linux-gnueabihf
Imports=lib/arm-linux-gnueabihf/qt5/imports
Libraries=lib/arm-linux-gnueabihf
LibraryExecutables=lib/arm-linux-gnueabihf/qt5/libexec
Plugins=lib/arm-linux-gnueabihf/qt5/plugins
Qml2Imports=lib/arm-linux-gnueabihf/qt5/qml
Settings=/etc/xdg
Translations=share/qt5/translations
TargetSpec=linux-g++

Now, I still need to override the compilers, but that's all I need to do to get a build:

$ qmake -makefile -qtconf /tmp/qmake.conf QMAKE_CC=arm-linux-gnueabihf-gcc QMAKE_CXX=arm-linux-gnueabihf-g++ QMAKE_LINK=arm-linux-gnueabihf-g++
$ make
…
$ file usr/bin/program
usr/bin/program: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=541e472a29a2b309af2fea40d9aa3439e6616bdd, not stripped

Can I get rid of needing to override the compilers? Let's try poking with TargetSpec:

# cd /usr/lib/arm-linux-gnueabihf/qt5/mkspecs/
# cp -r linux-g++ arm-linux-gnueabihf
# cd arm-linux-gnueabihf
# …tweak…
# cat qmake.conf
#
# qmake configuration for arm-linux-gnueabihf
#

MAKEFILE_GENERATOR      = UNIX
CONFIG                 += incremental
QMAKE_INCREMENTAL_STYLE = sublib

include(../common/linux.conf)
include(../common/gcc-base-unix.conf)
include(../common/g++-unix.conf)

QMAKE_COMPILER          = arm-linux-gnueabihf-gcc

QMAKE_CC                = arm-linux-gnueabihf-gcc

QMAKE_LINK_C            = $$QMAKE_CC
QMAKE_LINK_C_SHLIB      = $$QMAKE_CC

QMAKE_CXX               = arm-linux-gnueabihf-g++

QMAKE_LINK              = $$QMAKE_CXX
QMAKE_LINK_SHLIB        = $$QMAKE_CXX

load(qt_config)

Now, using arm-linux-gnueabihf as TargetSpec:

[Paths]
Prefix=/usr
ArchData=lib/arm-linux-gnueabihf/qt5
Binaries=lib/qt5/bin
Data=share/qt5
Documentation=share/qt5/doc
Examples=lib/arm-linux-gnueabihf/qt5/examples
Headers=include/arm-linux-gnueabihf/qt5
HostBinaries=bin
HostData=lib/arm-linux-gnueabihf/qt5
HostLibraries=lib/arm-linux-gnueabihf
Imports=lib/arm-linux-gnueabihf/qt5/imports
Libraries=lib/arm-linux-gnueabihf
LibraryExecutables=lib/arm-linux-gnueabihf/qt5/libexec
Plugins=lib/arm-linux-gnueabihf/qt5/plugins
Qml2Imports=lib/arm-linux-gnueabihf/qt5/qml
Settings=/etc/xdg
Translations=share/qt5/translations
TargetSpec=arm-linux-gnueabihf

It finally works:

$ qmake -makefile -qtconf /tmp/qmake.conf
$ make
$ file usr/bin/program
usr/bin/program: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=541e472a29a2b309af2fea40d9aa3439e6616bdd, not stripped

Skipping prl file '/usr/lib/arm-linux-gnueabihf/libQt5PlatformSupport.prl', because it cannot be opened (No such file or directory).

Turn this into a kit for Qt Creator

I could not find a way to configure -qtconf in a kit in Qt Creator. Since Qt Creator detects versions of Qt by pointing at their qmake, I resorted to making a custom qmake wrapper:

#!/usr/bin/python3

# /usr/local/bin/qmake-arm-linux-gnueabihf

import sys, os

QT_CONFIG = "/home/enrico/qt-arm-linux-gnueabihf.conf"

argv0 = os.path.join(os.path.dirname(sys.argv[0]), "qmake")

if len(sys.argv) == 1:
    os.execv("/usr/bin/qmake", [argv0] + sys.argv[1:])
else:
    os.execv("/usr/bin/qmake", [argv0] + sys.argv[1:] + ["-qtconf", QT_CONFIG])

Just calling qmake -qtconf /home/enrico/qt-arm-linux-gnueabihf.conf "$@" is not enough: for some invocations of qmake, providing -qtconf gives an error:

$ qmake -qtconf /home/enrico/qt-arm-linux-gnueabihf.conf
qmake: could not find a Qt installation of 'conf'

If needed I can look at qmake's sources to see what is going on, but for now the wrapper above seems to be enough to cover all the needs of Qt Creator, and if the wrapper is in the path, Qt Creator manages to autodetect it on startup.

These are the details of a working arm-linux-gnueabihf kit for Qt Creator:

  • Device type: Generic Linux Device
  • Device: my hi-fi
  • Sysroot: blank
  • C compiler: /usr/bin/arm-linux-gnueabihf-gcc
  • C++ compiler: /usr/bin/arm-linux-gnueabihf-g++
  • Qt version: /usr/local/bin/qmake-arm-linux-gnueabihf

And with this, software cross-builds locally and can be deployed and tested remotely.

Summary of the situation so far

This is what is needed to do cross-build now:

  • Depend on crossbuild-essential-armhf
  • Install :armhf versions of Qt and all build-dependencies
  • Deploy a custom mkspec
  • Deploy a custom qt.conf
  • Deploy qmake wrapper
  • Configure a kit

No dirty hacks, everything can be shipped by a single .deb package.

Further things to investigate:

  • Find out if there is a way to package and install a kit for Qt Creator
  • Find an easy way to use a chroot as the cross-build environment, to be able to use cross-build-dependencies that cannot be coinstalled together with their native version

Credits

This has been done as part of my work with Truelite.

pdo debian eng qt-cross

Qt Creator cross-platform development in Stretch: exploration

I have been asked to create a system that makes it easy to do cross-architecture Qt development with Debian Stretch.

The goal is this:

  • Do development with Qt Creator, including remote debugging on the target architecture
  • Compile using native code, to avoid running the compilers inside an emulator making the build slower
  • Leverage all of Debian as development environment, using existing Debian packages for build-dependencies
  • Set everything up just by installing a single .deb package, which ideally gives Qt Creator a new kit that just works

I did some exploration on this some time ago, and the Qt/KDE and cross-build people did some work on this, too.

Now I'm trying to build on all that. I need to target Stretch so I cannot build on the recent improvements in Qt packaging, but I can at least use the experience that went into those changes.

I have two sample Qt application to try and cross-build, one that depends on a non-Qt library (libmosquittopp-dev), and one that depends on a nasty Qt library (qtwebengine5-dev).

This post has the notes from the first day of trying out different strategies.

It begins

I imported the two projects in Qt creator, installed their amd64 dependencies and make sure they build for the current system, with the default kit, no cross-building yet.

That works, good.

Now let's see about the rest:

dpkg --add-architecture armhf
apt install crossbuild-essential-armhf

A cross-build kit for Qt Creator

I created a new armhf kit for Qt Creator:

  • Device type: Generic Linux Device
  • Device: my hi-fi
  • Sysroot: blank
  • C compiler: /usr/bin/arm-linux-gnueabihf-gcc
  • C++ compiler: /usr/bin/arm-linux-gnueabihf-g++
  • Qt version: /usr/lib/arm-linux-gnueabihf/qt5/bin/qmake

I ran qmake from Qt Creator and go this:

10:59:49: Starting: "/usr/lib/arm-linux-gnueabihf/qt5/bin/qmake" …/project.pro -spec linux-g++ CONFIG+=debug CONFIG+=qml_debug
sh: 1: /usr/lib/arm-linux-gnueabihf/qt5/bin/rcc: not found
10:59:50: The process "/usr/lib/arm-linux-gnueabihf/qt5/bin/qmake" exited normally.

rcc is provided by qtbase5-dev-tools, which cannot be coinstalled alongside other architectures' version of itself:

# apt install qtbase5-dev-tools:armhf
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  qtbase5-dev-tools
The following NEW packages will be installed:
  qtbase5-dev-tools:armhf
0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
Need to get 651 kB of archives.
After this operation, 985 kB disk space will be freed.
Do you want to continue? [Y/n] q

I'll try with a very dirty hack:

# cd /usr/lib/arm-linux-gnueabihf/qt5/bin
# ln -s `which rcc` .
# ln -s `which uic` .
# ln -s `which moc` .
# ls -la
total 1944
drwxr-xr-x 2 root root    4096 Nov 29 11:04 .
drwxr-xr-x 7 root root    4096 Nov 28 17:05 ..
lrwxrwxrwx 1 root root      12 Nov 29 11:04 moc -> /usr/bin/moc
-rwxr-xr-x 1 root root 1982208 Jan 11  2017 qmake
lrwxrwxrwx 1 root root      12 Nov 29 11:04 rcc -> /usr/bin/rcc
lrwxrwxrwx 1 root root      12 Nov 29 11:04 uic -> /usr/bin/uic

This is ugly:

  • It places files in /usr outside /usr/local that are not maintained by the package manager
  • It places amd64 executables in /usr/lib/arm-linux-gnueabihf which should contain armhf code

Let's see what happens in Qt Creator:

11:14:35: Starting: "/usr/lib/arm-linux-gnueabihf/qt5/bin/qmake" …/project.pro -spec linux-g++ CONFIG+=debug CONFIG+=qml_debug
11:14:35: The process "/usr/lib/arm-linux-gnueabihf/qt5/bin/qmake" exited normally.
11:14:35: Starting: "/usr/bin/make" qmake_all
make: Nothing to be done for 'qmake_all'.
11:14:36: The process "/usr/bin/make" exited normally.

and build:

11:15:29: Starting: "/usr/bin/make"
g++ -c -pipe -std=c++11 -g -Wall -W -D_REENTRANT -fPIC -DQT_DEPRECATED_WARNINGS -DQT_QML_DEBUG -DQT_SVG_LIB -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -I../project -I. -isystem /usr/include/arm-linux-gnueabihf/qt5 -isystem /usr/include/arm-linux-gnueabihf/qt5/QtSvg -isystem /usr/include/arm-linux-gnueabihf/qt5/QtWidgets -isystem /usr/include/arm-linux-gnueabihf/qt5/QtGui -isystem /usr/include/arm-linux-gnueabihf/qt5/QtCore -I. -I/usr/lib/arm-linux-gnueabihf/qt5/mkspecs/linux-g++ -o main.o ../project/main.cpp

Unfortunately, it is using g++ even though I configured the kit to use /usr/bin/arm-linux-gnueabihf-* instead.

Trying an explicit override in the .pro:

QMAKE_CXX = /usr/bin/arm-linux-gnueabihf-g++
QMAKE_LINK = /usr/bin/arm-linux-gnueabihf-g++

And the project builds and runs fine on the Raspberry Pi, using a kit and two simple overrides in the .pro, all build dependencies as packaged by Debian, and a toolchain that entirely runs on native code.

Entirely?

$ file /usr/lib/arm-linux-gnueabihf/qt5/bin/qmake
/usr/lib/arm-linux-gnueabihf/qt5/bin/qmake: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=1f5b063926570702f5568b1e5cec6c70d214fc73, stripped
$ dpkg-architecture -q DEB_HOST_ARCH
amd64
$ /usr/lib/arm-linux-gnueabihf/qt5/bin/qmake -v
QMake version 3.0
Using Qt version 5.7.1 in /usr/lib/arm-linux-gnueabihf

Indeed qmake is armhf code that runs happily on my amd64 laptop because I accidentally have qemu-user-static installed.

This strategy produces results, although it depends on a dirty hack. To summarise it:

  • Depend on crossbuild-essential-armhf, qemu-user-static, qemu-system-arm
  • Install :armhf versions of Qt and all build-dependencies
  • Symlink native moc, rcc, uic into the armhf bin directory (argh!)
  • Configure a kit as before
  • Override QMAKE_CXX and QMAKE_LINK in the .pro

Other things to investigate, besides removing the need for that dirty hack:

  • Find out if there is a way to package and install a kit for Qt Creator
  • Find out why qmake is ignoring the compiler settings from the kit and needs overrides in the .pro. Ideally the .pro should be unmodified, so that it can be used for all builds

A native qmake

Would it be possible to build a native version of qmake tweaked to point to all the right bits out of the box?

$ apt source qtbase-opensource-src-5.7.1+dfsg
$ qtbase-opensource-src-5.7.1+dfsg$
$ DEB_HOST_MULTIARCH=arm-linux-gnueabihf DEB_HOST_ARCH_BITS=32 debian/rules override_dh_auto_configure
MAKEFLAGS="-j1" ./configure \
            -confirm-license \
            -prefix "/usr" \
            -bindir "/usr/lib/arm-linux-gnueabihf/qt5/bin" \
            -libdir "/usr/lib/arm-linux-gnueabihf" \
            -docdir "/usr/share/qt5/doc" \
            -headerdir "/usr/include/arm-linux-gnueabihf/qt5" \
            -datadir "/usr/share/qt5" \
            -archdatadir "/usr/lib/arm-linux-gnueabihf/qt5" \
            -plugindir "/usr/lib/arm-linux-gnueabihf/qt5/plugins" \
            -importdir "/usr/lib/arm-linux-gnueabihf/qt5/imports" \
            -translationdir "/usr/share/qt5/translations" \
            -hostdatadir "/usr/lib/arm-linux-gnueabihf/qt5" \
            -sysconfdir "/etc/xdg" \
            -examplesdir "/usr/lib/arm-linux-gnueabihf/qt5/examples" \
            -opensource \
            -plugin-sql-mysql \
            -plugin-sql-odbc \
            -plugin-sql-psql \
            -plugin-sql-sqlite \
            -no-sql-sqlite2 \
            -plugin-sql-tds \
            -system-sqlite \
            -platform linux-g++ \
            -system-harfbuzz \
            -system-zlib \
            -system-libpng \
            -system-libjpeg \
            -system-doubleconversion \
            -openssl \
            -no-rpath \
            -verbose \
            -optimized-qmake \
            -dbus-linked \
            -no-strip \
            -no-separate-debug-info \
            -qpa xcb \
            -xcb \
            -glib \
            -icu \
            -accessibility \
            -compile-examples \
            -no-directfb \
            -gstreamer 1.0 \
            -plugin-sql-ibase -opengl desktop \
…
$ file bin/qmake
bin/qmake: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=ba868730cf34c54d7ddf6df0ab4b6ce5c7d6f2a0, not stripped
$ bin/qmake -v
QMake version 3.0
Using Qt version 5.7.1 in /usr/lib/arm-linux-gnueabihf

./configure in Qt builds qmake, and there is no need to run make afterwards. Good.

DEB_HOST_ARCH_BITS=32 is a hack I added to avoid debian/rules using -platform linux-g++-64 instead of -platform linux-g++.

Let's try to use that in Qt Creator:

sudo cp bin/qmake /usr/local/bin/qmake-arm-linux-gnueabihf

Qt Creator autodetects the new qmake and offers it as one of the available versions of Qt. Nice.

The need for the symlink hack is still there:

11:46:51: Starting: "/usr/local/bin/qmake-arm-linux-gnueabihf" ../project.pro -spec linux-g++-64 CONFIG+=debug CONFIG+=qml_debug
sh: 1: /usr/lib/arm-linux-gnueabihf/qt5/bin/rcc: not found

So is the need for the QMAKE_CXX and QMAKE_LINK overrides in the .pro.

Still, this way I could remove qemu-user-static from my system and the project still builds on my laptop and runs on my Raspberry Pi.

The qemu dependency is not needed anymore, the rest of the problems still stand. I wonder, since I'm rebuilding qmake, if there's a way to tell it to use the compilers I want, and the tools I want, removing the need for the dirty hack and the overrides in the .pro files.

qtwebengine5-dev

How about the other project that depends on qtwebengine5-dev?

# apt install qtwebengine5-dev qtwebengine5-dev:armhf
Reading package lists... Done
Building dependency tree
Reading state information... Done
qtwebengine5-dev is already the newest version (5.7.1+dfsg-6.1).
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 qtwebengine5-dev:armhf : Depends: libqt5webengine5:armhf (= 5.7.1+dfsg-6.1) but it is not going to be installed
                          Depends: libqt5webenginecore5:armhf (= 5.7.1+dfsg-6.1) but it is not going to be installed
                          Depends: libqt5webenginewidgets5:armhf (= 5.7.1+dfsg-6.1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

It turns out that something in the big chain of dependencies of qtwebengine5-dev makes the amd64 and armhf versions not coinstallable.

It seems that I have to abandon the idea of installing armhf build-dependencies in the main development system, and I need to start considering to leave the host system untouched to do the native builds, and use a chroot for the cross-compilers and the cross-build-dependencies.

In the next post, I'll see how that goes.

Credits

This has been done as part of my work with Truelite.

pdo debian eng qt-cross

Qt cross-architecture development in Debian

Update: this post is now obsolete: cross-building Qt applications now works in Debian!

Follows the old post:

Use case: use Debian Stable as an environment to run amd64 development machines to develop Qt applications for Raspberry Pi or other smallish armhf devices.

Qt Creator is used as Integrated Development Environment, and it supports cross-compiling, running the built source on the target system, and remote debugging.

Debian Stable (vanilla or Raspbian) runs on both the host and the target systems, so libraries can be kept in sync, and both systems have access to a vast amount of libraries, with security support.

On top of that, armhf libraries can be installed with multiarch also in the host machine, so cross-builders have access to the exact same libraries as the target system.

This sounds like a dream system. But. We're not quite there yet.

cross-compile attempts

I tried cross compiling a few packages:

$ sudo debootstrap stretch cross
$ echo "strech_cross" | sudo tee cross/etc/debian_chroot
$ sudo systemd-nspawn -D cross
# dpkg --add-architecture armhf
# echo "deb-src http://deb.debian.org/debian stretch main" >> /etc/apt/sources.list
# apt update
# apt install --no-install-recommends build-essential crossbuild-essential-armhf

Some packages work:

# apt source bc
# cd bc-1.06.95/
# apt-get build-dep -a armhf .
# dpkg-buildpackage -aarmhf -j2 -b
…
dh_auto_configure -- --prefix=/usr --with-readline
        ./configure --build=x86_64-linux-gnu --prefix=/usr --includedir=\${prefix}/include --mandir=\${prefix}/share/man --infodir=\${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-silent-rules --libdir=\${prefix}/lib/arm-linux-gnueabihf --libexecdir=\${prefix}/lib/arm-linux-gnueabihf --disable-maintainer-mode --disable-dependency-tracking --host=arm-linux-gnueabihf --prefix=/usr --with-readline
…
dpkg-deb: building package 'dc-dbgsym' in '../dc-dbgsym_1.06.95-9_armhf.deb'.
dpkg-deb: building package 'bc-dbgsym' in '../bc-dbgsym_1.06.95-9_armhf.deb'.
dpkg-deb: building package 'dc' in '../dc_1.06.95-9_armhf.deb'.
dpkg-deb: building package 'bc' in '../bc_1.06.95-9_armhf.deb'.
 dpkg-genbuildinfo --build=binary
 dpkg-genchanges --build=binary >../bc_1.06.95-9_armhf.changes
dpkg-genchanges: info: binary-only upload (no source code included)
 dpkg-source --after-build bc-1.06.95
dpkg-buildpackage: info: binary-only upload (no source included)

With qmake based Qt packages, qmake is not configured for cross-building, probably because it is not currently supported:

# apt source pumpa
# cd pumpa-0.9.3/
# apt-get build-dep -a armhf .
# dpkg-buildpackage -aarmhf -j2 -b
…
        qmake -makefile -nocache "QMAKE_CFLAGS_RELEASE=-g -O2 -fdebug-prefix-map=/root/pumpa-0.9.3=.
          -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2"
          "QMAKE_CFLAGS_DEBUG=-g -O2 -fdebug-prefix-map=/root/pumpa-0.9.3=. -fstack-protector-strong
          -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2"
          "QMAKE_CXXFLAGS_RELEASE=-g -O2 -fdebug-prefix-map=/root/pumpa-0.9.3=. -fstack-protector-strong
          -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2"
          "QMAKE_CXXFLAGS_DEBUG=-g -O2 -fdebug-prefix-map=/root/pumpa-0.9.3=. -fstack-protector-strong
          -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2"
          "QMAKE_LFLAGS_RELEASE=-Wl,-z,relro -Wl,-z,now"
          "QMAKE_LFLAGS_DEBUG=-Wl,-z,relro -Wl,-z,now" QMAKE_STRIP=: PREFIX=/usr
qmake: could not exec '/usr/lib/x86_64-linux-gnu/qt5/bin/qmake': No such file or directory
…
debian/rules:19: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

With cmake based Qt packages it goes a little better in that it finds the cross compiler, pkg-config and some multiarch paths, but then it tries to run armhf moc, which fails:

# apt source caneda
# cd caneda-0.3.0/
# apt-get build-dep -a armhf .
# dpkg-buildpackage -aarmhf -j2 -b
…
        cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=None
          -DCMAKE_INSTALL_SYSCONFDIR=/etc -DCMAKE_INSTALL_LOCALSTATEDIR=/var -DCMAKE_SYSTEM_NAME=Linux
          -DCMAKE_SYSTEM_PROCESSOR=arm -DCMAKE_C_COMPILER=arm-linux-gnueabihf-gcc
          -DCMAKE_CXX_COMPILER=arm-linux-gnueabihf-g\+\+
          -DPKG_CONFIG_EXECUTABLE=/usr/bin/arm-linux-gnueabihf-pkg-config
          -DCMAKE_INSTALL_LIBDIR=lib/arm-linux-gnueabihf
…
CMake Error at /usr/lib/arm-linux-gnueabihf/cmake/Qt5Core/Qt5CoreConfig.cmake:27 (message):
  The imported target "Qt5::Core" references the file

     "/usr/lib/arm-linux-gnueabihf/qt5/bin/moc"

  but this file does not exist.  Possible reasons include:

  * The file was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and contained

     "/usr/lib/arm-linux-gnueabihf/cmake/Qt5Core/Qt5CoreConfigExtras.cmake"

  but not all the files it references.

Note: Although I improvised a chroot to be able to fool around with it, I would use pbuilder or sbuild to do the actual builds.

Helmut suggests pbuilder --host-arch or sbuild --host.

Doing it the non-Debian way

This guide in the meantime explains how to set up a cross-compiling Qt toolchain in a rather dirty way, by recompiling Qt pointing it at pieces of the Qt deployed on the Raspberry Pi.

Following that guide, replacing the CROSS_COMPILE value with /usr/bin/arm-linux-gnueabihf- gave me a working qtbase, for which it is easy to create a Kit for Qt Creator that works, and supports linking applications with Debian development packages that do not use Qt.

However, at that point I need to recompile all dependencies that use Qt myself, and I quickly got stuck at that monster of QtWebEngine, whose sources embed the whole of Chromium.

Having a Qt based development environment in which I need to become the maintainer for the whole Qt toolchain is not a product I can offer to a customer. Cross compiling qmake based packages on stretch is not currently supported, so at the moment I had to suggest to postpone all plans for total world domination for at least two years.

Cross-building Debian

In the meantime, Helmut Grohne has been putting a lot of effort into making Debian packages cross-buildable:

helmut> enrico: yes, cross building is painful. we have ~26000 source packages. of those, ~13000 build arch-dep packages. of those, ~6000 have cross-satisfiable build-depends. of those, I tried cross building ~2300. of those 1300 cross built. so we are at about 10% working.

helmut> enrico: plus there are some 607 source packages affected by some 326 bugs with patches.

helmut> enrico: gogo nmu them

helmut> enrico: I've filed some 1000 bugs (most of them with patches) now. around 600 are fixed :)

He is doing it mostly alone, and I would like people not to be alone when they do a lot of work in Debian, so…

Join Helmut in the effort of making Debian cross-buildable!

Build any Debian package for any device right from the comfort of your own work computer!

Have a single development environment seamlessly spanning architecture boundaries, with the power of all that there is in Debian!

Join Helmut in the effort of making Debian cross-buildable!

Apply here, or join #debian-bootstrap on OFTC!

Cross-building Qt in Debian

mitya57 summarised the situation on the KDE team side:

mitya57> we have cross-building stuff on our TODO list, but it will likely require a lot of time and neither Lisandro nor I have it currently.

mitya57> see https://gobby.debian.org/export/Teams/KDE/qt-cross for a summary of what needs to be done.

mitya57> Any help or patches are always welcome :))

qemu-user-static

Helmut also suggested to use qemu-user-static to make the host system able to run binaries compiled for the target system, so that even if a non-cross-compiling Qt build tries to run moc and friends in their target architecture version, they would transparently succeed.

At that point, it would just be a matter of replacing compiler paths to point to the native cross-compiling gcc, and the build would not be slowed down by much.

Fixing bug #781226 would help in making it possible to configure a multiarch version of qmake as the qmake used for cross compiling.

I have not had a chance of trying to cross-build in this way yet.

In the meantime...

Having qtcreator able to work on an amd64 devel machine and deploy/test/debug remotely on an arm target machine, where both machine run debian stable and have libraries in sync, would be a great thing to have even though packages do not cross-build yet.

Helmut summarised the situation on IRC:

svuorela and others repeat that Qt upstream is not compatible with Debian's multiarch thinking, in that Qt upstream insists on having one toolchain for each pair of architectures, whereas the Debian way tends to be to make packages generic and split stuff such that it can be mixed and matched.

An example being that you need to run qmake (thus you need qmake for the build architecture), but qmake also embeds the relevant paths and you need to query it for them (so you need qmake for the host architecture)

Either you run it through qemu, or you have a particular cross qmake for your build/host pair, or you fix qt upstream to stop this madness

Building qmake in Debian for each host-target pair, even just limited to released architectures, would mean building Qt 100 times, and that's not going to scale.

I wonder:

  • can I have a qmake-$ARCH binary that can build a source using locally installed multiarch Qt libraries, do I need to recompile and ship the whole of Qt, or just qmake?
  • is there a recipe for building a cross-building Qt environment that would be able use Debian development libraries installed the normal multiarch way?
  • we can't do perfect yet, but can we do better than this?
pdo debian eng qt-cross
Archive
Top 15 tags
eng 650 pdo 546 debian 322 sw 235 tips 123 rant 79 life 76 links 74 talk 67 ita 64 songs 61 pub 60 cazzeggio 53 debtags 50 python 34
Recent series