Logging on the Freerunner

By default, the FreeRunner comes without a system log daemon, and rightfully so, because you don't want a daemon filling up /var, and a nightly cron job to rotate logs, on a mobile phone.

However, I do not want to give up logging. If there are bugs I want to investigate, I like to be able to look at the logs.

The solution: remote logging. Configure the phone to send all log info via UDP to my laptop, and configure my laptop to listen for UDP syslog messages from the phone only.

How to do it:

1. apt-get install rsyslog (see below how other logging daemons fail)
2. Add a /etc/hosts entry for the laptop on the phone: 192.168.0.200 base
3. Add a /etc/hosts entry for the phone on the laptop: 192.168.0.202 openmoko

4. Replace the phone's rsyslog rules to send everything to the laptop:

   ################
   ##### RULES ####
   ################
   
   
   # Everything goes to the main computer, via UDP, if available
   *.*     @base
   
   
   #
   # Emergencies are sent to everybody logged in.
   #
   *.emerg                         *
   
   

5. Tell the laptop's rsyslog to accept UDP messages from the phone only:

   # provides UDP syslog reception
   $ModLoad imudp
   $UDPServerRun 514
   $UDPServerAddress 192.168.0.200
   
   

That is all. When the phone is not connected, the UDP packets from the logger will go nowhere. When it gets connected to the laptop, a script is triggered that creates the 192.168.0.200 interface, rsyslogd will listen on that interface and the UDP packets will be logged.

This has the extra potential to be able to get error messages in case of bugs in the SD card driver, which I seem to hit from time to time.

Why rsyslog

sysklogd could receive the messages from the phone, but can only listen on all interfaces, and therefore requires me to set up firewall on my laptop in order to get packets from the phone only.

syslog-ng can and can be told to listen on a given interface, but it will refuse to start if that interface isn't available. Since the phone is on an hotplugged interface that comes and goes, this is totally unacceptable.

And finally, rsyslog is going to replace sysklogd both in Debian and in Fedora, so it's just as well a good excuse to switch.

Undoable apt-get build-dep

If I do apt-get build-dep foobar, all the dependencies will be installed, but they will not be marked as autoinstalled: if I then do not need to build the package anymore, I have a hard time finding out what to remove.

What I would like is a script that turns build dependencies of a package into a metapackage, so that I can later remove the metapackage and apt-get autoclean will remove everything from the system.

It turns out that it already exists: it's called mk-build-deps and is part of devscripts:

# apt-get install equivs
$ mk-build-deps lxlauncher

That will give you lxlauncher-build-deps_0.2-2_all.deb. It also works on arbitrary control files.

There is also a repository of such build-deps packages maintained by Frank Lichtenheld.

Running apt on the FreeRunner

I've already mentioned that I'm running approx in the laptop and I configured the FreeRunner to access the laptop's cache. Here are the other customisations needed to have a decently working apt:

# cat /etc/apt/apt.conf.d/99freerunner 
APT::Install-Recommends "false";
Acquire::PDiffs "false";

The rationale is that recommends would bloat a system that is supposed to be small, and pdiff requires more CPU, memory and disk space/time than it actually saves in bandwidth.

Thanks to Michael Banck and Peter Palfrader for helping me to find out how to disable pdiffs.

How to read the Freerunner's accelerometers

This code has been take from moko_eightball by Jakob Westhoff: it just continuously prints the value of the three accelerometers.

#include <stdio.h>
#include <stdint.h>

void processInputEvents(FILE* in)
{
    int x = 0, y = 0, z = 0;
    while (1)
    {
        char padding[16];
        uint16_t type, code;
        int32_t value;

        // Skip the timestamp
        fread(padding, 1, 8, in);

        // Read the type
        fread(&type, 1, 2, in);

        // Read the code
        fread(&code, 1, 2, in);

        // Read the value
        fread(&value, 1, 4, in);

        switch( type )
        {
            case 0:
                switch( code )
                {
                    case 0:
                        fprintf(stdout, "x%d y%d z%d\n", x, y, z);
                        break;
                    default:
                        //warning( "Unknown code ( 0x%02x ) for type 0x%02x\n", code, type );
                        break;
                }
                break;
            case 2:
                switch ( code )
                {
                    case 0:
                        // Update to the new value
                        x = value;
                        break;
                    case 1:
                        // Update to the new value
                        y = value;
                        break;
                    case 2:
                        // Update to the new value
                        z = value;
                        break;
                    default:
                        //warning( "Unknown code ( 0x%02x ) for type 0x%02x\n", code, type );
                        break;
                }
                break;

            default:
                //warning( "Unknown type ( 0x%02x ) in accelerometer input stream\n", type );
                break;
        }


    }
}


int main()
{
    FILE* in = fopen("/dev/input/event2", "r");
    processInputEvents(in);
    fclose(in);
    return 0;
}

Polysms

Here is my first software designed for the FreeRunner: polysms. It's a commandline tool: you pass it a polygen grammar name and a phone number, and it will send a SMS to that phone number using the polygen output for that grammar as the SMS text:

# polyrun manager 0012345678

And here is the code, that works on the http://www.freesmartphone.org dbus framework:

#!/usr/bin/python

# (C) 2008 Enrico Zini
# Most bits of this are stripped from zhone, which is:
# (C) 2007 Johannes 'Josch' Schauer
# (C) 2008 Michael 'Mickey' Lauer <mlauer@vanille-media.de>
# (C) 2008 Jan 'Shoragan' Luebbe
# (C) 2008 Daniel 'Alphaone' Willmann
# (C) 2008 Openmoko, Inc.
# GPLv2 or later

from dbus import SystemBus, Interface
from dbus.exceptions import DBusException
import logging
logger = logging.getLogger( __name__ )

from dbus.mainloop.glib import DBusGMainLoop
DBusGMainLoop(set_as_default=True)

import gobject
import sys
from subprocess import Popen, PIPE


class Phone:
    def tryGetProxy( self, busname, objname ):
        try:
            return self.bus.get_object( busname, objname )
        except DBusException, e:
            logger.warning( "could not create proxy for %s:%s" % ( busname, objname ) )

    def __init__(self):
        try:
            self.bus = SystemBus()
        except DBusException, e:
            logger.error( "could not connect to dbus_object system bus: %s" % e )
            return False

        # Phone
        self.gsm_device_obj = self.tryGetProxy( 'org.freesmartphone.ogsmd', '/org/freesmartphone/GSM/Device' )

        if ( self.gsm_device_obj is not None ):
            self.gsm_device_iface = Interface(self.gsm_device_obj, 'org.freesmartphone.GSM.Device')
            self.gsm_sim_iface = Interface(self.gsm_device_obj, 'org.freesmartphone.GSM.SIM')
            self.gsm_network_iface = Interface(self.gsm_device_obj, 'org.freesmartphone.GSM.Network')
            self.gsm_call_iface = Interface(self.gsm_device_obj, 'org.freesmartphone.GSM.Call')
            self.gsm_test_iface = Interface(self.gsm_device_obj, 'org.freesmartphone.GSM.Test')

        # Main loop
        self.loop = gobject.MainLoop()

    def send(self, number, message):
        def onSent():
            print "SENT"
            self.loop.quit()
        def onStore(index):
            print "STORED AS", index
            self.gsm_sim_iface.SendStoredMessage(
                index,
                reply_handler=onSent,
                error_handler=self.onError
            )
        self.gsm_sim_iface.StoreMessage(
            number, message,
            reply_handler=onStore,
            error_handler=self.onError
        )

    def onError(self, result):
        print "ERROR", result

    def mainloop(self):
        self.loop.run()

if len(sys.argv) != 3:
    print >>sys.stderr, "Usage: %s grammarname phonenumber"
    sys.exit(1)

message = Popen(["/usr/bin/polyrun", sys.argv[1]], stdout=PIPE).communicate()[0]
number = sys.argv[2]

print "Sending to %s:" % number
print message

phone = Phone()
phone.send(number, message)
phone.mainloop()

Docking the FreeRunner into the laptop

Earlier in the day, I wrote:

So yes, my laptop can now be turned into a phone charger with networking, DNS and apt cache services. I shall look into hooking that script into dbus to have it run automatically when the phone is plugged and unplugged.

It's not dbus, it's udev, and I've managed to do it.

It's an udev rule:

# cat /etc/udev/rules.d/z60_openmoko_net.rules
ACTION=="add", SUBSYSTEM=="net", ATTRS{idVendor}=="1457", ATTRS{idProduct}=="5122", RUN+="/root/bin/share-openmoko"
ACTION=="remove", SUBSYSTEM=="net", INTERFACE="usb0", RUN+="/root/bin/share-openmoko"

And a script:

# cat bin/share-openmoko 
#!/bin/sh

if [ "$ACTION" == "add" ]
then
    ACTION=start
fi

if [ "$ACTION" == "remove" ]
then
    ACTION=stop
fi

INTERFACE=${INTERFACE:-"usb0"}
ACTION=${ACTION:-"$1"}

case "$ACTION" in
    start)
        logger -t openmoko "Connected, setting up network"
        iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE -s 192.168.0.200/29
        ifconfig usb0 192.168.0.200 netmask 255.255.255.248
        /etc/init.d/dnsmasq start
        echo 1 > /proc/sys/net/ipv4/ip_forward
        ;;
    stop)
        logger -t openmoko "Disconnected, bringing down network"
        echo 0 > /proc/sys/net/ipv4/ip_forward
        /etc/init.d/dnsmasq stop
        iptables -t nat -F POSTROUTING
        ifconfig usb0 down
        ;;
esac

The script has extra cruft that makes it double as a udev script and as a init.d style script, just because I didn't feel like abandoning the init.d style interface. However, udev handles it perfectly, so there's probably no use at all for the start/stop part.

This means that the interface and all supporting services will be brought up and down when the phone is connected/disconnected, and also when the phone is suspended/resumed. Of course, more fancy things can be plugged into the script, like syncing PIM info, file systems, turning on applets in panels, mounting phone file systems and whatnot.

Unpacking the new FreeRunner

I got myself a FreeRunner. Here are some notes from the first few days:

Available operating systems

The FreeRunner comes with the Om 2007.2 distribution: it works for basic phone things, and it has an opkg package manager that you can use to install all sort of extra software. Only issue: the SMS application was rather unstable with my SIM card.

I then tried Om 2008.8, which is a major new redesign, partly based on Qtopia. It's definitely more advanced on how it looks, with smooth animations all around, but I did not manage to get the GSM part to work, because I did not manage to get it to ask for a PIN.

I then tried plain Qtopia and I got what looks and feels like a properly working mobile phone. The only issue that it has is that I did not manage to make it suspend, so battery life is much shorter than it could be. Apparently, this should improve as kernel 2.6.26 reaches the phone. I kept Qtopia installed in the phone flash as a "stable" phone system.

And finally, Debian. Debian is based on the freesmartphone.org software stack, which is an attempt to create a UI agnostic DBUS frontend to the hardware that supports multiple applications running on top of it. It is a young project, but it can already drive a mobile phone in a useful way. It has a demo interface that is basically a showcase of what is implemented in the DBUS frontend, but does most of the basic things you need from a mobile phone, including taking and making phone calls. And then it has Debian behind, all of it. I need to buy a bigger microsd card.

Tips and tricks

Flashing things

1. apt-get install dfu-util
2. Start the Freerunner bios/bootloader, by holding down Power and then pressing AUX (standard bootloader), or by holding down AUX and then pressing Power (factory, unbrickable, read only fail safe bootloader).
3. dfu-util -l shows you a list of devices it can access. If you see more than once, you need to specify in all dfu-util commands which one you want, using -d USBID. In my case, I have to always use dfu-util -d 0x1d50:0x5119
4. To flash the kernel: dfu-util -a kernel -R -D /path/to/uImage
5. To flash the root file system: dfu-util -a rootfs -R -D rootfs_filename.jffs2
6. To flash the bootloader: dfu-util -a u-boot -R -D uboot_filename.bin
7. To flash the u-boot configuration: dfu-util -a u-boot_env -D env.new

You can also download all of these things from the FreeRunner by using -U instead of -D.

Networking via USB

All of the distributions I tried, by default configure the USB as a gadget with ethernet over usb. You can form a lan with it using the cdc_ether module, and you will find your phone preconfigured as 192.168.0.202 expecting to find a gateway at 192.168.0.200.

I made myself this script to start and stop networking with the phone:

#!/bin/sh

case "$1" in
    start)
        iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE -s 192.168.0.200/29
        ifconfig usb0 192.168.0.200 netmask 255.255.255.248
        /etc/init.d/dnsmasq start
        echo 1 > /proc/sys/net/ipv4/ip_forward
        ;;
    stop)
        echo 0 > /proc/sys/net/ipv4/ip_forward
        /etc/init.d/dnsmasq stop
        iptables -t nat -F POSTROUTING
        ifconfig usb0 down
        ;;
esac

Besides doing masquerading, it also brings up dnsmasq so that the phone can always find a DNS together with the router.

Another useful trick is to configure the phone to share the approx cache with the laptop:

deb http://192.168.0.200:9999/debian unstable main
deb http://192.168.0.200:9999/debian experimental main
deb http://pkg-fso.alioth.debian.org/debian unstable main

So yes, my laptop can now be turned into a phone charger with networking, DNS and apt cache services. I shall look into hooking that script into dbus to have it run automatically when the phone is plugged and unplugged.

Changing the ringtone in Debian

In case you don't like the default ringtone, this is how to change it:

1. vi /usr/share/python-support/fso-frameworkd/framework/subsystems/oeventd/receiver.py
2. look for def _play( self ):
3. change the codec in decoder = gst.element_factory_make, if needed. The list of available plugins is on the gstreamer website.
4. change the path in filesrc.set_property
5. /etc/init.d/fso-frameworkd restart and maybe /etc/init.d/zhone-session restart

Yes, the ringtone is currently hardcoded, but it does say it's a prototype after all. Or, if you prefer, it's fully configurable and the configuration can be found in /usr/share/python-support/fso-frameworkd.

Configuring the bootloader

You can connect to the u-boot bootloader via a serial terminal on /dev/ttyACM0. Type help and you will find that it can do a lot of things. The OpenMoko wiki has pages on the bootloader itself, its commands and the environment.

The environment is the configuration of the bootloader, similar somehow to /boot/grub/menu.lst. Unlike grub, you can edit the environment from within the bootloader and then save it using the saveenv command.

What I did:

1. A boot entry for Debian, with the kernel on an ext2 partition instead of fat: setenv menu_3 Boot from microSD (FAT+ext2): setenv bootargs \${bootargs_base} rootfstype=ext2 root=/dev/mmcblk0p2 rootdelay=5 \${mtdparts} ro\; mmcinit\; fatload mmc 1 0x32000000 \${sd_image_name}\; bootm 0x32000000
2. Stay in the bootloader until a choice has been made: setenv bootdelay -1
3. Don't power down the bootloader when idle: setenv boot_menu_timeout 99999
4. Always show the menu at boot: setenv stop_in_menu yes
5. saveenv

So now my phone dual boots, and I can choose if I want a more reliable phone now (Qtopia) or if I want to play with my future phone (Debian).

Configuring ssh for two host keys on the same host

Minor issue, but annoying: since both QTopia and Debian show up on 192.168.0.202, ssh will complain about changed host keys. Here is how to configure ssh to avoid the problem (in ~/.ssh/config):

Host debian
HostName 192.168.0.202
User root
HostKeyAlias debian

Host qtopia
HostName 192.168.0.202
User root
HostKeyAlias qtopia

Introducing the Humongous Merged Packages File From Hell

Suppose you want to build some service that requires information about all packages in all architectures. Like, for example, the debtags web tagging interface.

If that is the case, then you may be interested in the Humongous Merged Packages File From Hell.

It contains a merge of all Packages and Source files of etch, lenny, sid and experimental, for main, contrib and non-free, in all architectures.

The merge is done according to completely arbitrary criteria of mine:

• One record per (package, version) couple found; multiple record per (package, version) are merged together
• Size and Installed-Size are the average of all values found
• MD5sum, SHA1, SHA256, Filename, Files, Directory, Checksums-Sha1, Checksums-Sha256, Binary are thrown away
• Information (such as Homepage, Vcs* fields, Uploaders) found in the Sources file is repeated for every binary package
• All other packages get as value a list of all the values found, with duplicates removed

It's there in case anyone other than me may find it useful.

Joys of schroot

Schroot is good! Try schroot. You have LVM set up and some free extents in a vg? Try this:

# Create a new logical volume for a sid chroot
lvcreate -n schroot-sid -L1G YOUR-VG-NAME
mke2fs -j /dev/YOUR-VG-NAME/schroot-sid

# Debootstrap sid into it
mount /dev/YOUR-VG-NAME/schroot-sid /mnt
debootstrap --keyring=/etc/apt/trusted.gpg sid /mnt http://ftp.uk.debian.org/debian
umount /mnt

Then apt-get install schroot and put this in your /etc/schroot/schroot.conf:

[sid]
type=lvm-snapshot
device=/dev/YOUR-VG-NAME/schroot-sid
description=Debian sid
priority=5
users=YOUR-USER-NAME
root-users=YOUR-USER-NAME
source-root-users=root
aliases=unstable
lvm-snapshot-options=--size 2G

Now, schroot -c sid will make a LVM snapshot of /dev/YOUR-VG-NAME/schroot-sid, mount it, set it up as a nice working system, and chroot into it running a shell. Have a look around: resolv.conf is set up, your home directory is bind mounted, /proc and /sys are mounted.

When you exit the chroot, the LVM snapshot disappears, and /dev/YOUR-VG-NAME/schroot-sid is unchanged.

If you want to modify /dev/YOUR-VG-NAME/schroot-sid, you just schroot -c sid-source.

In my laptop, I did this:

# schroot -c sid-source
(sid)# echo APT::Install-Recommends "false";' > /etc/apt/apt.conf.d/50enrico
(sid)# apt-get install build-essential, pbuilder, devscripts, fakeroot
(sid)# apt-get clean

At this point, to build a debian package like if I were using pbuilder, I do this:

pkgdir$ schroot -c sid
(sid)pkgdir$ su -c /usr/lib/pbuilder/pbuilder-satisfydepends
(sid)pkgdir$ debuild -us -uc

You can repeat this whole process with lenny and etch, for example, and if you are on an amd64 system, you can also do sid32, lenny32 and etch32.

If you don't want to redownload packages all the time, install approx and configure the -source chroots to use it. This also allows you to easily switch all systems from one mirror to another if, for example, you are traveling abroad.

If you want to do the same as pbuilder update, you can do this:

# schroot -c sid-source
(sid)# apt-get update
(sid)# apt-get dist-upgrade
(sid)# apt-get autoremove
(sid)# apt-get clean

You can use schroot -c sid-source for all sorts of permanent maintenance works.

You can also create persistent named snapshots:

$ schroot -b -c sid -n playground

This will only disappear when you do:

$ schroot -c playground -e

And you can enter the schroot as many time as you want with:

$ schroot -r -c playground

You can also enter a temporary snapshot as many times as you want, by using the random name that schroot generated for it (you can see it, for example, when running df or mount).

You fear that lvm snapshots are a little fragile and a little slow at the moment? It's not a problem: your home is bind-mounted inside the chroot, so you can do all the work in your normal home filesystem, without stressing the snapshot.

This is good not only to build packages. For example, now that svk is broken in sid and lenny, I just did schroot -b -c etch -n svk, installed svk on it, and then every time I need svk, I can do schroot -r -c svk and have it ready.

Added links: * using sbuild as a Debian maintainer * a similar article about schroot

Send a fax from the laptop

My bank sent me a PDF form via e-mail. I needed to fill it in, then send it back via fax. Send it back via e-mail would not work because it's not secure. The bank agrees that this is fantastically silly, but apparently this requirement is not their fault.

Step 1: send a fax with the laptop

1. apt-get source sl-modem-daemon efax-gtk
2. patch as instructed in the Debian BTS
3. pbuilder-satisfydepends, debuild, dpkg -i
4. slmodemd -c ITALY --alsa hw:0,6
5. echo ATDmymobilenumber > /dev/ttySL0 and my mobile phone rung
6. efax-gtk

Believe it or not, at this point I managed to successfully send a test fax.

Background: the laptop's modem is actually a sound card, and is ashamed to admit that it can also work as a modem:

$ lspci
00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio Controller (rev 03)

But the sound card actually has its own bus, which you can query with aplay -l:

$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: Intel [HDA Intel], device 0: ALC861VD Analog [ALC861VD Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: Intel [HDA Intel], device 6: Si3054 Modem [Si3054 Modem]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

Then you learn that sl-modem-daemon can drive it both on i386 and on amd64, but you get period size 48 is not supported by playback (64) when trying to dial. But then you find the patch to get rid of that, and it works.

The modem was the last device in the new laptop that I had not yet attempted to use. I can now claim that every single piece of hardware on my ASUS F9E-2P119E laptop can be made to work with Debian. Oh, yes!

Step 2: fill in the form

Much to my surprise, evince allowed me to just click in the form fields and type text. Even checkboxes worked. "Save a copy", however, did not retain the field contents: I had to print to file to get another PDF with the fields filled in. Update: this could be a limitation of that specific PDF, see this thread on the Adobe forums (thanks to Tomas Weber).

However, evince did not allow me to import an image with my signature and paste it in the right place. Inkscape, however, successfully managed to import the PDF as an editable vector drawing that I could change at will. Again, that was impressive.

From there, it was just a matter of pasting the signature in the right place, save as PostScript, give it to efax-gtk and phone the bank to learn that, in fact, the fax was received and was perfectly readable.