Pages about Debian.
Running apt on the FreeRunner
I've already mentioned that I'm running approx in the laptop and I configured the FreeRunner to access the laptop's cache. Here are the other customisations needed to have a decently working apt:
# cat /etc/apt/apt.conf.d/99freerunner
APT::Install-Recommends "false";
Acquire::PDiffs "false";
The rationale is that recommends would bloat a system that is supposed to be small, and pdiff requires more CPU, memory and disk space/time than it actually saves in bandwidth.
Thanks to Michael Banck and Peter Palfrader for helping me to find out how to disable pdiffs.
Posted Sun 31 Aug 2008 23:42:48 CESTHow to read the Freerunner's accelerometers
This code has been take from moko_eightball by Jakob Westhoff: it just continuously prints the value of the three accelerometers.
#include <stdio.h> #include <stdint.h> void processInputEvents(FILE* in) { int x = 0, y = 0, z = 0; while (1) { char padding[16]; uint16_t type, code; int32_t value; // Skip the timestamp fread(padding, 1, 8, in); // Read the type fread(&type, 1, 2, in); // Read the code fread(&code, 1, 2, in); // Read the value fread(&value, 1, 4, in); switch( type ) { case 0: switch( code ) { case 0: fprintf(stdout, "x%d y%d z%d\n", x, y, z); break; default: //warning( "Unknown code ( 0x%02x ) for type 0x%02x\n", code, type ); break; } break; case 2: switch ( code ) { case 0: // Update to the new value x = value; break; case 1: // Update to the new value y = value; break; case 2: // Update to the new value z = value; break; default: //warning( "Unknown code ( 0x%02x ) for type 0x%02x\n", code, type ); break; } break; default: //warning( "Unknown type ( 0x%02x ) in accelerometer input stream\n", type ); break; } } } int main() { FILE* in = fopen("/dev/input/event2", "r"); processInputEvents(in); fclose(in); return 0; }
Polysms
Here is my first software designed for the FreeRunner: polysms. It's a commandline tool: you pass it a polygen grammar name and a phone number, and it will send a SMS to that phone number using the polygen output for that grammar as the SMS text:
# polyrun manager 0012345678
And here is the code, that works on the http://www.freesmartphone.org dbus framework:
#!/usr/bin/python # (C) 2008 Enrico Zini # Most bits of this are stripped from zhone, which is: # (C) 2007 Johannes 'Josch' Schauer # (C) 2008 Michael 'Mickey' Lauer <mlauer@vanille-media.de> # (C) 2008 Jan 'Shoragan' Luebbe # (C) 2008 Daniel 'Alphaone' Willmann # (C) 2008 Openmoko, Inc. # GPLv2 or later from dbus import SystemBus, Interface from dbus.exceptions import DBusException import logging logger = logging.getLogger( __name__ ) from dbus.mainloop.glib import DBusGMainLoop DBusGMainLoop(set_as_default=True) import gobject import sys from subprocess import Popen, PIPE class Phone: def tryGetProxy( self, busname, objname ): try: return self.bus.get_object( busname, objname ) except DBusException, e: logger.warning( "could not create proxy for %s:%s" % ( busname, objname ) ) def __init__(self): try: self.bus = SystemBus() except DBusException, e: logger.error( "could not connect to dbus_object system bus: %s" % e ) return False # Phone self.gsm_device_obj = self.tryGetProxy( 'org.freesmartphone.ogsmd', '/org/freesmartphone/GSM/Device' ) if ( self.gsm_device_obj is not None ): self.gsm_device_iface = Interface(self.gsm_device_obj, 'org.freesmartphone.GSM.Device') self.gsm_sim_iface = Interface(self.gsm_device_obj, 'org.freesmartphone.GSM.SIM') self.gsm_network_iface = Interface(self.gsm_device_obj, 'org.freesmartphone.GSM.Network') self.gsm_call_iface = Interface(self.gsm_device_obj, 'org.freesmartphone.GSM.Call') self.gsm_test_iface = Interface(self.gsm_device_obj, 'org.freesmartphone.GSM.Test') # Main loop self.loop = gobject.MainLoop() def send(self, number, message): def onSent(): print "SENT" self.loop.quit() def onStore(index): print "STORED AS", index self.gsm_sim_iface.SendStoredMessage( index, reply_handler=onSent, error_handler=self.onError ) self.gsm_sim_iface.StoreMessage( number, message, reply_handler=onStore, error_handler=self.onError ) def onError(self, result): print "ERROR", result def mainloop(self): self.loop.run() if len(sys.argv) != 3: print >>sys.stderr, "Usage: %s grammarname phonenumber" sys.exit(1) message = Popen(["/usr/bin/polyrun", sys.argv[1]], stdout=PIPE).communicate()[0] number = sys.argv[2] print "Sending to %s:" % number print message phone = Phone() phone.send(number, message) phone.mainloop()
Docking the FreeRunner into the laptop
Earlier in the day, I wrote:
So yes, my laptop can now be turned into a phone charger with networking, DNS and apt cache services. I shall look into hooking that script into dbus to have it run automatically when the phone is plugged and unplugged.
It's not dbus, it's udev, and I've managed to do it.
It's an udev rule:
# cat /etc/udev/rules.d/z60_openmoko_net.rules
ACTION=="add", SUBSYSTEM=="net", ATTRS{idVendor}=="1457", ATTRS{idProduct}=="5122", RUN+="/root/bin/share-openmoko"
ACTION=="remove", SUBSYSTEM=="net", INTERFACE="usb0", RUN+="/root/bin/share-openmoko"
And a script:
# cat bin/share-openmoko
#!/bin/sh
if [ "$ACTION" == "add" ]
then
ACTION=start
fi
if [ "$ACTION" == "remove" ]
then
ACTION=stop
fi
INTERFACE=${INTERFACE:-"usb0"}
ACTION=${ACTION:-"$1"}
case "$ACTION" in
start)
logger -t openmoko "Connected, setting up network"
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE -s 192.168.0.200/29
ifconfig usb0 192.168.0.200 netmask 255.255.255.248
/etc/init.d/dnsmasq start
echo 1 > /proc/sys/net/ipv4/ip_forward
;;
stop)
logger -t openmoko "Disconnected, bringing down network"
echo 0 > /proc/sys/net/ipv4/ip_forward
/etc/init.d/dnsmasq stop
iptables -t nat -F POSTROUTING
ifconfig usb0 down
;;
esac
The script has extra cruft that makes it double as a udev script and as a init.d style script, just because I didn't feel like abandoning the init.d style interface. However, udev handles it perfectly, so there's probably no use at all for the start/stop part.
This means that the interface and all supporting services will be brought up and down when the phone is connected/disconnected, and also when the phone is suspended/resumed. Of course, more fancy things can be plugged into the script, like syncing PIM info, file systems, turning on applets in panels, mounting phone file systems and whatnot.
Posted Fri 22 Aug 2008 13:32:58 CESTUnpacking the new FreeRunner
I got myself a FreeRunner. Here are some notes from the first few days:
Available operating systems
The FreeRunner comes with the Om 2007.2 distribution: it works for basic phone things, and it has an opkg package manager that you can use to install all sort of extra software. Only issue: the SMS application was rather unstable with my SIM card.
I then tried Om 2008.8, which is a major new redesign, partly based on Qtopia. It's definitely more advanced on how it looks, with smooth animations all around, but I did not manage to get the GSM part to work, because I did not manage to get it to ask for a PIN.
I then tried plain Qtopia and I got what looks and feels like a properly working mobile phone. The only issue that it has is that I did not manage to make it suspend, so battery life is much shorter than it could be. Apparently, this should improve as kernel 2.6.26 reaches the phone. I kept Qtopia installed in the phone flash as a "stable" phone system.
And finally, Debian. Debian is based on the freesmartphone.org software stack, which is an attempt to create a UI agnostic DBUS frontend to the hardware that supports multiple applications running on top of it. It is a young project, but it can already drive a mobile phone in a useful way. It has a demo interface that is basically a showcase of what is implemented in the DBUS frontend, but does most of the basic things you need from a mobile phone, including taking and making phone calls. And then it has Debian behind, all of it. I need to buy a bigger microsd card.
Tips and tricks
Flashing things
apt-get installdfu-util- Start the Freerunner bios/bootloader, by holding down Power and then pressing AUX (standard bootloader), or by holding down AUX and then pressing Power (factory, unbrickable, read only fail safe bootloader).
dfu-util -lshows you a list of devices it can access. If you see more than once, you need to specify in alldfu-utilcommands which one you want, using-d USBID. In my case, I have to always usedfu-util -d 0x1d50:0x5119- To flash the kernel:
dfu-util -a kernel -R -D /path/to/uImage - To flash the root file system:
dfu-util -a rootfs -R -D rootfs_filename.jffs2 - To flash the bootloader:
dfu-util -a u-boot -R -D uboot_filename.bin - To flash the u-boot configuration:
dfu-util -a u-boot_env -D env.new
You can also download all of these things from the FreeRunner by
using -U instead of -D.
Networking via USB
All of the distributions I tried, by default configure the USB
as a gadget with
ethernet over usb. You can form a lan with it using the
cdc_ether module, and you will find your phone
preconfigured as 192.168.0.202 expecting to find a
gateway at 192.168.0.200.
I made myself this script to start and stop networking with the phone:
#!/bin/sh
case "$1" in
start)
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE -s 192.168.0.200/29
ifconfig usb0 192.168.0.200 netmask 255.255.255.248
/etc/init.d/dnsmasq start
echo 1 > /proc/sys/net/ipv4/ip_forward
;;
stop)
echo 0 > /proc/sys/net/ipv4/ip_forward
/etc/init.d/dnsmasq stop
iptables -t nat -F POSTROUTING
ifconfig usb0 down
;;
esac
Besides doing masquerading, it also brings up dnsmasq so that the phone can always find a DNS together with the router.
Another useful trick is to configure the phone to share the approx cache with the laptop:
deb http://192.168.0.200:9999/debian unstable main
deb http://192.168.0.200:9999/debian experimental main
deb http://pkg-fso.alioth.debian.org/debian unstable main
So yes, my laptop can now be turned into a phone charger with networking, DNS and apt cache services. I shall look into hooking that script into dbus to have it run automatically when the phone is plugged and unplugged.
Changing the ringtone in Debian
In case you don't like the default ringtone, this is how to change it:
vi /usr/share/python-support/fso-frameworkd/framework/subsystems/oeventd/receiver.py- look for
def _play( self ): - change the codec in
decoder = gst.element_factory_make, if needed. The list of available plugins is on the gstreamer website. - change the path in
filesrc.set_property /etc/init.d/fso-frameworkd restartand maybe/etc/init.d/zhone-session restart
Yes, the ringtone is currently hardcoded, but it does say it's a
prototype after all. Or, if you prefer, it's fully configurable and
the configuration can be found in
/usr/share/python-support/fso-frameworkd.
Configuring the bootloader
You can connect to the u-boot bootloader via a serial
terminal on /dev/ttyACM0. Type help and
you will find that it can do a lot of things. The OpenMoko wiki has
pages on the
bootloader itself, its
commands and the
environment.
The environment is the configuration of the bootloader, similar
somehow to /boot/grub/menu.lst. Unlike grub, you can
edit the environment from within the bootloader and then save it
using the saveenv command.
What I did:
- A boot entry for Debian, with the kernel on an ext2 partition
instead of fat:
setenv menu_3 Boot from microSD (FAT+ext2): setenv bootargs \${bootargs_base} rootfstype=ext2 root=/dev/mmcblk0p2 rootdelay=5 \${mtdparts} ro\; mmcinit\; fatload mmc 1 0x32000000 \${sd_image_name}\; bootm 0x32000000 - Stay in the bootloader until a choice has been made:
setenv bootdelay -1 - Don't power down the bootloader when idle:
setenv boot_menu_timeout 99999 - Always show the menu at boot:
setenv stop_in_menu yes saveenv
So now my phone dual boots, and I can choose if I want a more reliable phone now (Qtopia) or if I want to play with my future phone (Debian).
Configuring ssh for two host keys on the same host
Minor issue, but annoying: since both QTopia and Debian show up
on 192.168.0.202, ssh will complain about changed host keys. Here
is how to configure ssh to avoid the problem (in
~/.ssh/config):
Host debian
HostName 192.168.0.202
User root
HostKeyAlias debian
Host qtopia
HostName 192.168.0.202
User root
HostKeyAlias qtopia
Posted Fri 22 Aug 2008 10:46:43 CEST
apt-xapian-index: dynamically generated tag clouds
About apt-xapian-index, I have already posted:
- an introduction of apt-xapian-index;
- an example of how to query it;
- a way to add simple result filters to the query;
- a way to suggest keywords and tags to use to improve the query.
- a way to search for similar packages.
- a way to implement an adaptive cutoff on result quality.
- a smart way of querying tags
- how to implement search as you type
Today I'll show how to create tag clouds. Not only that, but I'll show how to implement tag clouds that change as the user types a query.
This example uses python-gtk, and has been created together with Matteo Zandi.
Generating a tag cloud out of any Xapian query is simple, and it is just a matter of presenting into a tag cloud the information that you get with the technique shown in a smart way of querying tags: you get the tags related to the query, and you lay out their names with a font size proportional to their Xapian rank.
For the presentation, we can load pretty names from the Debtags
vocabulary in /var/lib/debtags/vocabulary:
from debian_bundle import deb822 # Facet name -> Short description facets = dict() # Tag name -> Short description tags = dict() for p in deb822.Deb822.iter_paragraphs(open("/var/lib/debtags/vocabulary", "r")): if "Description" not in p: continue desc = p["Description"].split("\n", 1)[0] if "Tag" in p: tags[p["Tag"]] = desc elif "Facet" in p: facets[p["Facet"]] = desc
The query then goes on as usual, and when we get the tags from
the eset we also record their score and normalise it
between 0 and 1. I found that computing the logarithm of scores
helps to avoid having a tag cloud with a few huge tags and a lot of
tiny tiny tags:
class Filter(xapian.ExpandDecider): def __call__(self, term): return term[:2] == "XT" def format(k): if k in tags: facet = k.split("::", 1)[0] if facet in facets: return "<i>%s: %s</i>" % (facets[facet], tags[k]) else: return "<i>%s</i>" % tags[k] else: return k taglist = [] maxscore = None for res in enquire.get_eset(15, rset, Filter()): # Normalise the score in the interval [0, 1] weight = math.log(res.weight) if maxscore == None: maxscore = weight tag = res.term[2:] taglist.append( (tag, format(tag), float(weight) / maxscore) ) taglist.sort(key=lambda x:x[0])
Finally, you mark up a gtkhtml2.Document to display
in a gtkhtml2 widget:
def mark_text_up(result_list): # 0-100 score, key (facet::tag), description document = gtkhtml2.Document() document.clear() document.open_stream("text/html") document.write_stream("""<html><head> <style type="text/css"> a { text-decoration: none; color: black; } </style> </head><body>""") for tag, desc, score in result_list: document.write_stream('<a href="%s" style="font-size: %d%%">%s</a> ' % (tag, score*150, desc)) document.write_stream("</body></html>") document.close_stream() return document
That's it, try it out.
You can use the git web interface to get to the full source code and the module it uses.
Posted Tue 12 Aug 2008 11:21:18 CESTHandling concurrent update-apt-xapian-index runs
So, update-apt-xapian-index takes a bit of a long time to run. When you install the package, if the index isn't there then postinst will rebuild it in background, good.
Now, suppose you have an advanced package manager that after installing apt-xapian-index, detects that it's installed, detects that there is no database and runs an update with a nice GUI feedback. If this happened while the update is running in the background, the usual thing to do would be to abort the update; however, the package manager wouldn't be able to know when the indexing has finished and it can enable the advanced features.
Let's not do the usual thing, then:
# `progress' is the object that does progress reporting. Commandline options # can choose a silent one, a machine readable one or a user readable one. # Lock the session so that we prevent concurrent updates lockfd = os.open(XAPIANDBLOCK, os.O_RDWR | os.O_CREAT) lockpyfd = os.fdopen(lockfd) try: fcntl.lockf(lockpyfd, fcntl.LOCK_EX | fcntl.LOCK_NB) # `ServerProgress` is a proxy progress reporter that listens for # connections on a Unix socket, and broadcasts progress status updates both # to the local progress reporter and to whoever connects to the socket. progress = ServerProgress(progress) except IOError, e: if e.errno == errno.EACCES or e.errno == errno.EAGAIN: progress.notice("Another update is already running: showing its progress.") # `ClientProgress' is another proxy class that connects to the socket # forwards progress updates to whatever the user has chosen as the # local progress indicator. childProgress = ClientProgress(progress) childProgress.loop() sys.exit(0) else: raise
What happens here is that if the upload is in progress, the second update-apt-xapian-index will not be able to lock a lockfile. But instead of exiting, it connects to the other indexer via a unix socket, and shows the progress just as if it were the one doing the indexing.
So, now, suppose I run update-apt-xapian-index:
# update-apt-xapian-index
Reading Debtags database: done.
Rebuilding Xapian index... 20%
And suppose I run another one:
# update-apt-xapian-index
Another update is already running: showing its progress.
Rebuilding Xapian index... 20%
And suppose I run yet another one:
# update-apt-xapian-index --batch-mode
notice: Another update is already running: showing its progress.
begin: Rebuilding Xapian index
progress: 19/100
progress: 20/100
And suppose I run yet another one (running with -q shows nothing, but it will wait until the master indexer has finished):
# update-apt-xapian-index -q
This way, the advanced package manager (adept 3 does this) will show a progress indicator and correctly let the user know when indexing has finished... only, if an update is already running, the progress indicator will not start at 0% because the update was already in progress, and the user will think: "wow, I've been lucky!".
The server side of the socket only uses nonblocking I/O, so there's no risk of slave indexers causing the master indexer to hang.
It's been simple to implement, and brillant. It is in
apt-xapian-index 0.15, which I have just uploaded to
sid. Shame it's a bit late for lenny.
Introducing the Humongous Merged Packages File From Hell
Suppose you want to build some service that requires information about all packages in all architectures. Like, for example, the debtags web tagging interface.
If that is the case, then you may be interested in the Humongous Merged Packages File From Hell.
It contains a merge of all Packages and Source files of etch, lenny, sid and experimental, for main, contrib and non-free, in all architectures.
The merge is done according to completely arbitrary criteria of mine:
- One record per
(package, version)couple found; multiple record per(package, version)are merged together - Size and Installed-Size are the average of all values found
- MD5sum, SHA1, SHA256, Filename, Files, Directory, Checksums-Sha1, Checksums-Sha256, Binary are thrown away
- Information (such as Homepage, Vcs* fields, Uploaders) found in the Sources file is repeated for every binary package
- All other packages get as value a list of all the values found, with duplicates removed
It's there in case anyone other than me may find it useful.
Posted Thu 31 Jul 2008 21:53:54 CESTJoys of schroot
Schroot is good! Try schroot. You have LVM set up and some free extents in a vg? Try this:
# Create a new logical volume for a sid chroot lvcreate -n schroot-sid -L1G YOUR-VG-NAME mke2fs -j /dev/YOUR-VG-NAME/schroot-sid # Debootstrap sid into it mount /dev/YOUR-VG-NAME/schroot-sid /mnt debootstrap --keyring=/etc/apt/trusted.gpg sid /mnt http://ftp.uk.debian.org/debian umount /mnt
Then apt-get install schroot and put this in your
/etc/schroot/schroot.conf:
[sid] type=lvm-snapshot device=/dev/YOUR-VG-NAME/schroot-sid description=Debian sid priority=5 users=YOUR-USER-NAME root-users=YOUR-USER-NAME source-root-users=root aliases=unstable lvm-snapshot-options=--size 2G
Now, schroot -c sid will make a LVM snapshot of
/dev/YOUR-VG-NAME/schroot-sid, mount it, set it up as
a nice working system, and chroot into it running a shell. Have a
look around: resolv.conf is set up, your home directory is bind
mounted, /proc and /sys are mounted.
When you exit the chroot, the LVM snapshot disappears, and
/dev/YOUR-VG-NAME/schroot-sid is unchanged.
If you want to modify
/dev/YOUR-VG-NAME/schroot-sid, you just schroot
-c sid-source.
In my laptop, I did this:
# schroot -c sid-source (sid)# echo APT::Install-Recommends "false";' > /etc/apt/apt.conf.d/50enrico (sid)# apt-get install build-essential, pbuilder, devscripts, fakeroot (sid)# apt-get clean
At this point, to build a debian package like if I were using pbuilder, I do this:
pkgdir$ schroot -c sid (sid)pkgdir$ su -c /usr/lib/pbuilder/pbuilder-satisfydepends (sid)pkgdir$ debuild -us -uc
You can repeat this whole process with lenny and etch, for example, and if you are on an amd64 system, you can also do sid32, lenny32 and etch32.
If you don't want to redownload packages all the time, install
approx and
configure the -source chroots to use it. This also
allows you to easily switch all systems from one mirror to another
if, for example, you are traveling abroad.
If you want to do the same as pbuilder update, you
can do this:
# schroot -c sid-source (sid)# apt-get update (sid)# apt-get dist-upgrade (sid)# apt-get autoremove (sid)# apt-get clean
You can use schroot -c sid-source for all sorts of
permanent maintenance works.
You can also create persistent named snapshots:
$ schroot -b -c sid -n playground
This will only disappear when you do:
$ schroot -c playground -e
And you can enter the schroot as many time as you want with:
$ schroot -r -c playground
You can also enter a temporary snapshot as many times as you
want, by using the random name that schroot generated for it (you
can see it, for example, when running df or
mount).
You fear that lvm snapshots are a little fragile and a little slow at the moment? It's not a problem: your home is bind-mounted inside the chroot, so you can do all the work in your normal home filesystem, without stressing the snapshot.
This is good not only to build packages. For example, now that
svk is broken in sid
and lenny, I just did schroot -b -c etch -n svk,
installed svk on it, and then every time I need svk, I can do
schroot -r -c svk and have it ready.
Added links: * using sbuild as a Debian maintainer * a similar article about schroot
Posted Tue 17 Jun 2008 17:29:09 CESTSend a fax from the laptop
My bank sent me a PDF form via e-mail. I needed to fill it in, then send it back via fax. Send it back via e-mail would not work because it's not secure. The bank agrees that this is fantastically silly, but apparently this requirement is not their fault.
Step 1: send a fax with the laptop
apt-get source sl-modem-daemon efax-gtk- patch as instructed in the Debian BTS
pbuilder-satisfydepends,debuild,dpkg -islmodemd -c ITALY --alsa hw:0,6echo ATDmymobilenumber > /dev/ttySL0and my mobile phone rungefax-gtk
Believe it or not, at this point I managed to successfully send a test fax.
Background: the laptop's modem is actually a sound card, and is ashamed to admit that it can also work as a modem:
$ lspci
00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio Controller (rev 03)
But the sound card actually has its own bus, which you can query
with aplay -l:
$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: Intel [HDA Intel], device 0: ALC861VD Analog [ALC861VD Analog]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: Intel [HDA Intel], device 6: Si3054 Modem [Si3054 Modem]
Subdevices: 0/1
Subdevice #0: subdevice #0
Then you learn that sl-modem-daemon can drive it
both on i386 and on amd64, but you get period size 48 is not
supported by playback (64) when trying to dial. But then you
find the patch to get rid of that, and it works.
The modem was the last device in the new laptop that I had not
yet attempted to use. I can now claim that every single
piece of hardware on my ASUS F9E-2P119E laptop can be
made to work with Debian. Oh, yes!
Step 2: fill in the form
Much to my surprise, evince allowed me to just
click in the form fields and type text. Even checkboxes worked.
"Save a copy", however, did not retain the field contents: I had to
print to file to get another PDF with the fields filled in.
Update: this could be a limitation of that specific PDF,
see this thread
on the Adobe forums (thanks to Tomas Weber).
However, evince did not allow me to import an image with my
signature and paste it in the right place. Inkscape,
however, successfully managed to import the PDF as an editable
vector drawing that I could change at will. Again, that was
impressive.
From there, it was just a matter of pasting the signature in the
right place, save as PostScript, give it to efax-gtk
and phone the bank to learn that, in fact, the fax was received and
was perfectly readable.
