Category: Geek

Thought of the day

New features usually come with new versions. Before reporting a bug for a new feature, it may be a good idea to make sure you are using the latest version. apt-get can be really useful for that.

Learning IA-64 assembly

While testing EGLIBC 2.10.1 on all Debian architectures, I have discovered that the testsuite on IA-64 fails for the new POSIX 2008 math tests. I have reported the problem both upstream and on debian-ia64@lists.debian.org, but without success.

IA-64 being one of the last architecture (with HPPA) on which EGLIBC 2.10.1 fails to build from sources, I have decided to spend my day fixing the problem and digging into the corresponding IA-64 assembly code. The mathematical functions on this architecture are based on the Highly Optimized Mathematical Functions for the Intel® Itanium™ Architecture. Using the Intel® Itanium™ Architecture Software Developer’s Manual and after a lot of tries, I have been able to add the missing code paths needed for POSIX 2008 compliance, and also fixed a few bugs on the stack frame allocation (arguments of the alloc instruction).

The resulting patch is now in the pkg-glibc SVN and in the upstream bugzilla.

And more important, I have learned the basic about IA-64 assembly!

New GPG key
pub   4096R/1DDD8C9B 2009-05-09
      Key fingerprint = 7746 2642 A9EF 94FD 0F77  196D BA9C 7806 1DDD 8C9B
uid                  Aurelien Jarno <aurel32@debian.org>
uid                  Aurelien Jarno <aurelien@aurel32.net>
uid                  Aurelien Jarno <aurelien@jarno.fr>
sub   4096R/C3FCA1A8 2009-05-09

I’ll get it signed by other Debian Developers tomorrow, during the Debian France meeting.

Debian is switching to EGLIBC

I have just uploaded Embedded GLIBC (EGLIBC) into the archive (it is currently waiting in the NEW queue), which will soon replace the GNU C Library (GLIBC).

The EGLIBC is a variant of the GLIBC which stays source and binary compatible with the original GLIBC. While primarily targeted for embedded architectures, it has some really nice points:

We do not use some of these features yet, but this upload is a first step. From the user point of view, the package names are unchanged (except the source package and the binary package containing the sources) so no transition is needed.

Debian QEMU images updated

Following the release of Debian Lenny, I have updated my set of Debian QEMU images. The following images are now available:

There is no Debian Lenny SPARC image available, as QEMU does not fully support SPARC64 yet, and Debian Lenny now only supports 64-bit kernels.

Note also that the README.txt files (which among other things contain the md5sums of the images) is now GPG signed. Read carefully these files as they contain details on how to use the images, and especially the minimum QEMU version to use.

Re: emulated buildds

Wouter, I really doubt that the decision of having a Xen build daemon has been taken in a team, and the fact is that it’s causing problems.

The only goal of my post is to show we have double standards.

Emulated versus paravirtualized build daemons

There has been a few flam^Wdiscussions about emulated build daemons, each time coming to the conclusion that we should not upload packages built on an emulated machine to the archive.

However Debian has started to use at least one paravirtualized (Xen) build daemon, the i386 experimental one. The result is that one of the tests of the GNU libc testsuite is failing. On the other hand, the GNU libc and the GCC testsuites are giving the same results on a QEMU emulated machine and a real machine, for amd64, arm, armel, i386, mips, mipsel and powerpc. Same for KVM on amd64 and i386.

I wonder if we made the right choice

QEMU PowerPC

For a few weeks Laurent Vivier, Blue Swirl and myself have been working on getting QEMU PowerPC working correctly with recent distributions.

QEMU used to rely on OpenHackWare for the OpenFirmware implementation on PowerPC. It is a very limited implementation (for example it as no Forth support), which is unable to boot most 2.6.x kernels with the OldWorld emulation. It is able to boot recent kernels with the PReP emulation, but things like the PCI bus emulation are not working correctly. Moreover the PReP kernels are gone with the removal of the arch/ppc tree.

OpenBIOS was already used for the OpenFirmware implementation of Sparc 32 and Sparc 64 targets. It now supports PowerPC for the OldWorld emulation. As a result it is now possible to use Debian PowerPC under QEMU emulating an OldWorld machine.

What works?

  • Display (partly), keyboard, hard disk, network;
  • Booting from CD-ROM or from disk using Quik;
  • Installation of Debian Etch or Lenny (but due to a bug in debian-installer quik.conf has to be fixed manually);
  • Standard Debian kernels;
  • G3 CPU emulation: the testsuite results of the GNU libc 2.7 and of GCC 4.3 (Debian packages) are the same than on a real machine.
  • virtio devices

What doesn’t work / has to be done?

  • A few devices part of the MacIO chipset are not emulated and/or are replaced by other devices: IDE, SCSI, Ethernet and sound;
  • The red and green colors are reversed, in some modes only (in debian-installer for example);
  • X only outputs some strange images;
  • PCI devices using I/O ports don’t work (like the RTL8029 card, or the RTL8139 card with the 8139too driver).

For those who want to test, an Etch image is available. You will need to compile QEMU by manually given that the version in Debian is too old and that openbios-ppc is still in the NEW queue.

IPsec, MTU & NAT

Dear lazyweb, I encounter MTU problems with an IPsec setup and NAT.

Here is a simplified version of my setup:

remote host < --- internet ---> (eth0) gateway (eth1) < --- LAN

As you may have guess, the gateway has only one public IP address and thus the hosts on the LAN are connected to the internet through NAT.

The connection between the remote host and the gateway is secured using IPsec (kame tools), and this works as expected as long, as the connection is done between the remote host and the gateway. The problems arise when I try to make a connection between the remote host and one host from the LAN. Due to the use of IPsec, the MTU is reduced by 44 bytes, however “ICMP need to frag” packets are not emitted by the gateway, so the connection just hangs.

I have tried various solution from the web (setting MTU on the various interfaces, clamping MSS with iptables, defining advmss with ip route, etc.), and the only one which actually works is reducing the MTU on the LAN hosts. Not very useable given that they are a lot of hosts on the LAN.

Note that when IPsec is disabled, if I lower the MTU of eth0, the “ICMP need to frag” packets are correctly emitted, and the connection just works.

Suggestions?

MAC address strangeness

Today I upgraded my BIOS in the hope to solve various issues. When I rebooted the machine, it didn’t get an IP address through DHCP: the Ethernet MAC address has been changed by the BIOS upgrade.

Now compare the old and the new Ethernet MAC addresses:

old address: 0:1a:4d:60:72:e0
new address: e0:72:60:4d:1a:0

Time to laugh…

ARM code of the day


ldmeqib r9!, {r1, r8, ip}^
ldclsl 3, cr14, [r4, #-364]!
stmleda r1, {r0, r2, r6, r7, r9, sl, ip, lr}^
cmppl r6, #12582912

This program does not work, but it still has a meaning.

Hint: Each instruction is 32-bit long, the total length is 128 bits.

New ARM autobuilders

I am really upset by the way the ARM build daemons are managed. The packages are not uploaded regularly, with sometimes three days between two uploads. Well it should not be a problem, if packages that have failed to build due to some packages not uploaded fast enough (see for example python-gnome, or rkward) were requeued, but that’s actually not the case. Also last week, the build daemon named “cats” stopped to upload packages, despite it continued to build them. 11 days later, nothing has changed even after a mail to arm@buildd.debian.org. And I do not speak about build daemons building nothing.

All of that resulted in ARM being the slowest architecture to build packages. It looks like the ARM build daemon maintainer does not know the excellent page made by Jeroen.

As arm@buildd.debian.org is everything but responsive (well if you can assign a level of responsiveness to /dev/null), I have decided to act. I have installed QEMU on an 8-way Opteron machine, and created 8 emulated ARM machines, which 256MB of RAM and 10GB of disk for each, all running buildd + sbuild. Altogether those 8 emulated ARM machines should be faster than all the Debian ARM build daemons. I have setup a wanna-build database on my server. During the day it has built around 100 packages.

Yes I agree that real machines would be better, but I don’t have a stack of fast ARM machines at home. Anyway when you see random segfaults on the build daemons, or strange failures (that happen for weeks) on the build daemon named “netwinder“, you may think to that again.

Unfortunately the machine I use for that is only available for a few months, and I will have to stop the emulated machine then. I just hope that the situation with respect to the build daemons will improve, so that I can stop them even before.

Cheap MIPS/MIPSEL development machine

In addition to the ARM platform, it is now possible to emulate a MIPS (big endian) or MIPSEL (little endian) machine running Debian under QEMU. It could be used as a cheap MIPS/MIPSEL development machine to test your packages, provided that you have a not to slow i386, AMD64 or PowerPC computer.

Running on an Athlon 64 X2 3800+ the emulated system is around 10% faster than my SGI Indy R4400SC 200MHz and possibly with much more RAM (my emulated system has 512 MB of RAM). Also for the ones who know about the Indy SCSI controller, the transfer rate is around 13 MB/s on the emulated system.

I have written a small howto explaining how to install Debian Etch on such an emulated MIPS or MIPSEL machine, using Debian-Installer RC1, which has just been released.

Thanks to Daniel Jacobowitz who has recently improved QEMU/MIPS a lot, and to Thiemo Seufer who has started the integration of the MIPS QEMU platform in Debian-Installer and in the Debian kernel.

Note 1: I have written this howto very quickly, so there are probably some mistakes. Comments are welcome.
Note 2: I have updated the ARM howto to take in account improvements from Debian-Installer RC1