There has been a few flam\^Wdiscussions about emulated build daemons, each time coming to the conclusion that we should not upload packages built on an emulated machine to the archive.

However Debian has started to use at least one paravirtualized (Xen) build daemon, the i386 experimental one. The result is that one of the tests of the GNU libc testsuite is failing. On the other hand, the GNU libc and the GCC testsuites are giving the same results on a QEMU emulated machine and a real machine, for amd64, arm, armel, i386, mips, mipsel and powerpc. Same for KVM on amd64 and i386.

I wonder if we made the right choice ...

For a few weeks Laurent Vivier, Blue Swirl and myself have been working on getting QEMU PowerPC working correctly with recent distributions.

QEMU used to rely on OpenHackWare for the OpenFirmware implementation on PowerPC. It is a very limited implementation (for example it as no Forth support), which is unable to boot most 2.6.x kernels with the OldWorld emulation. It is able to boot recent kernels with the PReP emulation, but things like the PCI bus emulation are not working correctly. Moreover the PReP kernels are gone with the removal of the arch/ppc tree.

OpenBIOS was already used for the OpenFirmware implementation of Sparc 32 and Sparc 64 targets. It now supports PowerPC for the OldWorld emulation. As a result it is now possible to use Debian PowerPC under QEMU emulating an OldWorld machine.

What works?

• Display (partly), keyboard, hard disk, network;
• Booting from CD-ROM or from disk using Quik;
• Installation of Debian Etch or Lenny (but due to a bug in debian-installer quik.conf has to be fixed manually);
• Standard Debian kernels;
• G3 CPU emulation: the testsuite results of the GNU libc 2.7 and of GCC 4.3 (Debian packages) are the same than on a real machine.
• virtio devices

What doesn't work / has to be done?

• A few devices part of the MacIO chipset are not emulated and/or are replaced by other devices: IDE, SCSI, Ethernet and sound;
• The red and green colors are reversed, in some modes only (in debian-installer for example);
• X only outputs some strange images;
• PCI devices using I/O ports don't work (like the RTL8029 card, or the RTL8139 card with the 8139too driver).

For those who want to test, an Etch image is available. You will need to compile QEMU by manually given that the version in Debian is too old and that openbios-ppc is still in the NEW queue.

As I am one of those who have read "able" instead of "unable", I had to find a really cheap flight. This may sound crazy, but an Iberia flight to Buenos Aires from Berlin costs far less than from Lyon (where I live) or even from Madrid: 781.12 EUR from Berlin instead over 1,200.00 EUR from Madrid and over 1,300.00 EUR from Lyon (in both cases with a change in Madrid Barajas).

I wonder if Iberia pays you 420,00 EUR if you flight from Berlin to Madrid...

Now I have to find a cheap way to go to Berlin from Lyon (probably an EasyJet flight). I plan to spend a few days of holiday in Berlin before flying to Argentina.

We now have a second GNU/kFreeBSD build daemon building the kfreebsd-amd64 architecture. It has been kindly offered and is hosted in UK by The Positive Internet Company Ltd.

It adds redundancy to our buildd network, which is important given that the other kfreebsd-amd64 build daemon machine is on a simple ADSL line, and so less reliable than a machine in a datacenter.

For those who want to know more, here is the current status of the GNU/kFreeBSD buildd network:

• callisto.debian.net (kfreebsd-amd64)
  2 x Opteron 1.4GHZ, 1GB RAM
  Hosted by The Positive Internet Company
• shockley.aurel32.net (kfreebsd-amd64)
  Sempron 2600+, 1GB RAM
  Hosted by Aurelien Jarno (ADSL)
• hagrid.debian.net (kfreebsd-i386)
  PIII 800 MHz, 512MB RAM
  Hosted by ETH Zürich
• maxwell.aurel32.net (kfreebsd-i386)
  Sempron 2600+, 1GB RAM
  Hosted by Aurelien Jarno (ADSL)

My goal is to eventually get rid of all build daemons that I am hosting at home on my ADSL line, though that is less critical now.

Christian, this is the percentage of your blog entries concerning translation ratios in the last two weeks.

I am sure you can do better ;-)

I am really upset by the way the ARM build daemons are managed. The packages are not uploaded regularly, with sometimes three days between two uploads. Well it should not be a problem, if packages that have failed to build due to some packages not uploaded fast enough (see for example python-gnome, or rkward) were requeued, but that's actually not the case. Also last week, the build daemon named "cats" stopped to upload packages, despite it continued to build them. 11 days later, nothing has changed even after a mail to arm@buildd.debian.org. And I do not speak about build daemons building nothing.

All of that resulted in ARM being the slowest architecture to build packages. It looks like the ARM build daemon maintainer does not know the excellent page made by Jeroen.

As arm@buildd.debian.org is everything but responsive (well if you can assign a level of responsiveness to /dev/null), I have decided to act. I have installed QEMU on an 8-way Opteron machine, and created 8 emulated ARM machines, which 256MB of RAM and 10GB of disk for each, all running buildd + sbuild. Altogether those 8 emulated ARM machines should be faster than all the Debian ARM build daemons. I have setup a wanna-build database on my server. During the day it has built around 100 packages.

Yes I agree that real machines would be better, but I don't have a stack of fast ARM machines at home. Anyway when you see random segfaults on the build daemons, or strange failures (that happen for weeks) on the build daemon named "netwinder", you may think to that again.

Unfortunately the machine I use for that is only available for a few months, and I will have to stop the emulated machine then. I just hope that the situation with respect to the build daemons will improve, so that I can stop them even before.

The NEW queue is almost empty with 7 packages only. Thanks a lot Joerg.

In addition to the ARM platform, it is now possible to emulate a MIPS (big endian) or MIPSEL (little endian) machine running Debian under QEMU. It could be used as a cheap MIPS/MIPSEL development machine to test your packages, provided that you have a not to slow i386, AMD64 or PowerPC computer.

Running on an Athlon 64 X2 3800+ the emulated system is around 10% faster than my SGI Indy R4400SC 200MHz and possibly with much more RAM (my emulated system has 512 MB of RAM). Also for the ones who know about the Indy SCSI controller, the transfer rate is around 13 MB/s on the emulated system.

I have written a small howto explaining how to install Debian Etch on such an emulated MIPS or MIPSEL machine, using Debian-Installer RC1, which has just been released.

Thanks to Daniel Jacobowitz who has recently improved QEMU/MIPS a lot, and to Thiemo Seufer who has started the integration of the MIPS QEMU platform in Debian-Installer and in the Debian kernel.

Note 1: I have written this howto very quickly, so there are probably some mistakes. Comments are welcome.
Note 2: I have updated the ARM howto to take in account improvements from Debian-Installer RC1

Martin, I can't let you say that experimental is autobuilt. I have uploaded a few versions of the glibc to experimental in the last few months and only arm, hppa and alpha have tried to build them. Before the summer mips and mipsel also have tried to build the experimental glibc. And the last version has not been tried on alpha, despite the fact it includes a fix and should now build correctly on this architecture.

Hopefully I have most Debian architectures at home, but alpha, m68k and ia64, so I can build the glibc on my own machines. For m68k and ia64, I can use debian.org machines. Concerning alpha, the debian porting machine, ie escher.debian.org, is locked down since the compromise about 4 months ago. And no answer to my mail to debian-admin... Hopefully a user kindly gave me an access on his machine, thanks. The glibc 2.5 is now building correctly on this machine, but I can't upload the package because I can't trust the machine. That really sucks.

What about removing alpha from the release architectures? After all, it fails to comply with the etch candidate release architecture criteria for months.

Update: Some build daemons had glibc in "weak-no-autobuild". Thanks Martin for fixing that.

You want a cheap ARM development machine to test your packages? You can actually have one for free (both as in beer and as in speach :)) provided that you have a not to slow i386, AMD64 or PowerPC computer. The solution is called QEMU, a generic and open source processor emulator which can emulate i386, x86_64, ARM, MIPS, PowerPC and Sparc systems.

Running an Athlon 64 3800+ the emulated system is around 20% faster than the popular NSLU2 and possibly with much more RAM (my emulated system has 256 MB of RAM).

I have written a small howto explaining how to install Debian Etch on such an emulated ARM machine.

Note: I have written this howto very quickly, so there are probably some mistakes. Comments are welcome.