GNU libc 2.34 in unstable

The GNU libc version 2.34 has just been accepted into unstable. Getting it ready has been more challenging than other versions, as this version integrates a few libraries (libpthread, libdl, libutil, libanl) into libc. While this is handled transparently at runtime, there are a few corner cases at build time:

  • For backwards compatibility, empty static archives (e.g. libpthread.a) are provided, so that the linker options keep working. However a few cmake files shipped in some packages still reference the path to the shared library symlink (e.g. libpthread.so) which is now removed.

  • A few symbols have also been moved from libresolv to libc and their __ prefix removed. While compatibily symbols are provided in the shared library for runtime compatiblity, this does not work at link time for static libraries referencing this symbol.

The next challenge is to get it migrating into testing!

For the adventurous persons, GNU libc 2.35 is now available in experimental. And as people keep asking, the goal is to get the just released GNU libc 2.36 into Bookworm.

James Webb Space Telescope launched!

The long awaited James Webb Space Telescope has finally been successfully launched today. It is a Xmas gift for many people who have been waiting for it for many years.

On a more personal side, I am happy and proud to have contributed to a tiny part of a tiny piece of software of this huge project over the last 15 years: the Instrument Performance Simulator of the NIRSpec instrument.

10 years ago...

I joined the Debian GNU libc team and did my first glibc upload. At that time source-only upload were far from existing, and I was using a HP 9000 model 715/80 HPPA workstation for my Debian builds.

Still it seems to me like yesterday.

MIPS Creator CI20

I have received two MIPS Creator CI20 boards, thanks to Imagination Technologies. It's a small MIPS32 development board:

MIPS Creator CI20

As you can see it comes in a nice packaging with a world-compatible power adapter. It uses a Ingenic JZ4780 SoC with a dual core MIPS32 CPU running at 1.2GHz with a PowerVR SGX540 GPU. The board is fitted with 1GB of RAM, 8GB of NOR flash, HDMI output, USB 2.0 ports, Ethernet + Wi-Fi + BlueTooth, SD card slot, IR receiver, expansion headers and more. The schematics are available. The Linux kernel and the U-Boot bootloader sources are also available.

Powering this board with a USB keyboard, a USB mouse and a HDMI display, it boots off the internal flash on a Debian Wheezy up to the XFCE environment. Besides the kernel, the Wi-Fi + Bluetooth firmware, and very few configuration changes, it runs a vanilla Debian. Unfortunately I haven't found time to play more with it yet, but it looks already quite promising.

The board has not been formally announced yet, so I do not know when it will become available, nor the price, but if you are interested I'll bring it to DebConf14. Don't hesitate to ask me if you want to look at it or play with it.

Intel about to disable TSX instructions?

Last time I changed my desktop computer I bought a CPU from the Intel Haswell family, the one available on the market at that time. I carefully selected the CPU to make sure it supports as many instructions extensions as possible in this family (Intel likes segmentation, even high-end CPUs like the Core i7-4770k do not support all possible instructions). I ended-up choosing the Core i7-4771 as it supports the "Transactional Synchronization Extensions" (Intel TSX) instructions, which provide transactional memory support. Support for it has been recently added in the GNU libc, and has been activated in Debian. By choosing this CPU, I wanted to be sure that I can debug this support in case of bug report, like for example in bug#751147.

Recently some computing websites started to mention that the TSX instructions have bugs on Xeon E3 v3 family (and likely on Core i7-4771 as they share the same silicon and stepping), quoting this Intel document. Indeed one can read on page 49:

HSW136. Software Using Intel TSX May Result in Unpredictable System Behavior

Problem: Under a complex set of internal timing conditions and system events, software using the Intel TSX (Transactional Synchronization Extensions) instructions may result in unpredictable system behavior.
Implication: This erratum may result in unpredictable system behavior.
Workaround: It is possible for the BIOS to contain a workaround for this erratum.

And later on page 51:

Due to Erratum HSw136, TSX instructions are disabled and are only supported for software development. See your Intel representative for details.

The same websites tell that Intel is going to disable the TSX instructions via a microcode update. I hope it won't be the case and that they are going to be able to find a microcode fix. Otherwise it would mean I will have to upgrade my desktop computer earlier than expected. It's a bit expensive to upgrade it every year and that's a the reason why I skipped the Ivy Bridge generation which didn't bring a lot from the instructions point of view. Alternatively I can also skip microcode and BIOS updates, in the hope I won't need another fix from them at some point.

Debian is switching (back) to GLIBC

Five years ago Debian and most derivatives switched from the standard GNU C Library (GLIBC) to the Embedded GLIBC (EGLIBC). Debian is now about to take the reverse way switching back to GLIBC, as EGLIBC is now a dead project, the last release being the 2.19 one. At the time of writing the glibc package has been uploaded to experimental and sits in the NEW queue.

EGLIBC is dead for a good reason: the GLIBC development has changed a lot in the recent years, due to two major events: Ulrich Drepper leaving Red Hat and the GLIBC development, and the GLIBC steering committe self-dissolving. This has resulted in a much more friendly development based on team work with good cooperation. The development is now based on peer review, which results in less buggy code (humans do make mistakes). It has also resulted in things that were clearly impossible before, like using the same repository for all architectures, and even getting rid of the ports/ directory. Before we used to have two sets of architectures, the main ones in the glibc repository with architectures like x86, SuperH or SPARC, and the secondary ones in the glibc-ports repository with architectures like ARM or MIPS. As you can see the separation was quite arbitrary, and often leaded to missing changes on the secondary architectures. We also got real stable branches, with regular fixes.

The most important EGLIBC features have been merged to GLIBC, including for example the use of non-bash but POSIX shell, or the renaming of reserved keywords. The notable exception is the support for configurable components, which we originally planned to use for Debian-Installer, by building a smaller flavor using -Os and without NIS and RPC support. At the end we never worked on that, and it seems that the hardware targeted by Debian has grown faster than the GLIBC size, so that is not really a big loss. At the end, we ended up with only 5 missing patches from the EGLIBC tree:

The package names are unchanged (except the source package and the binary package containing the sources) so the transition is fully transparent for the users.

I would like to thank all the CodeSourcery employees who worked on EGLIBC, with a special thank to Joseph Myers who spent countless hours to merge the most important EGLIBC changes back to GLIBC, and sent regular emails about the merge status. I would also like to thanks all the people on the GLIBC side that made the change to happen, and all persons participating in the GLIBC development.

On configure systems

I will never understand the point of using autotools, cmake or whatever configure system, when later the code uses an hardcoded list of architectures to determine the size of a pointer... Unfortunately for porters this pattern is quite common.

Update: As people keep asking, the way to check for the size of a given type is explained in the autoconf manual. To check for the size of the pointer, the following entry has to be added to configure.ac:

AC_CHECK_SIZEOF(void *)

On a 64-bit system, this will lead to the following entry in config.h:

/* The size of `void *', as computed by sizeof. */ #define SIZEOF_VOID_P 8

Debian QEMU images updated

Following the release of Debian Wheezy, I have (finally) updated my set of Debian QEMU images for both Squeeze (6.0.8) and Wheezy (7.3). The following images are now available:

Each of these directories contains a GPG signed README.txt file with the md5sums all files, detailed instructions how to run these images and especially the minimum QEMU version to use.

The requirements to run the default desktop environment have increased a lot between Squeeze and Wheezy. An accelerated graphics card is now needed to be able to use Gnome (unless you use the fallback mode), which is not something provided by QEMU (actually there is now a QXL para-virtualized video card, but the driver is only in unstable). In addition GDM now needs more than 128MiB to start, while this is the default amount of memory provided for virtual machines. I have therefore decided to switch the default desktop environment on the Wheezy images to Xfce and the display manager to LightDM. Both Gnome and GDM are still installed on the images, and the original default can easily be restored using the following commands:

  • update-alternatives --auto x-session-manager
  • echo /usr/sbin/gdm3 > /etc/X11/default-display-manager

Beside this the new images only contain minor changes. The filesystems have been tweaked to not run fsck after a certain amount of days, and locales-all and openssh-server are now installed in all images. For MIPS and MIPSEL, 64-bit kernels are now also installed and provided, so that it is possible to choose between a 32-bit or a 64-bit kernel (see the README.txt for more details).

There is no Debian Wheezy SPARC image available, as QEMU does not fully support SPARC64 yet (it is actually possible to run it, but then the VM crashes often), and Debian Wheezy now only supports 64-bit kernels. I will also invest time to build an S390X image, but so far I haven't been successful on that.

The following images are still available at the same location, though they haven't been updated:

  • sparc (Etch)
  • SH4 (Sid from a few years ago)

Detecting code issues using multiple architectures

Sometimes building the same code on multiple architectures is useful to detect horrors like:

ExEnv::err0() << sprintf("AtomInfo: invalid name: %s\n",name.c_str());

This code builds using -Werror=format-security with only a few warnings onmost architectures, while GCC is correctly detects the issue on some others.

This has been reported as bug#728249.

How multiarch adds new RC bugs...

# dpkg --add-architecture kfreebsd-amd64
# dpkg -i libc0.1-dev_2.13-32_kfreebsd-amd64.deb
Selecting previously unselected package libc0.1-dev.
(Reading database ... 446113 files and directories currently installed.)
Unpacking libc0.1-dev (from libc0.1-dev_2.13-32_kfreebsd-amd64.deb) ...
dpkg: error processing libc0.1-dev_2.13-32_kfreebsd-amd64.deb (--install):
  trying to overwrite '/usr/include/_G_config.h', which is also in package libc6-dev 2.13-32
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Errors were encountered while processing:
  libc0.1-dev_2.13-32_kfreebsd-amd64.deb

Before multiarch the bug was not existing, and of course none of libc6-dev and libc0.1-dev are marked as Multi-Arch: something. People wanting to delay the release of Wheezy, I am sure you can find much more RC bugs like that.