Intel about to disable TSX instructions?

Last time I changed my desktop computer I bought a CPU from the Intel Haswell family, the one available on the market at that time. I carefully selected the CPU to make sure it supports as many instructions extensions as possible in this family (Intel likes segmentation, even high-end CPUs like the Core i7-4770k do not support all possible instructions). I ended-up choosing the Core i7-4771 as it supports the "Transactional Synchronization Extensions" (Intel TSX) instructions, which provide transactional memory support. Support for it has been recently added in the GNU libc, and has been activated in Debian. By choosing this CPU, I wanted to be sure that I can debug this support in case of bug report, like for example in bug#751147.

Recently some computing websites started to mention that the TSX instructions have bugs on Xeon E3 v3 family (and likely on Core i7-4771 as they share the same silicon and stepping), quoting this Intel document. Indeed one can read on page 49:

HSW136. Software Using Intel TSX May Result in Unpredictable System Behavior

Problem: Under a complex set of internal timing conditions and system events, software using the Intel TSX (Transactional Synchronization Extensions) instructions may result in unpredictable system behavior.
Implication: This erratum may result in unpredictable system behavior.
Workaround: It is possible for the BIOS to contain a workaround for this erratum.

And later on page 51:

Due to Erratum HSw136, TSX instructions are disabled and are only supported for software development. See your Intel representative for details.

The same websites tell that Intel is going to disable the TSX instructions via a microcode update. I hope it won't be the case and that they are going to be able to find a microcode fix. Otherwise it would mean I will have to upgrade my desktop computer earlier than expected. It's a bit expensive to upgrade it every year and that's a the reason why I skipped the Ivy Bridge generation which didn't bring a lot from the instructions point of view. Alternatively I can also skip microcode and BIOS updates, in the hope I won't need another fix from them at some point.

Debian is switching (back) to GLIBC

Five years ago Debian and most derivatives switched from the standard GNU C Library (GLIBC) to the Embedded GLIBC (EGLIBC). Debian is now about to take the reverse way switching back to GLIBC, as EGLIBC is now a dead project, the last release being the 2.19 one. At the time of writing the glibc package has been uploaded to experimental and sits in the NEW queue.

EGLIBC is dead for a good reason: the GLIBC development has changed a lot in the recent years, due to two major events: Ulrich Drepper leaving Red Hat and the GLIBC development, and the GLIBC steering committe self-dissolving. This has resulted in a much more friendly development based on team work with good cooperation. The development is now based on peer review, which results in less buggy code (humans do make mistakes). It has also resulted in things that were clearly impossible before, like using the same repository for all architectures, and even getting rid of the ports/ directory. Before we used to have two sets of architectures, the main ones in the glibc repository with architectures like x86, SuperH or SPARC, and the secondary ones in the glibc-ports repository with architectures like ARM or MIPS. As you can see the separation was quite arbitrary, and often leaded to missing changes on the secondary architectures. We also got real stable branches, with regular fixes.

The most important EGLIBC features have been merged to GLIBC, including for example the use of non-bash but POSIX shell, or the renaming of reserved keywords. The notable exception is the support for configurable components, which we originally planned to use for Debian-Installer, by building a smaller flavor using -Os and without NIS and RPC support. At the end we never worked on that, and it seems that the hardware targeted by Debian has grown faster than the GLIBC size, so that is not really a big loss. At the end, we ended up with only 5 missing patches from the EGLIBC tree:

The package names are unchanged (except the source package and the binary package containing the sources) so the transition is fully transparent for the users.

I would like to thank all the CodeSourcery employees who worked on EGLIBC, with a special thank to Joseph Myers who spent countless hours to merge the most important EGLIBC changes back to GLIBC, and sent regular emails about the merge status. I would also like to thanks all the people on the GLIBC side that made the change to happen, and all persons participating in the GLIBC development.

On configure systems

I will never understand the point of using autotools, cmake or whatever configure system, when later the code uses an hardcoded list of architectures to determine the size of a pointer... Unfortunately for porters this pattern is quite common.

Update: As people keep asking, the way to check for the size of a given type is explained in the autoconf manual. To check for the size of the pointer, the following entry has to be added to configure.ac:

AC_CHECK_SIZEOF(void *)

On a 64-bit system, this will lead to the following entry in config.h:

/* The size of `void *', as computed by sizeof. */ #define SIZEOF_VOID_P 8

Debian QEMU images updated

Following the release of Debian Wheezy, I have (finally) updated my set of Debian QEMU images for both Squeeze (6.0.8) and Wheezy (7.3). The following images are now available:

Each of these directories contains a GPG signed README.txt file with the md5sums all files, detailed instructions how to run these images and especially the minimum QEMU version to use.

The requirements to run the default desktop environment have increased a lot between Squeeze and Wheezy. An accelerated graphics card is now needed to be able to use Gnome (unless you use the fallback mode), which is not something provided by QEMU (actually there is now a QXL para-virtualized video card, but the driver is only in unstable). In addition GDM now needs more than 128MiB to start, while this is the default amount of memory provided for virtual machines. I have therefore decided to switch the default desktop environment on the Wheezy images to Xfce and the display manager to LightDM. Both Gnome and GDM are still installed on the images, and the original default can easily be restored using the following commands:

  • update-alternatives --auto x-session-manager
  • echo /usr/sbin/gdm3 > /etc/X11/default-display-manager

Beside this the new images only contain minor changes. The filesystems have been tweaked to not run fsck after a certain amount of days, and locales-all and openssh-server are now installed in all images. For MIPS and MIPSEL, 64-bit kernels are now also installed and provided, so that it is possible to choose between a 32-bit or a 64-bit kernel (see the README.txt for more details).

There is no Debian Wheezy SPARC image available, as QEMU does not fully support SPARC64 yet (it is actually possible to run it, but then the VM crashes often), and Debian Wheezy now only supports 64-bit kernels. I will also invest time to build an S390X image, but so far I haven't been successful on that.

The following images are still available at the same location, though they haven't been updated:

  • sparc (Etch)
  • SH4 (Sid from a few years ago)

Detecting code issues using multiple architectures

Sometimes building the same code on multiple architectures is useful to detect horrors like:

ExEnv::err0() << sprintf("AtomInfo: invalid name: %s\n",name.c_str());

This code builds using -Werror=format-security with only a few warnings onmost architectures, while GCC is correctly detects the issue on some others.

This has been reported as bug#728249.

How multiarch adds new RC bugs...

# dpkg --add-architecture kfreebsd-amd64
# dpkg -i libc0.1-dev_2.13-32_kfreebsd-amd64.deb
Selecting previously unselected package libc0.1-dev.
(Reading database ... 446113 files and directories currently installed.)
Unpacking libc0.1-dev (from libc0.1-dev_2.13-32_kfreebsd-amd64.deb) ...
dpkg: error processing libc0.1-dev_2.13-32_kfreebsd-amd64.deb (--install):
  trying to overwrite '/usr/include/_G_config.h', which is also in package libc6-dev 2.13-32
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Errors were encountered while processing:
  libc0.1-dev_2.13-32_kfreebsd-amd64.deb

Before multiarch the bug was not existing, and of course none of libc6-dev and libc0.1-dev are marked as Multi-Arch: something. People wanting to delay the release of Wheezy, I am sure you can find much more RC bugs like that.

10 years ago...

Date: Mon, 18 Mar 2002 18:22:10 +0000
From: James Troup <troup@samosa.debian.org>
To: Aurelien Jarno <aurelien@aurel32.net>
Cc: da-manager@debian.org
Subject: New Debian maintainer Aurelien Jarno

[ This is a long (automatically-generated) mail, but it contains
   important information, please read it all carefully. ]

Dear Aurelien Jarno!

An account has been created for you on developer-accessible machines
with username 'aurel32'.  The password for this account can be found 
encrypted with your PGP or GPG key and appended to this message. A list 
of machines available to Debian developers can be found at
<URL:http://db.debian.org/machines.cgi>. Please take a minute now to
familiarize yourself with the Debian Machine Usage Policy, available
at <URL:http://www.debian.org/devel/dmup>

You have been subscribed to the debian-private mailing list as
<aurel32@debian.org>.  Please respect the privacy of that list and don't forward
mail from it elsewhere.  E-mail to <aurel32@debian.org> will be
forwarded to <aurelien@aurel32.net>.  To change this, please see
<URL:http://db.debian.org/forward.html> Also, please subscribe to 
debian-devel-announce, if you haven't done so already.

We strongly suggest that you use your aurel32@debian.org address for
the maintainer field in your packages, because that one will be valid
as long as you are a Debian developer, even if you change jobs, leave
university or change Internet Service providers.  If you do so, please
add that address to your PGP/GPG key(s) (using `gpg --edit-key "YOUR
USER ID"') and send it to the keyring server at keyring.debian.org
with `gpg --keyserver keyring.debian.org --send-keys "YOUR USER ID"'.

You can find more information useful to developers at
<URL:http://www.debian.org/devel/> (in particular, see the subsection
titled "Debian Developer's reference").

We suggest that you subscribe to debian-mentors@lists.debian.org.
This list is for new maintainers who seek help with initial packaging
and other developer-related issues.  Those who prefer one-on-one help
can also post to the list, and an experienced developer may volunteer
to help you.  You can get online help on IRC, too, if you join the
channel #debian-devel on irc.debian.org.  Take a look at the support
section on www.debian.org in order to find out more information.

You should have read these documents before working on your packages.

  o The Debian Social Contract
    <URL:http://www.debian.org/social_contract.html>

  o The Debian Policy Manual
    <URL:http://www.debian.org/doc/debian-policy/>

If you have some spare time and want to contribute it to Debian you
may wish to take a look at the "Work-Needing and Prospective Packages
for Debian GNU/Linux" also known as WNPP that can be found at
<URL:http://www.debian.org/devel/wnpp/>

If you plan to make a Debian package from a not yet packaged piece of
software you *must* announce your intention on the debian-devel mailing
list to make sure nobody else is working on them.

The machine ftp-master.debian.org is our main archive server.  Every 
uploaded package finds it's way there (except for Packages covered by US 
crypto laws which go to non-us.debian.org) eventually. master.debian.org is 
the home of our bug tracking system. Project web pages and CVS archives are 
hosted on klecker.debian.org (aka cvs/www.debian.org), klecker is also our
general shell server. Web pages should be placed in public_html on klecker 
and refered to by http://people.debian.org/~aurel32

You should use ssh to log into the machines instead of regular telnet
or rlogin. Our LDAP directory is able to share ssh RSA keys among machines,
please see <URL:http://db.debian.org/doc-mail.html> Otherwise when you
first login a ~/.ssh directory will be created with the appropriate
permissions. Please be aware of the security implications of using RSA
authentication and ssh agents.

Finally, please take a minute to visit <URL:http://db.debian.org/>. 
Login using the password information appended to this email, and 
update your personal information. The information is used to maintain 
your accounts on various Debian machines, and also to allow other 
developers and general users to find out more about you. Many of 
the fields are only visible to other registered Debian developers. This
is also the only way to change your password. The passwd program does not 
yet work.

Welcome to the project!

-- 
The Debian New Maintainer Team

Performances of open-source Radeon driver

I am the happy owner of a new netbook with an AMD Fusion E-450 APU, which includes a Radeon graphics card. I am using the open-source driver on it, that is a 3.2-rc7 kernel for KMS, and xserver-xorg-video-radeon package from sid. I have to say I am not really happy about the performances.

No I don't speak about the graphical performances that are pretty good (especially compared to my Intel Atom N450 based previous netbook) but about the power consumption. With this setup and with the original battery I get 2h30 of autonomy. Switching to UMS and adding some power management options in xorg.conf improves it to 2h40, but breaks suspend to ram/disk (a pity for a netbook) and switch between VT. I then tried the non-free fglrx driver, it also suffers from the suspend to ram/disk issue, in addition to crashing xorg when playing videos... On the other hand I get an impressive 3h30 of autonomy, and additionally a silent netbook (contrary to the open-source driver, the fan doesn't spin at idle).

I have tried plenty of options, ranging from adding some power management options to xorg.conf, to passing dynclks=1 to the radeon module, including setting /sys/class/drm/card0/device/power\_method to dynpm. Right now I have worked around the issue by buying a bigger battery which brings me 5h30 of autonomy, but I would really appreciate any software way to improve it with the open-source driver.

Debian s390x port (aka 31 bits is not enough)

During Debconf 11, I got access to a fast s390 machine, and I have started to work on a Debian s390x port, the 64-bit version of the s390 port. One of my goal was to help the SPARC64 port, as some of the issues are the same: both are 64-bit big-endian, don't support unaligned access and behave differently between -fpic and -fPIC.

Why such a port?

When talking about 64-bit ports, we usually hear: "4GB is enough, handling 64-bit takes more memory". This really sounds like "640K ought to be enough for anybody". The s390 port is actually 31-bit from the address point of view (one bit is reserved for address space extension from 24 to 31 bits), so each process is limited to 2GB only. Nowadays applications which need more than 2GB are not that uncommon, especially on mainframes. Actually the 2GB limit already causes some problem in Debian: in some cases it's not possible to build haskell applications or even C applications using GCC. On the other hand, we already require a 64-bit kernel on the s390 port (only the userland is 32-bit), and applications are handling more and more 64-bit or greater values (files offset, time counters, uid, etc.).

What is the status?

Bootstrapping the architecture was not really easy (as for any other new architectures), due to a huge amount of dependencies and build-dependencies loops, as explained by Wookey during Debconf11. Now that this part is mostly done, an autobuilder has been started and currently more than 65% of the packages are built. The s390x port is hosted on debian-ports.org. Unfortunately it is not yet deboostrapable, though that should happen in the next few days (only a few packages are missing).

The main issues are currently packages which fail to build from source due to linker, gcc-4.6 and curl changes, or due to the libjpeg and multiarch transitions, and thus are not directly related to s390x. If your package is in this case, it would be a good idea to fix it. Otherwise if it has a lot of reverse dependencies and the bug is opened for a while, just expect an NMU (as allowed by the 0-day NMU policy). Of course for a few packages s390x specific fixes are needed, some of them are already in the BTS.

How can you help?

The list of bugs blocking the s390x port is available through the s390x usertag, fixing these bugs (a lot of them are general FTBFS) would help a lot. Alternatively if you have access to an s390x machine you can take a look at the packages failing to build.

Update: Fixed the explanation about the 32th bit, thanks to Bastian Blank for the comment.

Debian and the ARM hype

Thanks to the versatility of the Linux kernel, Debian has always been known for supporting a large number of architectures. It has also often been criticized for that as it is said to slow down the development of Debian.

Among these architectures, the ARM one was considered dead a few years ago, and some people wanted to get rid of it. Today all major distributions now have an ARM port, one of those distributions being even based on Debian. It seems Debian was right.

Now that Android has been ported to MIPS, we may see more and more MIPS based devices. Will the same scenario happen again?