gcc119 maintainance


The Compile Farm AIX 7.2 system (gcc119) will be offline and unavailable on Monday, May 18, to transfer the system to a different partition.  All user files should be available on the new partition when the transfer is complete and the system is restarted.  The system will be transferred back to the original partition at a later time.

We have a new machine in the compile farm: gcc203.  It is a POWER8 system in big-endian mode running Linux, that is, powerpc64-linux.

It runs Debian; it is a 8-cpu partition of a bigger machine (an S822).
That means you shouldn't normally use more than -j32 (and always much less for non-interactive runs, of course).

The Power ISA machines we now have are:

Name Arch OS CPU cores CPU threads RAM Disk
gcc110 POWER7 BE Linux 16 cores 64 threads 64 GB 1.8 TB
gcc203 POWER8 BE Linux 8 cores 64 threads 64 GB 1.0 TB
gcc112 POWER8 LE Linux 20 cores 160 threads 256 GB 1.8 TB
gcc135 POWER9 LE Linux 32 cores 128 threads 256 GB 24.9 TB
gcc111 POWER7 BE AIX 12 cores 48 threads 128 GB  
gcc119 POWER8 BE AIX 16 cores 128 threads 160 GB  


So, clearly, as a workhorse machine you should use gcc135!

The list of machines currently has some of the CPU/core numbers wrong for gcc203; we're working on it, various programs do not know how to handle virtualised systems well.

Enjoy, and please be mindful of others!

Following a 4-months outage period, gcc20 has been brought back online thanks to our host Inria.

We took this opportunity to perform a much-needed upgrade from wheezy to stretch.

User data has been kept through the upgrade, but please use this opportunity to clean up your old data! Currently, 729 GB of disk space is used out of a 825 GB capacity.

Two farm servers, gcc75 and gcc76, are going to be shut down and moved on May 6th. If you have data you care about on these machines, please fetch it now! The servers may be hosted somewhere else in the future, but there is no ETA.

Many thanks to INSA Rouen for the 7 years of hosting!

New OpenBSD machine


We are pleased to announce that a new machine is available in the farm: gcc220, a physical server running OpenBSD. With 8 cores and 8 GB of RAM, it is more powerful than the existing OpenBSD VM, gcc300.

Many thanks to openbsd.amsterdam for providing both the machine and the hosting!

The new machine can be reached over SSH at gcc220.fsffrance.org and is located in Amsterdam.

In November 2018, gcc118 suffered from a filesystem issue after upgrading the kernel.

Thanks to the excellent support from OSUOSL students, the machine has been reinstalled with a more recent OpenSUSE version (15.0) and is available again for farm usage! Unfortunately, we couldn't recover any user data.

The SSH key of the server has changed. You will need to clear the old key from your known_hosts file:

ssh-keygen -R gcc118.fsffrance.org

ssh-keygen -R

We will look at a way to backup the server SSH keys on all machines to avoid this kind of inconvenience in the future.

Live resource usage


We have been monitoring most farm machines with munin for some time. This allows anyone to check whether a machine is heavily loaded or not before starting to use it, but it is time-consuming to check many graphs from several machines.

To quickly see the load of a machine, we now display a usage bar directly in the list of machines! We currently show 3 metrics: CPU usage, memory usage, and disk I/O load. The values are based on the last 48 hours of munin data and should give a good overview of current usage and expected performance.

Nethertheless, these values are indicative and only reflect the average usage. If you plan to use a machine for heavy tasks, you should check the munin graphs to better understand the usage pattern and make sure you don't disrupt the tasks of existing users. In addition, a few values are missing or incorrect: this happens either because the machine was not recently monitored by munin, or because the total number of CPU is incorrectly detected.

The value displayed in the usage bars is computed as a weighted average that gives more importance to high usage values. For instance:

  • a machine that is 20% busy for 100% of the time will get a weighted average of 20%
  • a machine that is 50% busy for 40% of the time will get a weighted average of 43.6%
  • a machine that is 100% busy for 20% of the time will get a weighted average of 63.8%

In each case, the arithmetic average would be the same (20% usage). However, we consider that the machine is more loaded in the third case.  Intuitively, if you would like to run a new task, using a machine that is already 100% busy (even only 20% of the time) is generally a bad idea: the new task might significantly interfere with the existing ones.  In contrast, a machine that is 20% busy still has a lot of room for more tasks.

We are happy to announce that a new IBM server has been added to the farm, thanks to OSUOSL!

This POWER9 server extends the already available IBM hardware in the compile farm: POWER7 (gcc110, gcc111) and POWER8 (gcc112, gcc119). It features 128 CPU cores and 256 GB of RAM, as well as a high-performance I/O setup with 8 disks in a RAID5 configuration. Disk I/O was the main bottleneck for the existing POWER machines, so hopefully this new machine will provide much better performance in practice.

It is already possible to connect to this new machine through SSH at gcc135.fsffrance.org.

gcc123 recently suffered from a disk failure, just a few months after being operational. Today, OSUOSL changed the disk and reinstalled the machine. Because there is no RAID on this machine, any user data stored on the machine was lost.

This is a good opportunity to remind that the compile farm is "best effort": we make no promise whatsover about data integrity, so make sure to always have a copy of your important work somewhere else!

The SSH host keys of gcc123 have been restored, so SSH access should work just fine with the reinstalled system.

gcc12 just reached 3000 days of uptime, which amounts to a little more than 8 years!

Here is a bit of history on this machine hosted by FSF France:

2007-06-24 gcc11/12/13 installed
2007-07-22 gcc11/12/13 moved to datacenter, gcc01..09 stopped, gcc08 online at a temporary location
2007-11-25 gcc11/12/13 moved to new datacenter, downtime 1700 UTC to 2000 UTC
2008-05-18 gcc11 and gcc12 moved to new datacenter (same IP).
2009-08-14 planned downtime for gcc11/gcc12 at the end of august
2009-08-31 gcc12 is down, please use gcc13 until gcc12 is restored
2009-09-21 gcc11/12 are up in their new FSF France datacenter in Rennes

Since then, the machine (and its hosting facility) has been extremely stable. Of course, since it is still running Debian 5 "lenny" and has limited hardware resources for today's standards, its usage is quite low nowadays.

There were even older machines in the GCC compile farm, see the history of the project.