Xcode 4.0.2 and 4.1 distributed building

For some reason, the distributed building system in Xcode 4.x seems to be broken. You can search for distcc servers in your network, define them, turn on distributed building, but still it does everything on localhost.

If you take a closer look at the build log, it says:

setenv DISTCC_HOSTS localhost

..so can’t expect much network traffic after that calling distcc. This overrides everything starting from /Developer/usr/etc/distcc/hosts and ~/.distcc/hosts. I couldn’t find any way to change this, until googling revealed a trick: making a proxy script to call distcc. This is absolutely the wrongest way to do this, but it works. Everything you do will be on your own risk and be sure to have a backup Xcode installer image.

First, create a ~/.distcc/hosts file if you don’t already have one. This is what the distcc program should look up in the first place. Fill in as follows:

myfirstserver/n,cpp,lzo
mysecondserver/n,cpp,lzo
localhost/n,cpp,lzo

where server can be a hostname or IP address, and n is the number of distcc processes you want to use. Usually the number of cores on the computer or that +1. I tend to give less processes than the maximum for localhost, as this makes using the computer you develop on smoother during builds. Save the file.

Open up a console and:

cd /Developer/usr/bin
mv distcc distcc.orig
nano distcc

Fill in the new contents for distcc:

#!/bin/sh
MY_DISTCC_HOSTS=`cat ~/.distcc/hosts`
export DISTCC_HOSTS="--randomize $MY_DISTCC_HOSTS"
/Developer/usr/bin/distcc.orig "$@"
exit $?

Save the file and exit (ctrl-o, ctrl-x).

chmod a+x distcc

Now in theory it should work. Go to Xcode and test. The build system starts to distribute the build load to your network. But still it’s slow. You see that it doesn’t start more than a couple processes at a time and waits for them to finish, and none of the computers really use their CPU to the full.

In Xcode, there’s a built-in parameter that limits the build jobs. This used to be called PBXNumberOfParallelBuildSubtasks but that has changed in Xcode 4. So, for example, for 8 jobs you need to run:

defaults write com.apple.dt.xcode IDEBuildOperationMaxNumberOfConcurrentCompileTasks 8

…where you can put the total number of wanted processes or build jobs in place of 8. Usually this should be the sum of the /n values you specified in the .distcc/hosts file.

Restart Xcode.