For a while now I have been reporting bugs that I find in the iPhone SDK / iPhone OS to Apple because I realised that it’d be nice to help out. Some bugs have been small and some have been large, ranging from minor crashes of MobileSafari up to full blown problems in the iPhone SDK and associated frameworks.

One that I came across today had stumped me for a long time and it has to do with the GCC atomic builtins. If you’re unfamiliar with them, then a good bit of introductory reading is a great blog post on the ARM blog. Now, these atomic builtins have not been defined within the iPhone’s libc implementation, until the 3.2 SDK came along – it appears that Apple have added them. This is a good thing, because it means that we can start using them in our applications. But, we can only use them for applications running on iPhone OS >=3.2 of course. That’s where the fun begins…

I have an application which I have been developing that needs to run on both the 3G and the 3GS, i.e. both armv6 and armv7. I found that after upgrading to the 3.2 SDK I started running into a rather strange problem when running the application on an iPhone 3G (running iPhone OS 3.1.3). The error I was getting was this:

dyld: lazy symbol binding failed: Symbol not found: ___sync_fetch_and_add_4
  Referenced from: /var/mobile/Applications/xxx/MyApp.app/MyApp
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: ___sync_fetch_and_add_4
  Referenced from: /var/mobile/Applications/xxx/MyApp.app/MyApp
  Expected in: /usr/lib/libSystem.B.dylib

Now that’s really odd because __sync_fetch_and_add_4 is one of those GCC atomics which shouldn’t be being linked in as I am building for an iPhone OS deployment target of 3.1. It’s worth at this stage having a quick look at – http://developer.apple.com/iphone/library/documentation/Xcode/Conceptual/iphone_development/120-Running_Applications/running_applications.html – which says:

You specify the earliest iPhone OS release on which you want your application to run with the iPhone OS Deployment Target build setting. By default, this build setting is set to the iPhone OS release that corresponds to the Base SDK build-setting value. For example, when you set Base SDK to iPhone Device 2.2.1 or iPhone Simulator 2.2.1, the value of the iPhone Deployment Target build setting is iPhone OS 2.2.1, as shown in Figure 3-3.

So that means that if I set the base SDK to 3.2 and the iPhone OS deployment target to 3.1, then I should get code that will definitely run on 3.1, right? Running my app on 3.1.3 however causes a crash simply because __sync_fetch_and_add_4 isn’t available in its libc.

After a bit of inspection I found that even this simple program caused the crash:

#include <string>

int main(int argc, char *argv[]) {
    std::string strA = "yes";
    return 0;
}

That really is a very simple program! Why would that crash! Well, with a bit of inspection using ‘nm‘ we can work out what’s going on. ‘nm’ shows us a list of the symbols that a given object file, or binary references. Below are outputs of ‘nm’ on the resulting binary from 2 different combinations of base SDK and iPhone OS deployment target.

Base SDK = 3.1.3, iPhone OS Deployment Target = 3.1:

nm build/Debug-iphoneos/AtomicsBug.app/AtomicsBug

build/Debug-iphoneos/AtomicsBug.app/AtomicsBug (for architecture armv6):
00002f64 s  stub helpers
00002fdc s GCC_except_table0
00003048 D _NXArgc
0000304c D _NXArgv
         U __Unwind_SjLj_Register
         U __Unwind_SjLj_Resume
         U __Unwind_SjLj_Unregister
         U __ZN9__gnu_cxx18__exchange_and_addEPVii
         U __ZNSs4_Rep10_M_destroyERKSaIcE
         U __ZNSs4_Rep20_S_empty_rep_storageE
         U __ZNSsC1EPKcRKSaIcE
         U ___gxx_personality_sj0
00003054 D ___progname
00002f58 t ___restore_vfp_d8_d15_regs
00002f50 t ___save_vfp_d8_d15_regs
00001000 A __mh_execute_header
00003050 D _environ
         U _exit
00002e9c t _main
0000301c s _pvars
         U dyld_stub_binder
00002e70 T start

build/Debug-iphoneos/AtomicsBug.app/AtomicsBug (for architecture armv7):
00002f64 s  stub helpers
00002fdc s GCC_except_table0
00003048 D _NXArgc
0000304c D _NXArgv
         U __Unwind_SjLj_Register
         U __Unwind_SjLj_Resume
         U __Unwind_SjLj_Unregister
         U __ZN9__gnu_cxx18__exchange_and_addEPVii
         U __ZNSs4_Rep10_M_destroyERKSaIcE
         U __ZNSs4_Rep20_S_empty_rep_storageE
         U __ZNSsC1EPKcRKSaIcE
         U ___gxx_personality_sj0
00003054 D ___progname
00001000 A __mh_execute_header
00003050 D _environ
         U _exit
00002eb0 t _main
0000301c s _pvars
         U dyld_stub_binder
00002e84 T start

Base SDK = 3.2, iPhone OS Deployment Target = 3.1:

nm build/Debug-iphoneos/AtomicsBug.app/AtomicsBug 

build/Debug-iphoneos/AtomicsBug.app/AtomicsBug (for architecture armv6):
00002f64 s  stub helpers
00002fdc s GCC_except_table0
00003048 D _NXArgc
0000304c D _NXArgv
         U __Unwind_SjLj_Register
         U __Unwind_SjLj_Resume
         U __Unwind_SjLj_Unregister
         U __ZNSs4_Rep10_M_destroyERKSaIcE
         U __ZNSs4_Rep20_S_empty_rep_storageE
         U __ZNSsC1EPKcRKSaIcE
         U ___gxx_personality_sj0
00003054 D ___progname
00002f58 t ___restore_vfp_d8_d15_regs
00002f50 t ___save_vfp_d8_d15_regs
         U ___sync_fetch_and_add_4
00001000 A __mh_execute_header
00003050 D _environ
         U _exit
00002ea4 t _main
0000301c s _pvars
         U dyld_stub_binder
00002e78 T start

build/Debug-iphoneos/AtomicsBug.app/AtomicsBug (for architecture armv7):
00002f74 s  stub helpers
00002fe0 s GCC_except_table0
00003044 D _NXArgc
00003048 D _NXArgv
         U __Unwind_SjLj_Register
         U __Unwind_SjLj_Resume
         U __Unwind_SjLj_Unregister
         U __ZNSs4_Rep10_M_destroyERKSaIcE
         U __ZNSs4_Rep20_S_empty_rep_storageE
         U __ZNSsC1EPKcRKSaIcE
         U ___gxx_personality_sj0
00003050 D ___progname
00001000 A __mh_execute_header
0000304c D _environ
         U _exit
00002ea8 t _main
00003018 s _pvars
         U dyld_stub_binder
00002e7c T start

Notice how it’s referencing __sync_fetch_and_add_4 in the armv6 version of the binary created with base SDK 3.2? That’s bad! After a bit of digging into the header files supplied with the SDKs we find where the problem stems from – it’s in the c++config.h header file. Here is the difference between the file supplied with 3.1.3 SDK to the file supplied with 3.2 SDK:

$ diff -u /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.1.3.sdk/usr/include/c++/4.2.1/armv6-apple-darwin9/bits/c++config.h /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.2.sdk/usr/include/c++/4.2.1/armv6-apple-darwin10/bits/c++config.h

--- /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.1.3.sdk/usr/include/c++/4.2.1/armv6-apple-darwin9/bits/c++config.h  2009-12-18 09:19:17.000000000 +0000
+++ /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.2.sdk/usr/include/c++/4.2.1/armv6-apple-darwin10/bits/c++config.h   2010-03-16 05:39:05.000000000 +0000
@@ -879,7 +879,7 @@
 /* #undef _GLIBCXX_VERSION */

 /* Define if builtin atomic operations are supported on this host. */
-/* #undef _GLIBCXX_ATOMIC_BUILTINS */
+#define _GLIBCXX_ATOMIC_BUILTINS 1

 /* Define to use concept checking code from the boost libraries. */
 /* #undef _GLIBCXX_CONCEPT_CHECKS */

This means that for any file compiled with the 3.2 SDK, GCC is told that it has the atomic builtins and so it creates code that links against them. So, there’s the problem!

Apple are trying to get everyone to use base SDK and iPhone OS deployment target settings rather than just building for an old SDK, but they need to make sure 100% that their SDKs are sane enough to cope with the asymmetry.

I have uploaded a sample project that shows the problem: AtomicsBug Project.

EDIT: I’ve found that if you use gcc-4.0 rather than gcc-4.2 then the problem doesn’t appear. This is because gcc-4.0 doesn’t use __sync_fetch_and_add for its atomic functions. (Note: this isn’t a fix but is a quick workaround for anyone experiencing the problem).

Update [17/05/2010]: It appears that Apple are aware of the bug. Fingers crossed that it’ll be fixed in future SDKs.

I have been bashing my head against the fact that on the iPhone you seemingly cannot access /etc/resolv.conf, or use functions from the Mac such as SCDynamicStore stuff. I have seen many people asking this question, a few related to the same package I was trying to use – ares.

Eventually I managed to find a fix which uses the libresolv to find the DNS servers. This seems to work just fine and I have shown the patch to ares_init.c below for reference.

#if TARGET_OS_IPHONE
#include <resolv.h>
#endif

...

#if TARGET_OS_IPHONE
// XXX: On the iPhone we need to get the DNS servers using resolv.h magic
if ((_res.options & RES_INIT) == 0) res_init();
channel->nservers = _res.nscount;
channel->servers = malloc(channel->nservers * sizeof(struct server_state));
memset(channel->servers, '\0', channel->nservers * sizeof(struct server_state));

int i;
for (i = 0; i < channel->nservers; i++)
{
    memcpy(&channel->servers[i].addr, &_res.nsaddr_list[i].sin_addr, sizeof(struct in_addr));
}
#endif

[As per most of my posts these days, this is just a brain dump. I'll try to pad it out as soon as possible]

3rd party libraries used by iPhone applications are required to be linked statically. In order to build such a library for the device and simulator you need to set up your environment such that it will cross compile for the two different environments. This is as simple as using the following set of ‘exports’ as defined below.

# Defines to set up environment
export DEVROOT=${ROOTDIR}/Platforms/${PLATFORM}.platform/Developer
export SDKROOT=${DEVROOT}/SDKs/${PLATFORM}${MAX_VERSION}.sdk
export CC=$DEVROOT/usr/bin/gcc
export LD=$DEVROOT/usr/bin/ld
export CPP=$DEVROOT/usr/bin/cpp
export CXX=$DEVROOT/usr/bin/g++
export AR=$DEVROOT/usr/bin/ar
export LIBTOOL=$DEVROOT/usr/bin/libtool
export AS=$DEVROOT/usr/bin/as
export NM=$DEVROOT/usr/bin/nm
export CXXCPP=$DEVROOT/usr/bin/cpp
export RANLIB=$DEVROOT/usr/bin/ranlib
export OPTFLAG="-O${OPT}"
export COMMONFLAGS="${ARCH} -pipe $OPTFLAG -gdwarf-2 -no-cpp-precomp -mthumb -isysroot ${SDKROOT} -miphoneos-version-min=${MIN_VERSION}"
export LDFLAGS="${COMMONFLAGS} -L${HOME}${SDKROOT}/usr/lib"
export CFLAGS="${COMMONFLAGS} -fvisibility=hidden"
export CXXFLAGS="${COMMONFLAGS} -fvisibility=hidden -fvisibility-inlines-hidden"

This uses a few defines which need to be different for the device vs. simulator. Below are what you might want to use for each one. Note that MIN_VERSION, MAX_VERSION and OPT can all be set to control what SDK versions and the level of optimisation is used.

Device:

# Variables
export ROOTDIR="/Developer"
export PLATFORM="iPhoneOS"
export ARCH="-arch armv6 -arch armv7"
export MIN_VERSION="3.1"
export MAX_VERSION="4.0"
export OPT="3"

Simulator:

# Variables
export ROOTDIR="/Developer"
export PLATFORM="iPhoneSimulator"
export ARCH="-arch i386"
export MIN_VERSION="3.1"
export MAX_VERSION="4.0"
export OPT="3"

Once you have set the environment, you just need to use ‘make’ to build the library. If the library is using autoconf, then you will need to use the following ‘./configure’ options:

Device:

./configure --host=arm-apple-darwin9 --prefix=~$SDKROOT --enable-shared=no --disable-dependency-tracking

[Note: --disable-dependency-tracking is required because gcc can't use dependency tracking when building for 2 architectures at the same time (i.e. armv6 and armv7).]

Simulator:

./configure --host=i386-apple-darwin --prefix=~$SDKROOT --enable-shared=no

Then just use ‘make’ and ‘make install’ to build the library! It really is as easy as that .

[Other libraries may require other configure options, but I can't cover them all here as they are often different]

EDIT: I’ve edited the defines above to be a bit clearer and factor out the common bits from the device/simulator.

For a recent project I’ve had to compile the Boost C++ library for the iPhone. Much of the Boost library is header files so they are fine as nothing needs to be done, they just get copied into place, but the bits which do need compiling are a bit trickier. So I thought I’d share my experiences here.

First you need to download Boost (I went for version 1.40) from http://www.boost.org. Then you need to edit some of the Boost.Jam configuration. This involves creating a user-config.jam file in your home directory like so:

~/user-config.jam:
using darwin : 4.2.1~iphone
   : /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/gcc-4.2 -arch armv6
   : <striper>
   : <architecture>arm <target-os>iphone <macosx-version>iphone-3.0
   ;

using darwin : 4.2.1~iphonesim
   : /Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/gcc-4.2 -arch i386
   : <striper>
   : <architecture>x86 <target-os>iphone <macosx-version>iphonesim-3.0
   ;

Then edit tools/build/v2/tools/darwin.jam and add in the following under the macosx-versions variable:

.macosx-versions =
    10.6 10.5 10.4 10.3 10.2 10.1
    iphone-3.1 iphonesim-3.1
    iphone-3.0 iphonesim-3.0
    iphone-2.3 iphonesim-2.3
    iphone-2.2 iphonesim-2.2
    iphone-2.1 iphonesim-2.1
    iphone-2.0 iphonesim-2.0
    iphone-1.x
    ;

Then, you need to use the following lines to build Boost for iPhoneOS and iPhoneSimulator respectively:

./bjam --prefix=~/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.0.sdk/usr toolset=darwin architecture=arm target-os=iphone macosx-version=iphone-3.0 define=_LITTLE_ENDIAN link=static include=/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.0.sdk/usr/include/c++/4.2.1/armv6-apple-darwin9 install

./bjam --prefix=~/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator3.0.sdk/usr toolset=darwin architecture=x86 target-os=iphone macosx-version=iphonesim-3.0 link=static include=/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator3.0.sdk/usr/include/c++/4.2.1/i686-apple-darwin9 install

It’s worth noting that at this stage everything will compile for the simulator but not for the device. This is because the device is missing two header files – ‘bzlib.h’ and ‘crt_externs.h’. To make it compile for the device simply copy these files from ‘/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator3.0.sdk/usr/include/’ into your Boost folder and then everything should compile fine. I have no idea why Apple have omitted these header files from the device. The libraries are there, just the header files missing. I’ve created a bug on Radar for this, so please also do the same if you encounter this problem such that Apple will listen and include them.

That installs the files in ‘~/Developer/Platforms/iPhone(OS|Simulator).platform/Developer/SDKs/iPhone(OS|Simulator)3.0.sdk/usr’ which was where I wanted mine to go, but feel free to change that to wherever you want the libraries to end up. I did it this way because I created a custom SDK and then include that from iPhone projects within XCode.

boost-1.42:
Since Boost has been updated, I thought I’d extend this article to explain how to compile boost-1.42 for the iPhone. Boost themselves have gone a long way to do everything for you now, so there’s much less that needs to be done. You now need to do this:

~/user-config.jam
using darwin : 4.2.1~iphone
   : /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/gcc-4.2 -arch armv7 -mthumb -fvisibility=hidden -fvisibility-inlines-hidden
   : <striper>
   : <architecture>arm <target-os>iphone <macosx-version>iphone-3.1.3
   ;

using darwin : 4.2.1~iphonesim
   : /Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/gcc-4.2 -arch i386 -fvisibility=hidden -fvisibility-inlines-hidden
   : <striper>
   : <architecture>x86 <target-os>iphone <macosx-version>iphonesim-3.1.3
   ;

Then edit tools/build/v2/tools/darwin.jam and add in the following under the macosx-versions variable:

tools/build/v2/tools/darwin.jam
## The MacOSX versions we can target.
.macosx-versions =
    10.6 10.5 10.4 10.3 10.2 10.1
    iphone-3.2 iphonesim-3.2
    iphone-3.1.3 iphonesim-3.1.3
    iphone-3.1.2 iphonesim-3.1.2
    iphone-3.1 iphonesim-3.1
    iphone-3.0 iphonesim-3.0
    iphone-2.2.1 iphonesim-2.2.1
    iphone-2.2 iphonesim-2.2
    iphone-2.1 iphonesim-2.1
    iphone-2.0 iphonesim-2.0
    iphone-1.x
    ;

# <grab source from boost.org>
tar xzf boost_1_42_0.tar.gz
cd boost_1_42_0
./bootstrap.sh

# Install for device:
./bjam --prefix=${HOME}/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS3.1.3.sdk/usr toolset=darwin architecture=arm target-os=iphone macosx-version=iphone-3.1.3 define=_LITTLE_ENDIAN link=static install

# Install for simulator
./bjam –-prefix=${HOME}/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator3.1.3.sdk/usr toolset=darwin architecture=x86 target-os=iphone macosx-version=iphonesim-3.1.3 link=static install