Cross-building for the MIPS32 architecture

The FRITZ!Box itself doesn't include any build tools. Why should it! It's an end-consumer product which is usually never changed, beside the user is updating it with an official firmware update. So we have to do it ourself. Creating a build chain for another architecture is usually hard. There are several things you have to consider. This includes problems with the configuration, endianess differences between the host and the target, bit depths and depending shared libraries which have to be available in the target format. Fortunately this is a common problem, especially for embedded systems, which means there is a proper solution called Buildroot (look at the first line in dmsg, when the FRITZ!Box has booted and you know what I mean). Buildroot is a build system which allows, similar to the Linux kernel, configuring and generating complete embedded Linux systems. Regardless how configured, Buildroot builds a complete Linux system, always. This is of course a little bit overhead for us, but we just pick out the relevant files. First we need to know which architecture we target. You have to login into the FRITZ!Box either using telnet or a working ssh daemon. To get some information about the hardware used in your FRITZ!Box execute cat /proc/cpuinfo.

# cat /proc/cpuinfo
system type             : TI UR8 (7270)
processor               : 0
cpu model               : MIPS 4KEc V6.8
BogoMIPS                : 359.62
wait instruction        : yes
microsecond timers      : yes
tlb_entries             : 16
extra interrupt vector  : yes
hardware watchpoint     : no
ASEs implemented        :
shadow register sets    : 1
core                    : 0
VCED exceptions         : not available
VCEI exceptions         : not available

Here we learn that the FRITZ!Box is using a MIPS processor. In this case it uses the little-endian format. Next you need some information about the system C library. On normal Linux (desktop) systems the GNU C Library is used. This library is heavyweight and therefore the uClibc was invented for embedded systems. By executing ls /lib/libuClibc* you get the version installed. In my case it is 0.9.31. Now we have all information needed to configure Buildroot. Download and extract the source of Buildroot.

If you are working on a Mac like me (where Buildroot doesn't work), VirtualBox with a Linux system as a guest is a good alternative .

By executing make menuconfig in the root directory of Buildroot you are able to configure it. The most important options are Target Architecture, set this to mipsel and Target Architecture Variant, set this to mips 32r2. In Toolchain -> uClibc C library Version select a version which is close to the one on your FRITZ!Box. Next you have to select which packages should be build in Package Selection for the target. Choose whatever you want but for the beginning Networking applications -> openssh might be enough. If you prefer a more lightweight ssh implementation dropbear may also an option. In the following we use OpenSSH. You can also tweak the build options, like the gcc version to use or set the gcc optimization level to -O3 in favor of size. After saving the configuration and executing make, it is time for a coffee. When the build has been finished all relevant files are in output/target/. We only pack the directories and files which are necessary. For example the libuClibc was built by Buildroot, but we will use the one from the FRITZ!Box. The following create a zip archive of the files for OpenSSH.

zip -r ../../local.zip "lib/libutil* lib/libnsl* lib/libresolv* \
 lib/libcrypt* usr/lib usr/share usr/sbin usr/bin etc/ssh_config \
 etc/sshd_config"

Creating overlay directories on the root filesystem

The following is based on ideas invented by the guys at http://www.spblinux.de/. Our aim is it to have as little persistent changes on the FRITZ!Box as possible. This allows us to go back to the original state with a simple reboot. Therefore all writeable directories will be on a tmpfs filesystem. tmpfs filesystems are created in RAM and the content will be removed when the filesystem is unmounted. Because the FRITZ!Box has only limited RAM, we will not copy our applications into the tmpfs filesystem, but just create symbolic links to the files on the USB stick (where usually much more space is available). Because we want the applications from the FRITZ!Box and our own one be accessible from the root filesystem, we need to merge both into the tmpfs directory. Therefore first symbolic links into the original filesystem and second symbolic links to the USB stick are created. The second step is only performed when there isn't a file already. This makes sure the original files are always preferred over our own one. The last step is binding the directories to the root filesystem. Summarized the following will be done:

1. remount / readonly
2. mount / into /var/_ro_
3. create tmpfs in /var/_overlay_
4. symlink the files from /var/_ro_ into /var/_overlay_
5. symlink the files from the USB stick into /var/_overlay_
6. bind the directories in /var/_overlay_ to /

This script does the previous steps. It uses helper methods which are defined in common.sh. You will find all the necessary files in the package you can download at the end of this article.

#!/bin/sh
 
. common.sh
 
MNT=/var/_overlay_
BASE_RO=/var/_ro_
BASE_OVL="${BASE}/local"
 
DIRS="lib etc usr/bin usr/sbin usr/lib usr/share"
 
# remount root readonly
"${MOUNT}" | "${GREP}" "/dev/root" | "${GREP}" -q ro
[ $? -eq 1 ] && _lmt /dev/root / "-o remount -r"
# bind root into var
_lchkmnt "${BASE_RO}"
if [ $? -eq 1 ]; then
  _lmkdir "${BASE_RO}"
  _lmnt / "${BASE_RO}" "-o bind"
fi
 
# create our overlay dir
_lchkmnt "${MNT}"
if [ $? -eq 1 ]; then
  _lmkdir "${MNT}"
  _lmnt tmpfs "${MNT}" "-t tmpfs"
fi
 
# link/copy the base stuff
for i in ${DIRS}; do
  _lsymlnk_dir "${BASE_RO}" "${i}" "${MNT}"
done
 
# link/copy our own stuff
for i in ${DIRS}; do
  _lsymlnk_dir "${BASE_OVL}" "${i}" "${MNT}"
done
 
# now bind all overlay dirs to root
# (this is critical don't interupt)
for i in ${DIRS}; do
  _lmnt "${MNT}/${i}" "/${i}" "-o bind"
done
 
exit 0

Now copy the zip archive you have created above to the FRITZ!Box and unpack it on the USB stick in addons/local/. The scripts have to be placed into addons/. You need to configure the following in common.sh:

# the base of the usb stick goes here:
BASE="/var/media/ftp/FLASH-DISK-01/addons"
# add the encrypted root password here:
PASS=""

Point to the base directory within the USB stick in BASE. Also add your encrypted root password into PASS. This will create a root user in /etc/passwd automatically.

If you believe all is correct, you can start the installation by executing ./install.sh. If no errors are shown, you can try ssh. ssh -v should output something like this:

# ssh -v
OpenSSH_5.8p2, OpenSSL 1.0.0d 8 Feb 2011
...

Additional to the standard filesystems mounted, you should see the following when executing mount:

# mount
...
/dev/root on /var/_ro_ type squashfs (ro,relatime)
tmpfs on /var/_overlay_ type tmpfs (rw,relatime)
tmpfs on /lib type tmpfs (rw,relatime)
tmpfs on /etc type tmpfs (rw,relatime)
tmpfs on /usr/bin type tmpfs (rw,relatime)
tmpfs on /usr/sbin type tmpfs (rw,relatime)
tmpfs on /usr/lib type tmpfs (rw,relatime)
tmpfs on /usr/share type tmpfs (rw,relatime)

Configure and start the OpenSSH daemon

Now that we have our brand new OpenSSH on the FRITZ!Box working, we need some last steps to let the daemon running correctly. First create a private/public RSA host key pair for the server process by executing:

ssh-keygen -t rsa -f addons/config/ssh_host_rsa_key

addons/config/ have to be an existing directory below the USB stick base. Next create a file addons/config/sshd_config in the same directory with the following content:

Protocol               2
UsePrivilegeSeparation no
Subsystem              sftp /usr/lib/sftp-server

The following script could be use to start the sshd:

#!/bin/sh
 
. common.sh
 
SSHD="/usr/sbin/sshd"
SSH_SPKEY="${BASE}/config/ssh_host_rsa_key"
SSH_CFG="${BASE}/config/sshd_config"
SSH_TPKEY="/var/tmp/ssh_host_rsa_key"
 
# sanity check
[ ! -e "${SSHD}" ] && _lce $? "sshd not installed"
 
# start in tmp
cd /var/tmp   
 
mount -o remount devpts /dev/pts -t devpts
 
# cp key and make them user readable only
_lcp_file "${SSH_SPKEY}" "${SSH_TPKEY}"
_lchmod 400 "${SSH_TPKEY}"             
 
# start sshd
"${SSHD}" -f "${SSH_CFG}" -h "${SSH_TPKEY}"

Please note that you have to set up the root user first and maybe adjust your port forwarding rules to allow access from the Internet as described in this post. The package will contain a script startup.sh which will add the root user automatically.

Here is the obligatory screenshot which shows htop with color terminal support. You see OpenSSH and OpenVPN in the list of the running processes.

Conclusion

Thanks to Buildroot it is pretty easy to cross-build for embedded machines these days. With some tricky directory rebinding new applications could be injected into the FRITZ!Box root filesystem without overwriting the existing firmware. Now you should be able to build any software already bundled with Buildroot or even add new one to the build process. Happy cross-building!

Chain of trust

To ensure the validity of every component in a computer system (hardware or software) a chain of trust is build. This basically means there is some root institution (in computer cryptography this is called Certificate Authority (CA)) which is trusted per se. Any following part in this hierarchy is signed by the parent authority. This allows a flexible mechanism where only the connected parties have to make sure they trust each other to make the full chain trustworthy. This concept is also used for code signing, cause it allows to be trustworthy in the eyes of Microsoft without ever being in touch with them. As Microsoft don't trust (for code signing) every root CA they have in their certificate store, they explicit allow only a handful of root CA's to be in this trust of chain. They archive this by cross signing root CA's with their own CA. The full concept is described in this article. It basically means the software maker certificate has to be trusted by an official CA included in every Windows version and this root CA has to be cross signed by Microsoft for code signing. This is the point where the problems start with the new VeriSign Primary CA.

Finding the little differences

The old code signing CA is the VeriSign Class 3 Public Primary CA, available since 1996. This certificate uses an 1024 bit key, which isn't considered save anymore in the future. Therefor VeriSign decided to replace this root CA with a stronger one, which uses an 2048 bit key. If you simply replace an old software maker certificate (signed with the old root CA) with one which is signed by the new CA you get a surprise. Installing a kernel mode driver signed with the new certificate ends up with a message like this:

A look into the security log of the event viewer shows this error message:

Code integrity determined that the image hash of a file is not valid. The
file could be corrupt due to unauthorized modification or the invalid hash
could indicate a potential disk device error.

To be honest: This information doesn't contain any useful hint. If you compare the old signed driver with the new signed driver you will not see any difference. Both could be successful verified as shown next:

Even if you compare all the sub-dialogs side by side you will not find any difference, beside the different root CA of course. So whats the difference? Well, you can't rely on the graphical representation of this trust of chain. When you invoke the signtool of the Windows Driver Kit (WDK) with the verify option, you will see the difference:

C:\Program Files\Oracle\VirtualBox_Sun>signtool.exe verify /v /kp drivers/vboxdrv/VBoxDrv.sys
 
Verifying: drivers/vboxdrv/VBoxDrv.sys
SHA1 hash of file: 9E58611C764D5AE04140E4CC7782B3229D1BCB8A
Signing Certificate Chain:
    Issued to: Microsoft Code Verification Root
    Issued by: Microsoft Code Verification Root
    Expires:   01.11.2025 14:54:03
    SHA1 hash: 8FBE4D070EF8AB1BCCAF2A9D5CCAE7282A2C66B3
 
        Issued to: Class 3 Public Primary Certification Authority
        Issued by: Microsoft Code Verification Root
        Expires:   23.05.2016 18:11:29
        SHA1 hash: 58455389CF1D0CD6A08E3CE216F65ADFF7A86408
 
            Issued to: VeriSign Class 3 Code Signing 2004 CA
            Issued by: Class 3 Public Primary Certification Authority
            Expires:   16.07.2014 00:59:59
            SHA1 hash: 197A4AEBDB25F0170079BB8C73CB2D655E0018A4
 
                Issued to: Sun Microsystems, Inc.
                Issued by: VeriSign Class 3 Code Signing 2004 CA
                Expires:   12.06.2011 00:59:59
                SHA1 hash: 1D4458051589B47A06260125F6EC6BBB6C24472E
 
The signature is timestamped: 08.02.2011 00:23:54
Timestamp Verified by:
    Issued to: Thawte Timestamping CA
    Issued by: Thawte Timestamping CA
    Expires:   01.01.2021 00:59:59
    SHA1 hash: BE36A4562FB2EE05DBB3D32323ADF445084ED656
 
        Issued to: VeriSign Time Stamping Services CA
        Issued by: Thawte Timestamping CA
        Expires:   04.12.2013 00:59:59
        SHA1 hash: F46AC0C6EFBB8C6A14F55F09E2D37DF4C0DE012D
 
            Issued to: VeriSign Time Stamping Services Signer - G2
            Issued by: VeriSign Time Stamping Services CA
            Expires:   15.06.2012 00:59:59
            SHA1 hash: ADA8AAA643FF7DC38DD40FA4C97AD559FF4846DE
 
Successfully verified: drivers/vboxdrv/VBoxDrv.sys
 
Number of files successfully Verified: 1
Number of warnings: 0
Number of errors: 0
 
C:\Program Files\Oracle\VirtualBox_Sun>

and

C:\Program Files\Oracle\VirtualBox_Oracle_Wrong\>signtool.exe verify /v /kp drivers/vboxdrv/VBoxDrv.sys
 
Verifying: drivers/vboxdrv/VBoxDrv.sys
SHA1 hash of file: F398B7124B0A8C32DBFB262343AC1180807505D0
Signing Certificate Chain:
    Issued to: VeriSign Class 3 Public Primary Certification Authority - G5
    Issued by: VeriSign Class 3 Public Primary Certification Authority - G5
    Expires:   17.07.2036 00:59:59
    SHA1 hash: 4EB6D578499B1CCF5F581EAD56BE3D9B6744A5E5
 
        Issued to: VeriSign Class 3 Code Signing 2010 CA
        Issued by: VeriSign Class 3 Public Primary Certification Authority - G5
        Expires:   08.02.2020 00:59:59
        SHA1 hash: 495847A93187CFB8C71F840CB7B41497AD95C64F
 
            Issued to: Oracle Corporation
            Issued by: VeriSign Class 3 Code Signing 2010 CA
            Expires:   08.02.2014 00:59:59
            SHA1 hash: A88FD9BDAA06BC0F3C491BA51E231BE35F8D1AD5
 
The signature is timestamped: 10.02.2011 09:30:08
Timestamp Verified by:
    Issued to: Thawte Timestamping CA
    Issued by: Thawte Timestamping CA
    Expires:   01.01.2021 00:59:59
    SHA1 hash: BE36A4562FB2EE05DBB3D32323ADF445084ED656
 
        Issued to: VeriSign Time Stamping Services CA
        Issued by: Thawte Timestamping CA
        Expires:   04.12.2013 00:59:59
        SHA1 hash: F46AC0C6EFBB8C6A14F55F09E2D37DF4C0DE012D
 
            Issued to: VeriSign Time Stamping Services Signer - G2
            Issued by: VeriSign Time Stamping Services CA
            Expires:   15.06.2012 00:59:59
            SHA1 hash: ADA8AAA643FF7DC38DD40FA4C97AD559FF4846DE
 
Successfully verified: drivers/vboxdrv/VBoxDrv.sys
 
Number of files successfully Verified: 1
Number of warnings: 0
Number of errors: 0
 
C:\Program Files\Oracle\VirtualBox_Oracle_Wrong>

As you can see, the chain of trust of the old certificate contains the Microsoft Code Verification Root, the new signed driver not. As Microsoft released the cross certificate somewhere in 2006, it makes sense that the new VeriSign certificate isn't signed by them. So first of all, we have to blame the signtool for silently ignoring a cross certificate which, obviously, doesn't trust the root certificate. When you have this information you can search for additional information and probably find an advisory of VeriSign. There you learn you need intermediate certificates for the new root CA.

Installing the right certificates on the build machine

The rest is easy. The certificate store of any Windows installation contains the new VeriSign Root CA as shown here:

Delete this root CA and replace it by the intermediate certificates you fetched from the website shown above. Just place the certificate in a text file, add the extension .der and double-click to install it. Make sure to replace really all versions of this certificate, even the one in the global store. When you now sign your driver with your new certificate the Microsoft Code Verification Root is in the trust of chain, as shown in the following:

C:\Program Files\Oracle\VirtualBox_Oracle_Correct>signtool.exe verify /v /kp drivers/vboxdrv/VBoxDrv.sys
 
Verifying: drivers/vboxdrv/VBoxDrv.sys
SHA1 hash of file: 201B7F97473D7F015A104D7841371C5AE4F22FF2
Signing Certificate Chain:
    Issued to: Microsoft Code Verification Root
    Issued by: Microsoft Code Verification Root
    Expires:   01.11.2025 14:54:03
    SHA1 hash: 8FBE4D070EF8AB1BCCAF2A9D5CCAE7282A2C66B3
 
        Issued to: Class 3 Public Primary Certification Authority
        Issued by: Microsoft Code Verification Root
        Expires:   23.05.2016 18:11:29
        SHA1 hash: 58455389CF1D0CD6A08E3CE216F65ADFF7A86408
 
            Issued to: VeriSign Class 3 Public Primary Certification Authority - G5
            Issued by: Class 3 Public Primary Certification Authority
            Expires:   08.11.2021 00:59:59
            SHA1 hash: 32F30882622B87CF8856C63DB873DF0853B4DD27
 
                Issued to: VeriSign Class 3 Code Signing 2010 CA
                Issued by: VeriSign Class 3 Public Primary Certification Authority - G5
                Expires:   08.02.2020 00:59:59
                SHA1 hash: 495847A93187CFB8C71F840CB7B41497AD95C64F
 
                    Issued to: Oracle Corporation
                    Issued by: VeriSign Class 3 Code Signing 2010 CA
                    Expires:   08.02.2014 00:59:59
                    SHA1 hash: A88FD9BDAA06BC0F3C491BA51E231BE35F8D1AD5
 
The signature is timestamped: 10.02.2011 15:03:30
Timestamp Verified by:
    Issued to: Thawte Timestamping CA
    Issued by: Thawte Timestamping CA
    Expires:   01.01.2021 00:59:59
    SHA1 hash: BE36A4562FB2EE05DBB3D32323ADF445084ED656
 
        Issued to: VeriSign Time Stamping Services CA
        Issued by: Thawte Timestamping CA
        Expires:   04.12.2013 00:59:59
        SHA1 hash: F46AC0C6EFBB8C6A14F55F09E2D37DF4C0DE012D
 
            Issued to: VeriSign Time Stamping Services Signer - G2
            Issued by: VeriSign Time Stamping Services CA
            Expires:   15.06.2012 00:59:59
            SHA1 hash: ADA8AAA643FF7DC38DD40FA4C97AD559FF4846DE
 
Successfully verified: drivers/vboxdrv/VBoxDrv.sys
 
Number of files successfully Verified: 1
Number of warnings: 0
Number of errors: 0
 
C:\Program Files\Oracle\VirtualBox_Oracle_Correct>

You see the old certificate (which is trusted by Microsoft) is the parent of the new certificate, which completes the trust of chain again. I guess this is only some temporary solution as long as Microsoft doesn't release a new cross certificate (that's why it is called intermediate). Luckily, you only need the intermediate certificates on the build machine. For your end users nothing has to be changed.

Conclusion

This article shows how a service provider make an easy task hard to do. Signing a kernel mode driver with the new certificate isn't hard, but finding the right information is. Although there is an advisory from VeriSign, it doesn't really explain what to do. As I believe in the future many other people will be in the same situation, I hope this article will save them from some sleepless nights.

Why do you want to use file shortcuts

On the classical UNIX systems you have hard and soft links. These are implemented by the filesystem and make it possible to link to another file or folder without any trouble. Most of the time soft links are used, but it really depends on the use case. Unfortunately these kind of links are not available on Windows (yes, I know there are also hard links and junctions on NTFS, but they are not common and difficult to handle), these links doesn't allow any additional attributes. For example one like to add a different icon to the link or provide more information through a comment field. Beside on Mac OS X, shortcuts can also be work as an application launcher, where the link contain the information what application should be started and how. In contrast to filesystem links which are handled by the operation system, these shortcuts are handled by the window system (or shell) running on the host (which doesn't mean there is no filesystem support for it). On Windows this is the Explorer, on Mac OS X the Finder and on Linux a freedesktop.org conforming file manager.

Creating a Desktop file on Linux

Desktop files on Linux (or any other UNIX system which conforms to freedesktop.org) is easy. It's a simple text file which implement the Desktop Entry Specification. In version 1.0 there are 18 possible entries, where not all of them are mandatory. In the following example I use Qt to write these files, but it should be no problem to use any other toolkit or plain C.

bool createShortcut(const QString &strSrcFile,
                    const QString &strDstPath,
                    const QString &strName)
{
 QFile link(strDstPath + QDir::separator() + strName + ".desktop");
 if (link.open(QFile::WriteOnly | QFile::Truncate))
 {
  QTextStream out(&link);
  out.setCodec("UTF-8");
  out << "[Desktop Entry]" << endl
      << "Encoding=UTF-8" << endl
      << "Version=1.0" << endl
      << "Type=Link" << endl
      << "Name=" << strName << endl
      << "URL=" << strSrcFile << endl
      << "Icon=icon-name" << endl;
  return true;
 }
 return false;
}

Replace icon-name by a registered icon on the system and you are done.

Creating a Shell link on Windows

Windows provides an interface for IShellLink since Windows XP. The following example shows how to use it:

bool createShortcut(LPCSTR lpszSrcFile,
                    LPCSTR lpszDstPath,
                    LPCSTR lpszName)
{
 IShellLink *pShl = NULL;
 IPersistFile *pPPF = NULL;
 HRESULT rc = CoCreateInstance(CLSID_ShellLink,
                               NULL,
                               CLSCTX_INPROC_SERVER,
                               IID_IShellLink,
                               (void**)(&pShl));
 if (FAILED(rc))
  return false;
 do
 {
  rc = pShl->SetPath(lpszSrcFile);
  if (FAILED(rc))
   break;
  rc = pShl->QueryInterface(IID_IPersistFile, (void**)&pPPF);
  if (FAILED(rc))
   break;
  WORD wsz[MAX_PATH];
  TCHAR path[MAX_PATH] = { 0 };
  lstrcat(path, lpszDstPath);
  lstrcat(path, "\\");
  lstrcat(path, lpszName);
  lstrcat(path, ".lnk");
  MultiByteToWideChar(CP_ACP, 0, buf, -1, wsz, MAX_PATH);
  rc = pPPF->Save(wsz, TRUE);
 } while(0);
 if (pPPF)
  pPPF->Release();
 if (pShl)
  pShl->Release();
 return SUCCEEDED(rc);
}

As you may noticed this uses COM. Many API's on Windows using the COM interface to communicate between processes. If you don't use COM in your application you have to initialize it first. This is achieved by adding the following call to the front of the function:

 if (FAILED(CoInitialize(NULL))
  return false;

Depending on your application it might be worth to unitialize COM after usage by appending the following to the function:

 CoUninitialize();

The function itself isn't any magic. It gets a COM interface to the IShellLink interface and then work with it, by setting the source path and adding a target path by using the IPersistFile interface. As I wrote before you could do much more. Providing a path to a specific application or adding your own parameters is no problem. Have a look at the documentation.

Creating an Alias file on Mac OS X

Shortcut files on Mac OS X are a little bit different. At first, they aren't one. There are the classical filesystem links and Alias files. Alias files are links which targeting a specific file, but they haven't all the possibilities of shortcuts like on Windows or Linux. As the name suggest they are really only an alias for another file or directory. So specifying an application to start or things like that aren't possible. Anyway they allow changing the icon and they are more persistent than on Window or Linux cause they are working with several attributes of the target file. Even if you rename or move the target, an Alias file will resolve the target correctly (if it is possible). On the other side, being such special means also being hard to create. In principle there are two possibilities. The first one is, creating a file which is no file at all, but has several resources forks attached. Therefor you need to know exactly how Alias files are built of and make sure with every release of Mac OS X you are following the development. There is a free project which does exactly that: NDAlias. If you are like me and a little bit more lazy, you ask someone who should know how to create Alias files. This is Finder. Although writing the files itself isn't easy, asking the Finder to do the job is not really easier, cause the information about doing exactly that are really rare. The following code shows how to achieve it:

bool createShortcut(NSString *pstrSrcFile,
                    NSString *pstrDstPath,
                    NSString *pstrName)
{
 /* First of all we need to figure out which process Id the Finder
  * currently has. */
 NSWorkspace *pWS = [NSWorkspace sharedWorkspace];
 NSArray *pApps = [pWS launchedApplications];
 bool fFFound = false;
 ProcessSerialNumber psn;
 for (NSDictionary *pDict in pApps)
 {
  if ([[pDict valueForKey:@"NSApplicationBundleIdentifier"]
         isEqualToString:@"com.apple.finder"])
  {
   psn.highLongOfPSN = [[pDict
                          valueForKey:@"NSApplicationProcessSerialNumberHigh"] intValue];
   psn.lowLongOfPSN  = [[pDict
                          valueForKey:@"NSApplicationProcessSerialNumberLow"] intValue];
   fFFound = true;
   break;
  }
 }
 if (!fFFound)
  return false;
 /* Now the event fun begins. */
 OSErr err = noErr;
 AliasHandle hSrcAlias = 0;
 AliasHandle hDstAlias = 0;
 do
 {
  /* Create a descriptor which contains the target psn. */
  NSAppleEventDescriptor *finderPSNDesc = [NSAppleEventDescriptor
                                            descriptorWithDescriptorType:typeProcessSerialNumber
                                            bytes:&psn
                                            length:sizeof(psn)];
  if (!finderPSNDesc)
   break;
  /* Create the Apple event descriptor which points to the Finder
   * target already. */
  NSAppleEventDescriptor *finderEventDesc = [NSAppleEventDescriptor
                                              appleEventWithEventClass:kAECoreSuite
                                              eventID:kAECreateElement
                                              argetDescriptor:finderPSNDesc
                                              returnID:kAutoGenerateReturnID
                                              transactionID:kAnyTransactionID];
  if (!finderEventDesc)
   break;
  /* Create and add an event type descriptor: Alias */
  NSAppleEventDescriptor *osTypeDesc = [NSAppleEventDescriptor descriptorWithTypeCode:typeAlias];
  if (!osTypeDesc)
   break;
  [finderEventDesc setParamDescriptor:osTypeDesc forKeyword:keyAEObjectClass];
  /* Now create the source Alias, which will be attached to the event. */
  err = FSNewAliasFromPath(nil, [pstrSrcFile fileSystemRepresentation], 0, &hSrcAlias, 0);
  if (err != noErr)
   break;
  char handleState;
  handleState = HGetState((Handle)hSrcAlias);
  HLock((Handle)hSrcAlias);
  NSAppleEventDescriptor *srcAliasDesc = [NSAppleEventDescriptor
                                           descriptorWithDescriptorType:typeAlias
                                           bytes:*hSrcAlias
                                           length:GetAliasSize(hSrcAlias)];
  if (!srcAliasDesc)
   break;
  [finderEventDesc setParamDescriptor:srcAliasDesc
    forKeyword:keyASPrepositionTo];
  HSetState((Handle)hSrcAlias, handleState);
  /* Next create the target Alias and attach it to the event. */
  err = FSNewAliasFromPath(nil, [pstrDstPath fileSystemRepresentation], 0, &hDstAlias, 0);
  if (err != noErr)
   break;
  handleState = HGetState((Handle)hDstAlias);
  HLock((Handle)hDstAlias);
  NSAppleEventDescriptor *dstAliasDesc = [NSAppleEventDescriptor
                                           descriptorWithDescriptorType:t ypeAlias
                                           bytes:*hDstAlias
                                           length:GetAliasSize(hDstAlias)];
  if (!dstAliasDesc)
   break;
  [finderEventDesc setParamDescriptor:dstAliasDesc
    forKeyword:keyAEInsertHere];
  HSetState((Handle)hDstAlias, handleState);
  /* Finally a property descriptor containing the target
   * Alias name. */
  NSAppleEventDescriptor *finderPropDesc = [NSAppleEventDescriptor recordDescriptor];
  if (!finderPropDesc)
   break;
  [finderPropDesc setDescriptor:[NSAppleEventDescriptor descriptorWithString:pstrName]
    forKeyword:keyAEName];
  [finderEventDesc setParamDescriptor:finderPropDesc forKeyword:keyAEPropData];
  /* Now send the event to the Finder. */
  err = AESend([finderEventDesc aeDesc],
               NULL,
               kAENoReply,
               kAENormalPriority,
               kNoTimeOut,
               0,
               nil);
 } while(0);
 /* Cleanup */
 if (hSrcAlias)
  DisposeHandle((Handle)hSrcAlias);
 if (hDstAlias)
  DisposeHandle((Handle)hDstAlias);
 return err == noErr ? true : false;
}

Although the code above looks a little bit scary, it does not much. It fetch the process serial number of the current Finder process, creates an Application event for creating an Alias file and send this event to the Finder.

Conclusion

Beside showing how to create file shortcuts on different platforms, this article also shows which work is necessary to create platform independent code. It's a simple example. But it also makes clear that one simple solution for platform one, not necessarily mean it's such simple on platform two.

Making this easy accessible to any developer is the next step. I will leave this exercise to the reader, but have a look at the platform code of the VirtualBox GUI and the corresponding Makefile.

Providing the necessary information to Mac OS X

Applications on Mac OS X need some defined structure. They are so-called bundles, which means on the filesystem layer they are directories. You can prove this by checking the Applications directory within the Terminal.app. You could also right-click on an application and select "Show Package Contents". The content of the bundle directory is usually hidden from the user when he works with the Finder or any other high level function of Mac OS X (like the open dialog). Additional to this layout on the filesystem, an application provide information about itself to the system with a plist file. This file has to be named Application.app/Content/Info.plist. The following shows exemplary the content of the Tunnelblick application:

<xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>CFBundleDevelopmentRegion</key>     <string>English</string>
 <key>CFBundleExecutable</key>            <string>Tunnelblick</string>
 <key>CFBundleIconFile</key>              <string>tunnelblick.icns</string>
 <key>CFBundleIdentifier</key>            <string>com.openvpn.tunnelblick</string>
 <key>CFBundleInfoDictionaryVersion</key> <string>6.0</string>
 <key>CFBundlePackageType</key>           <string>APPL</string>
 <key>CFBundleShortVersionString</key>    <string>3.1beta20 (build 2132)</string>
 <key>CFBundleSignature</key>             <string>OVPN</string>
 <key>CFBundleVersion</key>               <string>2132</string>
 <key>NSMainNibFile</key>                 <string>MainMenu</string>
 <key>NSPrincipalClass</key>              <string>NSApplication</string>
 <key>NSUIElement</key>                   <string>1</string>
 <key>SUEnableSystemProfiling</key>       <true/>
 <key>SUFeedURL</key>                     <string>http://tunnelblick.net/appcast.rss</string>
 <key>SUPublicDSAKeyFile</key>            <string>dsa_pub.pem</string>
</dict>
</plist>

The keys are mostly self explaining, you can find a full list here.

To register a file type association you have to add an array of the type CFBundleDocumentTypes. Again, here is an extraction of the Tunnelblick application, which shows the registration of the tblk extension:

 <key>CFBundleDocumentTypes</key>
 <array>
  <dict>
   <key>CFBundleTypeExtensions</key>   <array><string>tblk</string></array>
   <key>CFBundleTypeIconFile</key>     <string>tunnelblick_package.icns</string>
   <key>CFBundleTypeName</key>         <string>Tunnelblick VPN Configuration</string>
   <key>CFBundleTypeRole</key>         <string>Editor</string>
   <key>LSTypeIsPackage</key>          <true/>
   <key>NSPersistentStoreTypeKey</key> <string>Binary</string>
  </dict>
 </array>

You have to provide a file extension or mime-type, can add an icon and give a hint what your application can do with this type of file (viewer, editor or nothing).

Beside this passive way of announcing this information, there is also an active way (e.g. for use in an installer). See here for further information.

Where is this information stored

Although Mac OS X hasn't a registry like windows, some information are stored in global databases, too. Applications are registered at the launchd. The launchd is the central place for starting all kind of programs, from a background service to any common application, like Thunderbird. E.g. background services can register itself and even made their start depending on different events. VirtualBox has an example configuration included, which let launchd start the vbox webservice on activity on a certain port. Although this isn't used by many users out there (and therefore disabled by default), it shows some generic usage of this functionality.

If you look around in Mac OS X you have the tool launchctl, which allows you to start jobs or register background services with launchd. But it seems there is no tool which is able to get some information about the knowledge of launchd. Well, there is one. It's a little bit hidden in /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister. To be more comfortable with it, we should include it into the user path by linking it to a known path, like this:

sudo ln -s /Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister /usr/local/bin/lsregister

With this tiny tool we are able to dump the launchd database. lsregister -dump | grep -n10 Tunnelblick displays something like this for Tunnelblick:

10823---------------------------------------------------------------------------------
10824-bundle    id:            6984
10825:  path:          /Applications/Tunnelblick.app
10826-  name:
10827-  identifier:    com.openvpn.tunnelblick (0x800097b8)
10828-  version:       2132
10829-  mod date:      10/31/2010 12:37:08
10830-  reg date:      11/19/2010 11:13:39
10831-  type code:     'APPL'
10832-  creator code:  'OVPN'
10833-  sys version:   0
10834-  flags:         relative-icon-path  ui-element
10835-  item flags:    container  package  application  extension-hidden  native-app  ppc  i386
10836-  icon:          Contents/Resources/tunnelblick.icns
10837:  executable:    Contents/MacOS/Tunnelblick
10838-  inode:         3800129
10839-  exec inode:    3800213
10840-  container id:  32
10841-  library:
10842-  library items:
10843-  --------------------------------------------------------
10844-  claim   id:            17864
10845:          name:          Tunnelblick VPN Configuration
10846-          rank:          Default
10847-          roles:         Editor
10848-          flags:         relative-icon-path  package
10849-          icon:          Contents/Resources/tunnelblick_package.icns
10850-          bindings:      .tblk
10851---------------------------------------------------------------------------------

As you see, all the information provided by the plist file is registered. Beside other information, it also has the registration and modification times stored. To let launchd reread the plist file, usual its only necessary to change the modification time of the application. This could be done by simply executing touch /Applications/Application.app. Anyway, it's sometimes necessary to reset the content of this database for the own application. This could be done by executing the following:

lsregister -u /Applications/Application.app

To register the application again, just remove the -u parameter. You should also check if your application isn't registered more than once, with different paths, by using the -dump parameter. To fully reset the launchd database, you could use -kill. This will remove any file type association and registered application. You have been warned.

Conclusion

This article shows how Mac OS X handle informations about installed applications. With this knowledge a developer is able to register the own application into the launchd ecosystem and see how launchd interpret this information. Furthermore, the usage of lsregister allows a developer to analyze the content of the launchd database and make changes to it.

The hardware used, as already said, is a MacBook Pro 3,1 and a Western Digital My Passport Essential 500GB USB hard disk. Other combination may work, but I don't guarantee this, as always. Also you should be warned that anything I describe here could destroy your existing installation and I'm not responsible for that. Doing some kind of backup might be a good idea. Time Machine is easy to use .

Before we start, as usual, the credits for some sites I get my information from. This is mainly the Produnis blog, the Blog of Chris, the Ubuntu wiki and of course the guys around the Grub2 development.

Creating the EFI boot loader

Apple doesn't use the legacy BIOS to boot their machines, they use the Extensible Firmware Interface (EFI). This new way of booting operation systems is very flexible, as the name suggest, but has several drawbacks, like most of the standard operation system doesn't speaks it language. Although Linux can be configured to use EFI directly we will emulate a legacy BIOS in the following. For this we need a connector which makes EFI and Linux happy and let them both work smoothly together. This connector is Grub2, which is in development for several years now. It's the successor of Grub and is the standard in many popular Linux distributions these days. We have to build a version our self, for which an existing Linux installation is really helpful. I used my 64-bit Gentoo installation. First you have to find out if the EFI installation on your target Apple machine is 64 or 32-bit. You can do this by executing:

ioreg -l -p IODeviceTree | grep firmware-abi

This will return EFI64 or EFI32 respectively. In my case I need the 64-bit version, which is a little bit surprising when I consider that my MacBook Pro isn't able to boot a 64-bit Snow Leopard. Anyway, grab the latest version of Grub2 and unpack it on the Linux machine. Please note that you need a gcc with multilib support if you are targeting an architecture which isn't the same as the host one. Use the following to configure Grub2 and to build it. Of course you have to adjust the target architecture if it is a different one.

./configure --with-platform=efi --target=x86_64 --disable-werror
make

When this is finished you create the EFI package by executing

./grub-mkimage -d . -o bootx86.efi *.mod

Here I included all modules which are available. If size matter for you, you could of course make a selective choice on the modules included. I didn't test this myself, so you have to find out yourself which one are important.

Whipping the USB hard disk into shape

Next we have to prepare the USB hard disk for the new installation. In the following I assume your USB hard disk doesn't contain any valid data and could be reformatted without data lose. Make a backup of your data first if this isn't the case on your side. Apple uses the GUID Partition Table scheme to organize their partitions on a hard disk. This specification is part of EFI and remove many limitations of the Master boot record (MBR) scheme, which is widely used in the PC world. That is e.g. the disk size limitation of 2TB or the maximum of 4 primary partitions. You reformat your disk, using the Disk Utility application of Mac OS X. Make sure all existing partitions on the disk are unmounted. When necessary, change the partition scheme from MBR to GUID in the Options dialog of the Partitions window. Select the partitions count you want to use. You need at least 3 partitions to make Linux works fine. My partition scheme looks like in the following: As you can see I have 5 partitions configured. The first one is an additional Snow Leopard installation for testing. I also added a Data partition at the end for making the data transfer between the different operation systems as easy as possibly. LINUXBOOT is a small partition which will contain the EFI boot loader (size it 50MB or something like that). Linux Swap, obviously, will become the swap partition of the Linux installation. DISK1S5 is the Linux root partition itself. The Data partition has to be formated as Mac OS Extended. Don't use the Journaled version of HFS+, cause this makes trouble on the Linux side. The other partitions have to be formatted as MS-DOS (FAT).

After applying the changes we can add the EFI boot loader to the LINUXBOOT partition. The Apple EFI implementation is searching for a file with the efi extensions on all bootable hard disks. Mount LINUXBOOT and create a efi/boot directory on the root path. Copy the bootx86.efi file into the boot/ directory. As bootx86.efi is a Grub2 boot loader we need a valid Grub2 configuration file. The following grub.cfg shows the configuration for a Ubuntu 9.10 i386 installation. For the 64-bit version or any other version of Ubuntu the settings might be slightly different.

menuviewer="text"
timeout=10
default=0
set F1=ctrl-x
menuentry "ubuntu-9.10-desktop-i386"
{
 fakebios
 search --set -f /boot/vmlinuz-2.6.31-14-generic
 linux /boot/vmlinuz-2.6.31-14-generic root=UUID=4e140981-4ab3-41a2-a2fb-26b1287beb87 ro quiet splash noefi video=efifb
 initrd /boot/initrd.img-2.6.31-14-generic
}
menuentry "ubuntu-9.10-desktop-i386 single"
{
 fakebios
 search --set -f /boot/vmlinuz-2.6.31-14-generic
 linux /boot/vmlinuz-2.6.31-14-generic root=UUID=4e140981-4ab3-41a2-a2fb-26b1287beb87 ro noefi video=efifb single
 initrd /boot/initrd.img-2.6.31-14-generic
}
menuentry "ubuntu-9.10-desktop-i386 text"
{
 fakebios
 search --set -f /boot/vmlinuz-2.6.31-14-generic
 linux /boot/vmlinuz-2.6.31-14-generic root=UUID=4e140981-4ab3-41a2-a2fb-26b1287beb87 ro noefi vga=normal
 initrd /boot/initrd.img-2.6.31-14-generic
}
menuentry "Mac OS X"
{
 search --set -f /usr/standalone/i386/boot.efi
 chainloader /usr/standalone/i386/boot.efi
}
menuentry "CD"
{
 appleloader CD
}
menuentry "mbr"
{
 appleloader HD
}
menuentry "reboot"
{
 reboot
}

You have to change the root UUID to the one the Ubuntu installer will assign to your hard disk after installation. Just check the fstab file when the installation has finished. The first entry boots Linux with a splash image enabled. The second one is for the single user mode in the case something went wrong. Please note the video=efifb option, which enables the graphical mode in the boot phase.

Installing Ubuntu

Most of the installation process is straight forward and doesn't need any special attention. Download the version of your choice from one of the mirrors, burn it on CD and start the installation. You can select the CD as boot medium by pressing Alt when your Mac starts. When the installer ask for the partition scheme, you have to switch to "manual choice". Select the DISK1S5 (your what it is in your case) as the root / partition and change the filesystem type to ext3. Also remember the path to the system partition, cause you will need it later again. Select the swap partition and change its type to swap. Proceed with the rest of the installation until the last dialog. There select "advanced settings" and change the boot loader target from hd0 to /dev/sdXX, where you replace XX to the path you used previously in the partition tool.

If all went right you should be able to select the LINUXBOOT partition by pressing Alt when your Mac starts. After that Grub2 should shows up, you will be able to boot into your freshly installed Ubuntu.

Conclusion

In this post I showed how to easily add the possibility to boot Linux on your MacBook Pro. With the external USB hard disk solution, no internal valuable space is wasted. Of course the speed isn't the same as if the OS would be installed on the internal drive, but for testing software on different operation systems this is satisfactory. To increase the speed a little bit more, an external FireWire hard disk could be used.