Installing Unison

Most modern Linux distributions include Unison in their package manage system. On Mac OS X you can use MacPorts. Alternatively you could download a binary version for Mac OS X or Windows here. To prevent surprises and unnecessary trouble it might be a good idea to make sure that every involved system use the same version of Unison. At least on Linux and Mac OS X it is relatively easy to compile Unison from the sources.

Setting up public/private key authentication for ssh

One advantage of Unison over Rsync is that you can use different communication channels for the file transfers. One is ssh. As I always prefer/demand encrypted communication this is a big plus of course. In the default setup you can just use ssh. But for a little bit more comfort I suggest to create a public/private key pair for the authentication.
The following creates public/private keys without a password. Although this is much more easier to use, it should be only used on hosts which are trusted. If you are in doubt, use the normal password approach or even better create a public/private key pair with a password. Create a new public/private key pair with the following command:

user@host-a ~ $ ssh-keygen -t rsa

When you are asked for a password just hit Enter twice. The command creates the private key in ~/.ssh/id_rsa and the public key in ~/.ssh/id_rsa.pub. Now copy the public key to host B:

user@host-a ~ $ scp ~/.ssh/id_rsa.pub host_B:.ssh/authorized_keys

If you already have some public keys on host B, make sure you append the new key and not overwrite the file by the above command. Make the file accessible by the user only with:

user@host-b ~ $ chmod 600 ~/.ssh/authorized_keys

Now you should be able to connect to host B without any interaction needed.

Configuring Unison

Like in the long UNIX tradition, Unison is configured using text files. The files are located in the ~/.unison directory. You can configure more than one synchronization target by choosing a meaningful name. There exists one default target which is configured in the file default.prf. Because I have more than one target I prefer to split the configuration into several files. You can include other project files with the include statement as shown here:

# directory on host a (this is where Unison will be executed)
root       = /mnt/data/projects
# directory on host b (this is the remote host)
root       = ssh://host-b//mnt/data/projects
# which directories to sync?
include projects_files.prf
# options
include options.prf
ignorecase = false
# unison executable on the server
servercmd  = /usr/local/bin/unison

We setup the root directories on both machines, including the configuration file for the project target and some generic option file. We also overwriting the default unison location, because this is a self compiled version. The file options.prf looks like this:

# No staled nfs and mac store files
ignore  = Name .nfs*
ignore  = Name .DS_Store
# options
log     = true
rsrc    = true
auto    = true
#debug   = verbose
#logfile = ~/.unison/unison.log

This just set some generic options which are valid for all my targets. For the specific target projects the file projects_files.prf contains mainly the directories and files which should be ignored:

# No ISOs
ignore    = Path vms/ISO
# Ignore VBox branches
ignore    = Path vbox-*
# No binary output from the other platforms
ignore    = Path vbox*/out/*
# One exception:
ignorenot = Path vbox/out/linux.amd64.additions
# No wine stuff
ignore    = Path vbox*/wine.*
# Tools
ignore    = Name vbox*/tools/{FetchDir,freebsd*,os2*}

So in general, you configure the directory to synchronize and later define directories or files which should be ignored. As you see, you can include or exclude paths as you like. Even simple bash wildcards are possible. As shown in this example I exclude all binary files of a VirtualBox build, because they are useless on another platform. Understanding how Unison decide which directories or files should be synchronized is sometimes difficult. So I suggest to carefully read the documentation and just use the "try and failure" approach . Another reason for splitting up the configuration files is you can synchronize these files as well. I have another target which synchronize several configuration files, e.g. .bashrc, .profile, .vim* and the sub-project files of Unison like the projects_files.prf. You can't synchronize e.g. default.prf, cause the root directories are different from host to host, but the general configuration is always the same. My home target looks like this:

# Which directories/files to sync?
path = .bashrc
path = .ion3
path = .gdbinit
path = .cgdb
path = .valgrind-vbox.supp
path = .vim
path = .vimrc
path = .gvimrc
path = .Xdefaults
path = .gnupg
path = .unison/options.prf
path = .unison/home_files.prf
path = .unison/projects_files.prf
# Do not sync:
ignore = Path .vim/.netrwhist
ignore = Path .ion3/default-session*
ignore = Path .cgdb/readline_history.txt

One of the strengths over other synchronization tools is, you can do this for others host as well. So if you synchronize between host a and host b you can also synchronize between host c and host b. However, a little bit of discipline is necessary. There should be one host which all other host synchronize again.
If you now execute unison the project target will be used. If you execute unison home the files of the home target will be synchronized.

Conclusion

Unison is a very powerful tool. You can synchronize between more than two hosts (OS independent), in a secure way and uni-directional. Currently there is no better tool and I use it on a daily basis.

Cross-building for the MIPS32 architecture

The FRITZ!Box itself doesn't include any build tools. Why should it! It's an end-consumer product which is usually never changed, beside the user is updating it with an official firmware update. So we have to do it ourself. Creating a build chain for another architecture is usually hard. There are several things you have to consider. This includes problems with the configuration, endianess differences between the host and the target, bit depths and depending shared libraries which have to be available in the target format. Fortunately this is a common problem, especially for embedded systems, which means there is a proper solution called Buildroot (look at the first line in dmsg, when the FRITZ!Box has booted and you know what I mean). Buildroot is a build system which allows, similar to the Linux kernel, configuring and generating complete embedded Linux systems. Regardless how configured, Buildroot builds a complete Linux system, always. This is of course a little bit overhead for us, but we just pick out the relevant files. First we need to know which architecture we target. You have to login into the FRITZ!Box either using telnet or a working ssh daemon. To get some information about the hardware used in your FRITZ!Box execute cat /proc/cpuinfo.

# cat /proc/cpuinfo
system type             : TI UR8 (7270)
processor               : 0
cpu model               : MIPS 4KEc V6.8
BogoMIPS                : 359.62
wait instruction        : yes
microsecond timers      : yes
tlb_entries             : 16
extra interrupt vector  : yes
hardware watchpoint     : no
ASEs implemented        :
shadow register sets    : 1
core                    : 0
VCED exceptions         : not available
VCEI exceptions         : not available

Here we learn that the FRITZ!Box is using a MIPS processor. In this case it uses the little-endian format. Next you need some information about the system C library. On normal Linux (desktop) systems the GNU C Library is used. This library is heavyweight and therefore the uClibc was invented for embedded systems. By executing ls /lib/libuClibc* you get the version installed. In my case it is 0.9.31. Now we have all information needed to configure Buildroot. Download and extract the source of Buildroot.

If you are working on a Mac like me (where Buildroot doesn't work), VirtualBox with a Linux system as a guest is a good alternative .

By executing make menuconfig in the root directory of Buildroot you are able to configure it. The most important options are Target Architecture, set this to mipsel and Target Architecture Variant, set this to mips 32r2. In Toolchain -> uClibc C library Version select a version which is close to the one on your FRITZ!Box. Next you have to select which packages should be build in Package Selection for the target. Choose whatever you want but for the beginning Networking applications -> openssh might be enough. If you prefer a more lightweight ssh implementation dropbear may also an option. In the following we use OpenSSH. You can also tweak the build options, like the gcc version to use or set the gcc optimization level to -O3 in favor of size. After saving the configuration and executing make, it is time for a coffee. When the build has been finished all relevant files are in output/target/. We only pack the directories and files which are necessary. For example the libuClibc was built by Buildroot, but we will use the one from the FRITZ!Box. The following create a zip archive of the files for OpenSSH.

zip -r ../../local.zip "lib/libutil* lib/libnsl* lib/libresolv* \
 lib/libcrypt* usr/lib usr/share usr/sbin usr/bin etc/ssh_config \
 etc/sshd_config"

Creating overlay directories on the root filesystem

The following is based on ideas invented by the guys at http://www.spblinux.de/. Our aim is it to have as little persistent changes on the FRITZ!Box as possible. This allows us to go back to the original state with a simple reboot. Therefore all writeable directories will be on a tmpfs filesystem. tmpfs filesystems are created in RAM and the content will be removed when the filesystem is unmounted. Because the FRITZ!Box has only limited RAM, we will not copy our applications into the tmpfs filesystem, but just create symbolic links to the files on the USB stick (where usually much more space is available). Because we want the applications from the FRITZ!Box and our own one be accessible from the root filesystem, we need to merge both into the tmpfs directory. Therefore first symbolic links into the original filesystem and second symbolic links to the USB stick are created. The second step is only performed when there isn't a file already. This makes sure the original files are always preferred over our own one. The last step is binding the directories to the root filesystem. Summarized the following will be done:

1. remount / readonly
2. mount / into /var/_ro_
3. create tmpfs in /var/_overlay_
4. symlink the files from /var/_ro_ into /var/_overlay_
5. symlink the files from the USB stick into /var/_overlay_
6. bind the directories in /var/_overlay_ to /

This script does the previous steps. It uses helper methods which are defined in common.sh. You will find all the necessary files in the package you can download at the end of this article.

#!/bin/sh
 
. common.sh
 
MNT=/var/_overlay_
BASE_RO=/var/_ro_
BASE_OVL="${BASE}/local"
 
DIRS="lib etc usr/bin usr/sbin usr/lib usr/share"
 
# remount root readonly
"${MOUNT}" | "${GREP}" "/dev/root" | "${GREP}" -q ro
[ $? -eq 1 ] && _lmt /dev/root / "-o remount -r"
# bind root into var
_lchkmnt "${BASE_RO}"
if [ $? -eq 1 ]; then
  _lmkdir "${BASE_RO}"
  _lmnt / "${BASE_RO}" "-o bind"
fi
 
# create our overlay dir
_lchkmnt "${MNT}"
if [ $? -eq 1 ]; then
  _lmkdir "${MNT}"
  _lmnt tmpfs "${MNT}" "-t tmpfs"
fi
 
# link/copy the base stuff
for i in ${DIRS}; do
  _lsymlnk_dir "${BASE_RO}" "${i}" "${MNT}"
done
 
# link/copy our own stuff
for i in ${DIRS}; do
  _lsymlnk_dir "${BASE_OVL}" "${i}" "${MNT}"
done
 
# now bind all overlay dirs to root
# (this is critical don't interupt)
for i in ${DIRS}; do
  _lmnt "${MNT}/${i}" "/${i}" "-o bind"
done
 
exit 0

Now copy the zip archive you have created above to the FRITZ!Box and unpack it on the USB stick in addons/local/. The scripts have to be placed into addons/. You need to configure the following in common.sh:

# the base of the usb stick goes here:
BASE="/var/media/ftp/FLASH-DISK-01/addons"
# add the encrypted root password here:
PASS=""

Point to the base directory within the USB stick in BASE. Also add your encrypted root password into PASS. This will create a root user in /etc/passwd automatically.

If you believe all is correct, you can start the installation by executing ./install.sh. If no errors are shown, you can try ssh. ssh -v should output something like this:

# ssh -v
OpenSSH_5.8p2, OpenSSL 1.0.0d 8 Feb 2011
...

Additional to the standard filesystems mounted, you should see the following when executing mount:

# mount
...
/dev/root on /var/_ro_ type squashfs (ro,relatime)
tmpfs on /var/_overlay_ type tmpfs (rw,relatime)
tmpfs on /lib type tmpfs (rw,relatime)
tmpfs on /etc type tmpfs (rw,relatime)
tmpfs on /usr/bin type tmpfs (rw,relatime)
tmpfs on /usr/sbin type tmpfs (rw,relatime)
tmpfs on /usr/lib type tmpfs (rw,relatime)
tmpfs on /usr/share type tmpfs (rw,relatime)

Configure and start the OpenSSH daemon

Now that we have our brand new OpenSSH on the FRITZ!Box working, we need some last steps to let the daemon running correctly. First create a private/public RSA host key pair for the server process by executing:

ssh-keygen -t rsa -f addons/config/ssh_host_rsa_key

addons/config/ have to be an existing directory below the USB stick base. Next create a file addons/config/sshd_config in the same directory with the following content:

Protocol               2
UsePrivilegeSeparation no
Subsystem              sftp /usr/lib/sftp-server

The following script could be use to start the sshd:

#!/bin/sh
 
. common.sh
 
SSHD="/usr/sbin/sshd"
SSH_SPKEY="${BASE}/config/ssh_host_rsa_key"
SSH_CFG="${BASE}/config/sshd_config"
SSH_TPKEY="/var/tmp/ssh_host_rsa_key"
 
# sanity check
[ ! -e "${SSHD}" ] && _lce $? "sshd not installed"
 
# start in tmp
cd /var/tmp   
 
mount -o remount devpts /dev/pts -t devpts
 
# cp key and make them user readable only
_lcp_file "${SSH_SPKEY}" "${SSH_TPKEY}"
_lchmod 400 "${SSH_TPKEY}"             
 
# start sshd
"${SSHD}" -f "${SSH_CFG}" -h "${SSH_TPKEY}"

Please note that you have to set up the root user first and maybe adjust your port forwarding rules to allow access from the Internet as described in this post. The package will contain a script startup.sh which will add the root user automatically.

Here is the obligatory screenshot which shows htop with color terminal support. You see OpenSSH and OpenVPN in the list of the running processes.

Conclusion

Thanks to Buildroot it is pretty easy to cross-build for embedded machines these days. With some tricky directory rebinding new applications could be injected into the FRITZ!Box root filesystem without overwriting the existing firmware. Now you should be able to build any software already bundled with Buildroot or even add new one to the build process. Happy cross-building!

We start with a tool called Apparix.

Console?

As you may know, I'm a keyboard addict. My preferred OS is Linux, I use Ion3 as the window manager, rxvt-unicode as terminal application and (g)Vim as the preferred editor. On the other side I have a MacBook Pro as laptop and love the possibilities of Mac OS X. How does this come together? Well, all these tools are available on Mac OS X as well. The X11 port of Mac OS X is getting better with each release and using it in fullscreen is good enough. Mac OS X is based on BSD and together with MacPorts, there is nothing I missed until today. As I have to work really platform independent, there is Windows. Vim is available, but working console based (read: command.com) isn't really much fun. There is of course mingw, but I never managed it to create a work environment like on Linux/UNIX. Anyway, I work not that much on Windows, so I can live with other tools there as well (a really good free editor is Notepad++ and if you hate Explorer have a look at Altap Salamander). So the question remains: Console? I have the feeling using the keyboard within a console application is much faster than clicking around with the mouse. This doesn't mean I like to type every single character when starting an application or changing the current working directory. Many modern command interpreters, like the Bash, have built in word expansion. This allows you to use the tabulator key for fast expansion of typed characters for a given set of commands. Of course this makes the console life much more easier. But there is more!

Bookmarking important directories

Usually, I'm working on projects. One project is VirtualBox. Other might be some private stuff, like personal letters or other development I do. As the regular user organize his projects in directory trees it make sense to bookmark them. Most people believe bookmarking is something Internet specific, but this isn't true. Bookmarking means getting access to an important information very fast, without remembering the exact location of this information. This is exactly what Apparix does. It bookmarks directories and make these bookmarks easy accessible on the console. So I have bookmarks for sub-projects within VirtualBox, like the GUI or Main. Getting there is as easy as writing to vgui. It's the same as writing cd /bin and it even provide word expansion of the available bookmarks. In this case the bookmark is vgui and it expand to the GUI directory within the VirtualBox source tree.

How does this work?

Installing Apparix is pretty simple. You will find all the information on the website. The important part is the integration within your shell. You need to extent your .bashrc. One way is to add the functions you need into .bashrc. Alternatively you can download the file .bash_apparix and source it in your .bashrc. When you have done this and start a new shell, you should navigate to a directory you like to bookmark. There you type bm to create your first bookmark. Afterwards, regardless what your current working directory is, you could execute to DIR_NAME to get there. Of course you can name the bookmark other than the current directory name, by using bm MY_NAME. By executing als all bookmarks are shown. There is much more possible of course. See the documentation for more information. Anyway, this is the main use case for Apparix, so there is not much more to say. Really?

Using the Apparix database for other applications

The cool thing on many Linux applications is, they are saving their data in simple text files. Also Apparix. The bookmark "database" is saved in a file called .apparixrc. It's a simple text file, which is easy to parse. It looks like this:

j,vbox,/Users/poetzsch/projects/vbox
j,VirtualBox,/Users/poetzsch/projects/vbox/src/VBox/Frontends/VirtualBox
j,Main,/Users/poetzsch/projects/vbox/src/VBox/Main
j,Runtime,/Users/poetzsch/projects/vbox/src/VBox/Runtime
j,Installer,/Users/poetzsch/projects/vbox/src/VBox/Installer/darwin
j,cv,/Users/poetzsch/projects/private/office/latex/cv
j,dow,/Users/poetzsch/Downloads

This allows the usage of this information in other contexts as well. One example is Vim. As I mentioned earlier, this is my main editor. So it would be cool to use the bookmarks, I already have, there as well. To easy change the current directory in Vim, add the following to your .vimrc:

command! -complete=custom,BmCe -nargs=1 To :call BmTo('<args>')
fun BmCe(A,L,P)
 return system("awk 'BEGIN{FS=\",\"}/j,/{print $2}' ~/.apparixrc")
endfun
fun BmLs(dir)
 return system('apparix '.a:dir.' | tr -d "\n"')
endfun
fun BmTo(dir)
 execute ':cd '.BmLs(a:dir)
endfun

After restarting Vim, it is possible to navigate with :To MY_BOOKMARK to any directory known by Apparix.

Conclusion

One simple task, getting to a directory where you are often have to be, with one simple solution. Thats all to say about Apparix. It does this in a very simple way, which is fully integrated into the Bash and the saved information could be used by other applications as well.

Why should I test my software

This is easy to answer. Humans write software and humans tend to make mistakes. Even if something is working perfect, a bug fix or new feature can break it. Testing software will not prevent you from bugs, but it can boost your user satisfaction a lot, by removing errors which are common and avoidable.

What can be tested

First of all, you should think about what should be tested at all. The following list tries to categorizes different tests from a technical perspective:

1. Static tests
2. Compile time tests
3. Runtime test

Also we have to differ between automatic tests and manual tests done by an operator. In the following I will prioritize the automatic tests, simply because once they are setup, they are easier to handle. Manuel tests means, setting up a test plan, making someone responsible for checking the test plan and forcing the test to be done before a release. On the other side automatic tests could be done by a machine at day and night with every software level your application is currently in.

Static tests

Static tests analyze the code base and try to predict error paths. There are several applications on the market, but most of them are not free. First you need a product which speaks your language. If you develop in C, C++, Java, C# or whatever makes a huge difference here. There are many static analyzer out there which can analyze C, but not many can successfully parse any of the widely used higher programming languages. One of them, which can analyze C++, is Fortify. Fortify uses rules to find different errors a developer can do. There are of course default rules, but a developer can define rules himself. This product is specialized in detecting errors which may end in security holes. One such error in terms of security considerations is file input handling. When one is reading a string directly out of a file, some assumptions are done. Strings in the C/C++ world are zero terminated. All string handling/manipulation function, which are available e.g. from the glibc, are working over the single characters as long as they didn't find the termination byte. Now, if a developer is reading a string out of a file and doesn't check if the terminating zero is there, he open a security hole. If an attacker is using this knowledge he is able to provoke a buffer overflow. Fortify can detect such blindly faith to input data. Of course it can detect much more. On the down side, as most software projects are really complex, static code analyzer tend to produce many false/positive. Working through this false/positive and defining filters for the own software project needs much time. On the other side these tools help in finding error paths, which even runtime tests doesn't show, cause they are so unusual. But exactly these errors are used by attackers to abuse your software for bad things.

Compile time tests

Compile tests make sure your application compiles on different architectures, bit depth, or with other resources like external dependencies (e.g. depending third-party libraries). They are important when you develop for and/or if your team is using different platforms. They make sure a software is at least compiling on every important system. This doesn't mean the current status works well, it just makes sure every developer involved in the progress of the product is able to build an actual version, even if he is working on a different part of the software itself. Such a situation happens often in open source products, like Mozilla, which invented tinderbox for making these tests. It allows you to set up different build environments for your software and watch the result of every check-in you made in terms of compatibility to this environment. The following shows a screenshot of the Firefox-Ports tinderbox with one broken build box. As everybody know, red is bad:

Red means a burning tree. It shows you that your last check-in doesn't compile on a specific system. The cool thing is that you can look at the log and spot the error with ease. You know why it doesn't compile and can fix it. This will prevent you from being blamed by other team members because they can't work with the current trunk.

Runtime tests

Runtime tests are of course the biggest part of testing. They try to make sure the application does the job it was written for. This also means these tests are the hardest one. It may help to try to separate them as well. I use the following separation:

1. Unit tests
2. Dynamic analyzer
3. GUI tests

Unit tests

Unit tests check that the specialized functionality a unit provide is working correctly. A well-known example is string handling. String handling is necessary in almost all software projects out there. May it printing the help section of a command line tool or the information presentation in an GUI application. A unit test tries to test anything a user of this unit may do with it. This could be adding a string to another, replacing parts of a string, converting a string to other formats (like to another encoding or to an integer) or much more complex operations like searching within a string with a regular expression. All these operations could be relatively easy tested with unit tests. There is no general usable software out there which could be used for unit testing, but most software frameworks offer some help to write unit tests. Qt has a unit test framework and VirtualBox has a integrated system, as well. Here is a screenshot from a just announced graphical tool by Nokia, which makes spotting errors in Qt unit tests much more easier. The second screenshot shows the output of a VirtualBox unit test.

Please also note that unit testing not just means testing functionality. It could also mean testing performance regression. If some function needed an amount of time in an existing version, it should be at least equal or faster in means of execution time in a new version. Unit tests are evolutionary like the software itself. It is illusionary to believe a unit test is finished sometime. The test will/should grow as long as users will report errors. Often a unit test is called by the bugtracker id, which showed some specific problem the first time.

Dynamic analyzer

This field of testing is a little bit different from the other one, cause the developer needs to interact here. Although it would be possible to automate some of these tests, they are more designed to be done by the developer from time to time (usually after a feature is completed) to check for some basic errors. The most common dynamic analyzer in the open source world is the Valgrind instrumentation framework. It has a tool for checking memory management errors, like I showed in this post. As already written there, memory leak errors can happen to everyone, so having a tool for finding and eliminating them, is very useful. On the other side there is a tool called Callgrind, which profiles a given software in detail. After profiling some software, a developer is able to see execution times relatively to each other. This makes it possible to find the places an algorithm spend the most time. With this information a developer has the chance to find bottlenecks, which maybe could be eliminated. Beside that, is the knowledge of the call graph sometimes from great information, especially if you develop graphical applications, like in Qt. Having events, slots or asynchrony window systems, like in X11, make it sometimes hard to predict how the code will flow. It may also be helpful in deciding if some subroutine is a candidate for parallel execution. The following screenshot shows the result of analyzing some parts of VirtualBox. To visualize the output of Callgrind (and to interactively work with this data), KCachegrind is the number one tool. The second screenshot shows some similar profiling with Instruments from Xcode.

GUI tests

GUI's have the advantage to do several tasks in an independent order and users tend to do tasks in a way a developer don't expect. Developer tends to do tasks in the way they have programmed it. On the other side GUI testing tools do only things they are programmed for. To be honest, we don't do any automatic GUI testing in VirtualBox. It's such a wide field and doing it right is hard. Anyway, there are products out there which can be helpful. Qt itself have possibilities for GUI testing build in. Another product is from froglogic. Squish is specialized in GUI testing, especially for Qt applications. But of course there are other tools as well. Just have a look at your preferred build environment.

Conclusion

Testing produces work. It is also something developers don't like. On the other side it helps to prevent further work by bypassing errors which are preventable. Everybody should ask himself how much time he will sacrifice in setting up test machines or unit tests to make a product much more stable. Errors in freshly released products which don't happen in an older release are always bad marketing. If there is a chance to prevent them by just adding some lines of code, everyone should use this chance. This post doesn't show all available tools for testing. For example, the big players like Apple and Microsoft have also testing and analyzing software build into there IDE's. I just want to give a brief overview what is possible, so that everybody could check the own environment for possibilities of doing testing.

As a last word which touches my personal work environment: Please be assured that we are trying hard to make VirtualBox as stable and save as much as possible .

Chain of trust

To ensure the validity of every component in a computer system (hardware or software) a chain of trust is build. This basically means there is some root institution (in computer cryptography this is called Certificate Authority (CA)) which is trusted per se. Any following part in this hierarchy is signed by the parent authority. This allows a flexible mechanism where only the connected parties have to make sure they trust each other to make the full chain trustworthy. This concept is also used for code signing, cause it allows to be trustworthy in the eyes of Microsoft without ever being in touch with them. As Microsoft don't trust (for code signing) every root CA they have in their certificate store, they explicit allow only a handful of root CA's to be in this trust of chain. They archive this by cross signing root CA's with their own CA. The full concept is described in this article. It basically means the software maker certificate has to be trusted by an official CA included in every Windows version and this root CA has to be cross signed by Microsoft for code signing. This is the point where the problems start with the new VeriSign Primary CA.

Finding the little differences

The old code signing CA is the VeriSign Class 3 Public Primary CA, available since 1996. This certificate uses an 1024 bit key, which isn't considered save anymore in the future. Therefor VeriSign decided to replace this root CA with a stronger one, which uses an 2048 bit key. If you simply replace an old software maker certificate (signed with the old root CA) with one which is signed by the new CA you get a surprise. Installing a kernel mode driver signed with the new certificate ends up with a message like this:

A look into the security log of the event viewer shows this error message:

Code integrity determined that the image hash of a file is not valid. The
file could be corrupt due to unauthorized modification or the invalid hash
could indicate a potential disk device error.

To be honest: This information doesn't contain any useful hint. If you compare the old signed driver with the new signed driver you will not see any difference. Both could be successful verified as shown next:

Even if you compare all the sub-dialogs side by side you will not find any difference, beside the different root CA of course. So whats the difference? Well, you can't rely on the graphical representation of this trust of chain. When you invoke the signtool of the Windows Driver Kit (WDK) with the verify option, you will see the difference:

C:\Program Files\Oracle\VirtualBox_Sun>signtool.exe verify /v /kp drivers/vboxdrv/VBoxDrv.sys
 
Verifying: drivers/vboxdrv/VBoxDrv.sys
SHA1 hash of file: 9E58611C764D5AE04140E4CC7782B3229D1BCB8A
Signing Certificate Chain:
    Issued to: Microsoft Code Verification Root
    Issued by: Microsoft Code Verification Root
    Expires:   01.11.2025 14:54:03
    SHA1 hash: 8FBE4D070EF8AB1BCCAF2A9D5CCAE7282A2C66B3
 
        Issued to: Class 3 Public Primary Certification Authority
        Issued by: Microsoft Code Verification Root
        Expires:   23.05.2016 18:11:29
        SHA1 hash: 58455389CF1D0CD6A08E3CE216F65ADFF7A86408
 
            Issued to: VeriSign Class 3 Code Signing 2004 CA
            Issued by: Class 3 Public Primary Certification Authority
            Expires:   16.07.2014 00:59:59
            SHA1 hash: 197A4AEBDB25F0170079BB8C73CB2D655E0018A4
 
                Issued to: Sun Microsystems, Inc.
                Issued by: VeriSign Class 3 Code Signing 2004 CA
                Expires:   12.06.2011 00:59:59
                SHA1 hash: 1D4458051589B47A06260125F6EC6BBB6C24472E
 
The signature is timestamped: 08.02.2011 00:23:54
Timestamp Verified by:
    Issued to: Thawte Timestamping CA
    Issued by: Thawte Timestamping CA
    Expires:   01.01.2021 00:59:59
    SHA1 hash: BE36A4562FB2EE05DBB3D32323ADF445084ED656
 
        Issued to: VeriSign Time Stamping Services CA
        Issued by: Thawte Timestamping CA
        Expires:   04.12.2013 00:59:59
        SHA1 hash: F46AC0C6EFBB8C6A14F55F09E2D37DF4C0DE012D
 
            Issued to: VeriSign Time Stamping Services Signer - G2
            Issued by: VeriSign Time Stamping Services CA
            Expires:   15.06.2012 00:59:59
            SHA1 hash: ADA8AAA643FF7DC38DD40FA4C97AD559FF4846DE
 
Successfully verified: drivers/vboxdrv/VBoxDrv.sys
 
Number of files successfully Verified: 1
Number of warnings: 0
Number of errors: 0
 
C:\Program Files\Oracle\VirtualBox_Sun>

and

C:\Program Files\Oracle\VirtualBox_Oracle_Wrong\>signtool.exe verify /v /kp drivers/vboxdrv/VBoxDrv.sys
 
Verifying: drivers/vboxdrv/VBoxDrv.sys
SHA1 hash of file: F398B7124B0A8C32DBFB262343AC1180807505D0
Signing Certificate Chain:
    Issued to: VeriSign Class 3 Public Primary Certification Authority - G5
    Issued by: VeriSign Class 3 Public Primary Certification Authority - G5
    Expires:   17.07.2036 00:59:59
    SHA1 hash: 4EB6D578499B1CCF5F581EAD56BE3D9B6744A5E5
 
        Issued to: VeriSign Class 3 Code Signing 2010 CA
        Issued by: VeriSign Class 3 Public Primary Certification Authority - G5
        Expires:   08.02.2020 00:59:59
        SHA1 hash: 495847A93187CFB8C71F840CB7B41497AD95C64F
 
            Issued to: Oracle Corporation
            Issued by: VeriSign Class 3 Code Signing 2010 CA
            Expires:   08.02.2014 00:59:59
            SHA1 hash: A88FD9BDAA06BC0F3C491BA51E231BE35F8D1AD5
 
The signature is timestamped: 10.02.2011 09:30:08
Timestamp Verified by:
    Issued to: Thawte Timestamping CA
    Issued by: Thawte Timestamping CA
    Expires:   01.01.2021 00:59:59
    SHA1 hash: BE36A4562FB2EE05DBB3D32323ADF445084ED656
 
        Issued to: VeriSign Time Stamping Services CA
        Issued by: Thawte Timestamping CA
        Expires:   04.12.2013 00:59:59
        SHA1 hash: F46AC0C6EFBB8C6A14F55F09E2D37DF4C0DE012D
 
            Issued to: VeriSign Time Stamping Services Signer - G2
            Issued by: VeriSign Time Stamping Services CA
            Expires:   15.06.2012 00:59:59
            SHA1 hash: ADA8AAA643FF7DC38DD40FA4C97AD559FF4846DE
 
Successfully verified: drivers/vboxdrv/VBoxDrv.sys
 
Number of files successfully Verified: 1
Number of warnings: 0
Number of errors: 0
 
C:\Program Files\Oracle\VirtualBox_Oracle_Wrong>

As you can see, the chain of trust of the old certificate contains the Microsoft Code Verification Root, the new signed driver not. As Microsoft released the cross certificate somewhere in 2006, it makes sense that the new VeriSign certificate isn't signed by them. So first of all, we have to blame the signtool for silently ignoring a cross certificate which, obviously, doesn't trust the root certificate. When you have this information you can search for additional information and probably find an advisory of VeriSign. There you learn you need intermediate certificates for the new root CA.

Installing the right certificates on the build machine

The rest is easy. The certificate store of any Windows installation contains the new VeriSign Root CA as shown here:

Delete this root CA and replace it by the intermediate certificates you fetched from the website shown above. Just place the certificate in a text file, add the extension .der and double-click to install it. Make sure to replace really all versions of this certificate, even the one in the global store. When you now sign your driver with your new certificate the Microsoft Code Verification Root is in the trust of chain, as shown in the following:

C:\Program Files\Oracle\VirtualBox_Oracle_Correct>signtool.exe verify /v /kp drivers/vboxdrv/VBoxDrv.sys
 
Verifying: drivers/vboxdrv/VBoxDrv.sys
SHA1 hash of file: 201B7F97473D7F015A104D7841371C5AE4F22FF2
Signing Certificate Chain:
    Issued to: Microsoft Code Verification Root
    Issued by: Microsoft Code Verification Root
    Expires:   01.11.2025 14:54:03
    SHA1 hash: 8FBE4D070EF8AB1BCCAF2A9D5CCAE7282A2C66B3
 
        Issued to: Class 3 Public Primary Certification Authority
        Issued by: Microsoft Code Verification Root
        Expires:   23.05.2016 18:11:29
        SHA1 hash: 58455389CF1D0CD6A08E3CE216F65ADFF7A86408
 
            Issued to: VeriSign Class 3 Public Primary Certification Authority - G5
            Issued by: Class 3 Public Primary Certification Authority
            Expires:   08.11.2021 00:59:59
            SHA1 hash: 32F30882622B87CF8856C63DB873DF0853B4DD27
 
                Issued to: VeriSign Class 3 Code Signing 2010 CA
                Issued by: VeriSign Class 3 Public Primary Certification Authority - G5
                Expires:   08.02.2020 00:59:59
                SHA1 hash: 495847A93187CFB8C71F840CB7B41497AD95C64F
 
                    Issued to: Oracle Corporation
                    Issued by: VeriSign Class 3 Code Signing 2010 CA
                    Expires:   08.02.2014 00:59:59
                    SHA1 hash: A88FD9BDAA06BC0F3C491BA51E231BE35F8D1AD5
 
The signature is timestamped: 10.02.2011 15:03:30
Timestamp Verified by:
    Issued to: Thawte Timestamping CA
    Issued by: Thawte Timestamping CA
    Expires:   01.01.2021 00:59:59
    SHA1 hash: BE36A4562FB2EE05DBB3D32323ADF445084ED656
 
        Issued to: VeriSign Time Stamping Services CA
        Issued by: Thawte Timestamping CA
        Expires:   04.12.2013 00:59:59
        SHA1 hash: F46AC0C6EFBB8C6A14F55F09E2D37DF4C0DE012D
 
            Issued to: VeriSign Time Stamping Services Signer - G2
            Issued by: VeriSign Time Stamping Services CA
            Expires:   15.06.2012 00:59:59
            SHA1 hash: ADA8AAA643FF7DC38DD40FA4C97AD559FF4846DE
 
Successfully verified: drivers/vboxdrv/VBoxDrv.sys
 
Number of files successfully Verified: 1
Number of warnings: 0
Number of errors: 0
 
C:\Program Files\Oracle\VirtualBox_Oracle_Correct>

You see the old certificate (which is trusted by Microsoft) is the parent of the new certificate, which completes the trust of chain again. I guess this is only some temporary solution as long as Microsoft doesn't release a new cross certificate (that's why it is called intermediate). Luckily, you only need the intermediate certificates on the build machine. For your end users nothing has to be changed.

Conclusion

This article shows how a service provider make an easy task hard to do. Signing a kernel mode driver with the new certificate isn't hard, but finding the right information is. Although there is an advisory from VeriSign, it doesn't really explain what to do. As I believe in the future many other people will be in the same situation, I hope this article will save them from some sleepless nights.

Getting the superview

In Cocoa all user displayed content is a NSView (most of the time; forget about the Dock). This is really nice, cause you could manipulate them. Although there are no public methods for getting the NSView of the toolbar or even the titlebar, they exist as an NSView. The NSView of the toolbar could be accessed by a private method. As always in Objective C you could ask for a particular method by using the respondsToSelector statement like in the following:

NSToolbar *tb = [pWindow toolbar];
if ([tb respondsToSelector:@selector(_toolbarView)])
{
 NSView *tbv = [tb performSelector:@selector(_toolbarView)];
 if (tbv)
 {
  /* do something with tbv */
 }
}

This return the NSView of the toolbar at least until 10.6. But be warned this is a private method and of course this could be changed in a future version of Mac OS X. At least the shown method will correctly fail in the case Apple change his mind which will result in doing nothing. Anyway this will not include the area of the titlebar. To get the NSView which covers both the title bar and the toolbar you need another trick. The titlebar usual has a close, minimize or maximize button. These buttons are NSButton's and added by the system depending of the window type. As a NSButton is also a NSView it also has a parent. This dependency allows us to access the view which is responsible for displaying the unified titlebar and the toolbar. The following code shows how to get the responsible view and how to add an additional NSImageView to it:

NSView *wv = [[pWindow standardWindowButton:NSWindowCloseButton] superview];
if (wv)
{
 /* We have to calculate the size of the title bar for the center case. */
 NSSize s = [pImage size];
 NSSize s1 = [wv frame].size;
 NSSize s2 = [[pWindow contentView] frame].size;
 /* Correctly position the label. */
 NSImageView *iv = [[NSImageView alloc] initWithFrame:
        NSMakeRect(s1.width - s.width - (fCenter ? 10 : 0),
                   fCenter ? s2.height + (s1.height - s2.height - s.height) / 2 : s1.height - s.height - 1,
  	           s.width, s.height)];
 /* Configure the NSImageView for auto moving. */
 [iv setImage:pImage];
 [iv setAutoresizesSubviews:true];
 [iv setAutoresizingMask:NSViewMinXMargin | NSViewMinYMargin];
 /* Add it to the parent of the close button. */
 [wv addSubview:iv positioned:NSWindowBelow relativeTo:nil];
}

This code needs a NSWindow, a NSImage as label and the flag fCenter for deciding if the image is vertically centered within the titlebar. Also the badge is pinned on the right side by setting the AutoresizingMask. The following shows how this could be look like:

Adding functionality to the badge

First we should add some information about this release by using the setToolTip method of an NSView. Simply add this call to the code:

 [iv setToolTip:@"Some info about the beta version."];

To make this beta hint more useful, we should make it clickable. Getting mouse down events in Cocoa is only possible by handling the mouseDown event of the NSResponder sub-class. Also the view must accept first responder events. How to do this is shown in the following code:

@interface BetaImageView: NSImageView
{}
- (BOOL)acceptsFirstResponder;
- (void)mouseDown:(NSEvent *)pEvent;
@end
@implementation BetaImageView
- (BOOL)acceptsFirstResponder
{
 return YES;
}
- (void)mouseDown:(NSEvent *)pEvent
{
 if ([pEvent clickCount] > 1)
  [[NSWorkspace sharedWorkspace] openURL:
        [NSURL URLWithString:@"http://www.virtualbox.org/"]];
 else
  [super mouseDown:pEvent];
}
@end

Replace NSImageView with BetaViewImage in the allocation call in listing 2 and you are done. Change the URL to one where the user can respond to the beta questions, like a bug tracker or forum.

Conclusion

This post showed how to add an nice looking badge to the unified toolbar on Mac OS X. The user is reminded he is using some pre-release software all the time, but at the same time has an easy way to report problems with this release. Of course could such a badge used for anything else.

Why do you want to use file shortcuts

On the classical UNIX systems you have hard and soft links. These are implemented by the filesystem and make it possible to link to another file or folder without any trouble. Most of the time soft links are used, but it really depends on the use case. Unfortunately these kind of links are not available on Windows (yes, I know there are also hard links and junctions on NTFS, but they are not common and difficult to handle), these links doesn't allow any additional attributes. For example one like to add a different icon to the link or provide more information through a comment field. Beside on Mac OS X, shortcuts can also be work as an application launcher, where the link contain the information what application should be started and how. In contrast to filesystem links which are handled by the operation system, these shortcuts are handled by the window system (or shell) running on the host (which doesn't mean there is no filesystem support for it). On Windows this is the Explorer, on Mac OS X the Finder and on Linux a freedesktop.org conforming file manager.

Creating a Desktop file on Linux

Desktop files on Linux (or any other UNIX system which conforms to freedesktop.org) is easy. It's a simple text file which implement the Desktop Entry Specification. In version 1.0 there are 18 possible entries, where not all of them are mandatory. In the following example I use Qt to write these files, but it should be no problem to use any other toolkit or plain C.

bool createShortcut(const QString &strSrcFile,
                    const QString &strDstPath,
                    const QString &strName)
{
 QFile link(strDstPath + QDir::separator() + strName + ".desktop");
 if (link.open(QFile::WriteOnly | QFile::Truncate))
 {
  QTextStream out(&link);
  out.setCodec("UTF-8");
  out << "[Desktop Entry]" << endl
      << "Encoding=UTF-8" << endl
      << "Version=1.0" << endl
      << "Type=Link" << endl
      << "Name=" << strName << endl
      << "URL=" << strSrcFile << endl
      << "Icon=icon-name" << endl;
  return true;
 }
 return false;
}

Replace icon-name by a registered icon on the system and you are done.

Creating a Shell link on Windows

Windows provides an interface for IShellLink since Windows XP. The following example shows how to use it:

bool createShortcut(LPCSTR lpszSrcFile,
                    LPCSTR lpszDstPath,
                    LPCSTR lpszName)
{
 IShellLink *pShl = NULL;
 IPersistFile *pPPF = NULL;
 HRESULT rc = CoCreateInstance(CLSID_ShellLink,
                               NULL,
                               CLSCTX_INPROC_SERVER,
                               IID_IShellLink,
                               (void**)(&pShl));
 if (FAILED(rc))
  return false;
 do
 {
  rc = pShl->SetPath(lpszSrcFile);
  if (FAILED(rc))
   break;
  rc = pShl->QueryInterface(IID_IPersistFile, (void**)&pPPF);
  if (FAILED(rc))
   break;
  WORD wsz[MAX_PATH];
  TCHAR path[MAX_PATH] = { 0 };
  lstrcat(path, lpszDstPath);
  lstrcat(path, "\\");
  lstrcat(path, lpszName);
  lstrcat(path, ".lnk");
  MultiByteToWideChar(CP_ACP, 0, buf, -1, wsz, MAX_PATH);
  rc = pPPF->Save(wsz, TRUE);
 } while(0);
 if (pPPF)
  pPPF->Release();
 if (pShl)
  pShl->Release();
 return SUCCEEDED(rc);
}

As you may noticed this uses COM. Many API's on Windows using the COM interface to communicate between processes. If you don't use COM in your application you have to initialize it first. This is achieved by adding the following call to the front of the function:

 if (FAILED(CoInitialize(NULL))
  return false;

Depending on your application it might be worth to unitialize COM after usage by appending the following to the function:

 CoUninitialize();

The function itself isn't any magic. It gets a COM interface to the IShellLink interface and then work with it, by setting the source path and adding a target path by using the IPersistFile interface. As I wrote before you could do much more. Providing a path to a specific application or adding your own parameters is no problem. Have a look at the documentation.

Creating an Alias file on Mac OS X

Shortcut files on Mac OS X are a little bit different. At first, they aren't one. There are the classical filesystem links and Alias files. Alias files are links which targeting a specific file, but they haven't all the possibilities of shortcuts like on Windows or Linux. As the name suggest they are really only an alias for another file or directory. So specifying an application to start or things like that aren't possible. Anyway they allow changing the icon and they are more persistent than on Window or Linux cause they are working with several attributes of the target file. Even if you rename or move the target, an Alias file will resolve the target correctly (if it is possible). On the other side, being such special means also being hard to create. In principle there are two possibilities. The first one is, creating a file which is no file at all, but has several resources forks attached. Therefor you need to know exactly how Alias files are built of and make sure with every release of Mac OS X you are following the development. There is a free project which does exactly that: NDAlias. If you are like me and a little bit more lazy, you ask someone who should know how to create Alias files. This is Finder. Although writing the files itself isn't easy, asking the Finder to do the job is not really easier, cause the information about doing exactly that are really rare. The following code shows how to achieve it:

bool createShortcut(NSString *pstrSrcFile,
                    NSString *pstrDstPath,
                    NSString *pstrName)
{
 /* First of all we need to figure out which process Id the Finder
  * currently has. */
 NSWorkspace *pWS = [NSWorkspace sharedWorkspace];
 NSArray *pApps = [pWS launchedApplications];
 bool fFFound = false;
 ProcessSerialNumber psn;
 for (NSDictionary *pDict in pApps)
 {
  if ([[pDict valueForKey:@"NSApplicationBundleIdentifier"]
         isEqualToString:@"com.apple.finder"])
  {
   psn.highLongOfPSN = [[pDict
                          valueForKey:@"NSApplicationProcessSerialNumberHigh"] intValue];
   psn.lowLongOfPSN  = [[pDict
                          valueForKey:@"NSApplicationProcessSerialNumberLow"] intValue];
   fFFound = true;
   break;
  }
 }
 if (!fFFound)
  return false;
 /* Now the event fun begins. */
 OSErr err = noErr;
 AliasHandle hSrcAlias = 0;
 AliasHandle hDstAlias = 0;
 do
 {
  /* Create a descriptor which contains the target psn. */
  NSAppleEventDescriptor *finderPSNDesc = [NSAppleEventDescriptor
                                            descriptorWithDescriptorType:typeProcessSerialNumber
                                            bytes:&psn
                                            length:sizeof(psn)];
  if (!finderPSNDesc)
   break;
  /* Create the Apple event descriptor which points to the Finder
   * target already. */
  NSAppleEventDescriptor *finderEventDesc = [NSAppleEventDescriptor
                                              appleEventWithEventClass:kAECoreSuite
                                              eventID:kAECreateElement
                                              argetDescriptor:finderPSNDesc
                                              returnID:kAutoGenerateReturnID
                                              transactionID:kAnyTransactionID];
  if (!finderEventDesc)
   break;
  /* Create and add an event type descriptor: Alias */
  NSAppleEventDescriptor *osTypeDesc = [NSAppleEventDescriptor descriptorWithTypeCode:typeAlias];
  if (!osTypeDesc)
   break;
  [finderEventDesc setParamDescriptor:osTypeDesc forKeyword:keyAEObjectClass];
  /* Now create the source Alias, which will be attached to the event. */
  err = FSNewAliasFromPath(nil, [pstrSrcFile fileSystemRepresentation], 0, &hSrcAlias, 0);
  if (err != noErr)
   break;
  char handleState;
  handleState = HGetState((Handle)hSrcAlias);
  HLock((Handle)hSrcAlias);
  NSAppleEventDescriptor *srcAliasDesc = [NSAppleEventDescriptor
                                           descriptorWithDescriptorType:typeAlias
                                           bytes:*hSrcAlias
                                           length:GetAliasSize(hSrcAlias)];
  if (!srcAliasDesc)
   break;
  [finderEventDesc setParamDescriptor:srcAliasDesc
    forKeyword:keyASPrepositionTo];
  HSetState((Handle)hSrcAlias, handleState);
  /* Next create the target Alias and attach it to the event. */
  err = FSNewAliasFromPath(nil, [pstrDstPath fileSystemRepresentation], 0, &hDstAlias, 0);
  if (err != noErr)
   break;
  handleState = HGetState((Handle)hDstAlias);
  HLock((Handle)hDstAlias);
  NSAppleEventDescriptor *dstAliasDesc = [NSAppleEventDescriptor
                                           descriptorWithDescriptorType:t ypeAlias
                                           bytes:*hDstAlias
                                           length:GetAliasSize(hDstAlias)];
  if (!dstAliasDesc)
   break;
  [finderEventDesc setParamDescriptor:dstAliasDesc
    forKeyword:keyAEInsertHere];
  HSetState((Handle)hDstAlias, handleState);
  /* Finally a property descriptor containing the target
   * Alias name. */
  NSAppleEventDescriptor *finderPropDesc = [NSAppleEventDescriptor recordDescriptor];
  if (!finderPropDesc)
   break;
  [finderPropDesc setDescriptor:[NSAppleEventDescriptor descriptorWithString:pstrName]
    forKeyword:keyAEName];
  [finderEventDesc setParamDescriptor:finderPropDesc forKeyword:keyAEPropData];
  /* Now send the event to the Finder. */
  err = AESend([finderEventDesc aeDesc],
               NULL,
               kAENoReply,
               kAENormalPriority,
               kNoTimeOut,
               0,
               nil);
 } while(0);
 /* Cleanup */
 if (hSrcAlias)
  DisposeHandle((Handle)hSrcAlias);
 if (hDstAlias)
  DisposeHandle((Handle)hDstAlias);
 return err == noErr ? true : false;
}

Although the code above looks a little bit scary, it does not much. It fetch the process serial number of the current Finder process, creates an Application event for creating an Alias file and send this event to the Finder.

Conclusion

Beside showing how to create file shortcuts on different platforms, this article also shows which work is necessary to create platform independent code. It's a simple example. But it also makes clear that one simple solution for platform one, not necessarily mean it's such simple on platform two.

Making this easy accessible to any developer is the next step. I will leave this exercise to the reader, but have a look at the platform code of the VirtualBox GUI and the corresponding Makefile.

Providing the necessary information to Mac OS X

Applications on Mac OS X need some defined structure. They are so-called bundles, which means on the filesystem layer they are directories. You can prove this by checking the Applications directory within the Terminal.app. You could also right-click on an application and select "Show Package Contents". The content of the bundle directory is usually hidden from the user when he works with the Finder or any other high level function of Mac OS X (like the open dialog). Additional to this layout on the filesystem, an application provide information about itself to the system with a plist file. This file has to be named Application.app/Content/Info.plist. The following shows exemplary the content of the Tunnelblick application:

<xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>CFBundleDevelopmentRegion</key>     <string>English</string>
 <key>CFBundleExecutable</key>            <string>Tunnelblick</string>
 <key>CFBundleIconFile</key>              <string>tunnelblick.icns</string>
 <key>CFBundleIdentifier</key>            <string>com.openvpn.tunnelblick</string>
 <key>CFBundleInfoDictionaryVersion</key> <string>6.0</string>
 <key>CFBundlePackageType</key>           <string>APPL</string>
 <key>CFBundleShortVersionString</key>    <string>3.1beta20 (build 2132)</string>
 <key>CFBundleSignature</key>             <string>OVPN</string>
 <key>CFBundleVersion</key>               <string>2132</string>
 <key>NSMainNibFile</key>                 <string>MainMenu</string>
 <key>NSPrincipalClass</key>              <string>NSApplication</string>
 <key>NSUIElement</key>                   <string>1</string>
 <key>SUEnableSystemProfiling</key>       <true/>
 <key>SUFeedURL</key>                     <string>http://tunnelblick.net/appcast.rss</string>
 <key>SUPublicDSAKeyFile</key>            <string>dsa_pub.pem</string>
</dict>
</plist>

The keys are mostly self explaining, you can find a full list here.

To register a file type association you have to add an array of the type CFBundleDocumentTypes. Again, here is an extraction of the Tunnelblick application, which shows the registration of the tblk extension:

 <key>CFBundleDocumentTypes</key>
 <array>
  <dict>
   <key>CFBundleTypeExtensions</key>   <array><string>tblk</string></array>
   <key>CFBundleTypeIconFile</key>     <string>tunnelblick_package.icns</string>
   <key>CFBundleTypeName</key>         <string>Tunnelblick VPN Configuration</string>
   <key>CFBundleTypeRole</key>         <string>Editor</string>
   <key>LSTypeIsPackage</key>          <true/>
   <key>NSPersistentStoreTypeKey</key> <string>Binary</string>
  </dict>
 </array>

You have to provide a file extension or mime-type, can add an icon and give a hint what your application can do with this type of file (viewer, editor or nothing).

Beside this passive way of announcing this information, there is also an active way (e.g. for use in an installer). See here for further information.

Where is this information stored

Although Mac OS X hasn't a registry like windows, some information are stored in global databases, too. Applications are registered at the launchd. The launchd is the central place for starting all kind of programs, from a background service to any common application, like Thunderbird. E.g. background services can register itself and even made their start depending on different events. VirtualBox has an example configuration included, which let launchd start the vbox webservice on activity on a certain port. Although this isn't used by many users out there (and therefore disabled by default), it shows some generic usage of this functionality.

If you look around in Mac OS X you have the tool launchctl, which allows you to start jobs or register background services with launchd. But it seems there is no tool which is able to get some information about the knowledge of launchd. Well, there is one. It's a little bit hidden in /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister. To be more comfortable with it, we should include it into the user path by linking it to a known path, like this:

sudo ln -s /Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister /usr/local/bin/lsregister

With this tiny tool we are able to dump the launchd database. lsregister -dump | grep -n10 Tunnelblick displays something like this for Tunnelblick:

10823---------------------------------------------------------------------------------
10824-bundle    id:            6984
10825:  path:          /Applications/Tunnelblick.app
10826-  name:
10827-  identifier:    com.openvpn.tunnelblick (0x800097b8)
10828-  version:       2132
10829-  mod date:      10/31/2010 12:37:08
10830-  reg date:      11/19/2010 11:13:39
10831-  type code:     'APPL'
10832-  creator code:  'OVPN'
10833-  sys version:   0
10834-  flags:         relative-icon-path  ui-element
10835-  item flags:    container  package  application  extension-hidden  native-app  ppc  i386
10836-  icon:          Contents/Resources/tunnelblick.icns
10837:  executable:    Contents/MacOS/Tunnelblick
10838-  inode:         3800129
10839-  exec inode:    3800213
10840-  container id:  32
10841-  library:
10842-  library items:
10843-  --------------------------------------------------------
10844-  claim   id:            17864
10845:          name:          Tunnelblick VPN Configuration
10846-          rank:          Default
10847-          roles:         Editor
10848-          flags:         relative-icon-path  package
10849-          icon:          Contents/Resources/tunnelblick_package.icns
10850-          bindings:      .tblk
10851---------------------------------------------------------------------------------

As you see, all the information provided by the plist file is registered. Beside other information, it also has the registration and modification times stored. To let launchd reread the plist file, usual its only necessary to change the modification time of the application. This could be done by simply executing touch /Applications/Application.app. Anyway, it's sometimes necessary to reset the content of this database for the own application. This could be done by executing the following:

lsregister -u /Applications/Application.app

To register the application again, just remove the -u parameter. You should also check if your application isn't registered more than once, with different paths, by using the -dump parameter. To fully reset the launchd database, you could use -kill. This will remove any file type association and registered application. You have been warned.

Conclusion

This article shows how Mac OS X handle informations about installed applications. With this knowledge a developer is able to register the own application into the launchd ecosystem and see how launchd interpret this information. Furthermore, the usage of lsregister allows a developer to analyze the content of the launchd database and make changes to it.

Let the kernel manage the graphical modes

As most people are working under X11 they don't see the output of an kernel panic. When a kernel panic happens the kernel prints the reason for the panic and a kernel backtrace to the console window and stops immediately its own execution. It is not written into a log file or somewhere else. In consequence you don't have the ability to look into the panic text, cause the graphical mode is still on. Historically the mode settings are done by the graphic driver of the X11 system. So the kernel has no idea that or which graphic mode is currently in use. Fortunately the kernel hackers invented a new infrastructure which let the kernel do the mode switch. This subsystem is called Kernel-Mode-Settings (KMS). As the kernel do the mode settings, he can switch back to the console on a panic, regardless which graphical mode is currently configured. Beside this, KMS has other improvements like Fast User Switching or a flicker free switch between text and graphic mode. On the other side is this highly hardware dependent and even if it was introduced with version 2.6.28, not all today available hardware can make use of it. If you are an owner of an Intel graphic card you are in good shape. Radeon and NVidia cards have limited support through the in kernel drivers radeonhd and nouveau. For an Intel i915 card you need to enable the following kernel options:

CONFIG_DRM_I915=y
Location:
-> Device Drivers
-> Graphics support
-> Direct Rendering Manager (XFree86 4.1.0 and higher DRI support) (DRM [=y])
-> Intel 830M, 845G, 852GM, 855GM, 865G ( [=y])
 
CONFIG_DRM_I915_KMS=y
Location:
-> Device Drivers
-> Graphics support
-> Direct Rendering Manager (XFree86 4.1.0 and higher DRI support) (DRM [=y])
-> Intel 830M, 845G, 852GM, 855GM, 865G ( [=y])
-> i915 driver (DRM_I915 [=y])

The kernel line in your favorite boot loader needs the following additional parameter:

i915.modeset=1

X11 should have this minimal configuration for the device section:

Section "Device"
 Identifier    "i915"
 Driver        "intel"
 Option        "DRI"   "true"
EndSection

Please note that you need of course some recent kernel, X11 version and Intel X11 driver to make this work. After a compile, install and boot of the new kernel, KMS should be in use. You will notice it, cause the boot messages will be printed in a much higher graphical resolution, than the usual text mode provide. The next time a kernel panic occurs, the kernel will switch back to the console before the panic is printed. This allows you to see the info printed and maybe you get a useful hint for the reason of the panic.

Post the panic

If you can't use KMS or don't want transcribe the panic text by hand into the bugtracker, it would be nice if the text could be made available on another computer. Kernel hackers usual use the serial port for that. Unfortunately most modern computers doesn't have such a serial port anymore. Also you need two hosts with a serial port and the setup is complex (you have to know about baud-rates, parity and stuff like this). But there is a simpler solution: netconsole. Netconsole is a kernel module, which sends kernel messages anywhere to the net using UDP. The setup is really simple. In the kernel configuration you need the following setting:

CONFIG_NETCONSOLE=m
Location:
-> Device Drivers
-> Network device support (NETDEVICES [=y])

I prefer to compile it as module, which allows me to turn it on only when I need it. Load it with the following command:

modprobe netconsole netconsole=@/,@192.168.220.10/

The ip has to be replaced by the one of your target computer. You can of course tune it much more, like setting source and target ports or even let netconsole send the text to more than one host. On your client you need a network tool which can read from a socket and print the read text to stdout. Netcat or nc are two tools which are able to do just that. The call for nc looks like the following:

nc -l -u 6666

Now if a kernel panic will happen you will see an output like this:

BUG: unable to handle kernel NULL pointer dereference at (null)
IP: [] rb_erase+0x15c/0x320
PGD 6942f067 PUD a1e4067 PMD 0
Oops: 0000 [#1] SMP
last sysfs file: /sys/devices/virtual/block/md1/dev
CPU 3
Modules linked in: vboxnetadp vboxnetflt vboxdrv netconsole ...
 
Pid: 18887, comm: VirtualBox Tainted: G        W   2.6.36-gentoo #4 DG33TL/
RIP: 0010:[]  [] rb_erase+0x15c/0x320
RSP: 0018:ffff8800b430db58  EFLAGS: 00010046
RAX: 0000000000000000 RBX: ffff880069557a68 RCX: 0000000000000001
RDX: ffff880069557a68 RSI: ffff880001d8ed58 RDI: 0000000000000000
RBP: ffff8800b430db68 R08: 0000000000000001 R09: 000000008edcb5d6
R10: 0000000000000000 R11: 0000000000000202 R12: ffff880001d8ed58
R13: 0000000000000000 R14: 000000000000ed00 R15: 0000000000000002
FS:  00007fffde457710(0000) GS:ffff880001d80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 00000000064f9000 CR4: 00000000000026e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process VirtualBox (pid: 18887, threadinfo ffff8800b430c000, task ffff880091e227f0)
Stack:
 ffff88000a03ba18 ffff880001d8ed48 ffff8800b430dba8 ffffffff8105bf06
<0> ffff8800b430dba8 ffffffff8105c97c ffff8800b430dbc8 ffff88000a03ba18
<0> 00004ff8a86ba455 ffff880001d8ed48 ffff8800b430dc48 ffffffff8105ce77
Call Trace:
 [] __remove_hrtimer+0x36/0xb0
 [] ? lock_hrtimer_base+0x2c/0x60
 [] __hrtimer_start_range_ns+0x2b7/0x3c0
 [] ? rtR0SemEventMultiLnxWait+0x250/0x3d0 [vboxdrv]
 [] ? RTLogLoggerExV+0x12f/0x180 [vboxdrv]
 [] hrtimer_start+0x13/0x20
 [] rtTimerLnxStartSubTimer+0x60/0x120 [vboxdrv]
 [] rtTimerLnxStartOnSpecificCpu+0x21/0x30 [vboxdrv]
 [] rtmpLinuxWrapper+0x23/0x30 [vboxdrv]
 [] RTMpOnSpecific+0x99/0xa0 [vboxdrv]
 [] ? rtTimerLnxStartOnSpecificCpu+0x0/0x30 [vboxdrv]
 [] RTTimerStart+0x2a6/0x2e0 [vboxdrv]
 [] ? g_abExecMemory+0x33665/0x180000 [vboxdrv]
 [] g_abExecMemory+0xc678/0x180000 [vboxdrv]
 [] g_abExecMemory+0x328d7/0x180000 [vboxdrv]
 [] supdrvIOCtlFast+0x6a/0x70 [vboxdrv]
 [] VBoxDrvLinuxIOCtl+0x47/0x1e0 [vboxdrv]
 [] ? pick_next_task_fair+0xde/0x150
 [] do_vfs_ioctl+0xa1/0x590
 [] ? sys_futex+0x76/0x170
 [] sys_ioctl+0x4a/0x80
 [] system_call_fastpath+0x16/0x1b
Code: 07 a8 01 75 9d eb 81 0f 1f 84 00 00 00 00 00 48 3b 78 10 0f 84 ...
RIP  [] rb_erase+0x15c/0x320
 RSP
CR2: 0000000000000000
---[ end trace 4eaa2a86a8e2da24 ]---

Normally only kernel panics are sent to the console. You can increase the verbosity level by executing dmesg -n 8 as root.

Conclusion

To continue with the story from the beginning: With the shown methods you can hope your colleague get enough information to find the reason for the kernel panic. To be more helpful, the next step would be to try to debug the problem yourself. Even if the KGDB was merged into the kernel in version 2.6.35, it is not really usable for me. The reason is that it seems kernel hackers usually have really old hardware which either has a serial port, a PS/2 keyboard or both. Otherwise I can't find a reason why USB keyboards don't work. I asked on the mailing list of KGDB about the status of USB keyboard support and I can only hope support will be integrated soon.

In the following post, I will show how to suppress such unwanted error messages to make it much more easier to analyze the output of Valgrind for your own application.

Installing Valgrind

On Mac OS X you can use MacPorts to install Valgrind. You have to use valgrind-devel if you are on Snow Leopard, because Snow Leopard is supported in the current development version only. It's as simply as typing sudo port install valgrind-devel.

On Gentoo it can become a bit harder. The current stable version is 3.5 (like in MacPorts). If you try this version (at least on an unstable Gentoo like mine) with valgrind ls, you will get the following error:

valgrind:  Fatal error at startup: a function redirection
valgrind:  which is mandatory for this platform-tool combination
valgrind:  cannot be set up.  Details of the redirection are:
valgrind:
valgrind:  A must-be-redirected function
valgrind:  whose name matches the pattern:      strlen
valgrind:  in an object with soname matching:   ld-linux-x86-64.so.2
valgrind:  was not found whilst processing
valgrind:  symbols from the object with soname: ld-linux-x86-64.so.2
valgrind:
valgrind:  Possible fixes: (1, short term): install glibc's debuginfo
valgrind:  package on this machine.  (2, longer term): ask the packagers
valgrind:  for your Linux distribution to please in future ship a non-
valgrind:  stripped ld.so (or whatever the dynamic linker .so is called)
valgrind:  that exports the above-named function using the standard
valgrind:  calling conventions for this platform.
valgrind:
valgrind:  Cannot continue -- exiting now.  Sorry.

The reason is a striped glibc. To work properly, Valgrind needs to overwrite some of the system functions the glibc provide. It does this by getting the symbols by name out of this library. This is of course not possible if all the symbol names are removed. You can prove this by executing nm /lib/ld-linux-x86-64.so.2. Gentoo provides a FEATURE=splitdebug, which adds debug libraries to the installation. Unfortunately setting this feature in /etc/make.conf, means setting it global. Gentoo is known as being configurable as no other distribution out there and of course we can set a feature for one program only. To do so, create a file called glibc in /etc/portage/env/sys-libs/ and add the following content to it.

FEATURES="splitdebug"

After a rebuild of glibc by executing emerge --oneshot glibc, we have a working Valgrind.

As all programs, Valgrind isn't perfect. Version 3.5 shows many false/positive hits on my system, but fortunately the development goes on. Currently there is no newer version available in the Gentoo tree. Anyway it is not necessary to build one yourself, to get a more recent version. Using layman and the overlay tree of Flameeyes will let you integrate the development version of Valgrind seamlessly into your system. For a general How-to of layman check out this Users' guide. In short, something like the following should be sufficient:

layman -a flameeyes-overlay
layman -s flameeyes-overlay
echo "=dev-util/valgrind-9999 **" >> /etc/portage/package.keywords
emerge valgrind

Installing the development version of Valgrind is optional of course.

Know your tools

One usage of Valgrind could be look like this:

valgrind --leak-check=full --leak-resolution=high ./VirtualBox

Beside other errors it also shows this error message on my system:

==27174==    at 0x4C26C09: memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==27174==    by 0x4C26CB9: posix_memalign (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==27174==    by 0xBA8967F: ??? (in /usr/lib64/libglib-2.0.so.0.2400.2)
==27174==    by 0xBA89E9D: g_slice_alloc (in /usr/lib64/libglib-2.0.so.0.2400.2)
==27174==    by 0xBA89F86: g_slice_alloc0 (in /usr/lib64/libglib-2.0.so.0.2400.2)
==27174==    by 0xC204847: g_type_create_instance (in /usr/lib64/libgobject-2.0.so.0.2400.2)
==27174==    by 0xC1EB8A5: ??? (in /usr/lib64/libgobject-2.0.so.0.2400.2)
==27174==    by 0xC1ECE5D: g_object_newv (in /usr/lib64/libgobject-2.0.so.0.2400.2)
==27174==    by 0xC1ED494: g_object_new (in /usr/lib64/libgobject-2.0.so.0.2400.2)
==27174==    by 0x72A495F: ??? (in /usr/lib64/qt4/libQtGui.so.4.6.3)
==27174==    by 0x72A0D4F: ??? (in /usr/lib64/qt4/libQtGui.so.4.6.3)
==27174==    by 0x7289264: QGtkStyle::QGtkStyle() (in /usr/lib64/qt4/libQtGui.so.4.6.3)
==27174==    by 0x7215DB6: QStyleFactory::create(QString const&) (in /usr/lib64/qt4/libQtGui.so.4.6.3)
==27174==    by 0x6F5B7FC: QApplication::style() (in /usr/lib64/qt4/libQtGui.so.4.6.3)
==27174==    by 0x6F61DFF: QApplicationPrivate::initialize() (in /usr/lib64/qt4/libQtGui.so.4.6.3)
==27174==    by 0x6F61E88: QApplicationPrivate::construct(_XDisplay*, unsigned long, unsigned long) (in /usr/lib64/qt4/libQtGui.so.4.6.3)
==27174==    by 0x6F61FF3: QApplication::QApplication(_XDisplay*, int&, char**, unsigned long, unsigned long, int) (in /usr/lib64/qt4/libQtGui.so.4.6.3)
==27174==    by 0x44BC38: TrustedMain (main.cpp:371)
==27174==    by 0x44C649: main (main.cpp:651)

If you analyze the backtrace, you see that something in libQtGui is leaking memory. I don't want to blame someone for it or make a statement if this is right or wrong, I just want to get rid of it, to be able to easily spot errors VirtualBox itself produce. To do so, add --gen-suppressions=all to the Valgrind call. This will produce something similar like this:

{
 Memcheck:Leak
 fun:memalign
 fun:posix_memalign
 obj:/usr/lib64/libglib-2.0.so.0.2400.2
 fun:g_slice_alloc
 fun:g_slice_alloc0
 fun:g_type_create_instance
 obj:/usr/lib64/libgobject-2.0.so.0.2400.2
 fun:g_object_newv
 fun:g_object_new
 obj:/usr/lib64/qt4/libQtGui.so.4.6.3
 obj:/usr/lib64/qt4/libQtGui.so.4.6.3
 fun:_ZN9QGtkStyleC1Ev
 fun:_ZN13QStyleFactory6createERK7QString
 fun:_ZN12QApplication5styleEv
 fun:_ZN19QApplicationPrivate10initializeEv
 fun:_ZN19QApplicationPrivate9constructEP9_XDisplaymm
 fun:_ZN12QApplicationC1EP9_XDisplayRiPPcmmi
 fun:TrustedMain
 fun:main
}

To let Valgrind ignore this error in the future, copy the text into a file vbox.supp and start Valgrind with --suppressions=vbox.supp. Viola, this specific error isn't shown anymore. The format used there is easy to understand and you can of course tweak this much more. E.g. you could replace some of the fun: entries by "...". This is a placeholder for one or more functions calls with any name. Beside making suppression rules more general you can of course add as much as you like. Adding a name at the top make it easy to identify the different rules. For all the possibilities have a look at the documentation. Just for the curious, Valgrind is using such a file itself. Have a look at /usr/lib/valgrind/default.supp. You may also have noted that the function names in the normal error message differ from the one in the suppression list. The former is in the demangled form and the later in the saved form. You could force Valgrind to print mangled function names by adding the --demangle=no parameter to the call. This becomes handy if you manually create suppression lists.

Conclusion

By using suppression rules for the own application, unimportant errors could be eliminated in the output of Valgrind. With this is in mind there is no excuse anymore for memory leaks in the self developed applications. Beside memory leaks, Valgrind also finds places where uninitialized variables are in use or where memory is used which isn't allocated by the application. Also these tests could be filtered out by suppression rules.