FRITZ!Box tuning part 4: Cross-building and installing additional applications

The articles I wrote about the FRITZ!Box are pretty popular. They are creating the most traffic on my website. I understand this, cause the FRITZ!Box is a really great piece of hardware and AVM is also a company which knows how to make their users happy by serving regular updates to the firmware. Although I didn’t tuned my FRITZ!Box any further, I updated it with the latest Labor firmware version regularly. At some point the sshd setup (with dropbear) doesn’t worked anymore and I decided it is the time to update my software as well. Beside that it didn’t work anymore it is always a good idea to update software which allow access to a host from everywhere very regularly. Anyway, it turned out this isn’t as simple as I initial thought. Therefore here is the next post in the FRITZ!Box tuning series, which shows how to cross-build software for the MIPS32 architecture used in the FRITZ!Box and in particular get the sshd software to life again. I use a FRITZ!Box Fon WLAN 7270 v2 and the firmware is 54.05.05. Please make sure you read the other FRITZ!Box articles as well, cause some of the information given there still applies.

As already mentioned in the other articles, there is no warranty that this will work on your side and I am not responsible for any damage which may happen. You have been warned!

Cross-building for the MIPS32 architecture

The FRITZ!Box itself doesn’t include any build tools. Why should it! It’s an end-consumer product which is usually never changed, beside the user is updating it with an official firmware update. So we have to do it ourself. Creating a build chain for another architecture is usually hard. There are several things you have to consider. This includes problems with the configuration, endianess differences between the host and the target, bit depths and depending shared libraries which have to be available in the target format. Fortunately this is a common problem, especially for embedded systems, which means there is a proper solution called Buildroot (look at the first line in dmsg, when the FRITZ!Box has booted and you know what I mean). Buildroot is a build system which allows, similar to the Linux kernel, configuring and generating complete embedded Linux systems. Regardless how configured, Buildroot builds a complete Linux system, always. This is of course a little bit overhead for us, but we just pick out the relevant files. First we need to know which architecture we target. You have to login into the FRITZ!Box either using telnet or a working ssh daemon. To get some information about the hardware used in your FRITZ!Box execute cat /proc/cpuinfo.

# cat /proc/cpuinfo
system type             : TI UR8 (7270)
processor               : 0
cpu model               : MIPS 4KEc V6.8
BogoMIPS                : 359.62
wait instruction        : yes
microsecond timers      : yes
tlb_entries             : 16
extra interrupt vector  : yes
hardware watchpoint     : no
ASEs implemented        :
shadow register sets    : 1
core                    : 0
VCED exceptions         : not available
VCEI exceptions         : not available

Here we learn that the FRITZ!Box is using a MIPS processor. In this case it uses the little-endian format. Next you need some information about the system C library. On normal Linux (desktop) systems the GNU C Library is used. This library is heavyweight and therefore the uClibc was invented for embedded systems. By executing ls /lib/libuClibc* you get the version installed. In my case it is 0.9.31. Now we have all information needed to configure Buildroot. Download and extract the source of Buildroot.

If you are working on a Mac like me (where Buildroot doesn’t work), VirtualBox with a Linux system as a guest is a good alternative ;).

By executing make menuconfig in the root directory of Buildroot you are able to configure it. The most important options are Target Architecture, set this to mipsel and Target Architecture Variant, set this to mips 32r2. In Toolchain -> uClibc C library Version select a version which is close to the one on your FRITZ!Box. Next you have to select which packages should be build in Package Selection for the target. Choose whatever you want but for the beginning Networking applications -> openssh might be enough. If you prefer a more lightweight ssh implementation dropbear may also an option. In the following we use OpenSSH. You can also tweak the build options, like the gcc version to use or set the gcc optimization level to -O3 in favor of size. After saving the configuration and executing make, it is time for a coffee. When the build has been finished all relevant files are in output/target/. We only pack the directories and files which are necessary. For example the libuClibc was built by Buildroot, but we will use the one from the FRITZ!Box. The following create a zip archive of the files for OpenSSH.

zip -r ../../local.zip "lib/libutil* lib/libnsl* lib/libresolv* 
 lib/libcrypt* usr/lib usr/share usr/sbin usr/bin etc/ssh_config 
 etc/sshd_config"

Creating overlay directories on the root filesystem

The following is based on ideas invented by the guys at http://www.spblinux.de/. Our aim is it to have as little persistent changes on the FRITZ!Box as possible. This allows us to go back to the original state with a simple reboot. Therefore all writeable directories will be on a tmpfs filesystem. tmpfs filesystems are created in RAM and the content will be removed when the filesystem is unmounted. Because the FRITZ!Box has only limited RAM, we will not copy our applications into the tmpfs filesystem, but just create symbolic links to the files on the USB stick (where usually much more space is available). Because we want the applications from the FRITZ!Box and our own one be accessible from the root filesystem, we need to merge both into the tmpfs directory. Therefore first symbolic links into the original filesystem and second symbolic links to the USB stick are created. The second step is only performed when there isn’t a file already. This makes sure the original files are always preferred over our own one. The last step is binding the directories to the root filesystem. Summarized the following will be done:

  1. remount / readonly
  2. mount / into /var/_ro_
  3. create tmpfs in /var/_overlay_
  4. symlink the files from /var/_ro_ into /var/_overlay_
  5. symlink the files from the USB stick into /var/_overlay_
  6. bind the directories in /var/_overlay_ to /

This script does the previous steps. It uses helper methods which are defined in common.sh. You will find all the necessary files in the package you can download at the end of this article.

#!/bin/sh
 
. common.sh
 
MNT=/var/_overlay_
BASE_RO=/var/_ro_
BASE_OVL="${BASE}/local"
 
DIRS="lib etc usr/bin usr/sbin usr/lib usr/share"
 
# remount root readonly
"${MOUNT}" | "${GREP}" "/dev/root" | "${GREP}" -q ro
[ $? -eq 1 ] && _lmt /dev/root / "-o remount -r"
# bind root into var
_lchkmnt "${BASE_RO}"
if [ $? -eq 1 ]; then
  _lmkdir "${BASE_RO}"
  _lmnt / "${BASE_RO}" "-o bind"
fi
 
# create our overlay dir
_lchkmnt "${MNT}"
if [ $? -eq 1 ]; then
  _lmkdir "${MNT}"
  _lmnt tmpfs "${MNT}" "-t tmpfs"
fi
 
# link/copy the base stuff
for i in ${DIRS}; do
  _lsymlnk_dir "${BASE_RO}" "${i}" "${MNT}"
done
 
# link/copy our own stuff
for i in ${DIRS}; do
  _lsymlnk_dir "${BASE_OVL}" "${i}" "${MNT}"
done
 
# now bind all overlay dirs to root
# (this is critical don't interupt)
for i in ${DIRS}; do
  _lmnt "${MNT}/${i}" "/${i}" "-o bind"
done
 
exit 0

Now copy the zip archive you have created above to the FRITZ!Box and unpack it on the USB stick in addons/local/. The scripts have to be placed into addons/. You need to configure the following in common.sh:

# the base of the usb stick goes here:
BASE="/var/media/ftp/FLASH-DISK-01/addons"
# add the encrypted root password here:
PASS=""

Point to the base directory within the USB stick in BASE. Also add your encrypted root password into PASS. This will create a root user in /etc/passwd automatically.

If you believe all is correct, you can start the installation by executing ./install.sh. If no errors are shown, you can try ssh. ssh -v should output something like this:

# ssh -v
OpenSSH_5.8p2, OpenSSL 1.0.0d 8 Feb 2011
...

Additional to the standard filesystems mounted, you should see the following when executing mount:

# mount
...
/dev/root on /var/_ro_ type squashfs (ro,relatime)
tmpfs on /var/_overlay_ type tmpfs (rw,relatime)
tmpfs on /lib type tmpfs (rw,relatime)
tmpfs on /etc type tmpfs (rw,relatime)
tmpfs on /usr/bin type tmpfs (rw,relatime)
tmpfs on /usr/sbin type tmpfs (rw,relatime)
tmpfs on /usr/lib type tmpfs (rw,relatime)
tmpfs on /usr/share type tmpfs (rw,relatime)

Configure and start the OpenSSH daemon

Now that we have our brand new OpenSSH on the FRITZ!Box working, we need some last steps to let the daemon running correctly. First create a private/public RSA host key pair for the server process by executing:

ssh-keygen -t rsa -f addons/config/ssh_host_rsa_key

addons/config/ have to be an existing directory below the USB stick base. Next create a file addons/config/sshd_config in the same directory with the following content:

Protocol               2
UsePrivilegeSeparation no
Subsystem              sftp /usr/lib/sftp-server

The following script could be use to start the sshd:

#!/bin/sh
 
. common.sh
 
SSHD="/usr/sbin/sshd"
SSH_SPKEY="${BASE}/config/ssh_host_rsa_key"
SSH_CFG="${BASE}/config/sshd_config"
SSH_TPKEY="/var/tmp/ssh_host_rsa_key"
 
# sanity check
[ ! -e "${SSHD}" ] && _lce $? "sshd not installed"
 
# start in tmp
cd /var/tmp   
 
mount -o remount devpts /dev/pts -t devpts
 
# cp key and make them user readable only
_lcp_file "${SSH_SPKEY}" "${SSH_TPKEY}"
_lchmod 400 "${SSH_TPKEY}"             
 
# start sshd
"${SSHD}" -f "${SSH_CFG}" -h "${SSH_TPKEY}"

Please note that you have to set up the root user first and maybe adjust your port forwarding rules to allow access from the Internet as described in this post. The package will contain a script startup.sh which will add the root user automatically.

Here is the obligatory screenshot which shows htop with color terminal support. You see OpenSSH and OpenVPN in the list of the running processes.

Conclusion

Thanks to Buildroot it is pretty easy to cross-build for embedded machines these days. With some tricky directory rebinding new applications could be injected into the FRITZ!Box root filesystem without overwriting the existing firmware. Now you should be able to build any software already bundled with Buildroot or even add new one to the build process. Happy cross-building!

You can download the scripts and binaries discussed in this post:
FRITZBox_OpenSSH_5.8p2.tar.gz (3.8MB, SHA1)

26 thoughts on “FRITZ!Box tuning part 4: Cross-building and installing additional applications

  1. Great article!

    I am wondering if there is another solution then using a USB-stick? I would like to test it on my Fritz!Box Fon WLAN 7113 which has no USB port. Can you install in a local directory or path?

    Thank you!

    1. Hi,

      you could of course try to copy the files from the USB stick into the overlay directory instead of sym linking them. But it seems your FRITZ!Box has even lesser RAM than mine. So this could lead to filling up all memory on the FRITZ!Box (the data is around 10MB). Unnecessary to say that this will end in an unresponsive box.

      Another possibility is using a network share. You could mount a NFS share on the FRITZ!Box and using the files from there. I gave that a try and it works. I was able to mount my NAS with the following (after copying the data to /volume1/backup/addons):
      mount -t nfs -o nolock,rsize=8192,wsize=8192,soft 192.168.220.2:/volume1/backup /var/tmp/nfs
      Just change the BASE path in common.sh to point to /var/tmp/nfs/addons and you are done.

      This is of course not a real solution, but for an initial try it might be enough.

      Chris

  2. Hallo Christian,

    ich habe meine FRITZ!Box Fon WLAN 7270 v2 letzte Woche ebenfalls auf die neue Firmware 54.05.05 upgedated. Leider funktioniert mein Dropbear nicht mehr. Ist dir eine Möglichkeit bekannt, Dropbear zum laufen zu bringen? OpenSSH möchte ich nur einsetzten, wenn es nicht anders geht.

    Viele Grüße
    Simon

    1. Noch ein anderer Tip: Ich weiß jetzt nicht genau was dein Problem ist, aber falls du Probleme mit dem PTY hast, versuche mal:

      mount -o remount devpts /dev/pts -t devpts

      Dropbear danach neu starten.

      Chris

  3. Bisher habe ich Dropbear wie du in Part 1 beschrieben hast eingebunden: http://www.64k-tec.de/2010/01/fritzbox-tuning-part-1-enable-remote-access-over-ssh/

    Allerdings kann ich mich seit dem Firmwareupdate nicht mehr connecten, es komm die Fehlermeldung
    “Server refused to allocate pty”. Außerdem ist die bei “http://www.spblinux.de/fbox.new/cfg_dropbear” hinterlegt Dropbear Version nicht die aktuellste und ich weiß nicht wie ich 0.53 installieren kann.

    Werde es nach deinem Tutorial versuchen, danke. Hätte aber auf eine einfachere Variante gehofft.

      1. in what format the usb flash must be? i try with fat32 (it’s the only fs i can mount in 7140) but when i copy the files i get that this filesystem doesn’t support symlink.

        1. Ext2 would be best of course. But I also use a fat32 formated stick. You need a compressor which is able to create regular files out of the symbolic links in the archive.

          Chris

  4. 1.i format the flash with fat32.
    2.i compress the target folder to an .tar file.
    3.when i try to uncompress through telnet the fritz works for ever and never and the proccess.

    what am i doing wrong?

    1. Make really sure to uncompress it on the stick. If you uncompress it e.g. in /tmp your FRITZ!Box will be run out of memory and become unresponsive.

      Chris

  5. I have put a link to your page into spblinux.de/fbox.new/0.9.31/readme.txt

    So others who are able to build their own binaries can use your detailed instructions.

    spblinux

  6. Great tutorial, I now have a running sshd (and rsyncd) – but unfortunately still can not connect:
    Generating a password does not work. In my hosts file there is always an x where the password should be. So I can not login with a password.
    I tried to use authentication with the authorized_keys but this does not work as well.
    Can you give me a hint here?
    Thank you, Holger

    FRITZ!Box Fon WLAN 7270 v3
    Firmware-Version 74.05.05

  7. Der Download funktioniert leider nicht mehr.
    Ich schaffe es leider ohne nicht, weil ich ein “Permission denied” in Zeile 12 bekomme, wenn ich versuche das erste große Script auszuführen.

  8. Hi,

    I was wondering if anyone has tried this for a FritzBox 7390? It reports the following details:

    # uname -a
    Linux fritz.fonwlan.box 2.6.28.10 #1 Fri Apr 20 12:19:44 CEST 2012 mips
    GNU/Linux
    # cat /proc/cpuinfo
    system type : Ikanos Fusiv Core
    processor : 0
    cpu model : MIPS 24Kc V4.12
    BogoMIPS : 330.75
    wait instruction : yes
    microsecond timers : yes
    tlb_entries : 64
    extra interrupt vector : yes
    hardware watchpoint : yes, count: 4, address/irw mask: [0x0000,
    0x0030, 0x0f40, 0x02b0]
    ASEs implemented : mips16 dsp
    shadow register sets : 1
    core : 0
    VCED exceptions : not available
    VCEI exceptions : not available

    Can anyone help me with what the Target Architecture and Target Architecture Variant should be set to when configuring buildroot to build for the FB 3790?

    Thanks,

    Dean

  9. Hi,

    I tried your instructions for a FRITZ!Box 7360 SL.

    # cat /proc/cpuinfo
    system type : VR9
    processor : 0
    cpu model : MIPS 34Kc V5.5
    BogoMIPS : 332.59
    wait instruction : yes
    microsecond timers : yes
    tlb_entries : 16
    extra interrupt vector : yes
    hardware watchpoint : yes, count: 4, address/irw mask: [0x0000, 0x0ff8, 0x0ff8, 0x0ff8]
    ASEs implemented : mips16 dsp mt
    shadow register sets : 1
    core : 0
    VCED exceptions : not available
    VCEI exceptions : not available

    # ls /lib/libuClibc*
    /lib/libuClibc-0.9.32.so

    First, I tried to cross-compile on my own using buildroot-2013.05 with (next to default settings)

    Target Architecture: MPIS (little endian)
    Target Architecture Variant: mips 32r2
    Toolchain -> uClibc C library Version: uClibc 0.9.32.x
    Build options -> gcc optimization level: optimization level 3
    Package Selection for the target -> Network applications: openssh

    Compiling finished without errors. I set BASE and PASS variable in common.sh (but not ROOT) and took the three additional scripts from the provided download FRITZBox_OpenSSH_5.8p2.tar.gz. I didn’t obtain error messages when running ./install.sh. When I succesively tried: ssh -v, I got:

    # ssh -v
    /usr/bin/ssh: line 1: syntax error: unexpected “(”

    I have no idea what to do now! Can someone please help?! When I check for output of mount, I get:

    # mount
    rootfs on / type rootfs (rw)
    /dev/root on / type squashfs (ro,relatime)
    proc on /proc type proc (rw,relatime)
    tmpfs on /var type tmpfs (rw,relatime)
    tmpfs on /dev type tmpfs (rw,relatime)
    sysfs on /sys type sysfs (rw,relatime)
    devpts on /dev/pts type devpts (rw,relatime,mode=600)
    /dev/loop0 on /var/media/ftp type ext2 (rw,relatime)
    usbfs on /proc/bus/usb type usbfs (rw,relatime)
    /dev/sda1 on /var/media/ftp/PNY-USB2-0FD-01 type vfat (rw,noatime,fmask=0000,dmask=0000,allow_utime=0022,codepage=cp437,iocharset=iso8859-1,shortname=winnt,errors=remount-ro)
    /dev/root on /var/_ro_ type squashfs (ro,relatime)
    tmpfs on /var/_overlay_ type tmpfs (rw,relatime)
    tmpfs on /lib type tmpfs (rw,relatime)
    tmpfs on /etc type tmpfs (rw,relatime)
    tmpfs on /usr/bin type tmpfs (rw,relatime)
    tmpfs on /usr/sbin type tmpfs (rw,relatime)
    tmpfs on /usr/lib type tmpfs (rw,relatime)
    tmpfs on /usr/share type tmpfs (rw,relatime)

    Which seems to look fine according to the descritption above, doesn’t it?!

    The same behaviour occurs btw., when I directly use the binaries from the download FRITZBox_OpenSSH_5.8p2.tar.gz, instead of my self compiled ones.

    I am thankful for any Hint.

    H.

  10. Hi,

    I’m getting exactly the same error as H. using a 7390 (I tried compiling tcpdump). Any clues as to what could be wrong would be greatly apprectiated.

    Many thanks,
    Ian

    1. Hi,

      /usr/bin/ssh: line 1: syntax error: unexpected “(” indicates the command interpreter thinks this is a shell script, but I think this should be a binary. You can check this with cat /usr/bin/ssh. If it shows some texts its a shell script and you need to look at line 1. But I guess its a binary which the interpreter doesn’t understand. So maybe this is an issue with the MIPS architecture/endian type.

      Chris

  11. Hi,

    same problem with 7330SL.
    I can’t select “34Kc” in menuconfig 🙁

    # cat /proc/cpuinfo
    system type : AR9
    processor : 0
    cpu model : MIPS 34Kc V4.12
    BogoMIPS : 261.32
    wait instruction : yes
    microsecond timers : yes
    tlb_entries : 16
    extra interrupt vector : yes
    hardware watchpoint : yes, count: 4, address/irw mask: [0x0ff8, 0x0ff8, 0x0ffb, 0x0ffb]
    ASEs implemented : mips16 dsp mt
    shadow register sets : 1
    core : 0
    VCED exceptions : not available
    VCEI exceptions : not available

  12. Yes, this is an issue that has already been rerteopd. More specifically, the multilib toolchain being used was a Codesourcery toolchain. A Buildroot user already made some investigation about the issue, but I haven’t had the time myself to dig into the issue. It’s definitely on my TODO-list, but I’m not sure when I’ll be able to release proper fixes for multilib toolchain support.Thanks for your interest in Buildroot! Don’t hesitate to share your issues with the list or to report bugs to our bug tracker.

Leave a Reply

Your email address will not be published. Required fields are marked *