Random Information ...others might find it interesting too.

Speed up building FreeBSD (-CURRENT / HEAD)

Just some quick notes building -CURRENT / HEAD (FreeBSD 11).



Disables debugging code in malloc, building (and installing) of debug files and also doesn't build 32-bit libs which you most likely doesn't need as you are compiling apps natively (64-bit) anyway.



Set CPU type, gives a nice little speed boost. Do not use native, it will break things (tm).
If unsure which model suits your CPU best read /usr/src/share/examples/etc/make.conf if unsure use core2.
Disable more debugging code and addtional files and don't add profiling data.

Following the regular procedure you might want to use these switches to automatically remove old files.

make -DBATCH_DELETE_OLD_FILES delete-old
make -DBATCH_DELETE_OLD_FILES delete-old-libs

OpenWRT on the Ubiquiti Networks EdgeRouter Lite


As the EdgeRouter doesn't seem to get any love (QoS) on the FreeBSD project I decided to have a look at other software solutions and as I've previously have good experience with OpenWRT I decided go with it a try. Information is pretty sparse at the moment but I decided to write a few lines on how you write a bootable USB flash stick on FreeBSD. This actually fits within 128M but as I have no USB flash drives that small I ended up grabbing the smallest one I could find which was a 8Gb thumb drive. Do note that this does not need any modification of u-boot.


  • USB Flash drive (128Mbyte minimum)
  • OpenWRT image (openwrt-octeon-erlite-sysupgrade.tar)


  • Plug in the drive and run dmesg, at the end you'll see something like this
ugen3.2: <Verbatim> at usbus3
umass0: <Verbatim STORE N GO, class 0/0, rev 2.00/1.00, addr 2> on usbus3
umass0:  SCSI over Bulk-Only; quirks = 0xc100
umass0:10:0: Attached to scbus10
da8 at umass-sim0 bus 0 scbus10 target 0 lun 0
da8: <Verbatim STORE N GO 1.00> Removable Direct Access SCSI-2 device
da8: Serial Number 1226000000009999
da8: 40.000MB/s transfers
da8: 7645MB (15656960 512 byte sectors: 255H 63S/T 974C)
da8: quirks=0x2<NO_6_BYTE>
  • Let's have a look at the partition layout on da8
    gpart show /dev/da8
=>       1  15656959  da8  MBR  (7.5G)
         1        31       - free -  (16K)
        32  15656928    1  fat32  (7.5G)
  • Delete the FAT32 partition which has ID 1 (hence 1 before the filesystem)
    gpart delete -i 1 /dev/da8
  • Verify that it's deleted
    gpart show /dev/da8
=>       1  15656959  da8  MBR  (7.5G)
         1  15656959       - free -  (7.5G)
  • Delete the partition table
    gpart destroy /dev/da8
  • Create a new MBR partition table
    gpart create -s MBR /dev/da8
    da8 created
  • Create a 32Mbyte large partition for the kernel and align it to 1M
    Note: I don't think it matters in this case in terms of performance but why not since we have the space.
    gpart add -a 1M -t fat32 -s 32M /dev/da8
    da8s1 added
  • Just to be sure, make it active (bootable)
    gpart set -a active -i 1 /dev/da8
    active set on da8s1
  • Create a partition for the root filesystem, in this case 256Mbyte but you'll be fine with 64Mbyte if you have space constraints.
    gpart add -a 1M -t linux-data -s 256M /dev/da8
    da8s2 added
  • Generate (format) the FAT partition
    newfs_msdos /dev/da8s1
newfs_msdos: trim 16 sectors to adjust to a multiple of 63
/dev/da8s1: 65416 sectors in 8177 FAT16 clusters (4096 bytes/cluster)
BytesPerSec=512 SecPerClust=8 ResSectors=1 FATs=2 RootDirEnts=512 Sectors=65520 Media=0xf0 FATsecs=32 SecPerTrack=63 Heads=255 HiddenSecs=0
  • Extract firmware
    tar xf openwrt-octeon-erlite-sysupgrade.tar
  • Mount the FAT partiton
    mount -t msdosfs /dev/da8s1 /mnt
  • Copy kernel to the FAT partition, you also need to rename it to vmlinux.64
    cp ./sysupgrade-erlite/kernel /mnt/vmlinux.64
  • Unmount the FAT partition
    umount /mnt
  • Write the root filesystem the second partition
    dd if=./sysupgrade-erlite/root of=/dev/da8s2 bs=1M
48+0 records in
48+0 records out
50331648 bytes transferred in 7.267285 secs (6925784 bytes/sec)

All set, just plug it in the EdgeRouter

Running Debian on FreeBSD using bhyve

As I do run OpenWRT on a few devices that FreeBSD doesn't run on I always had to keep a Linux based operating system around as OpenWRT doesn't really work well on FreeBSD even though "it should" and ongoing work is being done in that department. You can get the bare distribution to compile but as soon as you try to build third party applications you're in for a not so nice treat. So far I've used VirtualBox and VMware Player on my work laptop as VirtualBox on FreeBSD is clunky compared to Windows and requires quite a bit of work to make it somewhat usable. While bhyve was announced in FreeBSD 10 as stable I felt that it never really was and all machines that was capable still ran 9.X so I decided to wait until FreeBSD 10.1 was released to give it a go. While it isn't the most user-friendly solution around it works surprisingly well for being still in very active development. Documentation is still a bit light but I managed to get it all working thanks to a few tutorials and the virtualization mailling list.

Notes: You are adviced to create a separate directory to put everything in, also running this as root user is required. Following this will create a virtual machine with 40Gbyte disk space, 2 CPUs and 768Mbyte of RAM.

  • First you need to install grub2-bhyve as bhyve can't boot Linux on its own
    cd /usr/ports/sysutils/grub2-bhyve && make install clean
  • Download the distribution you want to run, in my case Debian
    fetch http://ftp.no.debian.org/pub/Linux/debian-iso/7.7.0/amd64/iso-cd/debian-7.7.0-amd64-netinst.iso
  • Create a file which will act as your VMs hard drive (40G in my case)
    You can use truncate but it'll cause more fragmentation
    dd if=/dev/zero of=./debian-buildbox.img bs=1M count=40960
  • Load modules for networking and virtualization
    Note: If you're using GENERIC kernel this are all included by default
kldload if_tap  
kldload if_bridge
kldload vmm
kldload nmdm
  • Enable networking
sysctl net.link.tap.up_on_open=1
sysctl net.inet.ip.forwarding=1
  • Create a network interface for the VM (em1 being my network card)
ifconfig tap1 create
ifconfig bridge0 create
ifconfig bridge0 addm tap1 addm em1 up
  • Create a file called device.map containing the paths to HDD image file and ISO file
echo "(hd0) ./debian-buildbox.img" > device.map
echo "(cd0) ./debian-7.7.0-amd64-netinst.iso.iso" >> device.map
  • Run the bootloader
    Here you'll see a menu, select install and you'll be back at the prompt after a few secs
    grub-bhyve -m device.map -r cd0 -M 768M debian-buildbox
  • Launch the virtual machine
bhyve -A -c 2 -m 768M -H \
-s 0:0,hostbridge \
-s 1:0,lpc \
-s 2:0,virtio-net,tap1 \
-s 3,ahci-hd,debian-buildbox.img \
-s 4,ahci-cd,debian-7.7.0-amd64-netinst.iso \
-l com1,/dev/nmdm0A debian-buildbox
  • Run the installer and when it's going to reboot you'll be back at the prompt shortly after
  • Close the VM
    bhyvectl --vm=debian-buildbox --destroy
  • Tell bhyve to boot from the HDD
    grub-bhyve -m device.map -r hd0,msdos1 -M 768M debian-buildbox
  • Launch the virtual machine
bhyve -A -c 2 -m 768M -H \
-s 0:0,hostbridge \
-s 1:0,lpc \
-s 2:0,virtio-net,tap1 \
-s 3,ahci-hd,debian-buildbox.img \
-l com1,/dev/nmdm0A debian-buildbox

Unless something went wrong you can access your Linux OS using serial device /dev/nmdm0B
cu -l /dev/nmdm0B -s 9600


Remote serial port connection on FreeBSD

For debugging purposes I needed to provide serial access to a device attached to a remote machine without giving any other type of remote access. This doesn't sound too hard but surprisingly it's quite confusing and information is sparse. Making Google do it's magic I ended up with these three options:

  • ser2net
  • socat
  • remserial

ser2net: Follows the RFC 2217 standard which is good, unfortunately there doesn't seem to be many clients around. The only ones I could find that were free was Kermit (Open Source) and HW VSP by the HW-Group (Freeware).

socat: Seems to be awesome and very versatile if you know what you're doing, honestly the documentation put me off...

remserial: Emulates a serial port remotely, proprietary, very minimalistic (doesn't really output any errors) and doesn't run on Windows unfortunately.

As Windows wasn't a requirement in this case I ended up running remserial as it just worked and did what it was supposed to do without any additional applications.

On the server (-p port (TCP), -s speed raw, /dev/cuau0 being the serial device in my case):
remserial -d -p 32323 -s "115200 raw" /dev/cuau0 &

On the client (-r remoteipadress, -p port (TCP), -l redirected-serial-port):

kldload ptmx
remserial -d -r <ipaddress> -p 32323 -l /dev/remser1 /dev/ptmx &

All set, now you just need to connect as you usually do but using /dev/remser1 as the serial port instead.

Tested on FreeBSD 10.1

Anonymous NFSv4 shares on FreeBSD

Tested on FreeBSD 10 and above:

  • /etc/sysctl.conf
  • /etc/exports
    /path/to/somewhere -ro -mapall=nobody -alldirs -network
  • /etc/rc.conf

This will share your directory to all clients on that have 192.168.1.X as IP address with read only permissions on your network. Have in mind that all clients don't support v4, OpenELEC for instance only supports up to v3 which you'll need to adjust in your sysctl accordingly.