Random Information ...others might find it interesting too.

Running Debian on FreeBSD using bhyve

As I do run OpenWRT on a few devices that FreeBSD doesn't run on I always had to keep a Linux based operating system around as OpenWRT doesn't really work well on FreeBSD even though "it should" and ongoing work is being done in that department. You can get the bare distribution to compile but as soon as you try to build third party applications you're in for a not so nice treat. So far I've used VirtualBox and VMware Player on my work laptop as VirtualBox on FreeBSD is clunky compared to Windows and requires quite a bit of work to make it somewhat usable. While bhyve was announced in FreeBSD 10 as stable I felt that it never really was and all machines that was capable still ran 9.X so I decided to wait until FreeBSD 10.1 was released to give it a go. While it isn't the most user-friendly solution around it works surprisingly well for being still in very active development. Documentation is still a bit light but I managed to get it all working thanks to a few tutorials and the virtualization mailling list.

Notes: You are adviced to create a separate directory to put everything in, also running this as root user is required. Following this will create a virtual machine with 40Gbyte disk space, 2 CPUs and 768Mbyte of RAM.

  • First you need to install grub2-bhyve as bhyve can't boot Linux on its own
    cd /usr/ports/sysutils/grub2-bhyve && make install clean
  • Download the distribution you want to run, in my case Debian
    fetch http://ftp.no.debian.org/pub/Linux/debian-iso/7.7.0/amd64/iso-cd/debian-7.7.0-amd64-netinst.iso
  • Create a file which will act as your VMs hard drive (40G in my case)
    You can use truncate but it'll cause more fragmentation
    dd if=/dev/zero of=./debian-buildbox.img bs=1M count=40960
  • Load modules for networking and virtualization
    Note: If you're using GENERIC kernel this are all included by default
kldload if_tap  
kldload if_bridge
kldload vmm
kldload nmdm
  • Enable networking
sysctl net.link.tap.up_on_open=1
sysctl net.inet.ip.forwarding=1
  • Create a network interface for the VM (em1 being my network card)
ifconfig tap1 create
ifconfig bridge0 create
ifconfig bridge0 addm tap1 addm em1 up
  • Create a file called device.map containing the paths to HDD image file and ISO file
echo "(hd0) ./debian-buildbox.img" > device.map
echo "(cd0) ./debian-7.7.0-amd64-netinst.iso.iso" >> device.map
  • Run the bootloader
    Here you'll see a menu, select install and you'll be back at the prompt after a few secs
    grub-bhyve -m device.map -r cd0 -M 768M debian-buildbox
  • Launch the virtual machine
bhyve -A -c 2 -m 768M -H \
-s 0:0,hostbridge \
-s 1:0,lpc \
-s 2:0,virtio-net,tap1 \
-s 3,ahci-hd,debian-buildbox.img \
-s 4,ahci-cd,debian-7.7.0-amd64-netinst.iso \
-l com1,/dev/nmdm0A debian-buildbox
  • Run the installer and when it's going to reboot you'll be back at the prompt shortly after
  • Close the VM
    bhyvectl --vm=debian-buildbox --destroy
  • Tell bhyve to boot from the HDD
    grub-bhyve -m device.map -r hd0,msdos1 -M 768M debian-buildbox
  • Launch the virtual machine
bhyve -A -c 2 -m 768M -H \
-s 0:0,hostbridge \
-s 1:0,lpc \
-s 2:0,virtio-net,tap1 \
-s 3,ahci-hd,debian-buildbox.img \
-l com1,/dev/nmdm0A debian-buildbox

Unless something went wrong you can access your Linux OS using serial device /dev/nmdm0B
cu -l /dev/nmdm0B -s 9600


Remote serial port connection on FreeBSD

For debugging purposes I needed to provide serial access to a device attached to a remote machine without giving any other type of remote access. This doesn't sound too hard but surprisingly it's quite confusing and information is sparse. Making Google do it's magic I ended up with these three options:

  • ser2net
  • socat
  • remserial

ser2net: Follows the RFC 2217 standard which is good, unfortunately there doesn't seem to be many clients around. The only ones I could find that were free was Kermit (Open Source) and HW VSP by the HW-Group (Freeware).

socat: Seems to be awesome and very versatile if you know what you're doing, honestly the documentation put me off...

remserial: Emulates a serial port remotely, proprietary, very minimalistic (doesn't really output any errors) and doesn't run on Windows unfortunately.

As Windows wasn't a requirement in this case I ended up running remserial as it just worked and did what it was supposed to do without any additional applications.

On the server (-p port (TCP), -s speed raw, /dev/cuau0 being the serial device in my case):
remserial -d -p 32323 -s "115200 raw" /dev/cuau0 &

On the client (-r remoteipadress, -p port (TCP), -l redirected-serial-port):

kldload ptmx
remserial -d -r <ipaddress> -p 32323 -l /dev/remser1 /dev/ptmx &

All set, now you just need to connect as you usually do but using /dev/remser1 as the serial port instead.

Tested on FreeBSD 10.1

Anonymous NFSv4 shares on FreeBSD

Tested on FreeBSD 10 and above:

  • /etc/sysctl.conf
  • /etc/exports
    /path/to/somewhere -ro -mapall=nobody -alldirs -network
  • /etc/rc.conf

This will share your directory to all clients on that have 192.168.1.X as IP address with read only permissions on your network. Have in mind that all clients don't support v4, OpenELEC for instance only supports up to v3 which you'll need to adjust in your sysctl accordingly.

Intel NUC DC3217BY and supported memory speeds

Intel claims that this model officially this model supports 1600Mhz aka PC3-12800, this can be seen pretty much everywhere on their site including here. Digging around a bit more on Intel's site you'll find this page which pretty much says that 1.35 and 1.5V memory should work fine. Most manufacturers also have a memory compatibility guide which helps you select compatible memory to your hardware which in most cases this information isn't actually verified but based on technical specifications. While I do understand that its impossible and unreasonable that they test every possible combination they do have some kind of warranty that if model X doesn't work with ram module Y you'll get a replacement or refund. Nothing wrong here as this as it's a good service towards the end-user unfortunately most models are only available in some regions and/or have a different model number which makes very hard/impossible to actually get the listed modules. That said you always have JEDEC specifications as a fallback which essentially are a set standard of operating speed, voltage and timings. In theory all modules should support this or at least fallback to in terms of compatibility unfortunately this is not always the case. So to be on the safe side I always go for memory where the manufacturer specifically lists JEDEC specs as default specs, this usually works out great irregardless of hardware but not in this case.

As the DC3217BY is based on the Ivy Bridge platform it supports 1600Mhz according to Intel, both 1.35V and 1.5V. Having used three different modules on three different NUCs I've come to the conclusion that it's really picky and close to non-working setup.

Crucial Ballistix Sport 8GB 1600MHz (2x4Gb) - BLS2C4G3N169ES4CEU
Runs at 1600Mhz (1.35V only) but memory corruption

Crucial Ballistix Sport 4GB 1600MHz (1x4Gb) - BLS4G3N169ES4CEU
Runs at 1600Mhz (1.35V only) but memory corruption

Crucial 8GB 1600Mhz (2x4Gb) - CT2KIT51264BF160B
Runs at 1600Mhz (1.35V only) but memory corruption

Given that I used the Sport 8Gb kit first I didn't expect the 4Gb to work but I figured if I tried another product line it might help. These modules are all dual voltage compatible according to Crucial's support and since Intel never really intended the NUCs to "hardcore" overclocking audience you have very few settings available regarding memory. Since BIOS updates may resolve memory issues/compatibility I always update to the latest version available but I would still have the same issues. As I figured that 1.35V might be a bit on the low side as voltage may vary slightly I also tried 1.5V but that enforces 1333Mhz. There's no way of changing the memory speed so you're stuck with 1333Mhz which in all cases worked fine. Interestingly 1600Mhz usually works "well enough" so you probably wont see any BSODs but you will experience odd issues such as applications crashing randomly and random screen corruption.

As any other major company Intel points to the supported memory list and have no interest in adding a setting per one persons request, they are however helpful and polite. Making Google work a bit you'll find other users having the same issues so I would call it a hardware design flaw. That said, it's a great box and everything but the 1600Mhz support is bad/non-existant.

Firmware update may fix your USB 3.0 hub

I've been trying to do my homework on USB 3.0 hubs and consensus seems to be that they all have quirks at some point irregardless of controller. Some certainly do work better than others but since most manufacturers believe that this information is not important you end up buying these blindly as there may be unannounced revisions that has a completely different controller.

After quite a bit of reading I ended up with the conclusion that VIA VL812 is the best known controller around but it's not without its flaws. Doing some more research I found that this is one of the few controllers some actually mention in a product spec sheet as a feature. As I just needed a few ports additionally I went with a 4-port hub by HooToo called HT-UH002 which was reasonably priced at that time and did work as a bus powered only hub in most cases. While it didn't work for my intended application (not caused by the USB hub itself as it turned out) I ended up using it in my desktop instead. Most of the time it worked fine but occasionally it would decide to stop working.

Bothered with the issues I decided to have a look again to find something that worked without any issues, it turned out that VL812 (revision B2) as of writing is still the best one around. I did find a company called Plugable that's very open about the hardware being used and they actually do try to improve their products. One of their earlier products utilized the same chipset as my HooToo and they also provide firmware updates. While you technically can brick your hub it worked fine in my case and after updating the firmware it's been working great.

The firmware can be found here and is used at your own risk.

USB storage on OpenWRT

While OpenWRT does have wiki page about this topic here it's a bit difficult to easily tell what do to in order to get it working. Here's very short tutorial on how to partition and enable auto mount of your storage. I've also added a swap partition as a few applications such as MiniDLNA and Transmission tends to hog quite a bit during some operations.

Requires: block-mount, fdisk (busybox's version) and filesystem and USB subsystem (modules).

In my case I used an empty (well, invalid partition table) USB HDD so there are no partitions available.

  • First when you plug it in you'll get a few kernel messages
[  152.791517] usb 1-1.1: new high-speed USB device number 3 using orion-ehci
[  152.935444] usb-storage 1-1.1:1.0: USB Mass Storage device detected
[  152.951619] scsi0 : usb-storage 1-1.1:1.0
[  153.952121] scsi 0:0:0:0: Direct-Access     Disk     Name             ES2O PQ: 0 ANSI: 2 CCS
[  153.963425] sd 0:0:0:0: [sda] 625142448 512-byte logical blocks: (320 GB/298 GiB)
[  153.971914] sd 0:0:0:0: [sda] Write Protect is off
[  153.976733] sd 0:0:0:0: [sda] Mode Sense: 28 00 00 00
[  153.977407] sd 0:0:0:0: [sda] No Caching mode page found
[  153.982777] sd 0:0:0:0: [sda] Assuming drive cache: write through
[  153.992033] sd 0:0:0:0: [sda] No Caching mode page found
[  153.997391] sd 0:0:0:0: [sda] Assuming drive cache: write through
[  154.455456]  sda: unknown partition table
[  154.464693] sd 0:0:0:0: [sda] No Caching mode page found
[  154.470056] sd 0:0:0:0: [sda] Assuming drive cache: write through
[  154.476224] sd 0:0:0:0: [sda] Attached SCSI disk
  • Start fdisk to add a partition table and partitions to the drive
    fdisk /dev/sda
  • Type o to a new partition table
  • Type n to add a new partition
  • Type p to select primary partition type
  • Type 1 to set it as first partition
  • Leave blank to use default cylinder value as we want to use the beginning of the drive
  • Type +512M to make it 512Mbyte in size (swap partition)
  • Type n to add another partition
  • Type p to select primary partition type
  • Type 2 to set it as second partition
  • Leave blank to use default cylinder value, this will set start point at the first free cylinder
  • Leave blank as we want to allocate the rest of the drive
  • Type t to alter partition type
  • Type 1 to select the first partition
  • Type 82 to change it to a Linux swap partition
  • Type w to write your change to disk and close fdisk
  • Create swap filesystem
    mkswap /dev/sda1
  • Enable swap
    swapon /dev/sda1
  • Create ext4 filesystem for storage
    mkfs.ext4 /dev/sda2
  • Open /etc/config/fstab to enable auto mounting of USB HDD
    nano /etc/config/fstab
  • Change values of anon_swap and anon_mount from 0 to 1
  • Save changes and exit

Done! You just need to reboot for changes to apply.

Newer posts → Home ← Older posts