Serial-PXE-TFTP install of FreeBSD(BSDRP,Xsense,NAS4Free)/OpenBSD/Centos

Objectives

Remote  installation of multiples Operating systems using only:

• FreeBSD server with a PXE and TFTP services
• Serial console: IPMI Serial-over-LAN (sol)

I didn't found an easy way for PXE+TFTP (only!) serial remote installation for NetBSD or DragonFly.
FreeBSD was very complex too (need to recompile bootloader for TFTP and serial usage), but hopefully mfsBSD hides this problem.
OpenBSD and CentOS, by providing ramdisk natively and easy way of configuring their bootloader, were the most admin-friendly.

dnsmasq

This step will install an all-in-once DHCP/TFTP server:
pkg install dnsmasq
Then, create a small configuration file (example with "bce1" as NIC and local subnet in 192.168.1.0/24)
cat > /usr/local/etc/dnsmasq.conf <<EOF
interface=bce1
dhcp-range=192.168.1.80,192.168.1.85
pxe-service=x86PC, "pxelinux", pxelinux
enable-tftp
tftp-root=/tftpboot
EOF

And start it:
sysrc dnsmasq_enable=yes
service dnsmasq start

pxelinux

This step will install pxelinux binaries and configure PXE menu:

mkdir /tftpboot
cd /tftpboot
fetch https://www.kernel.org/pub/linux/utils/boot/syslinux/syslinux-6.03.zip
unzip -d syslinux syslinux-6.03.zip
cp syslinux/bios/memdisk/memdisk /tftpboot
cp syslinux/bios/core/pxelinux.0 /tftpboot
cp syslinux/bios/com32/elflink/ldlinux/ldlinux.c32 /tftpboot
cp syslinux/bios/com32/menu/menu.c32 /tftpboot
cp syslinux/bios/com32/libutil/libutil.c32 /tftpboot
cp syslinux/bios/com32/modules/pxechn.c32 /tftpboot
cp syslinux/bios/com32/lib/libcom32.c32 /tftpboot
cp syslinux/bios/com32/chain/chain.c32 /tftpboot
cp syslinux/bios/com32/modules/reboot.c32 /tftpboot/
rm syslinux-6.03.zip
rm -rf syslinux
mkdir /tftpboot/pxelinux.cfg
cat > /tftpboot/pxelinux.cfg/default <<EOF
SERIAL 0 115200
CONSOLE 0
UI menu.c32
TIMEOUT 300
MENU TITLE PXE BOOT MENU
LABEL freebsd
 MENU DEFAULT
 MENU LABEL mfsbsd (FreeBSD, pfSense, BSDRP, NAS4Free, etc...)
 KERNEL memdisk
 APPEND initrd=/mfsbsd-10.1-RELEASE-amd64.img harddisk raw
LABEL openbsd
 MENU LABEL OpenBSD
 KERNEL pxechn.c32
 APPEND ::/openbsd/pxeboot
LABEL netbsd
 MENU LABEL NetBSD
 KERNEL pxechn.c32
 APPEND ::/netbsd/pxeboot_ia32_com0.bin
LABEL centos
 MENU LABEL Centos 7
 kernel centos/vmlinuz
 append initrd=centos/initrd.img method=http://mirror.centos.org/centos/7/os/x86_64/ devfs=nomount ip=dhcp console=ttyS0,115200 earlyprint=serial,ttyS0,115200
LABEL local
 MENU LABEL local disk
 KERNEL chain.c32
 APPEND hd0
LABEL reboot
 MENU LABEL reboot
 KERNEL reboot.c32
EOF

FreeBSD

Download mfsBSD image and enable serial port:

fetch -o /tftpboot/mfsbsd-10.1-RELEASE-amd64.img http://mfsbsd.vx.sk/files/images/10/amd64/mfsbsd-10.1-RELEASE-amd64.img
mdconfig -a -t vnode -f mfsbsd-10.1-RELEASE-amd64.img
mount /dev/md0a /mnt/
echo "-S115200 -h" > /mnt/boot.config
umount /mnt
mdconfig -d -u 0

OpenBSD

Download OpenBSD's pxeboot and RamDisk image, then enable serial port:
mkdir /tftpboot/openbsd/
fetch -o /tftpboot/openbsd/pxeboot http://ftp.openbsd.org/pub/OpenBSD/5.7/amd64/pxeboot
fetch -o /tftpboot/openbsd/bsd.rd http://ftp.openbsd.org/pub/OpenBSD/5.7/amd64/bsd.rd
mkdir /tftpboot/etc
cat > /tftpboot/etc/boot.conf <<EOF
stty com0 115200
set tty com0
boot tftp:/openbsd/bsd.rd
EOF

CentOS

Download CentOS kernel and RamDisk:
mkdir /tftpboot/centos
fetch -o /tftpboot/centos/initrd.img ftp://ftp.free.fr/mirrors/ftp.centos.org/7.1.1503/os/x86_64/images/pxeboot/initrd.img
fetch -o /tftpboot/centos/vmlinuz ftp://ftp.free.fr/mirrors/ftp.centos.org/7.1.1503/os/x86_64/images/pxeboot/vmlinuz

Installing BSDRP, pfSense, OPNsense, NAS4Free, or any nanoBSD

From mfsbsd, just dd their serial nanobsd/embedded image to the local hard drive.
For installing FreeBSD: just uses bsdinstall

Debugging PXE/TFTP process

From the server, start a tcpdump accepting only bootps and tftp packets:

tcpdump -ni bce1 -vv port bootps or port tftp

ipfw improvement on FreeBSD -current

Few days ago Alexander V. Chernikov posted on the FreeBSD -net mailing list an "HEADS UP: Merging projects/ipfw to HEAD" with lot's of promises:

• Tables are now identified by names, not numbers. There can be up to 65k tables with up to 63-byte long names.
• Tables are now set-aware (default off), so you can switch/move them atomically with rules.
• More functionality is supported (swap, lock, limits, user-level lookup, batched add/del) by generic table code.
• New table types are added (flow) so you can match multiple packet fields at once.
• Ability to add different type of lookup algorithms for particular table type has been added.
• New table algorithms are added (cidr:hash, iface:array, number:array and flow:hash) to make certain types of lookup more effective.
• Table value are now capable of holding multiple data fields for different tablearg users

I'm not an expert of ipfw(8), but I would check the impact of this improved-ipfw on forwarding performance. By "performance" I mean how this code impact the throughput (in term of packet-per-second) of my FreeBSD firewall (I didn't bench all the parameters requiered by RFC3511).
Once the code was committed as r272840 on -head, I've generated a new nanobsd(8) image on my 10gigabit bench lab… and here are the results:

More than 100K pps of differences! Now I dream of an ipfw_sync equivalent to pf_sync(4).
And here are the ministat output for statistician extremists.
Regarding ipfw in stateless mode:


x 272685.ipfw-stateless
+ 273009.ipfw-stateless
+----------------------------------------------------------------------+
|x      x     x    x                                  + + +      +    +|
|   |______A__M___|                                                    |
|                                                     |___M__A_____|   |
+----------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5       1585928       1619817       1608891     1604564.2     12728.878
+   5       1683246       1712607       1690405     1695508.6      12250.89
Difference at 95.0% confidence
        90944.4 +/- 18219.1
        5.66786% +/- 1.13546%
        (Student's t, pooled s = 12492.2)

And regarding ipfw in statefull mode:


x 272685.ipfw-statefull
+ 273009.ipfw-statefull
+----------------------------------------------------------------------+
|xx    x   x    x                       ++   +    +                   +|
||_____A______|                                                        |
|                                    |_______M___A____________|        |
+----------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5       1390415       1433678       1407058     1408663.4     18451.472
+   5       1502719       1589778       1517320     1529871.8     35404.181
Difference at 95.0% confidence
        121208 +/- 41172.4
        8.6045% +/- 2.9228%
        (Student's t, pooled s = 28230.4)

PuTTY and Solarized colors

I'm using the Solarized color palette on all my FreeBSD desktops, but at work I had to works from a MS Windows desktop :-(
Here are my PuTTY settings for a correct rendering of Solarized colors.

Softwares used

• PuTTY and it's authentication agent (There is KiTTY, an improved version of PuTTY too)
• MTPuTTY (Multi-Tabbed Putty) for adding tabbed feature to PuTTY/KiTTY
• Solarized PuTTY settings

Installation steps

Installing PuTTY or MTPuTTY didn't need specials instruction.
If you're using KiTTY, the Solarized PuTTY.reg files need to be adapted by opening them into a text editor and replacing the line:
[HKEY_CURRENT_USER\Software\SimonTatham\PuTTY\Sessions\Solarized%20Dark]
By this one:
[HKEY_CURRENT_USER\Software\9bis.com\KiTTY\Sessions\Solarized%20Dark]
Then import the .reg files.

KiTTY/PuTTY configuration

Now start KiTTY or PuTTY, load the Dark or Light Solarized session and modify this session with:

• Window - Colours : Enable "Allow terminal to use xterm 256-colour mode"
• Connection - Data - Terminal details : Terminal-type string : "xterm-256color"

You can then add more customization, for example:

• Terminal - Bell : Visual Bell
• Window - Appearance - Font Settings - Font: "Consolas" - 12point
• Window - Appearance - Font Settings - Font quality: ClearType
• Window - Translation - Remote character set: UTF-8
• Window - Lines of scrollback: 10000
• Window - Selection - Control use of mouse: xterm (Right extends, Middle Past)
• Connection : Second between keepalives: 25
• Connection : Enable TCP keepalives
• Connection - SSH - X11: Enable X11 forwarding

Once all your customization done, save the session as "Default Settings".

Checking parameters

Start a KiTTY/PuTTY, check that your preferences are loaded by default and open a SSH session to an *nix machine.
Once logged, the command "echo $TERM" should answer "xterm-256color".
And, if you've solarized your VIM (you don't need to use let g:solarized_termcolors=256!), you should correctly see the column after entering a "set colorcolumn=80".

tmux

tmux need to be configured for advertise a 256color term by adding in ~/.tmux.conf the line:
set -g default-terminal "screen-256color"

Configuration IPv6 propre d'une Kimsufi sous FreeBSD

Pour configurer une passerelle par défaut IPv6 sur un Kimsufi, le guide officiel se résume à:

1. Paramétrer l'IPv6 de votre interface avec votre préfixe /64 (2001:41D0:1:46e::/64 par exemple)
2. Suivre la règle IP:v:6:FF:FF:FF:FF:FF pour déduire votre passerelle par défaut (dans notre exemple elle est donc 2001:41D0:1:4FF:FF:FF:FF:FF).

Sauf qu'avec cette règle l'IP de la passerelle est en dehors de votre réseau (/64)… donc injoignable !
«À ce qu'il paraît» cela ne pose pas de problème aux GNU/Linux…no comment.
Une autre section du guide propose de récupérer les RA pour trouver la route par défaut "link-local" annoncée par le routeur, mais cela ne fonctionne plus car ils ont été désactivés.
La solution la plus commune à ce problème est de paramétrer un préfixe /56 à la place du /64 sur votre interface: du coup la passerelle par défaut se trouve dans votre réseau et le problème est résolus.
Mais OVH m'a donné un /64, je ne vois pas pourquoi je lui déclarerai un /56!

Une jolie solution (soufflée par flo@) permettant de paramétrer un /64 tout en utilisant cette route par défaut est la suivante:

ifconfig_re0_ipv6="inet6 2001:41D0:1:46e::1 prefixlen 64"
ipv6_static_routes="mac"
ipv6_route_mac="-host 2001:41D0:1:4FF:FF:FF:FF:FF -iface re0"
ipv6_defaultrouter="2001:41D0:1:4FF:FF:FF:FF:FF"

Ethernet-Wifi failover on FreeBSD

I want a simple behaviour with my laptop:

1. If Ethernet cable connected use this connectivity, otherwise use the wireless;
2. I want to kept the same IP addresses, event if I'm using DHCP client.

The solution is quiet simple:

1. set-up an aggregate interface in failover mode with Ethernet as primary and wireless as backup;
2. Clone the Wireless MAC NIC to the Ethernet (opposite is not always possible with wireless chipset restriction).

Here are how to do it:

# ifconfig -l
iwn0 bge0 lo0
=> My Ethernet NIC is "bge0" and wireless is "iwn0" here

# set MAC=`ifconfig wlan0 | grep ether | cut -d ' ' -f 2`
# sysrc ifconfig_bge0="ether $MAC"
ifconfig_bge0:  -> ether 00:1c:23:25:ab:45
# sysrc wlans_iwn0=wlan0
wlans_iwn0:  -> wlan0
# sysrc ifconfig_wlan0="WPA up"
ifconfig_wlan0: WPA DHCP -> WPA up
# sysrc cloned_interfaces=lagg0
cloned_interfaces:  -> lagg0
# sysrc ifconfig_lagg0="laggproto failover laggport bge0 laggport wlan0 DHCP"
ifconfig_lagg0:  -> laggproto failover laggport bge0 laggport wlan0 DHCP
# sysrc ifconfig_lagg0_ipv6="inet6 accept_rtadv"
ifconfig_lagg0_ipv6:  -> inet6 accept_rtadv
# service netif restart

And now with Ethernet cable unplugged:

# ifconfig lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 00:1c:23:25:ab:45
    inet6 fe80::41d:23ff:fe25:ab78%lagg0 prefixlen 64 scopeid 0x4
    inet6 2a01:e35:9b9d:a1a0:41d:23ff:fe25:ab45 prefixlen 64 autoconf
    inet 192.168.100.7 netmask 0xffffff00 broadcast 192.168.100.255
    nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: active
    laggproto failover lagghash l2,l3,l4
    laggport: wlan0 flags=4<ACTIVE>
    laggport: bge0 flags=1<MASTER>


Then If I plug the Ethernet cable:


# ifconfig lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 00:1c:23:25:ab:45
    inet6 fe80::41d:23ff:fe25:ab45%lagg0 prefixlen 64 scopeid 0x4
    inet6 2a01:e35:9b9d:a1a0:41d:23ff:fe25:ab45 prefixlen 64 autoconf
    inet 192.168.100.7 netmask 0xffffff00 broadcast 192.168.100.255
    nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: active
    laggproto failover lagghash l2,l3,l4
    laggport: wlan0 flags=0<>
    laggport: bge0 flags=5<MASTER,ACTIVE>

Generating custom FreeBSD installation media

Objective

Generating a custom -current memstick image without all the debug feature enabled.

Prerequisite

Have the head source installed (I will use /usr/src as example).
If not, here is an example for synchronizing up-to-date head (-current) sources on /usr/src
svnlite co svn://svn.freebsd.org/base/head /usr/src

Customizing build compilation options

A little debug feature to disable on -current:
echo "MALLOC_PRODUCTION=yes" > /etc/src.conf

Building world and kernel

New we start the classical building of world and our customized kernel.
For information this step takes about 4 hours on my PC.
cd /usr/src
make buildworld; make buildkernel KERNCONF=GENERIC-NODEBUG

Generating install media image

Last step: Generating the install media.
Here is an example for generating memstick install media without port tree (long live to pkgng! neither doc):
cd /usr/src/release
make -DNOPORTS -DNODOC memstick

Replace "memstick" by "cdrom" (bootonly.iso and release.iso) or "ftp" for other media.
You can add a -DNOSRC option for avoiding to include sources too on the media.

Then copy the image to your usb key:
dd if=memstick of=/dev/da0 bs=64k

Cleaning your mess

Your system has lot's of file that you don't need anymore, here is how to clean it:
cd /usr/src/release
make clean
cd /usr/src
make clean

Xorg for FreeBSD on Raspberry Pi

Thanks to Ray, we have a working X11 drivers that works on Raspberry pi too.
Here is how to install it:
Download the latest Daisuke's Raspberry pi FreeBSD image (login/password: pi/ raspberry and root password: raspberry).
Unzip, dd it on your 8GB SD card, boot your rspie and configure your network (Internet access mandatory for downloading pkg).
Install USB mouse drivers:
cd /usr/src/sys/modules/usb/ums
make clean all install
Install pkg and declare the repository:
fetch http://dev.bsdrp.net/pkg/freebsd:10:arm:32:el:oabi:softfp/Latest/pkg.txz
tar xf ./pkg.txz -s ",/.*/,,g" "*/pkg-static"
./pkg-static add ./pkg.txz
echo 'packagesite: http://dev.bsdrp.net/pkg/${ABI}' > /usr/local/etc/pkg.conf
Install Xorg, scfb drivers and some X apps:
pkg inst xorg-minimal xf86-video-scfb xorg-apps 
Configure /etc/X11/xorg.conf:
Section "Files"
EndSection 
Section "Module"

    Load        "dbe"

    Disable    "dri"

    Disable    "dri2"

    Disable    "glx"

    SubSection  "extmod"

       Option  "omit xfree86-dga"

    EndSubSection

EndSection

Section "ServerFlags"

    Option    "AIGLX"        "false"

    Option    "NoAccel"    "True"

    Option    "NoDRI"        "True"

    Option    "DRI"        "False"

    Option    "DRI2"        "False"

EndSection

Section "InputDevice"

    Identifier  "Keyboard1"

    Driver      "kbd"

EndSection

Section "InputDevice"

    Identifier  "Mouse1"

    Driver      "mouse"

    Option      "Protocol"      "auto"

    Option      "Device"        "/dev/sysmouse"

EndSection

Section "Monitor"

    Identifier  "Monitor"

EndSection

Section "Device"

    Identifier  "Generic FB"

    Driver      "scfb"

    Option    "NoAccel"    "True"

EndSection

Section "Screen"

    Identifier  "Screen"

    Device      "Generic FB"

    Monitor     "Monitor"

    DefaultDepth 16

    SubSection "Display"

       Depth           16

    EndSubsection

EndSection

Section "ServerLayout"

    Identifier  "layout"

    Screen      0 "Screen" 0 0

    InputDevice "Mouse1" "CorePointer"

    InputDevice "Keyboard1" "CoreKeyboard"

EndSection

Then a dummy ~/.xinitrc:
xterm &
xclock &
xcalc &
exec twm

And enjoy:
startx

A first ugly screenshoot:

Installation de FreeBSD sur partition chiffrée

Le concept

Cet article présente l'usage de geli pour l'installation d'un FreeBSD (9.1) sur un disque chiffré.
Le module geli permet:

1. D'utiliser l'assistance matériel si elle est disponible (comme l'aesni des processeurs Intel).
2. De gérer des clés «maitres» (cas typique d'usage en entreprise: Une clé «maitre» stockée dans le coffre du bureau sécurité, permettant l'accès d'un disque en cas de perte d'une clé «salarié»).
3. De générer des clés aléatoires: Utile pour avoir une clé différente pour la partition swap à chaque re-démarrage.

L'intégralité des partitions du disque ne sont pas chiffrées:

1. La partition contenant le boot-loader (utilisant GPT, on réserve une partition pour lui) est en clair
2. La partition contenant la configuration des boot-loader, le noyau et ses modules est en clair
3. Tout le reste est chiffré (le /)

Cet exemple présente un usage simple sur un PC portable personnel, c'est à dire avec cette configuration:

1. Utilisation d'un mot de passe uniquement (pas de clé)
2. Pas de partition swap

À noter:
Le mot de passe étant demandé pendant le chargement du noyau, le clavier sera obligatoirement en QWERTY (à moins de reconfigurer/recompiler un noyau)… Donc pensez-y lorsque vous rentrez votre mot de passe ;-)

Un dessin vaut mieux qu'un long discours

Voici l'organisation des partitions sur votre disque à la fin de cette installation:

La procédure d'installation détaillée

Démarrage de l'installateur

On commence par lancer l'installateur et sélectionner les options suivantes:

1. Install
2. Distribution Select: Vous pouvez décocher le «port tree», le fabuleux pkgng boostrapable depuis la 9.1 rend caduque l'ancienne méthode de compilation des ports (et fini les portmaster/portupgrade)
3. Partitioning: Attention ! C'est là qu'il faut choisir «Shell»

Partitionnement et chiffrage «en ligne de commande» du disque

Une fois dans le shell, afficher la liste des disques détectés par le système:

# sysctl kern.disks
kern.disks: ada0 cd0 

Facile ici, il n'y a que ada0 (cd0 étant le lecteur CD), on commence par le ré-initialiser et lui créer une table GPT (la MBR c'est pour les vieux):

# gpart destroy -F ada0
gpart: arg0 'ada0': Invalid argument (si disque vide)
ou
ada0 destroyed (si table MBR/GPT déja présente)
# gpart create -s gpt ada0
ada0 created

Créer la première partition (p1) contenant le boot-loader:

# gpart add -t freebsd-boot -s 64k -a 4k ada0
ada0p1 added
# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada0
bootcode written to ada0 

Puis créer la deuxième (p2) de 1G qui va contenir le noyau:

# gpart add -t freebsd-ufs -s 1G -l bootpart -a 4k ada0
ada0p2 added
# newfs -j gpt/bootpart
gpt/bootpart: 1024.0MB (2097152 sectors) block size 32768, fragment size 4096
        using 4 cylinder groups of 256.03MB, 8193 blks, 32896 inodes.
        with soft updates
super-block backups (for fsck_ffs -b #) at:
 192, 524544, 1048896, 1573248
newfs: Can't stat gpt/bootpart: No such file or directory
Using inode 4 in cg 0 for 8388608 byte journal
newfs: soft updates journaling set

Puis créer, initialiser le chiffrement (clé a 256bits, taille de secteur a 4k), attacher et formatter la troisième partition (p3) qui va contenir tout le reste.
En profiter pour charger votre drivers crypto (ici aesni car processeur Intel).

# gpart add -t freebsd-ufs -l rootfs -a 4k ada0
ada0p3 added
# kldload aesni
cryptosoft0: <software crypto> on motherboard
aesni0: <AES-CBC,AES-XTS> on motherboard
# geli init -s 4096 -bl 256 gpt/rootfs
Enter new passphrase: jemmerdeleBEFTIetOCLTIC
Reenter new passphrase: jemmerdeleBEFTIetOCLTIC

Metadata backup can be found in /var/backups/gpt_rootfs.eli and
can be restored with the following command:

        # geli restore /var/backups/gpt_rootfs.eli gpt/rootfs
# geli attach gpt/rootfs
Enter passphrase: jemmerdeleBEFTIetOCLTIC
GEOM_ELI: Device gpt/rootfs.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
# newfs -j gpt/rootfs.eli
gpt/rootfs.eli: 9215.9MB (18874168 sectors) block size 32768, fragment size 4096
        using 15 cylinder groups of 626.09MB, 20035 blks, 80256 inodes.
        with soft updates
super-block backups (for fsck_ffs -b #) at:
 192, 1282432, 2564672, 3846912, 5129152, 6411392, 7693632, 8975872, 10258112,
 11540352, 12822592, 14104832, 15387072, 16669312, 17951552
newfs: Can't stat gpt/rootfs.eli: No such file or directory
Using inode 4 in cg 0 for 33554432 byte journal
newfs: soft updates journaling set

Reste le montage de ces partitions au bon endroit avant de retourner à l'installateur:

# mount /dev/gpt/rootfs.eli /mnt/
# mkdir /mnt/bootpart
# cd /mnt
# ln -s bootpart/boot boot
# mount /dev/gpt/bootpart /mnt/bootpart
# mkdir bootpart/boot

Et quitter le shell pour revenir à l'installateur:
# exit

Retour à l'installateur 

Terminer l'installation et la configuration réseau, timezone, services à activer, puis:

1. Dumpdev Configuration: No, car il n'existe pas de partiton swap pour la récupérer
2. Final Configuration: Exit
3. Manual Configuration: Yes, pour lancer une dernière fois le shell avant de redémarrer la machine

Finalisation de la configuration par le shell

On commence par vérifier que l'installation s'est correctement déroulée puis à configurer correctement le fichier boot/loader.conf:

# test -L boot && echo "ok" || echo "NOK !"
ok
# test -f bootpart/boot/loader && echo "ok" || echo "NOK !"
ok
# cat > boot/loader.conf <<EOF
> geom_eli_load="YES"
> aesni_load="YES"
> vfs.root.mountfrom="ufs:/dev/ada0p3.eli"
> EOF

Note: Ne pas essayer d'utiliser les label gpt pour le vfs.root.mountfrom car ça ne fonctionne pas.

Puis la génération du etc/fstab:

# cat > etc/fstab <<EOF
> /dev/gpt/bootpart /bootpart ufs rw,noatime 1 1
> /dev/gpt/rootfs.eli / ufs rw,noatime 1 1
> EOF 

On en profite aussi pour générer le mtree qui récupére l'emprunte sha256 de l'ensemble des fichiers de la partition /bootpart que vous stockerez sur votre partition chiffrée.

# cd /bootpart
# mtree -x -ic -k sha256digest > /root/boot.mtree


Et enfin quitter le shell et rebooter:

# quit 

Le démarrage

Désormais, au démarrage, pendant le chargement du noyau, s'affichera le message suivant puis il attendra votre mot de passe:
Enter passphrase for ada0p3: 

Note: Il arrive que cette phrase soit mélangée dans le dmesg et pas très visible. Il suffit alors de taper votre mot de passe quand le démarrage semble en pause.

Pour les paranos

Lors de l'étape de finalisation, juste avant le reboot, nous avons créé un fichier mtree.
Cette étape sert à vous protéger d'un vecteur d'attaque assez vicieux:
 Pour obtenir votre mot de passe il est possible d'aller remplacer le noyau (car stocké sur partition en clair) par un noyau modifié (qui enregistrer votre mot de passe à votre insu).
Le fait d'avoir stocké l'emprunte sha256 des fichiers en clair sur la partition chiffrée, et de vérifier leur cohérence après chaque démarrage (un petit script fait très bien l'affaire) permet de limiter cette attaque.
Pour vérifier la cohérence des fichiers avec le mtree, c'est cette commande:
mtree -f /root/boot.mtree -p /bootpart
S'il n'affiche rien: C'est bon signe :-)

Sources

Je me suis appuyé sur les deux articles suivants:

• Full disk encryption in FreeBSD 9.x (well, almost!)
• Installing FreeBSD 9.0 with encrypted root fs (all ufs)

Using BCM4313 wireless chip under FreeBSD

I've got a new laptop at $DAY-JOB, and this laptop have a BCM4313 chip:

# pciconf -lv | grep Wireless
    device     = 'BCM4313 802.11b/g/n Wireless LAN Controller'

I've first try with net/bwn-firmware-kmod and if_bwn without success, but NDIS works great (on 9.1-RC2 at last).
Here here to proced for converting the Windows XP drivers to a kernel module:
# cd /tmp
# fetch  http://gugus69.free.fr/freebsd/Broadcom_BCM43xx_5.100.235.19_win5x.tgz
# tar xzvf Broadcom_BCM43xx_5.100.235.19_win5x.tgz
# cd Broadcom_BCM43xx_5.100.235.19_win5x/
# ndisgen

Inside ndisgen, here is to proceed:

1. Type 3 for "Convert driver"
2. INF Filename: bcmwl5.inf
3. SYS Filename: bcmwl564.sys (or bcmwl5.sys for FreeBSD-i386)
4. Firmware Files: kept empty

At the end, it should generated the bcmwl564_sys.ko module, now enable it:
# cp bcmwl564_sys.ko /boot/modules/
# kldload bcmwl564_sys

ndis0: mem 0xd4500000-0xd4503fff irq 19 at device 0.0 on pci36
ndis0: NDIS API version: 5.


And check that you have a new ndis0 interface:

# ifconfig | grep ndis
ndis0: flags=8843 metric 0 mtu 2290

You can now configure your rc.conf like that

kld_list='bcmwl564_sys'

wlans_ndis0="wlan0"

ifconfig_wlan0="WPA SYNCDHCP"

And enjoy !

FreeBSD kernel for Nintendo Wii

Few months ago I've subsrcibed to the FreeBSD "svn-src-head" mailing-list for being notified of every change to the -current source tree.
And some weeks ago I've seen this message: Initial support for running FreeBSD on the Nintendo Wii.

What ??? FreeBSD is going running on Wii ???

I've looked for more information and found a small page that explain how to build it on the wiki: FreeBSD/Wii.
The commit message includes an important detail: "There are still some MMU problems and to get a working system you need to  patch locore32.S. Since we haven't found the best way yet to address that problem, we're not committing those changes yet."

1. The -current code is incomplete, but meanwhile we can apply this locore32.S unofficial patch.
2. The result is not usable: You will not obtain a shell prompt, it will hang before.

Here is the video of the boot process:

For building the kernel you need -current source tree, then the steps for building this kernel are pretty simple:

cd /usr/src

fetch http://gugus69.free.fr/freebsd/wii/locore32.diff

patch < locore32.diff

mkdir /tmp/WII

setenv MAKEOBJDIRPREFIX /tmp/WII ; make TARGET=powerpc TARGET_ARCH=powerpc buildworld

setenv MAKEOBJDIRPREFIX /tmp/WII ; make TARGET=powerpc TARGET_ARCH=powerpc KERNCONF=WII buildkernel

It will generate a kernel for WII here:

/tmp/WII/powerpc.powerpc/usr/src/sys/WII/kernel

Then follow instructions from FreeBSD/wii wiki page for setup the SD card.

You can download my own FreeBSD/Wii archive (to be extracted on the root of your SD card).

My home NAS: HP ProLiant MicroServer

My old NAS system was dying, I need to found a new one…
Here is my new setup:

• HP ProLiant MicroServer (using a special 100€ french user cash back offer, valid until the 31/10/2012)
• 8 GB of ECC RAM
• One 3.5 hard drive mounting bracket adapter for 5.25 inch bay (for the CD/DVD bay)
• One SATA cable with right angle connector
• 5 SATA hard drives, 2TB each
• A customized BIOS that unlock the SATA mode of the CD/DVD bay (locked in PATA mode in the official BIOS)
• A 2GB usb key (there is an USB connector inside the box)
• A HP MicroServer remote access card (not mandatory)
• Lian Li EX-50 Expension box
• 5 SATA hard drives, 1TB each. They came from my old NAS.

I've found lot's of advices regarding the setup of the HP MicroServer on TechForum.
Before installing FreeNAS on it, I'm playing with a small FreeBSD 9.0 installation on the USB key.
FreeBSD was installed without any package neither the port-tree: I'm using pkgng with my own package repository (thanks to poudriere).
The only problem that I found, was to correctly disable moused: A simple moused_enable="NO" in the /etc/rc.conf is not enough, I need to add moused_ums0_enable="NO" for disable it.
I've create 2 raid-z pools: A first with the internal hard-drives, a second with the external drives.

Now I've got lot's of disk space for my files:

[root@nas]/#zpool list
NAME       SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
estorage  4,53T   912K  4,53T     0%  1.00x  ONLINE  -
storage   9,06T  6,56T  2,50T    72%  1.00x  ONLINE  -

OpenBSD 5.1 installation on softraid(4)

OpenBSD 5.1 brings new cool features like this one:

"boot(8/amd64) is teached how to access softraid(4) volumes, which allows a kernel to be loaded from a softraid RAID 1 volume. Furthermore it is sufficient to only boot from a disk that is a member of a bootable softraid volume, as this case will be detected and the boot will automatically be redirected to sr[0-9]a:/bsd."

This mean that the bootloader can boot directly from a softraid(4) partition: No more "standard" partitions needed with kernel copied on them on both hard drive!


Here is how to proceed:
During your CD-ROM boot, check that you have a minimum of 2 hard drives detected:

CD-ROM: E0

Loading /5.1/AMD64/CDBOOT

probing: pc0 com0 mem[639K 126M a20=on]

disk: hd0+* hd1+* cd0

>> OpenBSD/amd64 CDBOOT 3.16

boot>

Continue the boot process and select to launch a shell:

Welcome to the OpenBSD/amd64 5.1 installation program.

(I)nstall, (U)pgrade or (S)hell? s

Display the names of the disks detected by OpenBSD:

# sysctl hw.disknames

hw.disknames=wd0:7922594e8158ee03,wd1:49129150e28daf19,cd0:,rd0:7c8ac10ea613493f

On this example (Virtualbox VM), we've got 2 SATA hard-drives: wd0 and wd1 (cd0 is the CD-ROM and rd0 is the RAM drive created by the kernel).
By default OpenBSD create only one /dev for the first hard drive, we need to create a second and initialize a MBR to them:

# cd /dev/

# sh MAKEDEV wd1

# fdisk -iy wd0

Writing MBR at offset 0.

# fdisk -iy wd1

Writing MBR at offset 0.

Now, we need to create a BSD label named "raid" to both disks.
We will create a BSD label for the first drive, and dump/restore the label table to the second drive:

# disklabel -E wd0

Label editor (enter '?' for help at any prompt)

> a a
offset: [64]
size: [20964761]
FS type: [4.2BSD] raid
> q
Write new label?: [y] 

# disklabel wd0 > protofile

# disklabel -R wd1 protofile

Now we can create a softraid(4) volume using the two "a" partitions :

# bioctl -c 1 -l wd0a,wd1a softraid0 

sd0 at scsibus1 targ 1 lun 0: SCSI2 0/direct fixed
sd0: 10236MB, 512 bytes/sector, 20964233 sectors
softraid0: SR RAID 1 volume attached as sd0

This will create a new drive: sd0 in this example.
Return back to the OpenBSD installer (Ctrl+D), and select this new softraid disk for the destination root disk:

# ^D

erase ^?, werase ^W, kill ^U, intr ^C, status ^T

Welcome to the OpenBSD/amd64 5.1 installation program.
(I)nstall, (U)pgrade or (S)hell? i

(etc...)

Available disks are: wd0 wd1 sd0.
Which one is the root disk? (or 'done') [wd0] sd0

Once installed, simply reboot and enjoy:

>> OpenBSD/amd64 BOOT 3.18
boot>
booting sr0a:/bsd: 5648376+1600524+932384+0+616448 [89+497880+322605]=0xd2d0a8
entry point at 0x1001e0 [7205c766, 34000004, 24448b12, ccb8a304]

(etc...)

sd0 at scsibus2 targ 1 lun 0: SCSI2 0/direct fixed
sd0: 10236MB, 512 bytes/sector, 20964233 sectors
root on sd0a (f7748118e1f577a4.a) swap on sd0b dump on sd0b
Automatic boot in progress: starting file system checks.
(etc...)

Installing Samsung ML-2950 series printer drivers on FreeBSD

I've just replaced my inkjet printer with a laser one (Samsung ML-2955ND: Netword and Duplex).
Samsung provide drivers for lot's of OS (HP-UX, IBM AIX, GNU/Linux, MacOSX, etc…) but nothing for FreeBSD :-(

Hopefully the GNU/Linux drivers is CUPS based and use standard PPD file… But with a binary only cups-filter.
I didn't know if it was possible to use the Linux emulation layer of FreeBSD for using this cups-filter, but a previous experience was successful, then I try it too.


The major idea is to:

1. Install print/cups
2. Enable and install the FreeBSD Linux binary compatibility
3. Getting the Samsung Unified drivers for the ML-2950 series
4. Extracting the PPD file and the i386 binary cups-filter used by the PPD
5. Fixing the missing dependency needed by the samsung cups-filter

I will not explain the steps 1,2 and 3.

Extract the Unified Linux Drivers:
$ tar zxvf ULD_Linux_V1.00.06.tar.gz

Then check the filter used by the PPD file:

$ grep cupsFilter uld/noarch/share/ppd/Samsung_ML-2950_Series.ppd
*cupsFilter:  "application/vnd.cups-raster 0 rastertospl"

=> This PPD file use the "rastertospl" filter.

Now copy the PPD and the Samsung cups-filter to the local cups directories:
$ mkdir /usr/local/share/cups/model/samsung
$ cp uld/noarch/share/ppd/Samsung_ML-2950_Series.ppd/usr/local/share/cups/model/samsung/
$ cp uld/i386/rastertospl /usr/local/libexec/cups/filter/

Once copied, you can execute rastertosamsungspl from a CLI for displaying the missing Linux libs:

# /usr/local/libexec/cups/filter/rastertospl
/usr/local/libexec/cups/filter/rastertosamsungspl: error while loading shared libraries: libcups.so.2: cannot open shared object file: No such file or directory

There are two method here:

The first and easiest is simply to install print/linux-f10-cups-libs. It should install all dependency needed by the crappy Samsung binary filter (Thanks Patrick for this tips!)

The second, a lot's more complex, and that add lot's of file not managed by the FreeBSD package manager is to check for missing libs one by one: You need to install the missing libs, running a ldconfig and re-execute rastertosamsungspl for discovering that another libs is missing.
Here is the list of all libs that I've installed:
# cd /tmp
# fetch http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/10/Everything/i386/os/Packages/cups-libs-1.3.9-2.fc10.i386.rpm
# fetch http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/10/Everything/i386/os/Packages/gnutls-2.4.2-2.fc10.i386.rpm
# fetch http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/10/Everything/i386/os/Packages/libtasn1-1.5-1.fc10.i386.rpm
# fetch http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/10/Everything/i386/os/Packages/libgcrypt-1.4.3-2.fc10.i386.rpm
# fetch http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/10/Everything/i386/os/Packages/libgpg-error-1.6-2.i386.rpm
# cd /compat/linux
# rpm2cpio /tmp/cups-libs-1.3.9-2.fc10.i386.rpm | tar -zxf -
# rpm2cpio /tmp/gnutls-2.4.2-2.fc10.i386.rpm | tar -zxf -
# rpm2cpio /tmp/libtasn1-1.5-1.fc10.i386.rpm | tar -zxf -
# rpm2cpio /tmp/libgcrypt-1.4.3-2.fc10.i386.rpm | tar -zxf -
# rpm2cpio /tmp/libgpg-error-1.6-2.i386.rpm | tar -zxf -
# /compat/linux/sbin/ldconfig


Once all dependencies are installed, you should have this message when executing rastertosamsungspl:

# /usr/local/libexec/cups/filter/rastertosamsungspl 
INFO: Usage: rastertosamsungspl job-id user title copies options [file]
ERROR: Wrong number of arguments

Now you can add your printer from the CUPS admin panel and use it :-)

BSD Router Project: Version 1.0 disponible

Cela faisait plusieurs mois qu'il n'y avait pas eu de nouvelle version de ce routeur libre: C'est corrigé depuis cette nuit.
Et comme j'ai passé pas mal de temps à le tester dans tous les sens, je me suis dit que cette version méritait finallement de passer directement en 1.0.
J'ai par contre découvert une grosse limitation concernant la protection des protocoles de routages IPv6 sous FreeBSD (RIPng et OSPFv3): Ces protocoles n'incluent plus leur propre mécanismes de sécurité mais les délèguent à la couche IPSec d'IPv6. Cela veux dire que nous devons passer par setkey(8) pour configurer des SAD/SPD manuels les protégeant.… Or la couche IPSec de FreeBSD ne sait pas gérer les adresses multicast et l'usage du multicast est très courant en IPv6 (RIPng et OSPFv3 l'utilisent). Cette limitation est assez bloquante car elle ne permet pas d'avoir le même niveau de protection des réseaux RIP/OSPF en IPv6 qu'en IPv4 sous FreeBSD.

J'ai aussi amélioré les scripts VirtualBox (PowerShell et Shell) permettant de lancer les labs virtuels facilement pour qu'ils prennent en charge les fonctionnalités de linked-type-clone et Page Fusion: Le gain en ressource pour la machine hôte est impressionnant!
Par exemple sur le lab BGP route reflector/confederation qui nécessite le lancement de 7 routeurs, donc 7 VMs (avec 128Mo de RAM et 256Mo de disque alloué à chacune):

• La fonctionnalité «linked-type-clone» permet de ne consommer que 153Mo d'espace disque pour l'ensemble des VMs: Le template de référence utilise 80Mo et chacun des clones 10Mo.
• La fonctionnalité «Page Fusion», elle, permet de ne consommer que 112Mo de RAM pour l'ensemble des VMs (environ 16Mo par VM).

Pour la suite, va falloir que je travaille un système de centralisation/gestion des configurations et pourquoi pas permettre le pre-provisionning des configurations pour du déploiement grande échelle.

Pourquoi ne jamais utiliser iperf sur FreeBSD

J'ai enfin eu le temps de remonter mon petit lab pour faire quelques tests de performance réseaux avec BSDRP:

• Un PC Engines wrap1e203, qui va me servir de routeur (Geode à 233Mhz)
• Deux clients léger HP Compaq T5000 (Transmeta Crusoe à 800Mhz)
• Un switch gigabit

L'avantage d'utiliser des vieilles machines FastEthernet est qu'elles sont plus faciles à saturer que des nouvelles supportant le gigabit.
Avant de tester les perfs de routage du WRAP, j'ai commencé par noter ma valeur de référence, en mesurant les perfs réseaux maximum des HP Compaq T5000.
Pour cela il existe plusieurs logiciels, le plus connus étant iperf. Mais sur les différentes mailing-list FreeBSD, j'ai souvent lus que iperf «ça pue car codé par des linuxiens et donc incapable d'utiliser le pleins potentiel de FreeBSD».
Dans le doute, j'ai donc aussi utilisé netblast (les sources sont cachées dans /usr/src/tools/tools/netrate/netblast, et il n'est pas installé par défaut).
D'ailleur concernant netblast sur sparc64: Il faut le patcher pour corriger un problème d'alignement.
Bref, lancons nos tests (en UDP et avec les tailles de paquet au max), et voici les résultats:

• iperf: 60Mbit/s
• netblast : 80Mbit/s

Le verdict est sans appel en faveur de netblast!
Et après avoir activé le polling, j'ai pus monter jusqu’à 92Mbit/s: On arrive presque au 100Mbit/s de leur carte réseaux… Pas mal du tout pour ces petites bêtes.

Backporting Intel PRO/Wireless 6205a FreeBSD 9.0 modules to 8.2

My old professionnal Dell Latitude D630 laptop was replaced by a new Dell Latitude E5420… and this new one have a wireless chipset not supported on FreeBSD 8.2:

[root@laptop]~#dmesg | grep iwn
iwn0: <Intel(R) PRO/Wireless 6205a> mem 0xe5300000-0xe5301fff irq 17 at device 0.0 on pci2
0xe5300000-0xe5301fff irq 17 at device 0.0 on pci2
iwn0: MIMO 2T2R, MoW, address a0:88:b4:7d:40:9c
iwn0: [ITHREAD]
iwn0: 11a rates: 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
iwn0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
iwn0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
iwn6005fw: could not load firmware image, error 2
iwn0: iwn_read_firmware: could not load firmare image "iwn6005fw"
iwn0: iwn_init_locked: could not read firmware, error 22

But, this new chipset is supported on FreeBSD-current, then I've backported it to 8.2.
I had to do 3 small changes in if_iwn.c file : One function name to revert (pci_find_cap to pci_find_extcap), and remove 2 lines (use of ic_rxstream and ic_txstream).
You can install this drivers like this:

cd /tmp

fetch http://gugus69.free.fr/tools/FreeBSD-8.2-iwn-current.tgz

cd /

tar zxvf /tmp/FreeBSD-8.2-iwn-current.tgz

cd /usr/src

make buildkernel

make installkernel

reboot

I've tested Internet browsing and small files transfert with this drivers without problem.

Creating and starting VirtualBox machine with PowerShell

BSD Router Project provided a VBScript for starting MS Windows based virtualbox lab. But because this vbs script works only on XP, I had to found a more powerful solution: Why not discovering PowerShell and using the VirtualBox COM API ?
I didn't found very detailed examples, then here is a PowerShell script that explains how to create, configure and start a VirtualBox Machine.
The goal is:

1. Create a VM (OS type: FreeBSD_64)
2. Configure it (with a serial port that redirect to a pipe)
3. Convert a unzipped BSDRP-full-amd64-vga disk image file to a VDI file, and attach this VDI to the VM
4. Start the VM

And here are the big steps of this script:

1. Initialize the COM API
2. Create the VM
3. Configure the VM (excluding attaching medium to it)
4. Save and register the VM
5. Convert a RAW file to VDI (using VBoxManage.exe) and registering it
6. Lock the VM in write mode for attaching the medium
7. Launch the VM

Now the PowerShell code sample: