vendredi, janvier 15, 2016

Example of a FreeBSD bug hunting session by a simple user

Problem description

I've meet a problem with one of my FreeBSD-wireless-router, and a FreeBSD network developer (Alexander V. Chernikov, alias melifaro) helps me to identify the culprit kernel function. I've write-down all tips I've learn by this teacher during this session here.

Day 1: Facing a bug

You need a bug for starting your day. In my case, after updating the configuration of previously working wireless-router, my setup stop working correctly.

Day 2: Reducing my setup complexity

My wireless-router configuration was complex: it involves routing, wireless in hostap mode, ipfw, snort, bridge, openvpn, etc.
The first step was to reproduce my problem:

  1. In the minimum steps (this mean with the simplest configuration)
  2. On the latest FreeBSD -current (because developers works on -current)

Rules for getting help: Your call-for-help message needs to be short, because developers don't have lot's of free time. It's very important that you clearly demonstrate a non-attended behavior and the steps for reproduce it easily.
I had to to it twice: I've post a first call-for-help message with a still too complex configuration:

Then I had to work again for simplify my problem and post a new message few days later:

This second message was a good one: It catch some developer eyes :-)

A resume of my bug with this setup:

LAN 0 <--> (re0) fbsd router (bridge0 = re1 + wlan0) <--> LAN 1 and Wireless LAN

This FreeBSD (11-head r293631) is configured like that:

  • One IP address on re0, we will call this LAN 0
  • One IP address on bridge0 (that includes interfaces re1 and wlan0)
  • re1 enabled (put in UP state)
  • wlan0 configured in hostap mode
  • forwarding enabled

But this setup can forward between wireless clients and hosts on LAN 0 ONLY if interface re1 (that belong to bridge0) is in "connected" status !?!
If the Ethernet NIC is in "not connected" status, the FreeBSD router will consider all clients behind bridge0 "unreacheable"… Even if it can ping all wireless clients!
Here is a tcpdump output from the router dumping a ping generated by a wireless clients (, connected to wlan0 and forwarded by cbridge0) toward an host on LAN 0 (

root@fbsd-router:~ # tcpdump -pni re0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on re0, link-type EN10MB (Ethernet), capture size 262144 bytes
23:38:04.466866 ARP, Request who-has tell, length 28
23:38:04.467052 ARP, Reply is-at 00:08:a2:09:c4:a2, length 46
23:38:04.467090 IP > ICMP echo request, id 72, seq 1,length 64
23:38:04.467226 IP > ICMP echo reply, id 72, seq 1, length 64
23:38:04.467300 IP > ICMP host unreachable, length 36
23:38:05.483053 IP > ICMP echo request, id 72, seq 2,length 64
23:38:05.483259 IP > ICMP echo reply, id 72, seq 2, length 64
23:38:05.483318 IP > ICMP host unreachable, length 36
23:38:06.387304 IP > ICMP echo request, id 72, seq 3,length 64
23:38:06.387466 IP > ICMP echo reply, id 72, seq 3, length 64
23:38:06.387514 IP > ICMP host unreachable, length 36
For solving this problem, I just need to plug the Ethernet interface for changing its status to "active".

Checking interface status: a simple user's way

The only check I can do was to check the "status" of my interfaces:

root@fbsd-router# ifconfig re0
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 00:0d:b9:3c:ae:24
        inet netmask 0xffffff00 broadcast
        media: Ethernet autoselect (1000baseT <full-duplex,master>)
        status: active

root@fbsd-router# ifconfig wlan0
wlan0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 04:f0:21:17:3b:d7
        media: IEEE 802.11 Wireless Ethernet autoselect mode 11ng <hostap>
        status: running
        ssid tutu channel 6 (2437 MHz 11g ht/40+) bssid 04:f0:21:17:3b:d7
        country US ecm authmode OPEN privacy OFF txpower 27 scanvalid 60
        protmode CTS ampdulimit 64k ampdudensity 8 shortgi wme burst
        dtimperiod 1 -dfs
        groups: wlan

root@fbsd-router# ifconfig re1
        ether 00:0d:b9:3c:ae:25
        media: Ethernet autoselect (none)
        status: no carrier

root@fbsd-router# ifconfig bridge0
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:6b:c0:de:b8:00
        inet netmask 0xffffff00 broadcast
        nd6 options=9<PERFORMNUD,IFDISABLED>
        groups: bridge
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: re1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 2 priority 128 path cost 55
        member: wlan0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 33333

Nothing seems wrong here:

  • re0 has flag UP and status "active"
  • wlan0 has flag UP and status "running" (I will suppose it's okay because wireless clients paquets correctly enter this interface and even ping bridge0 IP address)
  • re1 has flag UP and status "no carrier" (no cable connected)
  • bridge0 has flag UP but did not display its status (I will suppose it's UP because wireless client can ping it)

But a developer will never "suppose" the status of these interfaces and will speak in their strange language directly to the kernel for asking the real status.

Checking kernel interface status: a developer's way

How FreeBSD kernel manage network interface ? As a simple user, let's read the (huge) "man ifconfig". I didn't found my answer, but the "see also" section mention a "man netintro".
An introduction to "network" should be comprehensive for me.
Inside netintro man page, I've skipped addressing and routing sections for the interfaces section, but nothing here about "interface status". But the "see also" section mention a "man ifnet": Let's try!
Trying to resume this man page:

     The kernel mechanisms for handling network interfaces reside primarily in
     the ifnet, if_data, ifaddr, and ifmultiaddr structures in <net/if.h> and
     <net/if_var.h> and the functions named above and defined in
The system keeps a linked list of interfaces using the TAILQ macros
     defined in queue(3); this list is headed by a struct ifnethead called
     ifnet. The elements of this list are of type struct ifnet...
     The structure additionally contains generic statistics applicable to a
     variety of different interface types (except as noted, all members are of
     type u_long):

           ifi_link_state      (u_char) The current link state of Ethernet
                               interfaces.  See the Interface Link States
                               section for possible values.
 Interface Link States
     The following link states are currently defined:

           LINK_STATE_UNKNOWN      The link is in an invalid or unknown state.
           LINK_STATE_DOWN         The link is down.
           LINK_STATE_UP           The link is up.

Wow, I reach my limits ;-) But here is my understanding:

  1. Each network interface have an index number assigned
  2. once known this index number we can read its state by variable ifi_link_state

I've got 3 new questions now :-(

How to know the "index" number of my interfaces ?

User used to uses "netstat -i" should know the "Link#" displayed:

root@fbsd-router:~ # netstat -iWW | grep Link# | tr -s ' ' | cut -d ' ' -f 1-3
re0 1500 <Link#1>
re1 1500 <Link#2>
re2* 1500 <Link#3>
lo0 16384 <Link#4>
wlan0 1500 <Link#5>
bridge0 1500 <Link#6>

This link number is the interface index number :-)

How to display this variable (ifi_link_state) ?

This answer came from melifaro@ : "use kernel debugger (kgdb) for printing value of variable called ifindex_table[INDEX].if_link_state".

How did this wizzard of code find variable ifindex_table[] ?

I had to use "grep ifindex_table /usr/src/sys/net/*" and found that ifindex_table is defined in if.c as "Table of ifnet by index."
Reading the source code is mandatory here for discovering this variable name.

Why variable is if_link_state and not ifi_link_state like written in ifnet(9) ?

It seems that ifnet(9) (the manual page) is not up-to-date.

Time to play with live kernel debugging

root@fbsd-router:~ # kgdb /boot/kernel/kernel /dev/mem
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.

(kgdb) print ifindex_table[1].if_link_state
$1 = 2 '\002'
(kgdb) p ifindex_table[2].if_link_state
$2 = 1 '\001'
(kgdb) p ifindex_table[5].if_link_state
$3 = 0 '\0'
(kgdb) p ifindex_table[6].if_link_state
$4 = 1 '\001'

Great: I've got some values!
We can even print the full structure with command "ptype":
(kgdb) ptype ifindex_table[1]
type = struct ifnet {
    struct {
        struct ifnet *tqe_next;
        struct ifnet **tqe_prev;
    } if_link;
    struct {
        struct ifnet *le_next;
        struct ifnet **le_prev;
    } if_clones;
    struct {
        struct ifg_list *tqh_first;
        struct ifg_list **tqh_last;
    } if_groups;
    u_char if_alloctype;
    void *if_softc;
    void *if_llsoftc;
    void *if_l2com;
    const char *if_dname;
    int if_dunit;
    u_short if_index;
    short if_index_reserved;
    char if_xname[0];
    char *if_description;
    int if_flags;
    int if_drv_flags;
    int if_capabilities;
    int if_capenable;
    void *if_linkmib;
    size_t if_linkmiblen;
    u_int if_refcount;
    uint8_t if_type;
    uint8_t if_addrlen;
    uint8_t if_hdrlen;
    uint8_t if_link_state;
    uint32_t if_mtu;
    uint32_t if_metric;
    uint64_t if_baudrate;
    uint64_t if_hwassist;
Funny but useless for my current task.

How to convert link_state values in Human language ?

Another "grep LINK_STATE /usr/src/sys/net/*" for the answer: they are described in if.h:

 * Values for if_link_state.
#define LINK_STATE_UNKNOWN      0       /* link invalid/unknown */
#define LINK_STATE_DOWN         1       /* link is down */
#define LINK_STATE_UP           2       /* link is up */

Now I can translate the "kernel view" of my interfaces:

  • re0 (index 1) is in state 2 (UP)
  • re1 (index 2) is in state 1 (DOWN)
  • wlan0 (index 5) is in state 0 (UNKNOWN)
  • bridge0 (index 6) is in state 1 (DOWN)

What?!! Wait a minute: bridge0 is in state DOWN ?!? This can't be correct because my wlan0 interface is working!
The bridge logic seems to be wrong: If the first interface is in DOWN state, and the second in UNKNOW state, then bridge is in DOWN state.

Let's check by plugging re1 and testing the "kernel view" again:

(kgdb) p ifindex_table[2].if_link_state
$6 = 2 '\002'
(kgdb) p ifindex_table[6].if_link_state
$7 = 2 '\002'

Confirmed: once re1 switch to "LINK UP" state, the bridge switch to "LINK UP" too!!
There is definitively a bug in the bridge logic here, full detailed explanation here:

But I was not able, as simple user, to found by myself the exact index table name and variable name to debug :-(

Could I did it with D-trace ?

Checking kernel interface status: modern's way

Dtrace guide ( mention:
"DTrace is a comprehensive dynamic tracing framework for the illumos™ Operating System. DTrace provides a powerful infrastructure to permit administrators, developers, and service personnel to concisely answer arbitrary questions about the behavior of the operating system and user programs"

=> As a "service personnel" I should be able to use it ;-)

First "freebsd dtrace" google answer is:
This tutorial explains how to display all available dtrace probes… but what is a probe?
"A probe is a location or activity to which DTrace can bind a request to perform a set of actions, like recording a stack trace, a timestamp, or the argument to a function. Probes are like programmable sensors scattered all over your illumos system in interesting places." (for official guide).

For my "link state" problem, I will start by searching probes named  "link_state" or "linkstate":

root@fbsd-router# dtrace -l | grep 'link.*state'
   ID   PROVIDER            MODULE                          FUNCTION NAME
16563        fbt            kernel              do_link_state_change entry
16564        fbt            kernel              do_link_state_change return
16740        fbt            kernel                   vlan_link_state entry
43390        fbt            kernel              if_link_state_change entry
43391        fbt            kernel              if_link_state_change return
53619        fbt            kernel      usbd_req_set_port_link_state entry
53620        fbt            kernel      usbd_req_set_port_link_state return
55751        fbt         if_bridge                  bridge_linkstate entry
55825        fbt         bridgestp                    bstp_linkstate entry
55826        fbt         bridgestp                    bstp_linkstate return

There are some interesting results. But before to use them I had to read the dtrace guide:
"dtrace use D programming language for scripting action when probe are triggered."

As example, if I want to display "dtrace probe triggered" each time the dtrace proble "bridge_linkstate" is triggered, I can use this command:
dtrace -f 'bridge_linkstate {trace("dtrace probe triggered")}'

Here is an example:

root@fbsd-router:~ # dtrace -f 'bridge_linkstate {trace("dtrace probe triggered")}'
dtrace: description 'bridge_linkstate ' matched 1 probe
=> now I plug re1 interface to a switch

CPU     ID                    FUNCTION:NAME
  1  55751           bridge_linkstate:entry   dtrace probe triggered
=> now I unplug re1 interface

  1  55751           bridge_linkstate:entry   dtrace probe triggered
=> now I plug-back re1

  1  55751           bridge_linkstate:entry   dtrace probe triggered
=> now I unplug re1 interface

  1  55751           bridge_linkstate:entry   drace probe triggered

I've remove the other lines, because I've play with this cable during about 1 hour :-)

I can see when this probe was triggered…but I have no idea of the variable values changed (or not) before and after this call, then this information is almost useless.

Here came melifaro@ again that brings me a full dtrace script (I've added the comments):

/* The BEGIN is a special probe triggered at the begining of the script
   The purpose here is to define a table giving link_state value=>description
        a[0] = "UNKNOWN";
        a[1] = "DOWN";
        a[2] = "UP";

/* Defining an action when probe if_bridge:bridge_linkstate:entry is triggered
   Notice that each probe had an :entry and :return
   and entry or return variable are called "arg0"
   If you need the return code of the function, :return is used
/* Need to read sys/net/if_bridge.c for understanding this dtrace function
  bridge_linkstate() is called with an ifnet structure pointer as argument (arg0).
           First step is to cast arg0 into ifnet struct for using it: this->m_ifp
  But we can't directly use this->m_ifp->if_link_state, because as a bridge interface,
  this ifnet struct includes a specific "software state for each bridge"
  (bridge_softc struct) as if_bridge.
  Second step is to cast this->m_ifp->if_bridge into a bridge_softc struct: this->sc
  The first member of a bridge_softc struc is a standard ifnet structure nammed ifp.
           Then, at last, we cast ifnet struct on it: self->ifp
  Now we can get the if_link_state with self->ifp->if_link_state
        this->m_ifp = (struct ifnet *)arg0;
        this->sc = (struct bridge_softc *)this->m_ifp->if_bridge;
        self->ifp = (struct ifnet *)this->sc->sc_ifp;

/* Defining an action when probe kernel:do_link_state_change is triggered
   Notice the /self->ifp/
   This is specific to D language that didn't include control flow (like if, while,).
   Then here, each time this probe is triggered, the condition between / is checked.
   If false (=0), code is not exectuted.
   Here the condition is self->ifp, this mean this condition is triggered only if
   this variable was set (non NULL) by the previous probe.

        /* originaly a stack trace was displayed but I've commented it
/* Need to read sys/net/if.c for understanding this dtrace function
   static void do_link_state_change(void *arg, int pending)
           the first argument (arg0) is an ifnet structure, the second argument is the "new" state to apply
        printf("linkstate changed for %s to %s", stringof(self->ifp->if_xname),
/* Then reset the triggering variable to NULL */
        self->ifp = NULL;

Let's try it:

root@fbsd-router:~ # dtrace -s bridge.d
dtrace: script 'bridge.d' matched 3 probes

=> now I plug re1:

CPU     ID                    FUNCTION:NAME
  0  16563       do_link_state_change:entry linkstate changed for bridge0 to UP
=> now I unplug re1:

  1  16563       do_link_state_change:entry linkstate changed for bridge0 to DOWN
=> now I plug re1 again:

  1  16563       do_link_state_change:entry linkstate changed for bridge0 to UP

Great, this tool give "live" view of my REAL current interface status from the kernel point of view.
For using Dtrace, like with kgbd, I had to read FreeBSD sources (man page is outdated here).
For this small "easy" troubleshooting, kgbd is a lot's more easy and faster choice.
But it's too late: I've started to read FreeBSD source code and learned how to use Dtrace now :-)

Small bugs, beware I'm coming!

mercredi, septembre 16, 2015

Recipe for building a 10Mpps FreeBSD based router

First you need a server, with a minimum of 8 cores and a good NIC.
My setup is this one:

  • HP ProLiant DL360p Gen8: 8 cores Intel Xeon E5-2650 @ 2.60GHz
  • Quad port 10 Gigabit Chelsio TS540-CR

The first step is to entering the UEFI and disabling Hyper-Threading.
(note to myself: Need to generate benchmark comparing HT impact for a router use)

Once done, you can install a FreeBSD on it… but not a classical 10.2!

Default behavior of FreeBSD multi-queue NIC drivers is to create a number of queue equal to number of core (with a maximum number of 16 for Chelsio).
This mean for a 8 cores server, it will create 8 queues:
  • Each queue will obtain its own IRQ
  • The NIC will load-balance in/out frames between these queues
The NIC load-balance algorithm kepts same flow on the same queue by default: Then you need lot's of differents flow (different src/dst IP addresss or TCP/UDP ports) for a correct distribution among all theses queues: Don't bench your setup with only one FTP flow as example.

And FreeBSD meet a problem here because the number of queue/core didn't scale well after 4 cores:

=> On this 8 cores setup, you need to reduce your NIC queue number to 4 for the best performance.

But recently this problem was resolved by Alexander V. Chernikov (melifaro) on its experimental projects/routing branchs (lot's of cleaning regarding fine locking and testing new ideas):

Well… almost resolved: We see a big improvement and perfect linear scale up to 4 cores but still not linear to 8 cores. 
And surprisingly this non-linear problem isn't related to the improvement in forwarding code, but to the new random entropy harvester brings recently in head that is collecting first 2 bytes of each frame under single mutex.
Disabling INTERRUPT and NET_ETHER entropy sources (by adding harvest_mask="351" in /etc/rc.conf) solve the problem:

=> 9.5Mpps on this setup !

And for non-network people who didn't understand value in "paquet-per-second", here is a different graph regarding impact on forwarding performance with ipfw or pf enabled. With the equivalent IMIX on the right side:

Now how to reach 10Mpps? Just use a little more powerful CPU ;-)

And if you want to test these new performance on your hardware you just need one USB flash disk and installing on it these BSD Router Project (nanobsd) EXPERIMENTAL images used for theses benchs. It's just a dd to the USB flash disk, and more installation instructions are on BSDRP web site.

mercredi, juin 24, 2015

Serial-PXE-TFTP install of FreeBSD(BSDRP,Xsense,NAS4Free)/OpenBSD/Centos


Remote  installation of multiples Operating systems using only:
  • FreeBSD server with a PXE and TFTP services
  • Serial console: IPMI Serial-over-LAN (sol)

I didn't found an easy way for PXE+TFTP (only!) serial remote installation for NetBSD or DragonFly.
FreeBSD was very complex too (need to recompile bootloader for TFTP and serial usage), but hopefully mfsBSD hides this problem.
OpenBSD and CentOS, by providing ramdisk natively and easy way of configuring their bootloader, were the most admin-friendly.


This step will install an all-in-once DHCP/TFTP server:
pkg install dnsmasq
Then, create a small configuration file (example with "bce1" as NIC and local subnet in
cat > /usr/local/etc/dnsmasq.conf <<EOF
pxe-service=x86PC, "pxelinux", pxelinux

And start it:
sysrc dnsmasq_enable=yes
service dnsmasq start


This step will install pxelinux binaries and configure PXE menu:
mkdir /tftpboot
cd /tftpboot
unzip -d syslinux
cp syslinux/bios/memdisk/memdisk /tftpboot
cp syslinux/bios/core/pxelinux.0 /tftpboot
cp syslinux/bios/com32/elflink/ldlinux/ldlinux.c32 /tftpboot
cp syslinux/bios/com32/menu/menu.c32 /tftpboot
cp syslinux/bios/com32/libutil/libutil.c32 /tftpboot
cp syslinux/bios/com32/modules/pxechn.c32 /tftpboot
cp syslinux/bios/com32/lib/libcom32.c32 /tftpboot

cp syslinux/bios/com32/chain/chain.c32 /tftpboot
cp syslinux/bios/com32/modules/reboot.c32 /tftpboot/
rm -rf syslinux
mkdir /tftpboot/pxelinux.cfg
cat > /tftpboot/pxelinux.cfg/default <<EOF

SERIAL 0 115200
UI menu.c32
LABEL freebsd
 MENU LABEL mfsbsd (FreeBSD, pfSense, BSDRP, NAS4Free, etc...)
 KERNEL memdisk
 APPEND initrd=/mfsbsd-10.1-RELEASE-amd64.img harddisk raw
LABEL openbsd
 KERNEL pxechn.c32
 APPEND ::/openbsd/pxeboot
LABEL netbsd
 KERNEL pxechn.c32
 APPEND ::/netbsd/pxeboot_ia32_com0.bin
LABEL centos
 MENU LABEL Centos 7
 kernel centos/vmlinuz
 append initrd=centos/initrd.img method= devfs=nomount ip=dhcp console=ttyS0,115200 earlyprint=serial,ttyS0,115200
LABEL local
 MENU LABEL local disk
 KERNEL chain.c32

LABEL reboot
 MENU LABEL reboot
 KERNEL reboot.c32


Download mfsBSD image and enable serial port:
fetch -o /tftpboot/mfsbsd-10.1-RELEASE-amd64.img
mdconfig -a -t vnode -f mfsbsd-10.1-RELEASE-amd64.img
mount /dev/md0a /mnt/
echo "-S115200 -h" > /mnt/boot.config

umount /mnt
mdconfig -d -u 0


Download OpenBSD's pxeboot and RamDisk image, then enable serial port:
mkdir /tftpboot/openbsd/
fetch -o /tftpboot/openbsd/pxeboot
fetch -o /tftpboot/openbsd/bsd.rd
mkdir /tftpboot/etc
cat > /tftpboot/etc/boot.conf <<EOF
stty com0 115200
set tty com0
boot tftp:/openbsd/bsd.rd


Download CentOS kernel and RamDisk:
mkdir /tftpboot/centos
fetch -o /tftpboot/centos/initrd.img
fetch -o /tftpboot/centos/vmlinuz

Installing BSDRP, pfSense, OPNsense, NAS4Free, or any nanoBSD

From mfsbsd, just dd their serial nanobsd/embedded image to the local hard drive.
For installing FreeBSD: just uses bsdinstall

Debugging PXE/TFTP process

From the server, start a tcpdump accepting only bootps and tftp packets:
tcpdump -ni bce1 -vv port bootps or port tftp

lundi, octobre 13, 2014

ipfw improvement on FreeBSD -current

Few days ago Alexander V. Chernikov posted on the FreeBSD -net mailing list an "HEADS UP: Merging projects/ipfw to HEAD" with lot's of promises:
  • Tables are now identified by names, not numbers. There can be up to 65k tables with up to 63-byte long names.
  • Tables are now set-aware (default off), so you can switch/move them atomically with rules.
  • More functionality is supported (swap, lock, limits, user-level lookup, batched add/del) by generic table code.
  • New table types are added (flow) so you can match multiple packet fields at once.
  • Ability to add different type of lookup algorithms for particular table type has been added.
  • New table algorithms are added (cidr:hash, iface:array, number:array and flow:hash) to make certain types of lookup more effective.
  • Table value are now capable of holding multiple data fields for different tablearg users
I'm not an expert of ipfw(8), but I would check the impact of this improved-ipfw on forwarding performance. By "performance" I mean how this code impact the throughput (in term of packet-per-second) of my FreeBSD firewall (I didn't bench all the parameters requiered by RFC3511).
Once the code was committed as r272840 on -head, I've generated a new nanobsd(8) image on my 10gigabit bench lab… and here are the results:

More than 100K pps of differences! Now I dream of an ipfw_sync equivalent to pf_sync(4).
And here are the ministat output for statistician extremists.
Regarding ipfw in stateless mode:

x 272685.ipfw-stateless
+ 273009.ipfw-stateless
|x      x     x    x                                  + + +      +    +|
|   |______A__M___|                                                    |
|                                                     |___M__A_____|   |
    N           Min           Max        Median           Avg        Stddev
x   5       1585928       1619817       1608891     1604564.2     12728.878
+   5       1683246       1712607       1690405     1695508.6      12250.89
Difference at 95.0% confidence
        90944.4 +/- 18219.1
        5.66786% +/- 1.13546%
        (Student's t, pooled s = 12492.2)

And regarding ipfw in statefull mode:

x 272685.ipfw-statefull
+ 273009.ipfw-statefull
|xx    x   x    x                       ++   +    +                   +|
||_____A______|                                                        |
|                                    |_______M___A____________|        |
    N           Min           Max        Median           Avg        Stddev
x   5       1390415       1433678       1407058     1408663.4     18451.472
+   5       1502719       1589778       1517320     1529871.8     35404.181
Difference at 95.0% confidence
        121208 +/- 41172.4
        8.6045% +/- 2.9228%
        (Student's t, pooled s = 28230.4)

mercredi, septembre 17, 2014

PuTTY and Solarized colors

I'm using the Solarized color palette on all my FreeBSD desktops, but at work I had to works from a MS Windows desktop :-(
Here are my PuTTY settings for a correct rendering of Solarized colors.

Softwares used

Installation steps

Installing PuTTY or MTPuTTY didn't need specials instruction.
If you're using KiTTY, the Solarized PuTTY.reg files need to be adapted by opening them into a text editor and replacing the line:
By this one:
Then import the .reg files.

KiTTY/PuTTY configuration

Now start KiTTY or PuTTY, load the Dark or Light Solarized session and modify this session with:
  • Window - Colours : Enable "Allow terminal to use xterm 256-colour mode"
  • Connection - Data - Terminal details : Terminal-type string : "xterm-256color"
You can then add more customization, for example:
  • Terminal - Bell : Visual Bell
  • Window - Appearance - Font Settings - Font: "Consolas" - 12point
  • Window - Appearance - Font Settings - Font quality: ClearType
  • Window - Translation - Remote character set: UTF-8
  • Window - Lines of scrollback: 10000
  • Window - Selection - Control use of mouse: xterm (Right extends, Middle Past)
  • Connection : Second between keepalives: 25
  • Connection : Enable TCP keepalives
  • Connection - SSH - X11: Enable X11 forwarding
Once all your customization done, save the session as "Default Settings".

Checking parameters

Start a KiTTY/PuTTY, check that your preferences are loaded by default and open a SSH session to an *nix machine.
Once logged, the command "echo $TERM" should answer "xterm-256color".
And, if you've solarized your VIM (you don't need to use let g:solarized_termcolors=256!), you should correctly see the column after entering a "set colorcolumn=80".


tmux need to be configured for advertise a 256color term by adding in ~/.tmux.conf the line:
set -g default-terminal "screen-256color"

mercredi, février 19, 2014

Configuration IPv6 propre d'une Kimsufi sous FreeBSD

Pour configurer une passerelle par défaut IPv6 sur un Kimsufi, le guide officiel se résume à:
  1. Paramétrer l'IPv6 de votre interface avec votre préfixe /64 (2001:41D0:1:46e::/64 par exemple)
  2. Suivre la règle IP:v:6:FF:FF:FF:FF:FF pour déduire votre passerelle par défaut (dans notre exemple elle est donc 2001:41D0:1:4FF:FF:FF:FF:FF).
Sauf qu'avec cette règle l'IP de la passerelle est en dehors de votre réseau (/64)… donc injoignable !
«À ce qu'il paraît» cela ne pose pas de problème aux GNU/Linux…no comment.
Une autre section du guide propose de récupérer les RA pour trouver la route par défaut "link-local" annoncée par le routeur, mais cela ne fonctionne plus car ils ont été désactivés.
La solution la plus commune à ce problème est de paramétrer un préfixe /56 à la place du /64 sur votre interface: du coup la passerelle par défaut se trouve dans votre réseau et le problème est résolus.
Mais OVH m'a donné un /64, je ne vois pas pourquoi je lui déclarerai un /56!

Une jolie solution (soufflée par flo@) permettant de paramétrer un /64 tout en utilisant cette route par défaut est la suivante:

ifconfig_re0_ipv6="inet6 2001:41D0:1:46e::1 prefixlen 64"
ipv6_route_mac="-host 2001:41D0:1:4FF:FF:FF:FF:FF -iface re0"

samedi, janvier 25, 2014

Ethernet-Wifi failover on FreeBSD

I want a simple behaviour with my laptop:
  1. If Ethernet cable connected use this connectivity, otherwise use the wireless;
  2. I want to kept the same IP addresses, event if I'm using DHCP client.
The solution is quiet simple:
  1. set-up an aggregate interface in failover mode with Ethernet as primary and wireless as backup;
  2. Clone the Wireless MAC NIC to the Ethernet (opposite is not always possible with wireless chipset restriction).
Here are how to do it:

# ifconfig -l
iwn0 bge0 lo0

=> My Ethernet NIC is "bge0" and wireless is "iwn0" here

# set MAC=`ifconfig wlan0 | grep ether | cut -d ' ' -f 2`
# sysrc ifconfig_bge0="ether $MAC"
ifconfig_bge0:  -> ether 00:1c:23:25:ab:45
# sysrc wlans_iwn0=wlan0
wlans_iwn0:  -> wlan0

# sysrc ifconfig_wlan0="WPA up"
ifconfig_wlan0: WPA DHCP -> WPA up
# sysrc cloned_interfaces=lagg0
cloned_interfaces:  -> lagg0
# sysrc ifconfig_lagg0="laggproto failover laggport bge0 laggport wlan0 DHCP"
ifconfig_lagg0:  -> laggproto failover laggport bge0 laggport wlan0 DHCP
# sysrc ifconfig_lagg0_ipv6="inet6 accept_rtadv"
ifconfig_lagg0_ipv6:  -> inet6 accept_rtadv

# service netif restart

And now with Ethernet cable unplugged:

# ifconfig lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 00:1c:23:25:ab:45
    inet6 fe80::41d:23ff:fe25:ab78%lagg0 prefixlen 64 scopeid 0x4
    inet6 2a01:e35:9b9d:a1a0:41d:23ff:fe25:ab45 prefixlen 64 autoconf
    inet netmask 0xffffff00 broadcast
    media: Ethernet autoselect
    status: active
    laggproto failover lagghash l2,l3,l4
    laggport: wlan0 flags=4<ACTIVE>
    laggport: bge0 flags=1<MASTER>

Then If I plug the Ethernet cable:

# ifconfig lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 00:1c:23:25:ab:45
    inet6 fe80::41d:23ff:fe25:ab45%lagg0 prefixlen 64 scopeid 0x4
    inet6 2a01:e35:9b9d:a1a0:41d:23ff:fe25:ab45 prefixlen 64 autoconf
    inet netmask 0xffffff00 broadcast
    media: Ethernet autoselect
    status: active
    laggproto failover lagghash l2,l3,l4
    laggport: wlan0 flags=0<>
    laggport: bge0 flags=5<MASTER,ACTIVE>

mercredi, juin 05, 2013

Generating custom FreeBSD installation media


Generating a custom -current memstick image without all the debug feature enabled.


Have the head source installed (I will use /usr/src as example).
If not, here is an example for synchronizing up-to-date head (-current) sources on /usr/src
svnlite co svn:// /usr/src

Customizing build compilation options

A little debug feature to disable on -current:
echo "MALLOC_PRODUCTION=yes" > /etc/src.conf

Building world and kernel

New we start the classical building of world and our customized kernel.
For information this step takes about 4 hours on my PC.
cd /usr/src
make buildworld; make buildkernel KERNCONF=GENERIC-NODEBUG

Generating install media image

Last step: Generating the install media.
Here is an example for generating memstick install media without port tree (long live to pkgng! neither doc):
cd /usr/src/release
make -DNOPORTS -DNODOC memstick

Replace "memstick" by "cdrom" (bootonly.iso and release.iso) or "ftp" for other media.
You can add a -DNOSRC option for avoiding to include sources too on the media.

Then copy the image to your usb key:
dd if=memstick of=/dev/da0 bs=64k

Cleaning your mess

Your system has lot's of file that you don't need anymore, here is how to clean it:
cd /usr/src/release
make clean
cd /usr/src
make clean

mercredi, mars 06, 2013

Xorg for FreeBSD on Raspberry Pi

Thanks to Ray, we have a working X11 drivers that works on Raspberry pi too.
Here is how to install it:
Download the latest Daisuke's Raspberry pi FreeBSD image (login/password: pi/ raspberry and root password: raspberry).
Unzip, dd it on your 8GB SD card, boot your rspie and configure your network (Internet access mandatory for downloading pkg).
Install USB mouse drivers:
cd /usr/src/sys/modules/usb/ums
make clean all install

Install pkg and declare the repository:
tar xf ./pkg.txz -s ",/.*/,,g" "*/pkg-static"
./pkg-static add ./pkg.txz

echo 'packagesite:${ABI}' > /usr/local/etc/pkg.conf
Install Xorg, scfb drivers and some X apps:
pkg inst xorg-minimal xf86-video-scfb xorg-apps 
Configure /etc/X11/xorg.conf:
Section "Files"
Section "Module"
    Load        "dbe"
    Disable    "dri"
    Disable    "dri2"
    Disable    "glx"
    SubSection  "extmod"
       Option  "omit xfree86-dga"

Section "ServerFlags"
    Option    "AIGLX"        "false"
    Option    "NoAccel"    "True"
    Option    "NoDRI"        "True"
    Option    "DRI"        "False"
    Option    "DRI2"        "False"

Section "InputDevice"
    Identifier  "Keyboard1"
    Driver      "kbd"

Section "InputDevice"
    Identifier  "Mouse1"
    Driver      "mouse"
    Option      "Protocol"      "auto"
    Option      "Device"        "/dev/sysmouse"

Section "Monitor"
    Identifier  "Monitor"

Section "Device"
    Identifier  "Generic FB"
    Driver      "scfb"
    Option    "NoAccel"    "True"

Section "Screen"
    Identifier  "Screen"
    Device      "Generic FB"
    Monitor     "Monitor"
    DefaultDepth 16
    SubSection "Display"
       Depth           16

Section "ServerLayout"
    Identifier  "layout"
    Screen      0 "Screen" 0 0
    InputDevice "Mouse1" "CorePointer"
    InputDevice "Keyboard1" "CoreKeyboard"
Then a dummy ~/.xinitrc:
xterm &
xclock &
xcalc &
exec twm

And enjoy:

A first ugly screenshoot: