Archive
Building FreeBSD Media With Custom Packages
Building FreeBSD Media With Custom Packages
Building a FreeBSD release can be as simple as executing:
# cd /usr/src # make buildworld # cd release # make release # PKG_TREE=$pkg_dir PKG_DEST=$CHROOTDIR/R/cdrom/dvd1 PKG_DVD=Yes make package-split
The Use Case

I generate company specific FreeBSD distributions based on our internal requirements including loading specific packages. The FreeBSD DVD release from freebsd.org includes a packages/ directory created by the package-split.py script. I use the same script to accomplish the same task with custom packages.
We generate an ISO from the release and import it into Cobbler. We’ve setup Cobbler to allow us to provision operating systems non-interactively using PXE, DHCP, TFTP, and HTTP.
Prerequisites
- A directory containing at least the packages to include. One can download packages for 8.3-RELEASE via ftp at ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.3-release/.
- An INDEX file. The INDEX file will already exist if one downloads the packages from the above ftp server.
Configuring package-split.py
package-split.py is a Python script that “generates a master INDEX file for the CD images built by the FreeBSD release engineers. Each disc is given a list of desired packages. Dependencies of these packages are placed on either the same disc or an earlier disc. The resulting master INDEX file is then written out”.
In our use case, the files are not split into separate discs because we are packaging for a DVD as identified by the PKG_DVD environment variable.
View the package-split.py here. Because I am currently working with 8.3-RELEASE, I link directly to the that version of the script.
Our use case involves commenting lines 43 – 81 and copying lines 63 – 81. The copied lines are modified with names of packages installed in our base distribution. An example might look similar to:
if doing_dvd:
pkgs.extend(['ftp/wget',
'devel/subversion',
'lang/perl5.12',
'lang/python',
'lang/ruby18',
'net/rsync'])
Caveats
- There are inconsistencies in the INDEX file where some package names and origins do not match. An example such as ruby18-iconv shows the package name (column 1) is ruby18-iconv-$version while the origin (column 2) is /usr/ports/converters/ruby-iconv. `make package-split` errored on this package citing “Unable to find package for converters/ruby18-iconv” (see image below). Changing origin from “ruby-iconv” to “ruby18-iconv” resolved this.
- use the origin field to identify the package in configuring the package-split.py script. For example, ruby18-iconv, would be specified as ‘converters/ruby19-iconv’ as opposed to ‘ruby/ruby18-iconv’ despite having multiple categories and existing multiple directories.

Executing `make package-split`
Once the release is built, create the packages directory and move it to the new release by executing the following. This would also be an appropriate time to make additional modifications to the image such as replacing the mfsroot.gz with a new custom mfsroot.
# PKG_TREE=/tmp/packages PKG_DEST=$CHROOTDIR/R/cdrom PKG_DVD=Yes make package-split # mv $CHROOTDIR/R/cdrom/disc1/packages $CHROOTDIR/R/cdrom/dvd1
Generate the ISO
The release is now ready to be made into an ISO that we use to import the distribution into our provisioning platform, Cobbler.
# mkisofs -R -no-emul-boot -b boot/cdboot -o /tmp/iso/FreeBSD-8.3-RELEASE-p4-amd64-CUSTOM.iso $CHROOTDIR/R/cdrom/dvd1
Building FreeBSD With Multiple Kernels
Building a Release With Multiple Kernels
The Use Case
FreeBSD installs a run-time kernel used during normal day-to-day operations. In the event of repeated kernel panics, there may be a need for a kernel capable of online debugging.
Patching sysinstall
sysinstall(8) lacks support for installing multiple kernel distributions in releases 8.3 or earlier. Patch r240972 was committed to stable/8 enabling support for a DEBUG kernel distribution. The procedure below patches the sources with bits necessary for the remainder of this post.
# cd /usr/src
# svn diff -c 240972 http://svn.freebsd.org/base/stable/8 > sysinstall_patch
# patch -p0 < sysinstall_patch
Replacing the string DEBUG in the patch changes the kernel distribution name.
Preparing The Build
The only prerequisite is having FreeBSD sources available and, optionally, ports. Ports is obtained via cvs or Subversion. Information on FreeBSD source trees are found here.
We assume a FreeBSD release build with two kernels, GENERIC and DEBUG, on the amd64 architecture. To avoid using CVS/SVN as part of the build process, it is helpful to have a local copy of ports and FreeBSD sources. In this example, ports is /usr/ports and the sources are /usr/src.
First we must buildworld:
# cd /usr/src && make buildworld
The Kernel Config
The FreeBSD release build process builds the GENERIC kernel by default. Adding an additional kernel is accomplished by first creating the kernel config file. We will create a kernel config called /usr/src/sys/amd64/conf/DEBUG. The kernel config should appear as follows:
include GENERIC
options KDB_UNATTENDED
options DDB
Compiling The Release
Now compile the release (as root, in bash):
# cd release
# CHROOTDIR=/usr/release CVSROOT=/home/ncvs EXTPORTSDIR=/usr/ports EXTSRCDIR=/usr/src MAKE_DVD=Yes NO_FLOPPIES=Yes NODOC=Yes KERNELS=DEBUG KERNCONF=DEBUG INSTKERNNAME=DEBUG make release
The CVSROOT environment variable is non-optional. It is used to download ports and sources. If the environment variables EXTPORTSDIR and EXTSRCDIR are set, `make release` will not consult CVSROOT, despite being non-optional.
Executing/Using The Kernel
From the loader menu
Press 6 at the loader menu to escape to the loader(8) prompt and type:
boot DEBUG
From the running system
The DEBUG kernel is loaded by default when the kernel line in loader.conf appears as follows. Each subsequent boot loads the DEBUG kernel.
kernel=DEBUG
Dropping into the debugger
A kernel panic induces a system reboot unless debug.debugger_on_panic is enabled as follows:
sysctl debug.debugger_on_panic=1
To induce a panic:
sysctl debug.kdb.panic=1
So You Want A FreeBSD Debugging Kernel…
Installing a debugging kernel in FreeBSD
The Use Case
There are numerous reasons for needing a debugging kernel. For example, during a kernel panic. This was the case with us. Therefore, we installed a kernel with DDB enabled. In the event of a kernel panic, the system would drop to the debugger and the software engineers were then able to begin diagnosing the problem.
Preparing To Build The Kernel
Prerequisites
- FreeBSD source tree
- A kernel config
FreeBSD Sources
Operators can use cvs or Subversion to install the sources. Information on FreeBSD source trees can be found here.
The Kernel Config
Create a kernel config by opening /usr/src/sys/${arch}/conf/DEBUG. Where ${arch} is the machine architecture the kernel is to be built for. In this case, we build an amd64 kernel on an amd64 machine. The kernel config should appear as follows:
include GENERIC
options KDB_UNATTENDED
options DDB
Compiling and Installing The Kernel
Compiling and installing the debugging kernel is as simple as executing:
# cd /usr/src
# make buildkernel KERNCONF=DEBUG INSTKERNNAME=DEBUG
# make installkernel KERCONF=DEBUG INSTKERNNAME=DEBUG
Executing/Using The Kernel
From the loader menu
At the loader menu, press 6 to escape to the loader(8) prompt and type:
boot DEBUG
From the running system
The DEBUG kernel can be loaded by default by adding/modifying the kernel directive in loader.conf as follows. Each subsequent boot will load the DEBUG kernel.
kernel=DEBUG
Dropping into the debugger
During a kernel panic, it will simply reboot unless debug.debugger_on_panic is enabled. To enable this execute:
sysctl debug.debugger_on_panic=1
To induce a panic:
sysctl debug.kdb.panic=1
Installing FreeBSD 8.x Distributions After Installation
Installing Distributions After The Fact
FreeBSD 8.x [ and earlier ] media contain distributions one can select to install via sysinstall(8). This is accomplished via GUI or automated scripting requiring an install.cfg.

This image shows a typical FreeBSD installation where a kernel/distribution is extracted onto disk, What is an admin to do when they later find that they need to install a distribution after the fact? There are two methods to installing media distributions after the fact.
Using the Command Line
This is my preferred method as it does not require using sysinstall‘s graphical interface, but I am sure both have their merits. In this example, we install lib32. First, download the distribution from FreeBSD.org…
echo CHECKSUM.SHA256 CHECKSUM.MD5 install.sh lib32.a{a,b,c,d,e,f,g,h,i,j,k,l,m} lib32.inf lib32.mtree | xargs -n 1 -I%s fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/8.2-RELEASE/lib32/%s
The next step is to install the distribution…
./install.sh
and you’re done…
Using sysinstall(8)
Start the procedure by executing sysinstall. It opens an interactive GUI interface. The Following procedure installs the distribution…
Select “Configure”…

Select “Distributions”…

Select the distribution to install…

Select install media (this example uses ftp)…

Select ftp location (This step may differ depending on the media selected above)…

If networking is configured, select yet. Otherwise, select no and sysinstall will prompt for network configuration…

The distribution installs…

…and when completed, exit sysinstall.
References
* The command line used can be referenced in the FreeBSD Forums
Encrypting User Password Strings
Encrypting User Password Strings
The Use Case
User account passwords can be set during systems provisioning in multiple ways. Setting a host’s root password is a common use case. During a FreeBSD 8.x install, one can set root’s password (or other user account password) executing the following within the install.cfg:
/bin/sh -c ‘echo passwd |pw mod user username -h 0’
Unfortunately, doing so exposes the user’s account password. Therefore, the recommendation is to use an encrypted hash as follows:
/usr/bin/chpass -p $1$nPUexUs5$O4JuN.Ed/LqWHJKmf8K0h0 root
Encrypting The Password
The use case begs the question: How does one ascertain an encrypted hash given a string?
On FreeBSD, there are varying methods to ascertain an encrypted hash. Here are just two of the many.
openssl
# openssl passwd -1 MySecret
$1$YRth1v3T$MvGupL8n.VBjvM12JhR4G0
openssl(1) returns the entire encrypted hash. The character between the first and second dollar signs identifies the crypt mechanism. The following eight characters are the salt. The remaining characters following the third dollar sign are the encrypted string.
md5
# md5 -s MySecret
MD5 (“MySecret”) = 4132d75e6cb04073cc7756707057027f
md5(1) returns a partial encrypted hash. In the key-value pair, the key is the crypt mechanism. The first 8 characters of the value are the salt. The remaining characters are the encrypted string.
In this method, one must prepend $1$ to the value before feeding it into chpass(1).
Related Documentation
* Chapter 15, Security, of the FreeBSD handbook explains identifying a host’s crypt mechanism.
Automating Generation of the mfsroot.gz
NOTE: This post is old. I no longer generate mfsroots in this fashion and my methods continue to evolve, but my latest approach to modifying mfsroots is documented in the Installing FreeBSD via Cobbler post.
Automating Generation of the mfsroot.gz
Like the crunchgen(1) script, this is a starting point only that makes many assumptions. In fact, while writing this I have come up with plans to combine them.
I store an uncompressed mfsroot.gz file in subversion. It is stored in a manner that includes the files in stand/. You may or may not be aware that this is where the boot_crunch file exists. It also happens to be the location of all the hardlinks that point back to the boot_crunch inode. Subversion stores these as if they were all binaries. Below you will see that I programmatically restore the hardlinks.
A side effect of this management structure is that generating the same mfsroot.gz multiple times results in differing file sizes each time despite using the same content. I’m not 100% sure, but believe that this is due to the file metadata being different each time the content is exported from the subversion repo.
NOTE: The URL in the script below is a dummy URL. The real URL is internally hosted with no connectivity to the Internet.
#! /bin/sh
#
# mkmfsroot.sh: Generate a FreeBSD mfsroot file. This script does not
# function on hosts that are not running FreeBSD.
#
# Author: Rick
# Date: 21 August 2012
# Modif:
#####
## Variables
#####
SELF="${0##*/}";
NULL="/dev/null";
TMP="/tmp";
PATH="${PATH}";
ZERO="/dev/zero";
MFSROOTDIR="${1}";
SVNREPO="https://dummy.subversion.com/ats/ad/freebsd-prov/mfsroot/${MFSROOTDIR}";
REPODIR="/data/freebsd-prov/mfsroot/${MFSROOTDIR}";
MFSROOTSTAND="${REPODIR}/stand";
SVN="/usr/local/bin/svn";
STANDFILES="doconfig.sh install.cfg boot_crunch";
MFSROOT="/tmp/mfsroot";
MFSROOTMNT="/tmp/mfsrootmnt";
#####
## Functions
#####
# Clean what this script has done
CLEANUP() {
[ "${MDCFG}" = "True" ] && mdconfig -d -u ${DEVNUM};
if [ -d ${MFSROOTMNT} ]; then
cd && umount ${MFSROOTMNT} > ${NULL} 2>&1;
rm -rf ${MFSROOTMNT};
fi
[ -e ${MFSROOT} ] && rm ${MFSROOT};
[ -e ${REPODIR} ] && rm -rf ${REPODIR};
}
# We are erroring and exiting the script
ERR_EXIT() {
echo "${1}" 1>&2;
CLEANUP;
exit 1;
}
#####
## Main
#####
# Force two command line arguments
[ -z ${1} ] && ERR_EXIT "Specify an mfsroot!";
# Verify we're on a FreeBSD host
HOSTOS=`uname -s`;
[ ${HOSTOS} != "FreeBSD" ] && ERR_EXIT "Build host is not FreeBSD! Exiting!";
# Ensure we're using the latest repo version
[ -d ${REPODIR} ] && rm -rf ${REPODIR};
${SVN} export ${SVNREPO} ${REPODIR};
# Does the mfsroot dir we asked for exist?
[ ! -d ${REPODIR} ] && ERR_EXIT "${1} does not exist! Exiting!";
# Remove .svn if it exists
[ -d ${REPODIR}/.svn ] && rm -rf ${REPODIR}/.svn
# Fix stand/
#
# What do we do here and why?
#
# Since the contents of the mfsroot are stored in subversion, when the content
# is checked out or exported, each of the files in stand/ are stored as
# individual files when they are actually supposed to be hardlinks back to
# boot_crunch. Therefore, the code below checks out the content, removes the
# files in stand/, and exports boot_crunch and a couple other scripts. It
# finishes by regenerating all the hardlinks back to boot_crunch.
#
# We could resolve this in any number of ways, but as a stop gap measure for
# now we will leave it this way and fix it in the future.
if [ -d ${MFSROOTSTAND} ]; then
rm ${MFSROOTSTAND}/*;
for ea in ${STANDFILES}; do
${SVN} export ${SVNREPO}/stand/${ea} ${MFSROOTSTAND};
done
cd ${MFSROOTSTAND};
for i in $(./boot_crunch 2>&1 | grep -v usage); do
if [ "$i" != "boot_crunch" ]; then
rm -f ./"$i";
ln ./boot_crunch "$i";
fi
done
fi
# Calculate the size of the mfsroot, prepare the file and populate it
MFSROOTSIZE=$(du -sk ${REPODIR} | awk '{ print $1 }');
[ -z ${MFSROOTSIZE} ] && ERR_EXIT "Unable to determine mfsroot size! Exiting!";
dd if=${ZERO} of=${MFSROOT} bs=1024 count=$((${MFSROOTSIZE} + 1024));
[ $? -ge 1 ] && ERR_EXIT "Unable to create ${MFSROOT}! Exiting!";
MFSROOTDEV=$(mdconfig -f ${MFSROOT});
[ -z ${MFSROOTDEV} ] && ERR_EXIT "Unable to mount ${MFSROOT}! Exiting!";
DEVNUM=$(echo ${MFSROOTDEV} | awk '{ print substr($0,length($0)-0,length($0)) }'
);
MDCFG="True";
newfs /dev/${MFSROOTDEV};
[ ! -d ${MFSROOTMNT} ] && mkdir ${MFSROOTMNT};
mount /dev/${MFSROOTDEV} ${MFSROOTMNT} && \
(cd ${REPODIR}; tar -cf - .) | (cd ${MFSROOTMNT}; tar -xf -);
# Now the new mfsroot has been populated, it's time to save it and clean up
umount ${MFSROOTMNT} && mdconfig -d -u ${DEVNUM} && gzip ${MFSROOT};
[ $? -ne 0 ] && ERR_EXIT "Error unmounting ${MFSROOT}! Exiting!";
MDCFG="False";
CLEANUP;
#####
## EOF
#####
FreeBSD vlan Tagging
FreeBSD vlan Tagging
This post explains vlan tagging configurations in FreeBSD. Network appliances making up your network also need to be configured with vlan tagging support, but is beyond the scope of this post.
If vlan tagging support is compiled into the kernel, it is not required to include the following in the loader.conf:
if_vlan_load=”YES”
Configuring vlan tagging on a running system is done by executing the following:
# ifconfig $int up
# ifconfig vlan0 create
# ifconfig vlan0 10.1.1.5 netmask 255.255.255.0 vlan $id vlandev $int
# ifconfig vlan0_alias0 10.1.2.5 netmask 255.255.255.0 vlan $id vlandev $int
Configuring so vlan tags persists across reboots is done by inserting the following in rc.conf:
ifconfig_$int=”up”
cloned_interfaces=”vlan0″
ifconfig_vlan0=”10.1.1.5 netmask 255.255.255.0 vlan $id vlandev $int”
ifconfig_vlan0_alias0=”10.1.2.5 netmask 255.255.255.0 vlan $id vlandev $int”
ifconfig_vlan0_alias1=”10.1.3.5 netmask 255.255.255.0 vlan $id vlandev $int”
In the above examples, $id is the vlan id and $int is the physical network interface.
Also note, additional alias’ added must be added by appending alias[0-9] to the cloned interface in numerical order. Each alias is brought up until it encounters the an alias which is not in sequential order. It then finishes the OS boot without those alias’ added.
For further reading, check out the vlan(4) manpage and the ifconfig(8) manpage.
FreeBSD uname(1) and the Environment
FreeBSD uname(1) and The Environment
An environment variable composed of the string UNAME_ followed by any flag to the uname utility (except for -a) will allow the corresponding data to be set to the contents of the environment variable.
I would like to call out the environment variables specifically and identify what is changed when these environment variables are utilized. They are as follows:
- UNAME_s: The name of the operating system implementation
- UNAME_n: The name of the system
- UNAME_r: The current release level of the operating system
- UNAME_v: The version level of this release of the operating system
- UNAME_m: The type of the current hardware platform
- UNAME_p: The type of the machine processor architecture
- UNAME_i: The kernel ident
GNU tar Header Errors?
Dealing with GNU/tar’s Extended Header Warnings

It seems that FreeBSD‘s tar binary utilizes additional headers that GNU tar does not recognize. There are many blog posts about this. The problem appears to affect tarballs created on various operating systems that employ varying versions of tar and extracting those tarballs with GNU tar.
Generally speaking, these warning can be ignored. On the command line, the files contained within the tarball will still extract (or at least they have every time I’ve encountered this). This does, however, present a problem if one is programmatically encountering this. One can resolve this by addressing the issue within their code or by using the same version of tar on each platform to create and extract the tarballs in question.
Automating FreeBSD’s crunchgen(1)
Automating FreeBSD’s crunchgen(1)
One of my main tasks in my role at work is to maintain FreeBSD provisioning bits. I have found myself generating boot_crunch files with various binaries and what not to extend the capabilities of the provisioning aspect of FreeBSD. So much so that I whipped up a quick script to automate the task for me.
Please bear in mind, it is a quick script which makes a number of assumptions. It’s not perfect and can certainly use some tweaking and the like, but this initial version gets one started.
I keep my boot_crunch.conf in a subversion repo, thus the calls to svn.
#! /bin/sh
#
# bldcrunch.sh: Generate a FreeBSD boot_crunch file. This script does not
# function on hosts that are not running FreeBSD.
#
# Author: Rick
# Date: 25 July 2012
# Modif:
#####
## Variables
#####
SELF="${0##*/}";
NULL="/dev/null";
TMP="/tmp";
PATH="${PATH}";
SVNREPO="http://svn.domain.com/freebsd-repo";
REPODIR="/freebsd-repo";
BOOTCRUNCHDIR="boot_crunch";
BOOTCRUNCHCONF="boot_crunch.conf";
BOOTCRUNCHMK="boot_crunch.mk";
SVNBIN="/usr/local/bin/svn";
CRUNCHGEN="/usr/bin/crunchgen";
MAKE="/usr/bin/make";
#####
## Functions
#####
ERROR_EXIT() {
echo "${1}" 1>&2;
[ -d ${TMP}/${BOOTCRUNCHDIR} ] && rm ${TMP}/${BOOTCRUNCHDIR};
exit 1;
}
#####
## Main
#####
# Verify we're on a FreeBSD host
HOSTOS=`uname -s`;
[ ${HOSTOS} != "FreeBSD" ] && ERROR_EXIT "Build host is not FreeBSD! Exiting!";
# Do all of our binaries exist?
for ea in ${SVNBIN} ${CRUNCHGEN} ${MAKE}; do
if [ -e ${ea} ]; then
continue;
else
ERROR_EXIT "${ea} is required! Exiting!";
fi
done
# Update SVN repo to make sure we're using the latest boot_crunch.conf
if [ -d ${REPODIR}/${BOOTCRUNCHDIR} ]; then
cd ${REPODIR}/${BOOTCRUNCHDIR} && ${SVNBIN} update;
else
${SVN} co ${SVNREPO} ${REPODIR};
fi
# Setup our build environment
[ -d ${TMP}/${BOOTCRUNCHDIR} ] && rm -rf ${TMP}/${BOOTCRUNCHDIR};
mkdir ${TMP}/${BOOTCRUNCHDIR} && cd ${TMP}/${BOOTCRUNCHDIR};
# Build the boot_crunch
${CRUNCHGEN} -o ${REPODIR}/${BOOTCRUNCHDIR}/${BOOTCRUNCHCONF};
[ $? -eq 0 ] && ${MAKE} -f ${BOOTCRUNCHMK};
[ $? -ne 0 ] && ERROR_EXIT "Error generating boot_crunch! Exiting!";
#####
## EOF
#####
BTW, I edited this because I found out about wordpress’s sourcecode tag…awesome!