Archive

Archive for August, 2012

Automating Generation of the mfsroot.gz

August 21, 2012 Leave a comment

NOTE: This post is old. I no longer generate mfsroots in this fashion and my methods continue to evolve, but my latest approach to modifying mfsroots is documented in the Installing FreeBSD via Cobbler post.

Automating Generation of the mfsroot.gz

This is a companion post to Automating FreeBSD crunchgen(1). In addition to boot_crunch files, I generate mfsroot.gz files for each FreeBSD release to be integrated into our provisioning platform, Cobbler. The mfsroot.gz file in FreeBSD, in this context, is a memory file system that contains the bits necessary to install a FreeBSD OS on a system. Explained here is how to automate generation of that file.

Like the crunchgen(1) script, this is a starting point only that makes many assumptions. In fact, while writing this I have come up with plans to combine them.

I store an uncompressed mfsroot.gz file in subversion. It is stored in a manner that includes the files in stand/. You may or may not be aware that this is where the boot_crunch file exists. It also happens to be the location of all the hardlinks that point back to the boot_crunch inode. Subversion stores these as if they were all binaries. Below you will see that I programmatically restore the hardlinks.

A side effect of this management structure is that generating the same mfsroot.gz multiple times results in differing file sizes each time despite using the same content. I’m not 100% sure, but believe that this is due to the file metadata being different each time the content is exported from the subversion repo.

NOTE: The URL in the script below is a dummy URL. The real URL is internally hosted with no connectivity to the Internet.

#! /bin/sh
# 
# mkmfsroot.sh:   Generate a FreeBSD mfsroot file.  This script does not
#                 function on hosts that are not running FreeBSD.
# 
# Author:   Rick
# Date:     21 August 2012
# Modif:

#####
## Variables
#####
      
SELF="${0##*/}";
NULL="/dev/null";
TMP="/tmp";
PATH="${PATH}";
ZERO="/dev/zero";
MFSROOTDIR="${1}";
SVNREPO="https://dummy.subversion.com/ats/ad/freebsd-prov/mfsroot/${MFSROOTDIR}";
REPODIR="/data/freebsd-prov/mfsroot/${MFSROOTDIR}";
MFSROOTSTAND="${REPODIR}/stand";
SVN="/usr/local/bin/svn";
STANDFILES="doconfig.sh install.cfg boot_crunch";
MFSROOT="/tmp/mfsroot";
MFSROOTMNT="/tmp/mfsrootmnt";

#####
## Functions
#####

# Clean what this script has done
CLEANUP() {

   [ "${MDCFG}" = "True" ] && mdconfig -d -u ${DEVNUM};

   if [ -d ${MFSROOTMNT} ]; then
      cd && umount ${MFSROOTMNT} > ${NULL} 2>&1;
      rm -rf ${MFSROOTMNT};
   fi
   [ -e ${MFSROOT} ] && rm ${MFSROOT};
   [ -e ${REPODIR} ] && rm -rf ${REPODIR};
   
}

# We are erroring and exiting the script
ERR_EXIT() {

   echo "${1}" 1>&2;
   CLEANUP;
   exit 1;

}

#####
## Main
#####

# Force two command line arguments
[ -z ${1} ] && ERR_EXIT "Specify an mfsroot!";

# Verify we're on a FreeBSD host
HOSTOS=`uname -s`;
[ ${HOSTOS} != "FreeBSD" ] && ERR_EXIT "Build host is not FreeBSD! Exiting!";

# Ensure we're using the latest repo version
[ -d ${REPODIR} ] && rm -rf ${REPODIR};
${SVN} export ${SVNREPO} ${REPODIR};

# Does the mfsroot dir we asked for exist?
[ ! -d ${REPODIR} ] && ERR_EXIT "${1} does not exist! Exiting!";

# Remove .svn if it exists
[ -d ${REPODIR}/.svn ] && rm -rf ${REPODIR}/.svn

# Fix stand/
#
# What do we do here and why?
#
# Since the contents of the mfsroot are stored in subversion, when the content
# is checked out or exported, each of the files in stand/ are stored as
# individual files when they are actually supposed to be hardlinks back to
# boot_crunch.  Therefore, the code below checks out the content, removes the
# files in stand/, and exports boot_crunch and a couple other scripts.  It
# finishes by regenerating all the hardlinks back to boot_crunch.
#
# We could resolve this in any number of ways, but as a stop gap measure for
# now we will leave it this way and fix it in the future.
if [ -d ${MFSROOTSTAND} ]; then
   rm ${MFSROOTSTAND}/*;
   for ea in ${STANDFILES}; do
      ${SVN} export ${SVNREPO}/stand/${ea} ${MFSROOTSTAND};
   done
 
   cd ${MFSROOTSTAND};
   for i in $(./boot_crunch 2>&1 | grep -v usage); do
      if [ "$i" != "boot_crunch" ]; then
         rm -f ./"$i";
         ln ./boot_crunch "$i";
      fi
   done
fi

# Calculate the size of the mfsroot, prepare the file and populate it
MFSROOTSIZE=$(du -sk ${REPODIR} | awk '{ print $1 }');
[ -z ${MFSROOTSIZE} ] && ERR_EXIT "Unable to determine mfsroot size! Exiting!";

dd if=${ZERO} of=${MFSROOT} bs=1024 count=$((${MFSROOTSIZE} + 1024));
[ $? -ge 1 ] && ERR_EXIT "Unable to create ${MFSROOT}! Exiting!";

MFSROOTDEV=$(mdconfig -f ${MFSROOT});
[ -z ${MFSROOTDEV} ] && ERR_EXIT "Unable to mount ${MFSROOT}! Exiting!";
DEVNUM=$(echo ${MFSROOTDEV} | awk '{ print substr($0,length($0)-0,length($0)) }'
);
MDCFG="True";

newfs /dev/${MFSROOTDEV};

[ ! -d ${MFSROOTMNT} ] && mkdir ${MFSROOTMNT};
mount /dev/${MFSROOTDEV} ${MFSROOTMNT} && \
   (cd ${REPODIR}; tar -cf - .) | (cd ${MFSROOTMNT}; tar -xf -);
   
# Now the new mfsroot has been populated, it's time to save it and clean up
umount ${MFSROOTMNT} && mdconfig -d -u ${DEVNUM} && gzip ${MFSROOT};
[ $? -ne 0 ] && ERR_EXIT "Error unmounting ${MFSROOT}! Exiting!";
MDCFG="False";
CLEANUP;
         
#####
## EOF
#####
Advertisements
Categories: FreeBSD

FreeBSD vlan Tagging

August 14, 2012 5 comments

FreeBSD vlan Tagging

Virtual LANs were introduced by IEEE 802.1Q, a standards definition for tagging of ethernet frames across vlan-aware appliances. It is also known as vlan tagging. Virtual LANs add the ability to separate data on a single transport media into logical networks.

This post explains vlan tagging configurations in FreeBSD. Network appliances making up your network also need to be configured with vlan tagging support, but is beyond the scope of this post.

If vlan tagging support is compiled into the kernel, it is not required to include the following in the loader.conf:

if_vlan_load=”YES”

Configuring vlan tagging on a running system is done by executing the following:

# ifconfig $int up
# ifconfig vlan0 create
# ifconfig vlan0 10.1.1.5 netmask 255.255.255.0 vlan $id vlandev $int
# ifconfig vlan0_alias0 10.1.2.5 netmask 255.255.255.0 vlan $id vlandev $int

Configuring so vlan tags persists across reboots is done by inserting the following in rc.conf:

ifconfig_$int=”up”
cloned_interfaces=”vlan0″
ifconfig_vlan0=”10.1.1.5 netmask 255.255.255.0 vlan $id vlandev $int”
ifconfig_vlan0_alias0=”10.1.2.5 netmask 255.255.255.0 vlan $id vlandev $int”
ifconfig_vlan0_alias1=”10.1.3.5 netmask 255.255.255.0 vlan $id vlandev $int”

In the above examples, $id is the vlan id and $int is the physical network interface.

Also note, additional alias’ added must be added by appending alias[0-9] to the cloned interface in numerical order. Each alias is brought up until it encounters the an alias which is not in sequential order. It then finishes the OS boot without those alias’ added.

For further reading, check out the vlan(4) manpage and the ifconfig(8) manpage.

Categories: FreeBSD

FreeBSD uname(1) and the Environment

August 9, 2012 Leave a comment

FreeBSD uname(1) and The Environment

The default values provided by uname(1) can be over-ridden on a running system utilizing environment variables. The uname(1) manpage identifies this behavior in the “Environment” section stating:

An environment variable composed of the string UNAME_ followed by any flag to the uname utility (except for -a) will allow the corresponding data to be set to the contents of the environment variable.

I would like to call out the environment variables specifically and identify what is changed when these environment variables are utilized. They are as follows:

  • UNAME_s: The name of the operating system implementation
  • UNAME_n: The name of the system
  • UNAME_r: The current release level of the operating system
  • UNAME_v: The version level of this release of the operating system
  • UNAME_m: The type of the current hardware platform
  • UNAME_p: The type of the machine processor architecture
  • UNAME_i: The kernel ident
Categories: FreeBSD