Home > FreeBSD > Automating Generation of the mfsroot.gz

Automating Generation of the mfsroot.gz

NOTE: This post is old. I no longer generate mfsroots in this fashion and my methods continue to evolve, but my latest approach to modifying mfsroots is documented in the Installing FreeBSD via Cobbler post.

Automating Generation of the mfsroot.gz

This is a companion post to Automating FreeBSD crunchgen(1). In addition to boot_crunch files, I generate mfsroot.gz files for each FreeBSD release to be integrated into our provisioning platform, Cobbler. The mfsroot.gz file in FreeBSD, in this context, is a memory file system that contains the bits necessary to install a FreeBSD OS on a system. Explained here is how to automate generation of that file.

Like the crunchgen(1) script, this is a starting point only that makes many assumptions. In fact, while writing this I have come up with plans to combine them.

I store an uncompressed mfsroot.gz file in subversion. It is stored in a manner that includes the files in stand/. You may or may not be aware that this is where the boot_crunch file exists. It also happens to be the location of all the hardlinks that point back to the boot_crunch inode. Subversion stores these as if they were all binaries. Below you will see that I programmatically restore the hardlinks.

A side effect of this management structure is that generating the same mfsroot.gz multiple times results in differing file sizes each time despite using the same content. I’m not 100% sure, but believe that this is due to the file metadata being different each time the content is exported from the subversion repo.

NOTE: The URL in the script below is a dummy URL. The real URL is internally hosted with no connectivity to the Internet.

#! /bin/sh
# 
# mkmfsroot.sh:   Generate a FreeBSD mfsroot file.  This script does not
#                 function on hosts that are not running FreeBSD.
# 
# Author:   Rick
# Date:     21 August 2012
# Modif:

#####
## Variables
#####
      
SELF="${0##*/}";
NULL="/dev/null";
TMP="/tmp";
PATH="${PATH}";
ZERO="/dev/zero";
MFSROOTDIR="${1}";
SVNREPO="https://dummy.subversion.com/ats/ad/freebsd-prov/mfsroot/${MFSROOTDIR}";
REPODIR="/data/freebsd-prov/mfsroot/${MFSROOTDIR}";
MFSROOTSTAND="${REPODIR}/stand";
SVN="/usr/local/bin/svn";
STANDFILES="doconfig.sh install.cfg boot_crunch";
MFSROOT="/tmp/mfsroot";
MFSROOTMNT="/tmp/mfsrootmnt";

#####
## Functions
#####

# Clean what this script has done
CLEANUP() {

   [ "${MDCFG}" = "True" ] && mdconfig -d -u ${DEVNUM};

   if [ -d ${MFSROOTMNT} ]; then
      cd && umount ${MFSROOTMNT} > ${NULL} 2>&1;
      rm -rf ${MFSROOTMNT};
   fi
   [ -e ${MFSROOT} ] && rm ${MFSROOT};
   [ -e ${REPODIR} ] && rm -rf ${REPODIR};
   
}

# We are erroring and exiting the script
ERR_EXIT() {

   echo "${1}" 1>&2;
   CLEANUP;
   exit 1;

}

#####
## Main
#####

# Force two command line arguments
[ -z ${1} ] && ERR_EXIT "Specify an mfsroot!";

# Verify we're on a FreeBSD host
HOSTOS=`uname -s`;
[ ${HOSTOS} != "FreeBSD" ] && ERR_EXIT "Build host is not FreeBSD! Exiting!";

# Ensure we're using the latest repo version
[ -d ${REPODIR} ] && rm -rf ${REPODIR};
${SVN} export ${SVNREPO} ${REPODIR};

# Does the mfsroot dir we asked for exist?
[ ! -d ${REPODIR} ] && ERR_EXIT "${1} does not exist! Exiting!";

# Remove .svn if it exists
[ -d ${REPODIR}/.svn ] && rm -rf ${REPODIR}/.svn

# Fix stand/
#
# What do we do here and why?
#
# Since the contents of the mfsroot are stored in subversion, when the content
# is checked out or exported, each of the files in stand/ are stored as
# individual files when they are actually supposed to be hardlinks back to
# boot_crunch.  Therefore, the code below checks out the content, removes the
# files in stand/, and exports boot_crunch and a couple other scripts.  It
# finishes by regenerating all the hardlinks back to boot_crunch.
#
# We could resolve this in any number of ways, but as a stop gap measure for
# now we will leave it this way and fix it in the future.
if [ -d ${MFSROOTSTAND} ]; then
   rm ${MFSROOTSTAND}/*;
   for ea in ${STANDFILES}; do
      ${SVN} export ${SVNREPO}/stand/${ea} ${MFSROOTSTAND};
   done
 
   cd ${MFSROOTSTAND};
   for i in $(./boot_crunch 2>&1 | grep -v usage); do
      if [ "$i" != "boot_crunch" ]; then
         rm -f ./"$i";
         ln ./boot_crunch "$i";
      fi
   done
fi

# Calculate the size of the mfsroot, prepare the file and populate it
MFSROOTSIZE=$(du -sk ${REPODIR} | awk '{ print $1 }');
[ -z ${MFSROOTSIZE} ] && ERR_EXIT "Unable to determine mfsroot size! Exiting!";

dd if=${ZERO} of=${MFSROOT} bs=1024 count=$((${MFSROOTSIZE} + 1024));
[ $? -ge 1 ] && ERR_EXIT "Unable to create ${MFSROOT}! Exiting!";

MFSROOTDEV=$(mdconfig -f ${MFSROOT});
[ -z ${MFSROOTDEV} ] && ERR_EXIT "Unable to mount ${MFSROOT}! Exiting!";
DEVNUM=$(echo ${MFSROOTDEV} | awk '{ print substr($0,length($0)-0,length($0)) }'
);
MDCFG="True";

newfs /dev/${MFSROOTDEV};

[ ! -d ${MFSROOTMNT} ] && mkdir ${MFSROOTMNT};
mount /dev/${MFSROOTDEV} ${MFSROOTMNT} && \
   (cd ${REPODIR}; tar -cf - .) | (cd ${MFSROOTMNT}; tar -xf -);
   
# Now the new mfsroot has been populated, it's time to save it and clean up
umount ${MFSROOTMNT} && mdconfig -d -u ${DEVNUM} && gzip ${MFSROOT};
[ $? -ne 0 ] && ERR_EXIT "Error unmounting ${MFSROOT}! Exiting!";
MDCFG="False";
CLEANUP;
         
#####
## EOF
#####
Advertisements
Categories: FreeBSD
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: