Archive

Author Archive

Installing FreeBSD 8.x Distributions After Installation

September 17, 2012 Leave a comment

Installing Distributions After The Fact

While I did not end up needing to perform this procedure, I found the command line option intriguing…

FreeBSD 8.x [ and earlier ] media contain distributions one can select to install via sysinstall(8). This is accomplished via GUI or automated scripting requiring an install.cfg.

This image shows a typical FreeBSD installation where a kernel/distribution is extracted onto disk, What is an admin to do when they later find that they need to install a distribution after the fact? There are two methods to installing media distributions after the fact.

Using the Command Line

This is my preferred method as it does not require using sysinstall‘s graphical interface, but I am sure both have their merits. In this example, we install lib32. First, download the distribution from FreeBSD.org

echo CHECKSUM.SHA256 CHECKSUM.MD5 install.sh lib32.a{a,b,c,d,e,f,g,h,i,j,k,l,m} lib32.inf lib32.mtree | xargs -n 1 -I%s fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/8.2-RELEASE/lib32/%s

The next step is to install the distribution…

./install.sh

and you’re done…

Using sysinstall(8)

Start the procedure by executing sysinstall. It opens an interactive GUI interface. The Following procedure installs the distribution…

Select “Configure”…

Select “Distributions”…

Select the distribution to install…

Select install media (this example uses ftp)…

Select ftp location (This step may differ depending on the media selected above)…

If networking is configured, select yet. Otherwise, select no and sysinstall will prompt for network configuration…

The distribution installs…

…and when completed, exit sysinstall.

References

* The command line used can be referenced in the FreeBSD Forums

Categories: FreeBSD

Encrypting User Password Strings

September 10, 2012 Leave a comment

Encrypting User Password Strings

The Use Case

User account passwords can be set during systems provisioning in multiple ways. Setting a host’s root password is a common use case. During a FreeBSD 8.x install, one can set root’s password (or other user account password) executing the following within the install.cfg:

/bin/sh -c ‘echo passwd |pw mod user username -h 0’

Unfortunately, doing so exposes the user’s account password. Therefore, the recommendation is to use an encrypted hash as follows:

/usr/bin/chpass -p $1$nPUexUs5$O4JuN.Ed/LqWHJKmf8K0h0 root

Encrypting The Password

The use case begs the question: How does one ascertain an encrypted hash given a string?

On FreeBSD, there are varying methods to ascertain an encrypted hash. Here are just two of the many.

openssl

# openssl passwd -1 MySecret
$1$YRth1v3T$MvGupL8n.VBjvM12JhR4G0

openssl(1) returns the entire encrypted hash. The character between the first and second dollar signs identifies the crypt mechanism. The following eight characters are the salt. The remaining characters following the third dollar sign are the encrypted string.

md5

# md5 -s MySecret
MD5 (“MySecret”) = 4132d75e6cb04073cc7756707057027f

md5(1) returns a partial encrypted hash. In the key-value pair, the key is the crypt mechanism. The first 8 characters of the value are the salt. The remaining characters are the encrypted string.

In this method, one must prepend $1$ to the value before feeding it into chpass(1).

Related Documentation

* Chapter 15, Security, of the FreeBSD handbook explains identifying a host’s crypt mechanism.

Automating Generation of the mfsroot.gz

August 21, 2012 Leave a comment

NOTE: This post is old. I no longer generate mfsroots in this fashion and my methods continue to evolve, but my latest approach to modifying mfsroots is documented in the Installing FreeBSD via Cobbler post.

Automating Generation of the mfsroot.gz

This is a companion post to Automating FreeBSD crunchgen(1). In addition to boot_crunch files, I generate mfsroot.gz files for each FreeBSD release to be integrated into our provisioning platform, Cobbler. The mfsroot.gz file in FreeBSD, in this context, is a memory file system that contains the bits necessary to install a FreeBSD OS on a system. Explained here is how to automate generation of that file.

Like the crunchgen(1) script, this is a starting point only that makes many assumptions. In fact, while writing this I have come up with plans to combine them.

I store an uncompressed mfsroot.gz file in subversion. It is stored in a manner that includes the files in stand/. You may or may not be aware that this is where the boot_crunch file exists. It also happens to be the location of all the hardlinks that point back to the boot_crunch inode. Subversion stores these as if they were all binaries. Below you will see that I programmatically restore the hardlinks.

A side effect of this management structure is that generating the same mfsroot.gz multiple times results in differing file sizes each time despite using the same content. I’m not 100% sure, but believe that this is due to the file metadata being different each time the content is exported from the subversion repo.

NOTE: The URL in the script below is a dummy URL. The real URL is internally hosted with no connectivity to the Internet.

#! /bin/sh
# 
# mkmfsroot.sh:   Generate a FreeBSD mfsroot file.  This script does not
#                 function on hosts that are not running FreeBSD.
# 
# Author:   Rick
# Date:     21 August 2012
# Modif:

#####
## Variables
#####
      
SELF="${0##*/}";
NULL="/dev/null";
TMP="/tmp";
PATH="${PATH}";
ZERO="/dev/zero";
MFSROOTDIR="${1}";
SVNREPO="https://dummy.subversion.com/ats/ad/freebsd-prov/mfsroot/${MFSROOTDIR}";
REPODIR="/data/freebsd-prov/mfsroot/${MFSROOTDIR}";
MFSROOTSTAND="${REPODIR}/stand";
SVN="/usr/local/bin/svn";
STANDFILES="doconfig.sh install.cfg boot_crunch";
MFSROOT="/tmp/mfsroot";
MFSROOTMNT="/tmp/mfsrootmnt";

#####
## Functions
#####

# Clean what this script has done
CLEANUP() {

   [ "${MDCFG}" = "True" ] && mdconfig -d -u ${DEVNUM};

   if [ -d ${MFSROOTMNT} ]; then
      cd && umount ${MFSROOTMNT} > ${NULL} 2>&1;
      rm -rf ${MFSROOTMNT};
   fi
   [ -e ${MFSROOT} ] && rm ${MFSROOT};
   [ -e ${REPODIR} ] && rm -rf ${REPODIR};
   
}

# We are erroring and exiting the script
ERR_EXIT() {

   echo "${1}" 1>&2;
   CLEANUP;
   exit 1;

}

#####
## Main
#####

# Force two command line arguments
[ -z ${1} ] && ERR_EXIT "Specify an mfsroot!";

# Verify we're on a FreeBSD host
HOSTOS=`uname -s`;
[ ${HOSTOS} != "FreeBSD" ] && ERR_EXIT "Build host is not FreeBSD! Exiting!";

# Ensure we're using the latest repo version
[ -d ${REPODIR} ] && rm -rf ${REPODIR};
${SVN} export ${SVNREPO} ${REPODIR};

# Does the mfsroot dir we asked for exist?
[ ! -d ${REPODIR} ] && ERR_EXIT "${1} does not exist! Exiting!";

# Remove .svn if it exists
[ -d ${REPODIR}/.svn ] && rm -rf ${REPODIR}/.svn

# Fix stand/
#
# What do we do here and why?
#
# Since the contents of the mfsroot are stored in subversion, when the content
# is checked out or exported, each of the files in stand/ are stored as
# individual files when they are actually supposed to be hardlinks back to
# boot_crunch.  Therefore, the code below checks out the content, removes the
# files in stand/, and exports boot_crunch and a couple other scripts.  It
# finishes by regenerating all the hardlinks back to boot_crunch.
#
# We could resolve this in any number of ways, but as a stop gap measure for
# now we will leave it this way and fix it in the future.
if [ -d ${MFSROOTSTAND} ]; then
   rm ${MFSROOTSTAND}/*;
   for ea in ${STANDFILES}; do
      ${SVN} export ${SVNREPO}/stand/${ea} ${MFSROOTSTAND};
   done
 
   cd ${MFSROOTSTAND};
   for i in $(./boot_crunch 2>&1 | grep -v usage); do
      if [ "$i" != "boot_crunch" ]; then
         rm -f ./"$i";
         ln ./boot_crunch "$i";
      fi
   done
fi

# Calculate the size of the mfsroot, prepare the file and populate it
MFSROOTSIZE=$(du -sk ${REPODIR} | awk '{ print $1 }');
[ -z ${MFSROOTSIZE} ] && ERR_EXIT "Unable to determine mfsroot size! Exiting!";

dd if=${ZERO} of=${MFSROOT} bs=1024 count=$((${MFSROOTSIZE} + 1024));
[ $? -ge 1 ] && ERR_EXIT "Unable to create ${MFSROOT}! Exiting!";

MFSROOTDEV=$(mdconfig -f ${MFSROOT});
[ -z ${MFSROOTDEV} ] && ERR_EXIT "Unable to mount ${MFSROOT}! Exiting!";
DEVNUM=$(echo ${MFSROOTDEV} | awk '{ print substr($0,length($0)-0,length($0)) }'
);
MDCFG="True";

newfs /dev/${MFSROOTDEV};

[ ! -d ${MFSROOTMNT} ] && mkdir ${MFSROOTMNT};
mount /dev/${MFSROOTDEV} ${MFSROOTMNT} && \
   (cd ${REPODIR}; tar -cf - .) | (cd ${MFSROOTMNT}; tar -xf -);
   
# Now the new mfsroot has been populated, it's time to save it and clean up
umount ${MFSROOTMNT} && mdconfig -d -u ${DEVNUM} && gzip ${MFSROOT};
[ $? -ne 0 ] && ERR_EXIT "Error unmounting ${MFSROOT}! Exiting!";
MDCFG="False";
CLEANUP;
         
#####
## EOF
#####
Categories: FreeBSD

FreeBSD vlan Tagging

August 14, 2012 5 comments

FreeBSD vlan Tagging

Virtual LANs were introduced by IEEE 802.1Q, a standards definition for tagging of ethernet frames across vlan-aware appliances. It is also known as vlan tagging. Virtual LANs add the ability to separate data on a single transport media into logical networks.

This post explains vlan tagging configurations in FreeBSD. Network appliances making up your network also need to be configured with vlan tagging support, but is beyond the scope of this post.

If vlan tagging support is compiled into the kernel, it is not required to include the following in the loader.conf:

if_vlan_load=”YES”

Configuring vlan tagging on a running system is done by executing the following:

# ifconfig $int up
# ifconfig vlan0 create
# ifconfig vlan0 10.1.1.5 netmask 255.255.255.0 vlan $id vlandev $int
# ifconfig vlan0_alias0 10.1.2.5 netmask 255.255.255.0 vlan $id vlandev $int

Configuring so vlan tags persists across reboots is done by inserting the following in rc.conf:

ifconfig_$int=”up”
cloned_interfaces=”vlan0″
ifconfig_vlan0=”10.1.1.5 netmask 255.255.255.0 vlan $id vlandev $int”
ifconfig_vlan0_alias0=”10.1.2.5 netmask 255.255.255.0 vlan $id vlandev $int”
ifconfig_vlan0_alias1=”10.1.3.5 netmask 255.255.255.0 vlan $id vlandev $int”

In the above examples, $id is the vlan id and $int is the physical network interface.

Also note, additional alias’ added must be added by appending alias[0-9] to the cloned interface in numerical order. Each alias is brought up until it encounters the an alias which is not in sequential order. It then finishes the OS boot without those alias’ added.

For further reading, check out the vlan(4) manpage and the ifconfig(8) manpage.

Categories: FreeBSD

FreeBSD uname(1) and the Environment

August 9, 2012 Leave a comment

FreeBSD uname(1) and The Environment

The default values provided by uname(1) can be over-ridden on a running system utilizing environment variables. The uname(1) manpage identifies this behavior in the “Environment” section stating:

An environment variable composed of the string UNAME_ followed by any flag to the uname utility (except for -a) will allow the corresponding data to be set to the contents of the environment variable.

I would like to call out the environment variables specifically and identify what is changed when these environment variables are utilized. They are as follows:

  • UNAME_s: The name of the operating system implementation
  • UNAME_n: The name of the system
  • UNAME_r: The current release level of the operating system
  • UNAME_v: The version level of this release of the operating system
  • UNAME_m: The type of the current hardware platform
  • UNAME_p: The type of the machine processor architecture
  • UNAME_i: The kernel ident
Categories: FreeBSD

GNU tar Header Errors?

July 30, 2012 1 comment

Dealing with GNU/tar’s Extended Header Warnings

I tar’d files up on a FreeBSD host and transferred them to a linux host. When extracting the tarball, I was presented with the following warnings:

GNU/tar Header Errors

It seems that FreeBSD‘s tar binary utilizes additional headers that GNU tar does not recognize. There are many blog posts about this. The problem appears to affect tarballs created on various operating systems that employ varying versions of tar and extracting those tarballs with GNU tar.

Generally speaking, these warning can be ignored. On the command line, the files contained within the tarball will still extract (or at least they have every time I’ve encountered this). This does, however, present a problem if one is programmatically encountering this. One can resolve this by addressing the issue within their code or by using the same version of tar on each platform to create and extract the tarballs in question.

Automating FreeBSD’s crunchgen(1)

July 27, 2012 2 comments

Automating FreeBSD’s crunchgen(1)

Hey, Joe…this one’s for you…thanks for reading!

One of my main tasks in my role at work is to maintain FreeBSD provisioning bits. I have found myself generating boot_crunch files with various binaries and what not to extend the capabilities of the provisioning aspect of FreeBSD. So much so that I whipped up a quick script to automate the task for me.

Please bear in mind, it is a quick script which makes a number of assumptions. It’s not perfect and can certainly use some tweaking and the like, but this initial version gets one started.

I keep my boot_crunch.conf in a subversion repo, thus the calls to svn.

#! /bin/sh
#
# bldcrunch.sh:   Generate a FreeBSD boot_crunch file.  This script does not
#                 function on hosts that are not running FreeBSD.
#
# Author:   Rick
# Date:     25 July 2012
# Modif:

#####
## Variables
#####

SELF="${0##*/}";
NULL="/dev/null";
TMP="/tmp";
PATH="${PATH}";
SVNREPO="http://svn.domain.com/freebsd-repo";
REPODIR="/freebsd-repo";
BOOTCRUNCHDIR="boot_crunch";
BOOTCRUNCHCONF="boot_crunch.conf";
BOOTCRUNCHMK="boot_crunch.mk";
SVNBIN="/usr/local/bin/svn";
CRUNCHGEN="/usr/bin/crunchgen";
MAKE="/usr/bin/make";

#####
## Functions
#####

ERROR_EXIT() {

   echo "${1}" 1>&2;
   [ -d ${TMP}/${BOOTCRUNCHDIR} ] && rm ${TMP}/${BOOTCRUNCHDIR};
   exit 1;

}

#####
## Main
#####

# Verify we're on a FreeBSD host
HOSTOS=`uname -s`;
[ ${HOSTOS} != "FreeBSD" ] && ERROR_EXIT "Build host is not FreeBSD! Exiting!";

# Do all of our binaries exist?
for ea in ${SVNBIN} ${CRUNCHGEN} ${MAKE}; do
   if [ -e ${ea} ]; then
      continue;
   else
      ERROR_EXIT "${ea} is required! Exiting!";
   fi
done

# Update SVN repo to make sure we're using the latest boot_crunch.conf
if [ -d ${REPODIR}/${BOOTCRUNCHDIR} ]; then
   cd ${REPODIR}/${BOOTCRUNCHDIR} && ${SVNBIN} update;
else
   ${SVN} co ${SVNREPO} ${REPODIR};
fi

# Setup our build environment
[ -d ${TMP}/${BOOTCRUNCHDIR} ] && rm -rf ${TMP}/${BOOTCRUNCHDIR};
mkdir ${TMP}/${BOOTCRUNCHDIR} && cd ${TMP}/${BOOTCRUNCHDIR};

# Build the boot_crunch
${CRUNCHGEN} -o ${REPODIR}/${BOOTCRUNCHDIR}/${BOOTCRUNCHCONF};
[ $? -eq 0 ] && ${MAKE} -f ${BOOTCRUNCHMK};
[ $? -ne 0 ] && ERROR_EXIT "Error generating boot_crunch! Exiting!";

#####
## EOF
#####

BTW, I edited this because I found out about wordpress’s sourcecode tag…awesome!

Categories: FreeBSD

FreeBSD 8.x and 4K sector disks

July 10, 2012 1 comment

FreeBSD 8.x and 4K sector drives

FreeBSD 8.x (and older) use sysinstall as the install mechanism and fdisk for disk slicing/partitioning. Unfortunately, these utilities perform calculations based on a 512B sector size. This causes problems with drives of a sector size greater than this. 4K sector size drives are the most popular at this point.

I cannot explain the problem and solutions any better than Ivan Voras (FreeBSD source contributor since 2008), so I link to his blog here.

There are good tidbits of information in the comments such as sysinstall not understanding gpart, etc. It would be a good idea to read through the comments as well.

Categories: FreeBSD

FreeBSD + Dell 12G + X520-DA2 10G NIC

June 30, 2012 2 comments

Intel 82599 based NICs on Dell 12G Servers w/ FreeBSD?

Dell offers the X520-DA2 10G Fiber NIC based on the Intel 82599 chipset for it’s 12G line of servers. It seems that there were two batches of this NIC manufactured. One batch was Dell branded, the other Intel branded.

According to this post, the FreeBSD source appears to be missing PCI ID’s etc that would enable the Dell branded card to work under FreeBSD on it’s 12G servers resulting in the card not being recognized by the OS. Inspection of dmesg shows that there is an ethernet card with no driver attached.

This is currently being corrected in HEAD, but for the time being swapping in an Intel branded equivalent which is seen by FreeBSD without a problem is sufficient. Unfortunately, the Dell BIOS does not recognize the card. having said that, the card is recognized in the OS and does work using the ixgbe(4) driver. The interface name begins with “ix”.

Categories: FreeBSD

Using screen to connect to serial ports

June 22, 2012 2 comments

Using screen to connect to serial ports

screen is cool. screen is an indespencable tool to any unix administrator. My challenge was to find a solution to connect to the serial port on a server chassis. The solution I found was to utilize a USB-Serial adapter (and associated Mac OS X Lion Driver), null modem cable, and rj45-db9 adapter in conjunction with screen.

screen, being a key tool in an administrator’s tool box, was found to have the capability of connecting to serial ports!

To connect to the serial port:

screen /dev/tty.PL2303-000013FD

Specifying baud rate:

screen /dev/tty.PL2303-000013FD 9600

Enabling CTS/RTS:

screen /dev/tty.PL2303-000013FD 9600,ctsrts

Terminate the session using screen command ^a,k

Notes

  • Your tty will likely differ. I was using a PL2303 based USB-Serial adapter within Mac OS X 10.7 (aka Lion)
  • A driver for this USB-Serial adapter were difficult to find, so I will link to it here
  • I downloaded the driver from here
Categories: Technical Miscellany