Archive
Starting A Daemon via daemon(8) in FreeBSD
Starting A Daemon via daemon(8) in FreeBSD
Daemonizing an application that must persist across reboots while including native process management capability is easily accomplished by combining the functions of rc(8) scripts and commands with daemon(8), a utility that detaches itself from the controlling terminal and executing a program. For example, a service written in Python implementing an HTTP API front-end.
Combing these two utilities enables automated process initialization during system boot and provides the ability to stop, start, and restart services manually using either the rc(8) script or service(8). For example:
/usr/local/etc/rc.d/httpapi start
or
service httpapi start
Create an rc(8) script in /usr/local/etc/rc.d with a name descriptive of the application. httpapi is a descriptive name in this case. Writing the rc(8) script is covered extensively well in the “Practical rc.d scripting in BSD” article on FreeBSD.org and is a very good read. Those reading this blog are encouraged to go read that article though, for expediency, a basic implementation may look very similar to:
#!/bin/sh# REQUIRE: LOGIN
. /etc/rc.subr
name=httpapi
rcvar=httpapi_enable
command=/usr/local/bin/httpapi
command_interpreter=python
httpapi_user=httpapi
start_cmd=”/usr/sbin/daemon -u $httpapi_user $command”load_rc_config $name
run_rc_command “$1”
The value of the key rcvar is used to automate the startup of the daemon at system boot within /etc/rc.conf. Append a line to the file containing a combination of this value as key a key with a boolean “YES” or “NO” similarly to:
echo "httpapi_enable=\"YES\"" >> /etc/rc.conf
Disclaimer
Data and information described on this blog are for informational purposes only. The author of this blog provides no warranty/guarantee, expressed or implied, that this data and information will function as described here. Readers are expected to exercise due diligence when researching, developing, and deploying techniques and methods for use within their environments.
Comments posted are the explicit opinions of the comment poster themselves and does not necessarily reflect the views and opinions of the author of this blog.
Troubleshooting Large, Stalling git/ssh Transfers
Why is git/ssh stalling?
While working with the freebsd-ports repo on github in order to track and combine it with an internal remote repo, it was found to apparently stall during a git push to the internal remote repo. Thinking this was likely a fluke in the network or something, the process was re-executed, but exhibited the same behavior. The OS in this case is FreeBSD 10.0-RELEASE-p6.
Troubleshooting
Identifying the problem started by observing the git process in top which was in a “piperd” wait state. An email thread on freebsd-questions@[1] suggested the process was waiting on a read from a pipe. I only needed to identify the process it was waiting for.
Using lsof, according to the same reference, one could identify the offending process by matching the hex value of the pipe file descriptor to the piped process, in this case ssh. Attaching truss to ssh showed that it had apparently stalled on a getsockopt() operation.
A web search turned up related results[2] (found in References below) which indicated the likely problem was a bug in OpenSSH related to TcpRcvBufPoll. Explicitly disabling this in sshd_config and restarting the service did appear to work around the problem. Upon re-execution of my git push, the process didn’t stall and completed successfully.
Oddly enough, after working around this, I was unable to duplicate the behavior on the same host later. Therefore, there is some doubt that this was indeed the problem, but I post this anyway as it did appear to successfully work around it.
References
[1] FreeBSD Mailing List Archives
[2] stackoverflow post
Disclaimer
Data and information described on this blog are for informational purposes only. The author of this blog provides no warranty/guarantee, expressed or implied, that this data and information will function as described here. Readers are expected to exercise due diligence when researching, developing, and deploying techniques and methods for use within their environments.
Comments posted are the explicit opinions of the comment poster themselves and does not necessarily reflect the views and opinions of the author of this blog.
FreeBSD Ports Batch Install
FreeBSD Ports Batch Install
Many ports interactively prompt for various configuration options when installing. This is both annoying and time consuming. Fortunately, it can also be avoided.
`make config-recursive` interactively prompts users for options prior to `make install` or `make package-recursive`. This, however, is still interactive. Alternatively, one could add ‘-DBATCH’ to the command and install or create packages accepting default configuration options.
The following examples install and create packages including dependencies, respectively. In the second example, packages will be written to /usr/ports/packages if it exists.
# cd /usr/ports/sysutils/puppet # make -DBATCH install
or
# cd /usr/ports/sysutils/puppet # make -DBATCH package-recursive
Disclaimer
This blog is posted for informational purposes only. Extensive testing is recommended prior to implementing changes discussed here.
FreeBSD Partitions and Filesystems With GPART
FreeBSD Partitions and Filesystems With GPART
The Problem
Installing FreeBSD 8.x on a machine with two disk volumes of 100GB and slightly more than 3TB. The quantity of cylinders on the 3TB volume is greater than the 65,535 cylinders supported by bsdlabel. Therefore, the filesystem does not consume the total available disk space.
This is resolved by installing and implementing gpart. I won’t go into the procedure here as Warren Block blogs on this topic very well at http://www.wonkity.com/~wblock/docs/html/disksetup.html
Installing gpart
GPART is easily installed from FreeBSD.org package repos using pkg_add(1) as follows:
# pkg_add -r gpte
It may also be installed from ports by executing the following:
# cd /usr/ports/sysutils/gpte # make install clean
Disclaimer
This blog is posted for informational purposes only. Extensive testing is recommended prior to implementing changes discussed here.
So You Want A New FreeBSD Boot Loader Option…
Creating A New FreeBSD Boot Loader Option
The Use Case
There are multiple kernels installed on the FreeBSD host and an option to boot to this kernel is necessary.
Prerequisites
- This post assumes FreeBSD 8.x, but should apply with any version running loader(8)
- This post assumes that a secondary/debug kernel is installed
The Patch
This is the patch. Copy/paste it to /boot/beastie.patch for use in the procedure below.
Index: beastie.4th =================================================================== --- beastie.4th +++ beastie.4th @@ -41,6 +41,7 @@ variable bootkey variable bootacpikey variable bootsafekey +variable bootdebugkey variable bootverbosekey variable bootsinglekey variable escapekey @@ -194,6 +195,7 @@ printmenuitem ." Boot FreeBSD in Safe Mode" bootsafekey ! printmenuitem ." Boot FreeBSD in single user mode" bootsinglekey ! printmenuitem ." Boot FreeBSD with verbose logging" bootverbosekey ! + printmenuitem ." Boot FreeBSD with debug kernel" bootdebugkey ! printmenuitem ." Escape to loader prompt" escapekey ! printmenuitem ." Reboot" rebootkey ! menuX @ 20 at-xy @@ -286,6 +288,9 @@ s" YES" s" boot_single" setenv 0 boot then + dup bootdebugkey @ = if + s" /boot/DEBUG/kernel" 1 boot + then dup escapekey @ = if 2drop s" NO" s" autoboot_delay" setenv
Applying The Patch
Follow these steps to apply the patch to your target hosts:
# cd /boot # cp beastie.4th beastie.4th.orig # patch -p0 < beastie.patch
The new option will be loaded when loader called beastie.4th.
So You Want A FreeBSD Debugging Kernel…
Installing a debugging kernel in FreeBSD
The Use Case
There are numerous reasons for needing a debugging kernel. For example, during a kernel panic. This was the case with us. Therefore, we installed a kernel with DDB enabled. In the event of a kernel panic, the system would drop to the debugger and the software engineers were then able to begin diagnosing the problem.
Preparing To Build The Kernel
Prerequisites
- FreeBSD source tree
- A kernel config
FreeBSD Sources
Operators can use cvs or Subversion to install the sources. Information on FreeBSD source trees can be found here.
The Kernel Config
Create a kernel config by opening /usr/src/sys/${arch}/conf/DEBUG. Where ${arch} is the machine architecture the kernel is to be built for. In this case, we build an amd64 kernel on an amd64 machine. The kernel config should appear as follows:
include GENERIC
options KDB_UNATTENDED
options DDB
Compiling and Installing The Kernel
Compiling and installing the debugging kernel is as simple as executing:
# cd /usr/src
# make buildkernel KERNCONF=DEBUG INSTKERNNAME=DEBUG
# make installkernel KERCONF=DEBUG INSTKERNNAME=DEBUG
Executing/Using The Kernel
From the loader menu
At the loader menu, press 6 to escape to the loader(8) prompt and type:
boot DEBUG
From the running system
The DEBUG kernel can be loaded by default by adding/modifying the kernel directive in loader.conf as follows. Each subsequent boot will load the DEBUG kernel.
kernel=DEBUG
Dropping into the debugger
During a kernel panic, it will simply reboot unless debug.debugger_on_panic is enabled. To enable this execute:
sysctl debug.debugger_on_panic=1
To induce a panic:
sysctl debug.kdb.panic=1
Installing FreeBSD 8.x Distributions After Installation
Installing Distributions After The Fact
FreeBSD 8.x [ and earlier ] media contain distributions one can select to install via sysinstall(8). This is accomplished via GUI or automated scripting requiring an install.cfg.
This image shows a typical FreeBSD installation where a kernel/distribution is extracted onto disk, What is an admin to do when they later find that they need to install a distribution after the fact? There are two methods to installing media distributions after the fact.
Using the Command Line
This is my preferred method as it does not require using sysinstall‘s graphical interface, but I am sure both have their merits. In this example, we install lib32. First, download the distribution from FreeBSD.org…
echo CHECKSUM.SHA256 CHECKSUM.MD5 install.sh lib32.a{a,b,c,d,e,f,g,h,i,j,k,l,m} lib32.inf lib32.mtree | xargs -n 1 -I%s fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/8.2-RELEASE/lib32/%s
The next step is to install the distribution…
./install.sh
and you’re done…
Using sysinstall(8)
Start the procedure by executing sysinstall. It opens an interactive GUI interface. The Following procedure installs the distribution…
Select “Configure”…
Select “Distributions”…
Select the distribution to install…
Select install media (this example uses ftp)…
Select ftp location (This step may differ depending on the media selected above)…
If networking is configured, select yet. Otherwise, select no and sysinstall will prompt for network configuration…
The distribution installs…
…and when completed, exit sysinstall.
References
* The command line used can be referenced in the FreeBSD Forums
Encrypting User Password Strings
Encrypting User Password Strings
The Use Case
User account passwords can be set during systems provisioning in multiple ways. Setting a host’s root password is a common use case. During a FreeBSD 8.x install, one can set root’s password (or other user account password) executing the following within the install.cfg:
/bin/sh -c ‘echo passwd |pw mod user username -h 0’
Unfortunately, doing so exposes the user’s account password. Therefore, the recommendation is to use an encrypted hash as follows:
/usr/bin/chpass -p $1$nPUexUs5$O4JuN.Ed/LqWHJKmf8K0h0 root
Encrypting The Password
The use case begs the question: How does one ascertain an encrypted hash given a string?
On FreeBSD, there are varying methods to ascertain an encrypted hash. Here are just two of the many.
openssl
# openssl passwd -1 MySecret
$1$YRth1v3T$MvGupL8n.VBjvM12JhR4G0
openssl(1) returns the entire encrypted hash. The character between the first and second dollar signs identifies the crypt mechanism. The following eight characters are the salt. The remaining characters following the third dollar sign are the encrypted string.
md5
# md5 -s MySecret
MD5 (“MySecret”) = 4132d75e6cb04073cc7756707057027f
md5(1) returns a partial encrypted hash. In the key-value pair, the key is the crypt mechanism. The first 8 characters of the value are the salt. The remaining characters are the encrypted string.
In this method, one must prepend $1$ to the value before feeding it into chpass(1).
Related Documentation
* Chapter 15, Security, of the FreeBSD handbook explains identifying a host’s crypt mechanism.
FreeBSD vlan Tagging
FreeBSD vlan Tagging
This post explains vlan tagging configurations in FreeBSD. Network appliances making up your network also need to be configured with vlan tagging support, but is beyond the scope of this post.
If vlan tagging support is compiled into the kernel, it is not required to include the following in the loader.conf:
if_vlan_load=”YES”
Configuring vlan tagging on a running system is done by executing the following:
# ifconfig $int up
# ifconfig vlan0 create
# ifconfig vlan0 10.1.1.5 netmask 255.255.255.0 vlan $id vlandev $int
# ifconfig vlan0_alias0 10.1.2.5 netmask 255.255.255.0 vlan $id vlandev $int
Configuring so vlan tags persists across reboots is done by inserting the following in rc.conf:
ifconfig_$int=”up”
cloned_interfaces=”vlan0″
ifconfig_vlan0=”10.1.1.5 netmask 255.255.255.0 vlan $id vlandev $int”
ifconfig_vlan0_alias0=”10.1.2.5 netmask 255.255.255.0 vlan $id vlandev $int”
ifconfig_vlan0_alias1=”10.1.3.5 netmask 255.255.255.0 vlan $id vlandev $int”
In the above examples, $id is the vlan id and $int is the physical network interface.
Also note, additional alias’ added must be added by appending alias[0-9] to the cloned interface in numerical order. Each alias is brought up until it encounters the an alias which is not in sequential order. It then finishes the OS boot without those alias’ added.
For further reading, check out the vlan(4) manpage and the ifconfig(8) manpage.
FreeBSD uname(1) and the Environment
FreeBSD uname(1) and The Environment
An environment variable composed of the string UNAME_ followed by any flag to the uname utility (except for -a) will allow the corresponding data to be set to the contents of the environment variable.
I would like to call out the environment variables specifically and identify what is changed when these environment variables are utilized. They are as follows:
- UNAME_s: The name of the operating system implementation
- UNAME_n: The name of the system
- UNAME_r: The current release level of the operating system
- UNAME_v: The version level of this release of the operating system
- UNAME_m: The type of the current hardware platform
- UNAME_p: The type of the machine processor architecture
- UNAME_i: The kernel ident
hostileadmin 