Archive

Archive for November, 2012

FreeBSD: Computing kern.maxfilesperproc and kern.maxfiles

November 12, 2012 Leave a comment

Computing kern.maxfilesperproc and kern.maxfiles

Many thanks to Julien Charbon, a colleague of mine, for providing this information. This is specific to releng/8.3 because we currently utilize this version.

kern.maxfiles is derived from max.maxproc (max number of process) which is derived from max.maxusers (not the maximum number of users, but more a global order of magnitude of machine process/file descriptor usage). max.maxusers is derived from the amount of physical memory and capped up to 384:

sys/kern/subr_param.c:

        /* Base parameters */
        maxusers = MAXUSERS;
        TUNABLE_INT_FETCH("kern.maxusers", &maxusers);
        if (maxusers == 0) {
                maxusers = physpages / (2 * 1024 * 1024 / PAGE_SIZE);
                if (maxusers < 32)
                        maxusers = 32;
                if (maxusers > 384)
                        maxusers = 384;
        }

A machine with a large amount of memory, 72 GB for example, gets the maximum:

$ sysctl kern.maxusers
kern.maxusers: 384

in sys/kern/subr_param.c:

#define NPROC (20 + 16 * maxusers)
#ifndef NBUF
#define NBUF 0
#endif
#ifndef MAXFILES
#define MAXFILES (maxproc * 2)
#endif

[...]

     /*
      * The following can be overridden after boot via sysctl.  Note:
      * unless overriden, these macros are ultimately based on maxusers.
      */
     maxproc = NPROC;
     TUNABLE_INT_FETCH("kern.maxproc", &maxproc);
     /*
      * Limit maxproc so that kmap entries cannot be exhausted by
      * processes.
      */
     if (maxproc &gt; (physpages / 12))
             maxproc = physpages / 12;
     maxfiles = MAXFILES;
     TUNABLE_INT_FETCH("kern.maxfiles", &maxfiles);
     maxprocperuid = (maxproc * 9) / 10;
     maxfilesperproc = (maxfiles * 9) / 10;

Thus:

kern.maxproc = (20 + 16 * maxusers) = (20 + 16 * 384) = 6164
kern.maxfiles = (maxproc * 2) = (2 * 6164) = 12328
kern.maxfilesperproc = (maxfiles * 9) / 10 = (12328 * 9) / 10 = 11095

The math confirmed:

$ sysctl kern.maxproc
kern.maxproc: 6164
$ sysctl kern.maxfiles
kern.maxfiles: 12328
$ sysctl kern.maxfilesperproc
kern.maxfilesperproc: 11095

Thus these low default maximum values are the result of a tentative from FreeBSD to “auto-tune” the default maximum from memory size. However by limiting the kern.maxusers to 384, it concerns only system with less 192 KB of memory (embedded system?).

We located this patch which has the effect of increasing these tunables. See below.

$ sysctl kern.maxfiles
kern.maxfiles: 98312
$ sysctl kern.maxfilesperproc
kern.maxfilesperproc: 88480
Advertisements
Categories: FreeBSD

MeetBSD California 2012

November 7, 2012 1 comment
I am a fairly recent newcomer to the FreeBSD community and had the opportunity to attend my first BSD related conference. This is the account of my experience at MeetBSD California 2012.

MeetBSD California 2012 was coordinated by iX Systems and hosted at Yahoo! corporate headquarters in Sunnyvale, CA. The Yahoo! facility easily accommodated the 100 – 125 attendees in multiple conference rooms for varying purposes across the two days allocated.

Conference Format

The conference was spread across two days. Day 1 revolved around speakers and presentations while day 2 followed what was called “The unConference” agenda. “The unConference” included lightning talks/speed geeking sessions and break out sessions. The topics of the second day were chosen by conference attendees as opposed to the organizers.

The Conference: Day 1

Presentations

Impressions

All of the speakers were knowledgeable on their topics and well prepared with slide decks accompanying their presentations some even including demos. Attendees participated with questions, comments, and general discussion. The environment was engaging and lively and provided much information to absorb.

In the large conference room, however, it was difficult for some in the back to hear the speakers. I think an audio sound system would have been an excellent idea for this conference.

The Conference: Day 2

Day 2: The unConference

The idea of the speed geeking session was of particular interest to most in attendance and broke up the monotony of a constant stream of speakers and presentations. This session was a set of speakers covering a topic with a group of individuals for approximately 10 minutes. At the conclusion of that 10 minutes, each group rotated to a different speaker who spoke on their topic for 10 minutes. The rotation was repeated until each group had heard all of the speakers.

Later, Conference attendees voted on subjects that would later become the topics of the breakout sessions. These breakout sessions enabled groups of interested attendees to interactively discuss subjects with which they had definitive interest. These sessions occurred at the latter part of the day. However, as the day wore on, I began to lose focus.

Overall Experience

In conclusion, my experience at the conference was extremely positive. The conference content was very informative and presented well. Additionally, much was gained outside the confines of the conference itself. I had the opportunity to meet and socialize with multiple individuals who are BSD enthusiasts and others with whom I’ve communicated with digitally previously.

FreeBSD kernel panic in udp_input()

November 5, 2012 1 comment

FreeBSD kernel panic in udp_input()

The Problem

Running FreeBSD 8.3, this kernel panic was produced under UDP load:

Fatal trap 12: page fault while in kernel mode
cpuid = 4; apic id = 08
fault virtual address = 0x0
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff807a08a2
stack pointer = 0x28:0xffffffa40c2689b0
frame pointer = 0x28:0xffffffa40c268a80
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 12 (swi1: netisr 0)
trap number = 12
panic: page fault
cpuid = 4
KDB: stack backtrace:
#0 0xffffffff80642b3e at kdb_backtrace+0x5e
#1 0xffffffff8060fd57 at panic+0x187
#2 0xffffffff80905990 at trap_fatal+0x290
#3 0xffffffff80905ce1 at trap_pfault+0x201
kernel trap 12 with interrupts disabled
#4 0xffffffff8090619f at trap+0x3df

#5 0xffffffff808ed674 at calltrap+0x8

#6 0xffffffff8072000c at ip_input+0xac
Fatal trap 12: page fault while in kernel mode
cpuid = 5; apic id = 0a
#7 0xffffffff806cba29 at swi_net+0x149
fault virtual address = 0xc
#8 0xffffffff805e7794 at intr_event_execute_handlers+0x104
fault code = supervisor read data, page not present
#9 0xffffffff805e8e25 at ithread_loop+0x95
instruction pointer = 0x20:0xffffffff8064f795
#10 0xffffffff805e49af at fork_exit+0x11f
stack pointer = 0x28:0xffffffa41cd969a0
#11 0xffffffff808edbbe at fork_trampoline+0xe
frame pointer = 0x28:0xffffffa41cd969d0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = resume, IOPL = 0
current process = 8 (pagedaemon)
trap number = 12

Marc de la Gerroniere and Julien Charbon, developer colleagues of mine, commenced an intense troubleshooting exercise that lasted several days. Thanks to them for providing the patch below.

The Issue

The kernel panic occurred when a process executes `sysctl net.inet.udp.pcblist` followed by a UDP socket being closed and then receiving a packet on that socket before the process executing sysctl had completed.

The Patch

Applying this patch to FreeBSD sources and building resolves the issue above.

commit 497aa2bc6f4b7383344ab7fd812d296c6997298d
Author: Marc de la Gueronniere 
Date:   Wed Oct 24 19:59:41 2012 +0000

    kern/172963: Kernel panic in udp_input()

diff --git a/sys/netinet/udp_usrreq.c b/sys/netinet/udp_usrreq.c
index b720364..f9d0245 100644
--- a/sys/netinet/udp_usrreq.c
+++ b/sys/netinet/udp_usrreq.c
@@ -494,6 +494,12 @@ udp_input(struct mbuf *m, int off)
 
                       INP_RLOCK(inp);
 
+                     // in_pcbdrop does not unlink from V_udb
+                      if (inp->inp_socket == NULL) {
+                             INP_RUNLOCK(inp);
+                             continue;
+                     }
+
                    /*
                      * Handle socket delivery policy for any-source
                         * and source-specific multicast. [RFC3678]
@@ -1559,6 +1565,7 @@ udp_detach(struct socket *so)
        KASSERT(up != NULL, ("%s: up == NULL", __func__));
   inp->inp_ppcb = NULL;
       in_pcbdetach(inp);
+    in_pcbdrop(inp);
       in_pcbfree(inp);
       INP_INFO_WUNLOCK(&V_udbinfo);
      udp_discardcb(up);
diff --git a/sys/netinet6/udp6_usrreq.c b/sys/netinet6/udp6_usrreq.c
index 22ddde4..2e8b564 100644
--- a/sys/netinet6/udp6_usrreq.c
+++ b/sys/netinet6/udp6_usrreq.c
@@ -271,6 +271,10 @@ udp6_input(struct mbuf **mp, int *offp, int proto)
                                    inp->inp_fport != uh->uh_sport)
                                      continue;
                      }
+                     // in_pcbdrop does not unlink from V_udb
+                      if (inp->inp_socket == NULL) {
+                             continue;
+                     }
 
                    /*
                      * Handle socket delivery policy for any-source
@@ -964,6 +968,7 @@ udp6_detach(struct socket *so)
     up = intoudpcb(inp);
   KASSERT(up != NULL, ("%s: up == NULL", __func__));
   in_pcbdetach(inp);
+    in_pcbdrop(inp);
       in_pcbfree(inp);
       INP_INFO_WUNLOCK(&V_udbinfo);
      udp_discardcb(up);

References

A “Problem Report” (aka PR) was submitted to the FreeBSD project; it can be viewed at http://www.freebsd.org/cgi/query-pr.cgi?pr=172963.

Categories: FreeBSD