Archive
FreeBSD: Computing kern.maxfilesperproc and kern.maxfiles
Computing kern.maxfilesperproc and kern.maxfiles
Many thanks to Julien Charbon, a colleague of mine, for providing this information. This is specific to releng/8.3 because we currently utilize this version.
kern.maxfiles is derived from max.maxproc (max number of process) which is derived from max.maxusers (not the maximum number of users, but more a global order of magnitude of machine process/file descriptor usage). max.maxusers is derived from the amount of physical memory and capped up to 384:
/* Base parameters */
maxusers = MAXUSERS;
TUNABLE_INT_FETCH("kern.maxusers", &maxusers);
if (maxusers == 0) {
maxusers = physpages / (2 * 1024 * 1024 / PAGE_SIZE);
if (maxusers < 32)
maxusers = 32;
if (maxusers > 384)
maxusers = 384;
}
A machine with a large amount of memory, 72 GB for example, gets the maximum:
$ sysctl kern.maxusers kern.maxusers: 384
#define NPROC (20 + 16 * maxusers)
#ifndef NBUF
#define NBUF 0
#endif
#ifndef MAXFILES
#define MAXFILES (maxproc * 2)
#endif
[...]
/*
* The following can be overridden after boot via sysctl. Note:
* unless overriden, these macros are ultimately based on maxusers.
*/
maxproc = NPROC;
TUNABLE_INT_FETCH("kern.maxproc", &maxproc);
/*
* Limit maxproc so that kmap entries cannot be exhausted by
* processes.
*/
if (maxproc > (physpages / 12))
maxproc = physpages / 12;
maxfiles = MAXFILES;
TUNABLE_INT_FETCH("kern.maxfiles", &maxfiles);
maxprocperuid = (maxproc * 9) / 10;
maxfilesperproc = (maxfiles * 9) / 10;
Thus:
kern.maxproc = (20 + 16 * maxusers) = (20 + 16 * 384) = 6164
kern.maxfiles = (maxproc * 2) = (2 * 6164) = 12328
kern.maxfilesperproc = (maxfiles * 9) / 10 = (12328 * 9) / 10 = 11095
The math confirmed:
$ sysctl kern.maxproc kern.maxproc: 6164 $ sysctl kern.maxfiles kern.maxfiles: 12328 $ sysctl kern.maxfilesperproc kern.maxfilesperproc: 11095
Thus these low default maximum values are the result of a tentative from FreeBSD to “auto-tune” the default maximum from memory size. However by limiting the kern.maxusers to 384, it concerns only system with less 192 KB of memory (embedded system?).
We located this patch which has the effect of increasing these tunables. See below.
$ sysctl kern.maxfiles kern.maxfiles: 98312 $ sysctl kern.maxfilesperproc kern.maxfilesperproc: 88480
MeetBSD California 2012
MeetBSD California 2012 was coordinated by iX Systems and hosted at Yahoo! corporate headquarters in Sunnyvale, CA. The Yahoo! facility easily accommodated the 100 – 125 attendees in multiple conference rooms for varying purposes across the two days allocated.
Conference Format
The conference was spread across two days. Day 1 revolved around speakers and presentations while day 2 followed what was called “The unConference” agenda. “The unConference” included lightning talks/speed geeking sessions and break out sessions. The topics of the second day were chosen by conference attendees as opposed to the organizers.
The Conference: Day 1
Presentations
- David Maxwell: A Trip into the Collective Unconscious of BSD Developers
- Adrian Chadd: FreeBSD Embedded – Why isn’t it everywhere?
- Kris Moore: The Warden® – FreeBSD and Linux Jail Management
- Sean Bruno: QEMU on FreeBSD for fun and profit
- Scott Long: Netflix, FreeBSD, and the world’s largest content delivery network
- Josh Paetzel: FreeNAS™: Storage for Open Source
- Michael Dexter: The BHyVe Hypervisor In Depth
- Mark Linimon: The FreeBSD Ports Collection: 2012
Impressions
All of the speakers were knowledgeable on their topics and well prepared with slide decks accompanying their presentations some even including demos. Attendees participated with questions, comments, and general discussion. The environment was engaging and lively and provided much information to absorb.
In the large conference room, however, it was difficult for some in the back to hear the speakers. I think an audio sound system would have been an excellent idea for this conference.
The Conference: Day 2
Day 2: The unConference
The idea of the speed geeking session was of particular interest to most in attendance and broke up the monotony of a constant stream of speakers and presentations. This session was a set of speakers covering a topic with a group of individuals for approximately 10 minutes. At the conclusion of that 10 minutes, each group rotated to a different speaker who spoke on their topic for 10 minutes. The rotation was repeated until each group had heard all of the speakers.
Later, Conference attendees voted on subjects that would later become the topics of the breakout sessions. These breakout sessions enabled groups of interested attendees to interactively discuss subjects with which they had definitive interest. These sessions occurred at the latter part of the day. However, as the day wore on, I began to lose focus.
Overall Experience
In conclusion, my experience at the conference was extremely positive. The conference content was very informative and presented well. Additionally, much was gained outside the confines of the conference itself. I had the opportunity to meet and socialize with multiple individuals who are BSD enthusiasts and others with whom I’ve communicated with digitally previously.
FreeBSD kernel panic in udp_input()
FreeBSD kernel panic in udp_input()
The Problem
Running FreeBSD 8.3, this kernel panic was produced under UDP load:
Fatal trap 12: page fault while in kernel mode
cpuid = 4; apic id = 08
fault virtual address = 0x0
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff807a08a2
stack pointer = 0x28:0xffffffa40c2689b0
frame pointer = 0x28:0xffffffa40c268a80
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 12 (swi1: netisr 0)
trap number = 12
panic: page fault
cpuid = 4
KDB: stack backtrace:
#0 0xffffffff80642b3e at kdb_backtrace+0x5e
#1 0xffffffff8060fd57 at panic+0x187
#2 0xffffffff80905990 at trap_fatal+0x290
#3 0xffffffff80905ce1 at trap_pfault+0x201
kernel trap 12 with interrupts disabled
#4 0xffffffff8090619f at trap+0x3df#5 0xffffffff808ed674 at calltrap+0x8
#6 0xffffffff8072000c at ip_input+0xac
Fatal trap 12: page fault while in kernel mode
cpuid = 5; apic id = 0a
#7 0xffffffff806cba29 at swi_net+0x149
fault virtual address = 0xc
#8 0xffffffff805e7794 at intr_event_execute_handlers+0x104
fault code = supervisor read data, page not present
#9 0xffffffff805e8e25 at ithread_loop+0x95
instruction pointer = 0x20:0xffffffff8064f795
#10 0xffffffff805e49af at fork_exit+0x11f
stack pointer = 0x28:0xffffffa41cd969a0
#11 0xffffffff808edbbe at fork_trampoline+0xe
frame pointer = 0x28:0xffffffa41cd969d0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = resume, IOPL = 0
current process = 8 (pagedaemon)
trap number = 12
Marc de la Gerroniere and Julien Charbon, developer colleagues of mine, commenced an intense troubleshooting exercise that lasted several days. Thanks to them for providing the patch below.
The Issue
The kernel panic occurred when a process executes `sysctl net.inet.udp.pcblist` followed by a UDP socket being closed and then receiving a packet on that socket before the process executing sysctl had completed.
The Patch
Applying this patch to FreeBSD sources and building resolves the issue above.
commit 497aa2bc6f4b7383344ab7fd812d296c6997298d
Author: Marc de la Gueronniere
Date: Wed Oct 24 19:59:41 2012 +0000
kern/172963: Kernel panic in udp_input()
diff --git a/sys/netinet/udp_usrreq.c b/sys/netinet/udp_usrreq.c
index b720364..f9d0245 100644
--- a/sys/netinet/udp_usrreq.c
+++ b/sys/netinet/udp_usrreq.c
@@ -494,6 +494,12 @@ udp_input(struct mbuf *m, int off)
INP_RLOCK(inp);
+ // in_pcbdrop does not unlink from V_udb
+ if (inp->inp_socket == NULL) {
+ INP_RUNLOCK(inp);
+ continue;
+ }
+
/*
* Handle socket delivery policy for any-source
* and source-specific multicast. [RFC3678]
@@ -1559,6 +1565,7 @@ udp_detach(struct socket *so)
KASSERT(up != NULL, ("%s: up == NULL", __func__));
inp->inp_ppcb = NULL;
in_pcbdetach(inp);
+ in_pcbdrop(inp);
in_pcbfree(inp);
INP_INFO_WUNLOCK(&V_udbinfo);
udp_discardcb(up);
diff --git a/sys/netinet6/udp6_usrreq.c b/sys/netinet6/udp6_usrreq.c
index 22ddde4..2e8b564 100644
--- a/sys/netinet6/udp6_usrreq.c
+++ b/sys/netinet6/udp6_usrreq.c
@@ -271,6 +271,10 @@ udp6_input(struct mbuf **mp, int *offp, int proto)
inp->inp_fport != uh->uh_sport)
continue;
}
+ // in_pcbdrop does not unlink from V_udb
+ if (inp->inp_socket == NULL) {
+ continue;
+ }
/*
* Handle socket delivery policy for any-source
@@ -964,6 +968,7 @@ udp6_detach(struct socket *so)
up = intoudpcb(inp);
KASSERT(up != NULL, ("%s: up == NULL", __func__));
in_pcbdetach(inp);
+ in_pcbdrop(inp);
in_pcbfree(inp);
INP_INFO_WUNLOCK(&V_udbinfo);
udp_discardcb(up);
References
A “Problem Report” (aka PR) was submitted to the FreeBSD project; it can be viewed at http://www.freebsd.org/cgi/query-pr.cgi?pr=172963.
hostileadmin 