I installed Apache on a diskless FreeBSD-9/amd64 server. Once I added SSL, the web server wouldn’t start. It died with:
[Mon Mar 21 15:37:16 2011] [emerg] (13)Permission denied: couldn't grab the accept mutex
[Mon Mar 21 15:37:16 2011] [emerg] (13)Permission denied: couldn't grab the accept mutex
[Mon Mar 21 15:37:16 2011] [emerg] (13)Permission denied: couldn't grab the accept mutex
[Mon Mar 21 15:37:16 2011] [emerg] (13)Permission denied: couldn't grab the accept mutex
[Mon Mar 21 15:37:16 2011] [emerg] (13)Permission denied: couldn't grab the accept mutex
[Mon Mar 21 15:37:17 2011] [alert] Child 1733 returned a Fatal error... Apache is exiting!
Google says that the fix for this is to define AcceptMutex flock
somewhere in httpd.conf. Google shows dozens of mailing list discussions that give this advice, but without explanation. Every so often, someone whinges about it not working, and doesn’t get a response.
This is an example of what I call “Occult IT.” There are certain recipes that people just follow, without understanding why. We stumble around, grasping for invocations and incantations that will fix our problem. I don’t just want the ritual that will solve the problem; I want to know why the problem happened and deepen my understanding of the issue.
Besides, adding AcceptMutex flock
didn’t work.
To identify the problem, I set LogLevel to debug in httpd.conf and tried starting httpd again.
[Mon Mar 21 16:03:18 2011] [debug] prefork.c(1018): AcceptMutex: flock (default: flock)
I didn’t have that configuration setting in my httpd.conf, but it’s the Apache default anyway. So, what are my other choices?
Apache documentation says that the AcceptMutex setting determines how Apache serializes incoming requests. The flock setting dictates how Apache locks the LockFile.
My configuration doesn’t have a defined LockFile, so choosing how we lock it isn’t going to help.
I don’t like lock files. Bad stuff happens to them. Let’s try a locking method without a lockfile. The documentation lists two different classes of locking mechanisms. flock
and fcntl
work on lock files. posixsem
, pthread
, and sysvsem
use the in-memory mechanisms of semaphores and/or mutexes to provide locking.
As I don’t like lock files, I’ll try one of the in-memory mechanisms.
AcceptMutex posixsem
And Apache starts and runs perfectly.
I can’t find any details on the differences between these in-memory mechanisms, from a system administrator’s point of view. I imagine that the System V mechanism wouldn’t work if you’d removed that support from your kernel. But the point is:
Do not rely on occult IT. Read The Fine Manual.
Found that site searching for that problem, now i’m rss subscribing it…
By the way, that “couldn’t grab the accept mutex” may be related to eccessive concurrency ?
Good research, even better solution! Thanks a lot!
Very Nice!
AcceptMutex posixsem
add to httpd.conf
Cpu(s): 8.5%us, 0.6%sy, 0.0%ni, 90.7%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 6104728k total, 801408k used, 5303320k free, 10808k buffers
Swap: 2031608k total, 107380k used, 1924228k free, 98796k cached
Thank You.
Saved the day. Thanks!