This fancy sounding heading was totally authored after I understood what I wanted to do, which happened from writing up all these notes.
Disclaimer & tl;dr 👉 I have yet to figure out how to do this, because I don’t want to throw a bunch of commands at my computer I don’t understand the consequences of. Wish me luck for 2020.
The annoyance I want to improve
The project I’m working on uses OpenShift, and apparently I need to ramp up the system-wide resources for Minishift to run happily. Specifically: increase the number of open files per process. I’ve set a thing in my
.bashrc and run some commands and it’s all working fine. But after every restart, I get an error in a new shell and need to repeat the process. This is in my
ulimit -n 500000
But after a reboot, the terminal will complain that:
-bash: ulimit: open files: cannot modify limit: Invalid argument
When I check, I can see that the number has returned to 256:
launchctl limit maxfiles
maxfiles 256 unlimited
Then I can run this command to increase it:
sudo launchctl limit maxfiles 5000000 5000000
And check that it increased:
launchctl limit maxfiles
maxfiles 5000000 5000000
But after the next reboot, I need to repeat the command. So let’s dig into what all this stuff is, so I can try to improve the set up and learn what’s actually going on.
launchctl interfaces with launchd to manage and inspect daemons, agents and XPC services. launchctl allows for detailed examination of launchd endpoints.
— BSD General Commands Manual
launchd manages processes, both for the system as a whole and for individual users. The primary and preferred interface to launchd is via the launchctl(1) tool which (among other options) allows the user or administrator to load and unload jobs. Where possible, it is preferable for jobs to launch on demand based on criteria specified in their respective configuration files. — BSD General Commands Manual
A-ha. These are shell
builtin commands. Just like
a command or a function that is executed directly in the shell itself, instead of an external executable program which the shell would load and execute. — wiki/Shell_builtin
In the man page for
launchctl I can find this description about
but do I wonder why it’s listed under Legacy Subcommands 🤔
With no arguments, this command prints all the resource limits of launchd as found via getrlimit. When a given resource is specified, it prints the limits for that resource. With a third argument, it sets both the hard and soft limits to that value. With four arguments, the third and forth argument represent the soft and hard limits respectively.
But description makes sense, this is what the sudo command above does.
More words and concepts to take note of
init is the first process started during booting of the computer system. Read more: wiki/Init
Operating system service management examples:
- launchd - Used by Apple macOS
- systemd - Used by many Linux distributions
daemon is a computer program that runs as a background process, rather than being under the direct control of an interactive user. Read more: wiki/Daemon
Traditionally, the process names of a daemon end with the letter d, for clarification that the process is in fact a daemon
the parent process of a daemon is often, but not always, the init process
But back to figuring out what to do…
Now I understand way more what those commands up there do. It works, just not permanently.
Reading: Increase the maximum number of open file descriptors in Snow Leopard?
- Previously you could set the limits in a
/etc/launchd.confbut this is not longer supported
- You can still use
launchd.plistbut is it recommended? 🧐
sysctl -a | grep ^kern.max
Cool command! What does it do?
sysctl | a software utility of some Unix-like operating systems that reads and modifies the attributes of the system kernel such as its version number, maximum limits, and security settings |
-a | List all the currently available non-opaque values |
grep ^kern.max | …but instead of outputting the entire list, run a search and just return results that include
kern.maxvnodes: 263168 kern.maxproc: 4176 kern.maxfiles: 49152 kern.maxfilesperproc: 24576 kern.maxprocperuid: 2784 kern.maxnbuf: 16384
Reading: How to persistently control maximum system resource consumption on Mac? While I don’t understand the details, the answer below sounds like good advice. It explains how you’re likely to run into problems by changing these limits, and there is an alternative:
In general, rather than tuning individual parameters on the system and throwing the system out of balance (and potentially letting a single program crash the system by taking up all the resources), if the default system limits are insufficient for your needs, I recommend turning on "Server Performance Mode", or at least giving it a try.
Official Apple advice how to: Turn on performance mode for macOS Server
performance mode can be enabled even without macOS Server being installed to achieve similar benifits for other high-performance services
– Enable macOS Server Performance Mode
🤷🏻♀️ Tried this, but it didn’t work. Running Minishift still fails due to too many open files.
Should I leave it on anyway?
Server Performance Mode does take up more memory, and makes the system much more likely to suffer if some program goes out of control consuming resources, but greatly increases the capability of the system to handle a lot more background tasks. I think Apple made the right call by not turning it on by default but also making it easy to enable.
– What does serverperfmode=1 actually do on macOS?
Hm. I don’t entirely understand this. I’ll leave it on for now, and made a note to revisit later.
What are hard and soft limits?
The two numbers represent soft and hard limits. When the soft limit is reached, the process may receive a signal but will be allowed to continue. When it reaches the hard limit, it will be blocked. – Persist ulimit settings in Mac OS X
sysctl – configure kernel parameters at runtime
The numbers are higher, and round?! 🤔 with
serverperfmode=1 turned on.
sysctl -a | grep ^kern.max kern.maxvnodes: 300000 kern.maxproc: 5000 kern.maxfiles: 300000 kern.maxfilesperproc: 150000 kern.maxprocperuid: 3750 kern.maxnbuf: 16384
…and I learnt so much about how my machine works. But for now I will have to stick with re-running the
sudo launchctl limit maxfiles 5000000 5000000 command after each reboot.