man8alexd

Why do you think it is a better default? What does it solve?

You have a lot of unused anonymous memory that is better swapped out and used as a file cache. Specifically for Linux swap, see https://chrisdown.name/2018/01/02/in-defence-of-swap.html

You can't fix the fact that you can't predict the future. The software will always allocate more memory than it needs because it can't predict its future resource usage, so limiting memory allocation with strict overcommit is meaningless.

About shared memory included in the memory size calculations https://lkml.iu.edu/1902.2/05674.html

The kernel keeps track of active/inactive pages by scanning page tables during the reclaim process. It only scans until it finds enough inactive pages to reclaim. Scanning all pages all the time is very expensive from a performance point of view.

You can't know when the OOM killer is awake and hunting. The kernel doesn't know that.

OOM is better. If a program doesn't handle ENOMEM properly, then its state is unpredictable and can lead to data corruption.

Linux MemAvailable from /proc/meminfo is just an estimation calculated as an arbitrary percentage (50%) of free and potentially reclaimable memory.

You can't determine how much of the memory can actually be reclaimed under memory pressure until you try to reclaim it.

There is no point in managing memory allocations as they have little relation to actual memory usage. There are also other methods than the OOM killer to handle OOM, like process throttling using cgroups "memory.high" limits.

In this specific case, the correct behaviour would be to drop a part of the buffer pool until the memory pressure is gone. The context-dependent question is how much and how fast to drop. The current implementation drops to a single configurable level but I suspect it could have implemented better heuristics.

Because no one can predict the future, and they don't know how many resources they will need.

Maybe you should skip running a useless child process and just use PSI to monitor memory pressure.

You can always tell the system not to kill your Photoshop (or whatever else) by setting the OOM Score Adjust. This mechanism has existed for almost two decades and systemd has supported it for over a decade.

You will be surprised by how inaccurate memory measurements are.

See this retrocomputing.SE question https://retrocomputing.stackexchange.com/questions/32492/ori...

FreeBSD didn’t have memory overcommit and instead used strict swap reservation - each allocated anonymous memory page was supposed to have a corresponding swap page. This required 2x RAM swap space, otherwise you would get “out of swap” when forking a large process. FreeBSD implemented memory overcommit around 2000.

No, it doesn't handle failing allocations gracefully - https://unix.stackexchange.com/questions/797841/firefox-died...

Is Firefox low quality?

People like to reinvent things that they are not aware of. Original BSDs used to use strict swap reservation - every anonymous memory page had to have an associated swap page. You had to have the swap 2x of RAM to allow large processes to fork - otherwise you would get an "out of swap" error. FreeBSD implemented overcommit around 2000, I think version 4.x or 5.x.

MariaDB recently implemented memory PSI monitoring but failed with that in a curious way and disabled it afterwards by default. The failure is that under memory pressure, they flushed the entire InnoDB buffer pool.

The last paragraph:

On the modern desktop, where programmers don't care about failing malloc(), disabling overcommit is shooting yourself in the foot. As you can observe, the memory allocations start failing long before the memory is exhausted.

They do have sidecars like prometheus, node_exporter running alongside Postgres and they include them in their MemoryLimit calculations.

There is some kind of illusion or myth that strict overcommit solves memory management issues.

cgroups have nothing to do with overcommit and memory allocation. They limit actual memory usage for a specific program or group of programs. If this program tries to use more memory than the cgroup memory limit, the program gets OOM killed.

Yes, many have tried to use strict overcommit on the desktop. It is a good footgun. https://unix.stackexchange.com/a/797888/1027

Mode 0 (Heuristic) is described incorrectly. All this complex heuristic was removed almost a decade ago. Currently, the kernel refuses a single allocation that exceeds the physical memory. That is all.

The article ignores the proper modern solution to prevent OOM killing of critical processes - OOM Score Adjust.

Tuning CommitLimit manually is an archaic, imprecise, and error-prone way to handle memory limits, only suitable for single-process workloads that can handle ENOMEM properly. It completely ignores dynamic file page cache memory allocation. You still can get OOM if you get unusually high file activity. On the other hand, under low file activity, it wastes memory on the same page cache, because it can't be reclaimed without memory pressure, and memory pressure can't be created because workload hits ENOMEM earlier. Don't use strict overcommit.

Windows doesn't need to fork, and you can't fork a large process without overcommit.

disabling overcommit is trading OOM for random program crashes due to the inability to handle ENOMEM. It also wastes a lot of system memory.

https://unix.stackexchange.com/questions/797835/disabling-ov...

The Linux Kernel OOM killer kills random things.

By default, the Linux kernel kills the largest process in the system (unless OOM adjust was applied).

Location: Belgrade, Serbia (UTC+2)

Remote: yes (working exclusively remotely for the past 20 years)

Willing to relocate: yes

Technologies: AWS, Kubernetes, Nomad, Docker, podman, Hashicorp Consul, Vault, Packer, Terraform, Ansible, Chef, Puppet, Grafana, Loki, Prometheus, ELK stack, Jenkins, GitHub Actions, Flux, GitLab, Postfix, HAProxy, MySQL, MariaDB, PostgreSQL, Ruby, Python, bash, C

Résumé/CV: https://alexeydemidov.com/cv

Email: in the resume

GitHub: https://github.com/AlexeyDemidov

LinkedIn: https://www.linkedin.com/in/alekseydemidov/

ServerFault: https://serverfault.com/users/23022/alexd

Have been playing with Linux and FreeBSD for over two decades. Run an ISP engineering team for a decade, now building and maintaining cloud infras.