“POSIX advisory locks are broken by design”

https://www.sqlite.org/src/artifact/c230a7a24?ln=994-1081

by okket • 8 years ago

150 99 8 years ago

tonysdg• 8 years ago

I'm pretty sure this is a well-known problem in general on Linux -- file locking is almost never a good solution. It sucks, since there are definitely times where it would be useful.

In any event, this is probably a low priority for glibc/kernel devs -- the fact that syncing doesn't actually behave as expected is much more important (https://lwn.net/Articles/752063/).

mjw1007• 8 years ago

They added a solution for this three or four years ago, by adding "Open File Description Locks" which don't have the misbehaviour that that SQLite comment is complaining about.

See https://www.gnu.org/software/libc/manual/html_mono/libc.html...

jwilk• 8 years ago

Page with only the relevant chapter (rather than 5MB monstrosity): https://www.gnu.org/software/libc/manual/html_node/Open-File...

wahern2• 8 years ago

You can also use the BSD extension, flock(2), which is supported everywhere except commercial SysV clones like Solaris and HP/UX. flock has the semantics you would expect[1] and isn't Linux-only.

This LWN article has a great explanation of all the behaviors: https://lwn.net/Articles/586904/

[1] Effectively identical to the Linux extension/POSIX proposal except that (1) they don't support ranges, (2) don't contend with POSIX locks (they do on real BSDs but not Linux), and (3) BSD flock won't work across NFS.

SQLite• 8 years ago

Yes. And there is code in the SQLite source tree (currently on a branch) that supports OFD locks. The problem we have is that SQLite is so widely deployed and is in so many systems and on so many platforms, that we have to continue to support Posix Advisory Locks (PAL) for the foreseeable future, for compatibility. It will be great if someday we can remove the PAL code. But for now, it has to stay in the tree.

bch• 8 years ago

GNU != POSIX.

Glibc work won’t fix (e.g.) the BSDs.

kazinator5• 8 years ago

Actually, it wasn't a problem on Linux under the initial pthreads implementation. The API worked as you would expect for threads; the advisory locks were owned by individual threads. Eventually (probably after the change to NPTL and improved kernel support for threads) it was fixed to conform to the POSIX design whereby the locks are owned by the process.

mmt19• 8 years ago

the advisory locks were owned by individual threads. Eventually (probably after the change to NPTL and improved kernel support for threads)

Isn't that because, in the time prior to that "improved kernel support", threads were really processes, and it was once they became full-fledge (i.e. fully lightweight) threads that they lost the ability to own the advisory locks? Or am I misremembering?

blablabla123• 8 years ago

File locking seems good enough to me. I wasn't consiciously aware of this short-coming but when you think about it, it makes total sense: the syscalls for file locks (at least on Linux and macOS) are related to the file descriptors. So when you close the file, the lock is gone.

Why would you in any case hold 2 file descriptors open of the same file in the same process, especially when you use locking? For a "multi-process application" (this sounds so antiquated) probably one thread should hold the fd and exclusively write on it.

mmt19• 8 years ago

My (admittedly quick) reading of this is that the issues stem, primarily, if not exclusively, from using multiple file descriptors for the same file, necessitated by using/supporting threading for concurrency.

If that's the case, then perhaps it's not so much that PAL was broken by design but that the original design merely didn't anticipate threading. I'm not sure how the advisory locking scheme compares in terms of timeline with threading support, but Unix file locking in general easily predates even SMP.

Hello71• 8 years ago

I don't think that's correct. According to Wikipedia, SMP dates to approximately the early 1960s, whereas according to the FreeBSD man pages, flock and fcntl were both introduced in 4.2BSD, released in 1983, two decades later. It's at most fair to say that Unix file locking came around the same time as Unix SMP.

throwaway765433• 8 years ago

"Unix SMP" is not synonymous with a system threading interface -- processes give parallelism as well and predate the introduction of threading interfaces. The pthread interface came about in the 1990s.

The unix interface is broken with respect to threads in many places besides this. Signals. Forking. Problems abound because the concept was bolted on decades later.

mmt19• 8 years ago

As the sibling comment implies, I don't mean SMP anywhere in computing, I mean in the Unix world, which didn't even exist in the early 1960s.

I'll grant that my early Unix knowledge is far from comprehensive, primarily skewed toward hardware at certain universities and in the database industry, as well as mostly BSD, with little from the "AT&T side" before SVR3. Given that, I'd be hard-pressed to name a Unix port that supported SMP prior to 1983.

And I didn't mean to equate SMP with threads, only that thread support wouldn't make much sense without hardware concurrency, of which multi-processor hardware was an early practical example in the Unix world.

As such, it was merely a (failed) rhetorical device to illustrate how designers of filesystem locking might not even have a notion such as intra-process concurrency on their mental radar.

Even if you are, pedantically, correct, it doesn't refute my overall suggestion.

zAy0LfpBZLC8mAC7• 8 years ago

Well, threading is a major problem area, but it is most definitely broken by design in any case, in that it has sort-of "spooky action at a distance" semantics, which is just completely uncomposable. Imagine you call a function from a library that simply scans a directory for files with a specific magic value, say ... if any other part of your program happens to hold a POSIX lock on any of those files (could even be another unrelated library), that will release those locks. That's just braindead.

mmt19• 8 years ago

I can't claim to speak for the designers, but my intuition (having had my first Unix exposure in the early/mid-80s) is that their assumptions were along the lines that it would be used consistent with the part of the Unix philosophy stating:

Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".

This seems inconsistent with the (now totally common) programming styles that use multiple, unrelated libraries or have a single program do something that requires keeping files with locks open, while going off on a tangent to scan a directory. Back then, I suspect the standard practice would be to fork() if not outright delegate the task to existing tools like find/xargs.

I certainly agree that hidden side effects like closing a duplicate fd causing all the locks to disappear is undesirable, but it might not even have occurred to them that anyone would care. Maybe they thought about it and the alternative would be too memory intensive (remember when memory was actually scarce?). It would be interesting if anyone had pointers to something like actual mailing list archives.

Characterizing it as "just braindead" as uncharitable, when it's decades later, without at least being able to point to contemporaneous pressures and constraints.

iforgotpassword• 8 years ago

That design might be for portability, since pread() and pwrite() ought to be thread safe, so one global FD per file for your multi threaded app should be enough, at least on Linux and freebsd. It does however not help if some other part of your application "accidentally" opens that file too (some lib for example).

Aaargh20318• 8 years ago

If that's the case, then perhaps it's not so much that PAL was broken by design but that the original design merely didn't anticipate threading.

I don't think it's even that. It's simply because the API describes the contract between the kernel and processes, not the kernel and threads. It's a case of determining who is responsible for what.

In this case the process is responsible for coordinating file locks with the kernel, not individual threads. From the kernel's perspective, 2 requests to the locking API from different threads still come from the same actor: the process. From the perspective of the locking API threads are a process's private business.

A developer failing to understand the abstraction levels in an API is not the fault of the API's design, but it could be caused by documentation.

Dylan168073• 8 years ago

Being unable to control a lock from multiple threads isn't a great API either. Per file descriptor^H^Hion is closer to what I'd call ideal. It gives you a locking context that can map m:n to threads. Even if it is a bit fussy with things like fseek.

emmelaich2• 8 years ago

If you want locks from the same user you'd use other mechanisms that file locks. File locks are for processes that don't have a lighter weight way to cooperate. (e.g. semaphores.)

Also, there's a reason that databases have processes dedicated to writing (e.g. Postgres writer process, Oracle dbwr)

Fun fact - IBM's MVS didn't have file locks.

pvg• 8 years ago

there's a reason that databases have processes dedicated to writing

That's going to be a little tricky in an embedded db.

emmelaich2• 8 years ago

Sure, but you could still make sure writes are done in a mutex?

CaliforniaKarl• 8 years ago

An unrelated comment: Looking through the rest of the file, I love how many comments there are in general! I am able to get a sense of what each function does, and what the individual code blocks within the function are doing.

This is something that I do not see as often as I want, and I'm really glad to see it here!

Annatar8• 8 years ago

When he gets a file descriptor, he could also get an inode. Since on hard links inode will be the same, the prior lock state, if any, can be obtained and action taken accordingly.

tinus_hn• 8 years ago

As far as I know these alternative thread libraries are ancient history. I can understand you want to defend against sabotage like people using hard links to open a database under two names but is that really a dealbreaker?

“POSIX advisory locks are broken by design”

Related Stories