Tmpfs inode corruption: introducing inode64

https://chrisdown.name/2021/07/02/tmpfs-inode-corruption-introducing-inode64.html

by r4um • 5 years ago

161 27 5 years ago

cpcallen• 5 years ago

Can someone explain how tmpfs / /dev/shm track files if not by inode number?

esjeon• 5 years ago

Please read the article. TMPFS was internally using 32bit unsigned int for counting inode number, which got overflow-ed. Kernel already has 64bit wide ino_t, the dev migrated to it.

puppet-master• 5 years ago

The question was clear enough: per the article tmpfs is able to track two distinct files with the same inode number, implying it does not use the inode internally to differentiate between files

rav• 5 years ago

Since the contents are in-memory, you could imagine a straightforward implementation where the directory hierarchy in the tmpfs is stored in a pointer-based in-memory tree.

A file handle just needs a pointer to where in memory the contents are stored, and the inode numbers are computed on the side because it's part of the interface.

You don't want to use the pointers as inode numbers directly, since for security reasons the kernel doesn't want to export valid kernel pointers to user space.

yencabulator• 5 years ago

To play with these concepts, you can write a FUSE filesystem where each file has inode 42, while being wholly separate nodes & handles. Linux VFS doesn't really use the inode number internally for anything while a direntry is alive.

pengaru• 5 years ago

Since there's no need for these inode numbers to survive a reboot, the inodes are simply generated on-the-fly for a given tmpfs mount, and not persisted anywhere durable.

TFA makes this clear...

formerly_proven3• 5 years ago

inodes are actual objects in the kernel. Their inode number is supposed to be such that (device, inode, generation) uniquely identifies it, but the kernel itself doesn't really care about this (as long as your filesystem is not using the inode cache, which does use the inode number as a key). The way tmpfs works is that it uses the dirent cache as it's actual data structure (forcing the refcount of all cache entries >1 means they can't be evicted); those just directly contain pointers to inode objects. File data is stored in the page cache.

jabl• 5 years ago

Similarly, XFS has the inode32/64 mount options. Since kernel 3.7 inode64 is the default: https://man7.org/linux/man-pages//man5/xfs.5.html

esgwpl• 5 years ago

"The inode64 and inode32 names are used based on existing precedent from XFS."

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

kzrdude3• 5 years ago

Is the mount option needed for backwards compat? Any system that creates that many files, should be large file aware, hopefully? Maybe it comes down to many files being created (and deleted) during the lifetime of the tmpfs mount, i.e usage time, not size, without any counter reset.

A system that uses "our identifier space is approximately infinite" can be fragile in a surprising way. A system that uses "our identifier space is proportional to the physical size of the medium" is more humanly predictable in its limitations.

rini17• 5 years ago

And what can you do here? Either suddenly fail with "inodes exhausted" error (but here they aren't really exhausted only the numbers are), or try to reuse numbers (which would slow the allocation and potentially cause confusion), or...?

kzrdude3• 5 years ago

Reusing inodes is not that uncommon among filesystems, so I think tmpfs should have had an implementation like that.

smitty1e• 5 years ago

I have not read an article in quite some time that takes a complex topic and lays it out so clearly.

Thank you!

amelius• 5 years ago

While we're at it, can we also increase the default max number of hardlinks on ext4 filesystems? And the max number of commandline arguments?

tyingq• 5 years ago

I'm somewhat intrigued that this problem has remained for decades after 64 bit inodes were available in Linux.

opk2• 5 years ago

I enabled `pam_namespace` for per-user `/tmp` and `/var/tmp` on the systems I administer with those filesystems being tmpfs and xfs respectively. We get odd problems weekly where one or other user can't write to `/tmp` (but nsenter fails to reproduce it) but have never had issues with the `/var/tmp`. We came from Solaris where tmpfs was standard for `/tmp` and it surprised me to find that it seems to be an unusual choice on Linux. Are there reasons it isn't widely deployed on Linux? I can't think why inode reuse could cause the problems I experience but if I can enable this inode64 option, I'll give it a try.

scottlamb• 5 years ago

Iirc several distributions tried out tmpfs-based /tmp a few years ago then reverted. I think the problem was just that some Linux-specific software and sysadmins had grown up expecting to be able to write a lot of data to /tmp. Limiting it to 50% of RAM (the default iirc) didn't go well. The distribution people didn't want to deal with converting those things to use /var/tmp so they backed off. I don't think it helped politically that the systemd people were pushing it, as those folks were already perceived in some circles as dictating too much about the direction of Linux.

This is all from memory and I'm on my phone, so no citations. In any case, I don't think inode numbers were the problem.

opk2• 5 years ago

I've not come across FS_IOC_GETVERSION before. Should the many user-mode applications that rely on st_dev, st_ino being enough to identify an inode also be calling this ioctl? Or is it something you only need to worry about in limited circumstances?

Tmpfs inode corruption: introducing inode64

Related Stories