A minimum complete tutorial of Linux ext4 file system (2017)

https://metebalci.com/blog/a-minimum-complete-tutorial-of-linux-ext4-file-system/

by giis • 3 years ago

169 29 3 years ago

Does anyone know if there exists a tool that can convert tarballs to filesystems and back. I know you can make a loopback device, but it can be pretty hard/impossible to do inside a container, and often requires special flags and privileges.

st_goliath• 3 years ago

For squashfs there is `tar2sqfs`[1] and `sqfs2tar`[2] (and now also `sqfstar`[3]) that can go from tarball to filesystem and back again.

For ext4, I recently saw a patch set on the mailing list[4].

Not sure about other filesystems tough.

[1] https://manpages.debian.org/unstable/squashfs-tools-ng/tar2s...

[2] https://manpages.debian.org/unstable/squashfs-tools-ng/sqfs2...

[3] https://manpages.debian.org/unstable/squashfs-tools/sqfstar....

[4] https://lore.kernel.org/linux-ext4/20230812150204.462962-2-j...

loeg• 3 years ago

guestfish and libarchive are sort of in this space. Neither is a canned tool for exactly this.

cedws2• 3 years ago

You can extract the tarball to a directory, then run mkfs.ext4 -d DIRECTORY FILENAME BLOCKS to create a filesystem[0]. You'll need to know how many blocks your filesystem needs to be in advance.

Unfortunately, mkfs.ext4 only works on Linux. There is no port for other operating systems.

[0] https://github.com/cedws/concrete-ubuntu/blob/0ae3f076c5a20d...

guappa• 3 years ago

If you want to create read-only, use ext2. No point in using ext4 if you don't want journaling.

chociej• 3 years ago

7z can list and extract from ext4 images. Respectively:

$ 7z l <image>

$ 7z x <image>

chungy• 3 years ago

You can use fuse2fs to mount any ext[24] file system in user space. You can even use "-o fakeroot" to get complete control over UIDs and modes.

ravenstine• 3 years ago

I've been doing something similar recently, though it doesn't directly convert a tar to an ext4 FS. But maybe this can help you get to where you want to be.

On Alpine Linux:

```

apk add --no-cache coreutils e2fsprogs

```

#!/bin/sh

# Untar the tar file

mkdir -p /tmp/my_untarred_files_dir

tar -xvf my_tar_file.tar -C /tmp/my_untarred_files_dir

# Make an empty image file.

dd if=/dev/zero of="fs.raw" bs=1M count=1024

# Format the file as ext4 (with journaling) and copy untarred files into it

mke2fs -t ext4 -j -d "/tmp/my_untarred_files_dir" "fs.raw"

```

If you want to make a qcow2 image, you can then do this:

```

apk add --no-cache qemu-img

qemu-img convert -O qcow2 "fs.raw" "fs.qcow2"

```

torcete• 3 years ago

Maybe you are looking for something like fsarchiver?

https://www.fsarchiver.org

sitzkrieg2• 3 years ago

i am baffled this article used a usb stick instead of a file where it could have saved a lot of unnecessary i/o, but a good writeup. i had to implement ext4 support in something back in like 2017 and wish something like this existed

yjftsjthsd-h• 3 years ago

Maybe they wanted to avoid having to deal with setting up the loopback device?

mike_hock• 3 years ago

You mean the one losetup command?

contingencies• 3 years ago

Definitely that would be less portable syntax.

fsckboy• 3 years ago

what definition of i/o are you using, does file i/o not count?

sitzkrieg2• 3 years ago

i apologize, i meant external (and prob slower)

buredoranna2• 3 years ago

Awhile back I put together a visualization of ext4.

https://buredoranna.github.io/linux/ext4/2020/01/09/ext4-viz...

tptacek• 3 years ago

This is fantastic.

buredoranna2• 3 years ago

Thanks for saying so :)

doubloon• 3 years ago

thats very cool. i feel like a long time ago we used to be able to cat /dev/hda1 > /dev/video ? but maybe im imagining things. i know you could do it with audio.

loxias• 3 years ago

Yeah, it would work with /dev/dsp, the primary interface for audio IO in Open Sound System (OSS) -- the first media API for nx operating systems. Around the turn of the century it was replaced in the Linux kernel by the Advanced Linux Sound Architecture (ALSA). Which, to this day, is still the "lowest level" high performance way to do audio w/ linux.

As for _visualizing_ the audio as video, no, that was never possible w/o writing code. =) And to do it _well_, you need to do some math as well. I say this because I wrote that code in 2001, 2003, then again in 2008, then again later... it's a ... hobby of mine :)

colanderman• 3 years ago

/dev/fb0, yes you could do that if you configured your system to use framebuffers.

mixmastamyk• 3 years ago

Thanks. I really do miss the disk visualizations of the DOS and Norton Disk Doctor diagnostics and defragmenters from the old days. The one for the original Mac was pretty incredible due to high res graphics, even had color coded filetypes, etc if memory serves.

Linux never had a good one to my knowledge. Improved filesystems reduced the need and then SSDs delivered the deathblow.

But I feel like the visualization was useful in itself to see what was happening on the storage device and unfortunately forgotten due to those improvements.

fuzzfactor2• 3 years ago

really do miss the disk visualizations of the DOS

Even Windows XP had a graphical representation of the disk contents, and it animated during defrag.

killjoywashere• 3 years ago

#keeper

rollcat• 3 years ago

It's absolutely fantastic, how gracefully ext2/3/4 stood the test of time, for something designed in early 90s by people who hadn't had 30+ years of kernel hacking in their pockets (yet). While I generally prefer ZFS, ext4 remains a rock solid choice - I rarely have a reason to pick anything else (on Linux, at least).

sipofwater• 3 years ago

Uses the ext4 file system --> "LUKS encryption and decryption: In the cryptsetup-laboratory with Termux (running under the Android 11 operating system), "cryptsetup reencrypt --disable-locks --type luks2", no root access, no loop device, and an unusable "mount" command.": https://old.reddit.com/r/termux/comments/18am78j/luks_encryp... (old.reddit.com/r/termux/comments/18am78j/luks_encryption_and_decryption_in_the/)

A minimum complete tutorial of Linux ext4 file system (2017)

Related Stories