Installing Every Arch Package

https://ta180m.exozy.me/posts/installing-every-arch-package/

by kinduff • 4 years ago

271 86 4 years ago

ta180m.exozy.me

ShortStretto• 4 years ago

https://archive.is/lQHf0

Site was barely loading for me, here is a mirror.

metahost• 4 years ago

The screenshots are not mirrored. :(

iqanq2• 4 years ago

They have cloned cloudflare's captcha page. How shady.

Tildey• 4 years ago

I think what you’re referring to is a known issue with how their server handles cloudflares 1.1.1.1 DNS, not them intentionally being shady.

miyuru• 4 years ago

ta180m.exozy.me seem to hosted on a homelab as its being served from a spectrum IP address. and its seems only the IPv6 is down. Accessing the IPv4 address gives a nginx login prompt.

Will be interesting to read about the incident if the author is here.

steerablesafe2• 4 years ago

If you only install supposedly non-conflicting packages, and there are conflicting file locations, isn't that a packaging bug?

ginko• 4 years ago

Sounds like it, yeah.

jackosdev• 4 years ago

Some packages give you options for different packages that conflict. E.g. if you install i3 it will let you choose between standard i3, i3-gaps or 5 others. You can only have one installed at a time.

I've been using Pacman for years never had a single problem with conflicts or anything else.

dpwm• 4 years ago

Not if you disable file conflict checking, as the author did.

As we used a lot of unsafe hacks (disabling dependency and file conflict checking, for instance) to get this to actually work, I wouldn’t recommend using this system for anything other than proving it’s possible.

faho• 4 years ago

Yes, and they did say they were going to file bugs:

Investigate the file conflicts and file some Arch Linux bugs.

with a link to https://bugs.archlinux.org/task/73574, which is indeed a bug in the packaging script (it fails to consider python version 3.10 because it only globs 3.?).

steerablesafe2• 4 years ago

Ah, I missed that.

no_time2• 4 years ago

Now install every AUR package. Bonus points for heuristically looking for cryptominers after it's done.

RealStickman_• 4 years ago

What would be the order of preference for conflicting packages (compiling, binary, git)?

no_time2• 4 years ago

I'd start with binary packages in the order of ascending popularity

https://aur.archlinux.org/packages/?O=0&SeB=nd&K=bin&SB=p&SO...

brightball2• 4 years ago

Is this a real problem?

Qub3d• 4 years ago

While there have been malicious packages found in the AUR before[0], I haven't heard of a cryptominer sneaking in... yet.

One was found in the Ubuntu Snap store at one point though[1]

[0] https://www.bleepingcomputer.com/news/security/malware-found...

[1] https://www.bleepingcomputer.com/news/linux/malicious-packag...

seanw4443• 4 years ago

It's only a matter of time. Linux really only has a good malware track record due to lack of popularity.

grapescheesee3• 4 years ago

Better yet! Test the different tools for installing AUR. By installing all of them with each! Also, how much time do you have? Going to bet, it won't take too long before something breaks. Most of the AUR admins do a great job, however dependence issues can happen and compound quickly.

I know AUR package managers are bad juju. Still it sounds fun.

Abishek_Muthian• 4 years ago

I recently found that brltty/orca package on Arch conflicted with CH340 driver[1] and prevented nodemcu from being detected over USB(tty). I uninstalled those packages, But it might not be an option for visually impaired.

Installing every Arch/AUR package could break some hardware compatibility which we've taken for granted(Story link is not opening as of writing, So not sure whether this case has been covered in it).

[1] https://unix.stackexchange.com/questions/670636/unable-to-us...

blahgeek• 4 years ago

IIRC archlinux is pretty conservative at enabling services by default after installing, so installing every package should not take too much system resource aside from disk space.

I’m curious about doing the same thing in Ubuntu

__del__2• 4 years ago

i believe it's a rite of passage for any new linux user on a dpkg-based linux distribution to install synaptic and with it every possible package.

grapescheesee3• 4 years ago

It is a lot more fun trying to see how many you can uninstall before it really breaks and how. Well mainly tried on Ubuntu. Best done from a separate TTY and watch the main X user window.

Macha• 4 years ago

Arch won't auto enable systemd services, but that doesn't mean e.g. Plasma won't load all installed plugins.

fran-penedo• 4 years ago

Now is this useful? The short answer is no. The long answer is also no. I can think of exactly zero uses of this experiment (and I must be pretty crazy for doing it).

Well, at the very least it's a very computationally expensive test that managed to catch a few packaging bugs.

ianai• 4 years ago

Which is definitely the reason to do it. Probably fleshes out most of the cornercases possible to experience. Further, HDD sizes are huge so just calling the whole enchilada install a standard install class makes a kind of sense. Only need to worry about picking and choosing until the whole enchilada becomes small enough to not be worth the decision process to minimize.

jandrese• 4 years ago

In the bad old days we used to do the "kitchen sink" installs of Redhat because our machines weren't connected to the internet nor did they have access to the install media after install, so if we needed a package and it wasn't already installed it was a big problem.

Even today it's kind of a headache to install packages on machines with no Internet access, usually the easiest solution is to download everything and setting up a package repo on the LAN.

If you want real fun, try setting up a nontrivial node.js project without direct Internet access.

Felitio• 4 years ago

This is very nicely done.

From the weird idea (why would that be interesting) to optimal solutions, np hard, disk issues and then an interesting result.

zibzab• 4 years ago

I didn't really understand the fun of using Arch until I watched a few episodes of the "monthly Arch install" on YouTube.

It is very useful seeing how the "pros" do things, which looks substantially easier than when I try to install Arch and end up with a mess only slightly better than in this article ;)

Edit: the channels name is "EF - Linux made simple".

ms-fellag• 4 years ago

Hey thank you for this discovery. After every arch-install i think to myself i should probably automate this, and i only go as far as logging all the commands typed (mostly doing '# history > log.sh' and adding a few comments) and said file is is forgotten or i am trying something different for the next install (like desperately trying to make a CB3-131 Chromebook work nicely)

Here are the links for the lazy

[channel] https://www.youtube.com/channel/UCX_WM2O-X96URC5n66G-hvw

[the latest installment of Arch Monthly Install:Jan 2022] https://www.youtube.com/watch?v=7btEUHjECAo

seanw4443• 4 years ago

I had a script that set up my Arch environment. And then I switched up some of my software for alternatives, and changed a lot of my configurations. Now I'm too lazy.

Datagenerator2• 4 years ago

Fun stuff.

Reminds me of one former coworker, which didn't like me introducing FreeBSD. So he started installing all ports packages.. only to say the following morning: Look it's broken! ( disk was obviously full and this plot was irrational to people with some common sense ).

nix232• 4 years ago

"Nice" former coworker...hope you found your way to freebsd anyway :)

Datagenerator2• 4 years ago

Sure did, still in active production use mostly for the excellent ZFS integration hosting many PB's of research data.

ldksjf• 4 years ago

It was a joy to read.

Also, what theme is this? Looks pretty clean

yashmehrotra• 4 years ago

The theme is Gokarna. Its a hugo theme (https://github.com/526avijitgupta/gokarna)

Thanks for the kind words, I am the author of this theme :)

reshie• 4 years ago

now try with gentoo with no prebuilt pkg's. the end really says a lot about arch though even if it does have edges ;).

conaclos4• 4 years ago

Please, stop using heavy images: bloated-kde.png is 3.2Mb... The image is still loading in the moment of writing...

Same note for the website logo: render-small.gif is 4Mb. Still loading...

CarVac• 4 years ago

I was halfway into the article and then the fonts started loading.

numlock86• 4 years ago

There's people like me that already get a bad feeling about their entire landing page including all assets breaking the 1MB barrier: "Having an image in the >150kB range? Ewww, let's put that quality slider further to the left!"

But this guy? Wow! Having 4MB as a logo that makes up only 38x38 pixels on a 1080p screen is an absolute chad move when it comes to bandwidth. And yes, the website is still loading here, too. I wonder why ... :^)

conaclos4• 4 years ago

Strangely, one of the image is okish: bloated-lxdm.png is 216Kb.

zinekeller• 4 years ago

Same note for the website logo: render-small.gif is 4Mb. Still loading...

I'm still in process of saving that one, but I can tell that's a GIF animation.

Update: It's 240 frames of a spinning dodecahedron.

kevincox2• 4 years ago

Even applying better PNG compression reduces them to less than a third of the size. Beyond that you can consider if the full resolution and quality is actually critical to the article.

bmn__• 4 years ago

Applying a bare amount of image optimisation:

    original                      3_389_491 B
    pngout                        2_739_324 B
    quantised 256 colours+pngout    636_135 B

    webp lossless                 2_210_112 B
    webp default settings (lossy)   210_452 B

Spivak• 4 years ago

This kind of advice will never work because it requires people to remember to do it. The right approach is "whatever SSG you use add some code to generate different size images and then use srcset."

conaclos4• 4 years ago

Another solution could be to test the website with different bandwidths. Modern browser have dedicated tools for that.

nimbius• 4 years ago

the guy just installed every package for arch. im...not entirely sure they comprehend the concept of bloat...

perihelions• 4 years ago

This page is an advertisement for uMatrix. I am seeing a perfectly readable, very elegant webpage in 37 kB of total network transfer. (It really is nice: the syntax highlighting is tasteful).

Permitting: 1p CSS; denying: everything else, including fonts (in uBlock).

theandrewbailey• 4 years ago

It seems that the site has changed. I'm loading less than 600k for this page (scripts, images, fonts... everything).

conaclos4• 4 years ago

Yep: every image is now in webp.

I am glad that my comment improved this website :)

titaniumtown• 4 years ago

The site doesn't even load lol.

willsi• 4 years ago

I might have heard of Julia, but I don't remember. Thanks for sparking an interest, and the entertainment.

heywire• 4 years ago

Learned about two new utilities today: ncdu and bashtop

cmg• 4 years ago

I just installed ncdu via homebrew and ran in in my home directory - very snappy and helped me clean out some old caches.

yjftsjthsd-h2• 4 years ago

Install every package of a different distro? In particular, I’d like to see someone do this for Ubuntu, Debian, and NixOS and watch them suffer. This was painful enough.

I'm not sure that would cause any suffering in NixOS other that running out of disk space; one of the big claims of nix is that it happily lets you install whatever you want without conflicts. Although they also have a somewhat different idea of what "install" means; you might have to still put in some work to surface as many programs as possible to a single shell's PATH.

arianvanp• 4 years ago

Replit actually has every nix package installed: https://blog.replit.com/nix

They mount a 1TB volume with all packages preinstalled

yjftsjthsd-h2• 4 years ago

I'm actually shocked that this only installed 12,232 packages and uses 250GB of disk. I mean, that's not small, but it's actually sort of reasonable.

jonringer1172• 4 years ago

Copying my comment from reddit[0]:

For clarity, Arch has about 10k packages, AUR has around 60k packages. I believe this post is "just" about the 10k.

I’d like to see someone do this for Ubuntu, Debian, and NixOS and watch them suffer.

Speaking for NixOS:

I have. I would sometimes do a nixpkgs-review[1] of the mass "rebuild" PRs for Nixpkgs[2]. Hard to know how long it took to build as I would just let it "cook" on my build server while I did other things. The other thing is that nix gives unique names to all built packages and utilizes "maximal sharing" thereof, so everything gets memo-ized[3] on future runs.

The scale of the official nixpkgs repository is 4-6x greater than that of Arch (AUR is the user repository). 9.6k Arch packages vs 59.4k Nixpkgs packages according to repology[4]

Lastly, installing packages in nix is different. Everything goes into the nix store, which is relatively "inert". I don't need to worry about "hooks" or stateful logic being executed affecting my system. "But then how do you create services and other meaningful abstractions needed to make an OS? I thought NixOS was a distribution" It is, and it's down through NixOS modules[5] in the form of a configuration.nix. The NixOS modules can compose the verticals in my system to deliver something coherent and amazing.

Server used:

    OS: NixOS 22.05 (Quokka) x86_64
    Kernel: 5.10.91
    CPU: AMD Ryzen Threadripper 3990X (128) @ 2.900GHz
    Memory: 125913MiB / 257687MiB

[0]: https://www.reddit.com/r/linux/comments/shxq12/comment/hv5by... [1]: https://github.com/Mic92/nixpkgs-review [2]: https://github.com/NixOS/nixpkgs/pull/144730#pullrequestrevi... [3]: https://en.wikipedia.org/wiki/Memoization [4]: https://repology.org/repositories/statistics/newest [5]: https://nixos.wiki/wiki/Module

Diti• 4 years ago

Installing every single Nix package is a piece of cake. Just run nix-env -i without argument.

Nix has been behaving that way for at least 8 years: https://github.com/NixOS/nix/issues/308

jonringer1172• 4 years ago

Oh, nix-env.... The tool which attempted to bridge the worlds of nix and traditional package managers. But has all of the quirks of both.

lilboiluvr69• 4 years ago

Can anybody tell me which terminal is next to cool-retro-term (on the right) in this image? https://ta180m.exozy.me/images/bloated-kde.webp

qbasic_forever• 4 years ago

Looks like just a different preset or setting for cool-retro-term to me. It doesn't have to be amber and super curved, the IBM/DOS preset is pretty tame IIRC. Or just go into the settings and turn off and tweak things as appropriate.

Regardless, if you want a nice terminal that isn't retro shaded and has zero UI cruft like tabs, menus, etc. check out Alacritty.

favourable• 4 years ago

Wouldn't this lead to a dependency hell? [0]

'The OS: Oh you have three versions of Python, let's use the earliest version and make your program insecure'

[0] https://en.wikipedia.org/wiki/Dependency_hell

anthk• 4 years ago

Slackware minus slackbuilds it's like that. But contrary to arch, it doesn't autostart any service.

cryptonector• 4 years ago

Install pkg -> start service == bad, very bad, don't ever do it, no, just no.

vondur• 4 years ago

Oh man, seeing the desktop environment selector on the login page is pretty funny.

bee_rider• 4 years ago

For some reason I feel bad downloading too many unnecessary packages from these volunteer projects. I guess any given mirror must have enough users that even something like this is just a raindrop in the ocean, though?

majewsky• 4 years ago

I run an Arch Linux mirror. [1] The server hosting the mirror is averaging 1 MiB/s (8 Mib/s) outgoing traffic. There's quite a bit of fluctuation since package downloads are a very spotty load, but overall I don't think the server is breaking a sweat at all. CPU usage is pretty flat at 5% (of 1 vCPU). The same server is also hosting approx. 10 other websites, but the mirror causes most of the load by far from what I can tell.

[1] This is n=1 data because I can only speak for my own machine, so obviously take this with a grain of salt.

native_samples• 4 years ago

I can think of a use for this.

If you can make a system work properly with every package installed, then you could in theory get rid of the concept of installation entirely. Use a FUSE file system to make it appear as if every package is installed, and then block any program that opens a non-installed file until the package has been downloaded and installed for real. Now you don't have to know about packages or pacman anymore. Just do ls /usr/bin to see what's available in the index, and then type in what you want and hit enter. Wait a few seconds and the program will start.

However, for this to work, does require that the system be stable with every package "installed" simultaneously, so it'd mean fixing UIs that assume only a few options are available.

walrus01• 4 years ago

now show us what your username directory's dot file directories look like. I bet it's a mess.

Installing Every Arch Package

Related Stories