How Proxmox VE shreds your SSDs – with details

https://old.reddit.com/r/homelab/comments/1hisbw3/followup_on_the_earlier_why_proxmox_ve_shreds/

by free-pmx • 2 years ago

9 4 2 years ago

old.reddit.com

ggm2• 2 years ago

It writes too many logs. It's always log writes, except when it's the DNS.

I've not yet tried out Proxmox. Can you define where the Proxmox logs go? If so one could mount tmpfs ramdisks at startup and point the logs there with an aggressive log rotation cron to keep the usage low. If this is just syslog one could mount /var/log as tmpfs accepting that logs go away if one reboots or send syslog elsewhere over the network to cheap storage.

    # grep "/var/log" /etc/fstab
    tmpfs /var/log tmpfs defaults,nosuid,nodev,noatime,size=1g 0 0

ggm2• 2 years ago

Not a proxmox user but that's certainly where I'd go, if I was: write to memory via an fs abstraction, get data off into another place via network if you need it.

There's also the noatime flag, for unavoidable writes to local disk: avoid the update to dir stat() state if you can.

m463• 2 years ago

I have a small system running proxmox with a disk activity LED on the fron. And even when quiet with no containers or vms, it is hitting the disk.

a LOT.

I turned off a bunch of stuff (I don't run replication):

  systemctl disable pve-ha-crm
  systemctl disable pve-ha-lrm

But stuff like /var/lib/pve-firewall and /var/lib/rrdcached was still written to every second.

I played around with mount options like commit=n

It kind of helped to use log2ram:

https://github.com/azlux/log2ram

but it sure didn't eliminate the problem.

I think with a little effort from the proxmox guys, they could gather a lot of low-hanging fruit.

How Proxmox VE shreds your SSDs – with details

Related Stories