ticklyjunk

I built this arvhiver because i was maxing out my drives and i wanted to be better at compressing and managing files without the constant headache of manual archives. it is basically a hybrid user space storage kernel written in rust that makes cold storage like tapes, old hard drives, or s3 feel like a transparent extension of your local ssd. when your ssd hits a high water mark huskhoard moves cold files to n way replicas but keeps the file entry visible in your directory using hole punching with fallocate. if a process tries to read a file that has been moved the software intercepts the call using the linux fanotify api and restores the blocks in the background. unlike fuse this does not add overhead to your active files because it only wakes up when a stubbed file is accessed. i included native scsi tape support using ioctl and mtio commands and there is an http gateway so you can stream media directly from a tape to vlc or plex without writing the full file back to your drive first. cloud tiering is handled via rclone so you can use s3 or backblaze as a backend too. every block is blake3 hashed and zstd compressed and the catalog is stored in sqlite but i made it so you can rebuild the whole thing by deep scanning the raw metadata headers on the media if you lose the database. i chose rust because i needed memory safety for the low level pointer stuff with o direct block alignment. i am looking for feedback on the fanotify implementation especially regarding inode locks during restores and i would love to hear from anyone else running physical tape or SMR drives. github.com/huskhoard/huskhoard there are a few more technical notes in the blog huskhoard.com/blog.html

I think there is a pretty sound case for a tanking economy before 26. There are a few high flyers that are propping up the market but scrape away the foam and it is not looking healthy. The weakness is already showing

we are writing blobs/ objects to zfs tape volumes. it gives us an extra layer of defense from ransom attacks and satisfies our 321 requirement. We make the blobs transparent with some metadata tags. The objects are recorded in the catalog and we can pull individual files out of the blob. deepspace storage manages the tape gateway and catalog for the objects. short answer yes storing send streams to tape is doable robust workflow.

nice. tell me that was dreamed and vibe coded over a cup of coffee and I will believe you

We are using deepspace storage for this. We can get/put objects into the cloud as one of our target volumes. It works as an auto archive writing anything in the fs that is over 90 to a compressed object and leaving behind a stub. You can point Harbor (or other tools like Mimir/Loki ) to a DeepSpace endpoint which looks like a standard S3 target. Then add policies and the files get moved, replicated, versioned to tape, cloud, disk array in the background. The users just interact with the file system as usual and admins have a UI with a catalog which shows where everything actually is.