Gentle Guide to Self-Hosting

https://knhash.in/gentle-guide-to-self-hosting/

by kn81198 • 2 years ago

325 136 2 years ago

asar3• 2 years ago

Love self-hosting and really got into it over the last couple of months. I run a bunch of services for my company now and also in my home lab. I use a Hetzner VPS and provision things either via ansible + docker compose files or via https://github.com/coollabsio/coolify/.

The awesome-selfhosted repository is also a great place to find projects to self-host but lacks some features for ease-of-use, which is why I've created a directory with some UX improvements on https://selfhostedworld.com. It has search, filters projects by stars, trending, date and also has a dark-mode.

b_shulha3• 2 years ago

Any chance to get my SaaS into "Heroku alternatives" section as well?

https://ptah.sh

asar3• 2 years ago

Just added it!

user_7832• 2 years ago

Since you seem knowledgeable on this topic I'd like to ask - how risky is it to expose a computer on your network to the internet, if you're somewhat tech-savvy but not very familiar with networking? Is it relatively "safe" with modern tools and VMs or do you need to stay on top and (for eg) always ensure you're updating software weekly?

I've thought of setting up and running a server for a long time and finally have a spare laptop so I'm thinking of actually running a NAS at least.

voidUpdate• 2 years ago

You'll want to make sure everything stays up to date in case someone finds a vulnerability in whatever software you're currently using. If you have to expose stuff to the outside world, only open the ports you need to. Only allow access to a specific user with a non-default username (or at the very least disable root ssh access), and use long passwords or ssh keys. I think that's generally the bare minimum, but there are online guides to harden your stuff further like using wireguard and fail2ban and stuff

asar3• 2 years ago

The approach most people use is to tunnel into the server. You install a daemon on your computer which establishes a tunnel to log-into from outside your network. Cloudflare and Tailscale have solutions for this that are very popular among the self-hosted crowd.

https://developers.cloudflare.com/cloudflare-one/application...

https://tailscale.com/kb/1151/what-is-tailscale

conradklnspl• 2 years ago

A god option is to setup a wireguard connection between workstation and servers. All traffic has to go through wireguard.

Because wireguard is UDP and only responds to valid requests, there isn't any open port from the outside. Not even ssh.

packetlost5• 2 years ago

I've been doing it for about 13 years now with HTTP/s (80, 443), SSH (22), MOSH (lol idk), and IRC (6697) exposed to the internet. You don't need it, but something like fail2ban or crowdsec is a good idea. You will get spammed with attempts to break in using default passwords for commodity routers (Ubiquiti's `ubnt` is rather popular), but if you're up to date and take a few minor precautions it's not all that hard and/or dangerous. That being said, there are alternatives such as Tailscale that are strictly more secure but far less flexible. I've heard of people using Cloudflare tunnels as well, but I'd rather not rely on big players for stuff like that if I'm going through the effort to self host (and don't have any real risk of DDoS).

I would try to set up automatic updates for critical security patches or update about weekly. I know people that self host and do it monthly and they seem fine too. Most anything super scary vulnerability wise is on the front page here for awhile, so if you read regularly you'll probably see when a quick update is prudent. I personally use NixOS for all of my servers and have auto-updates configured to run daily.

An old laptop is exactly how I got started 13 years ago, they're great because they tend to be pretty power efficient and quiet too.

cutler3• 2 years ago

Just add `sudo apt update && apt upgrade` to your crontab.

the_gastropod• 2 years ago

I sat on the fence for a long time wanting to do this, and finally pulled the trigger and picked up a Synology NAS last year. I've had a blast setting up a handful of handy little self-hosted services on the thing. Highly recommend giving it a go!

I haven't had any security issues yet (knock on wood). But it seems pretty low-risk if you follow basic best practices. The only thing I have exposed to the internet is a reverse proxy that proxies to a handful of docker containers.

diggan2• 2 years ago

Is it relatively "safe" with modern tools and VMs or do you need to stay on top and (for eg) always ensure you're updating software weekly?

First step to figure out if you actually need to be able to access it from the outside at all. If you just want a NAS, chances are you can put it on a separate VLAN/network that is only accessible within your LAN, so it wouldn't even be accessible from the outside.

If you really need it to be accessible from the outside, make sure you start with everything locked down/not accessible at all from the beginning, then step-by-step open up exactly what you want, and nothing else. Make sure the endpoints accessible is run by software you keep up to date, at least weekly if not daily.

unethical_ban• 2 years ago

Keep things up to date and ideally, having your public facing servers in a DMZ/their own VLAN (separate network from your private stuff).

Administrative things like SSH and RDP are best accessed with a VPN but you can configure SSH in particular to be key-based authentication only, which is very secure.

0xfeba• 2 years ago

I've been at it for over a decade. Home router has firewall exceptions for SSH (not port 22 though), TLS IRC, and 80/443, which are forwarded to my home server with fail2ban.

I run SSH (requires PKI outside local network), IRC, nextcloud, and ampache (though don't really use ampache anymore :( ).

Home server is encrypted RAID6 Arch Linux. If I had to do it again I'd forego rolling releases and use something more stable, like Debian.

Encrypted backups are done to backblaze once a month. I also have a backup drive that I plug in on occasion, encrypted of course.

Which reminds me my RAID6 drives are getting old now... I'm tempted to move to a VPS.

ang_cire5• 2 years ago

It is very service-dependent. If you are wanting to run a NAS for e.g. a media server, you may want to look into Cloudflare Tunnels or Tailscale.

I set up Jellyfin and Kavita, and those are internet-exposed, but also Nextcloud, and Portainer, and Calibre, and those are behind github SSO auth, via Cloudflare. Basically, before you can hit the nextcloud login page, you have to auth to github (as me) with 2FA first, so no one can sit there and try to brute-force my nextcloud login.

bongobingo1• 2 years ago

Hm, is there a name for the type of software that Coolify is, where it presents a management plane for other servers, vs Dokku where it runs on the server?

b_shulha3• 2 years ago

Coolify and others mentioned on that website can run on the server itself as well.

It happened that Coolify provides the paid option to sponsor the development, but it is not mandatory.

apitman8• 2 years ago

This is pretty nice. I see sish and inlets. I have a lot more similar tools on my list here: https://github.com/anderspitman/awesome-tunneling

For auth, I also made a comparison of OIDC servers here: https://github.com/lastlogin-net/obligator#comparison-is-the...

leosanchez2• 2 years ago

Miniflux is very good. It even has a telegram integration which will send you notification whenever a new article is published

loremm• 2 years ago

in general, it's worth noting telegram bots are easy (free) to make and messages can be sent with one cURL command. Very useful, you can even set it up to send after long terminal commands so you know to check back

leosanchez2• 2 years ago

I am aware. I use it with my backup scripts. But I felt it was cool integration

Havoc• 2 years ago

But then someone whispers K8S into your ear…

jasonjayr• 2 years ago

That reminds me ... I have to go feed my homelab K3S cluster some updated CRDs to plan for the next upgrade window ...

infogulch• 2 years ago

k8s was born from devs unconscious desire for another tamagotchi pet

arrty88• 2 years ago

I'm a big fan of self hosting. I have learned a lot on a small hobby project.

for those who are curious about my setup, I bought a used Dell R630 on ebay for cheap. 1tb raid 1 on ssds, 32gb ram, 32 cores, and i am enjoying running a few small hobby apps with docker, virsh, and minikube (yes i learned all 3). I have a 1gbps fiber connection. I use a 1m cronjob to detect if my IP changes, and i use the linode api to change my DNS A records.

chadsix2• 2 years ago

I am part of a company that promotes self hosting and provides external routing for self hosting [1]

We made Cloud Seeder [2] an open source application that makes deploying and managing your self-hosted server a 1-click issue!

Hope this comes in handy for someone! :-)

[1] https://ipv6.rs

[2] https://ipv6.rs/cloudseeder https://github.com/ipv6rslimited/cloudseeder

DanAtC• 2 years ago

I like the concept, but only 5 IPs? With IPv6 you should be offering at least a /64 per tunnel.

chadsix2• 2 years ago

Great point!

We offer 5 because we're geared toward helping people host appliances as opposed to raw network setup! We also offer automatic RDNS with this as well as the Cloud Seeder appliance!

Thanks again for your comments and as well thoughts!

xiconfjs• 2 years ago

From the FAQ: * Q: "What about IPv4?"

* A: "While IPv4 is still widely used, its necessity is diminishing as the world transitions to IPv6. (...)"

;)

cutler3• 2 years ago

1.5Gb RAM/10Gb disk? Hetzner's basic cloud VPS comes with 4Gb RAM and 40Gb disk for E4.51.

Ringz• 2 years ago

Do you have an eye on the (potential) price difference?

cutler3• 2 years ago

More the resource difference.

tarruda2• 2 years ago

AFAIK Nothing seems to beat Oracle cloud: https://www.oracle.com/cloud/costestimator.html

For compute:

"Each tenancy gets the first 3,000 OCPU hours and 18,000 GB hours per month for free to create Ampere A1 Compute instances. This free-tier usage is shared across Bare Metal, Virtual Machine, and Container Instances."

For block storage:

"Block Volume service Free tier allowance is for entire tenancy, 200GB total/tenancy. If there are multiple quotes for the same tenancy, only total 200GB can be applied"

In other words: you have a 4-core ARM CPU + 24GB RAM + 200GB space for free.

sandos• 2 years ago

Yep, I was running one of these for the longest time.. until they blocked idle instances! Hah.. thats the kind of usage free gets you, lot of people hoarding it for... nothing. I mean, I could easily have thought of stuff to load the instance up slightly but, eeh.

account42• 2 years ago

The cost in this is your soul. I prefer not to make deals with the devil.

massysett• 2 years ago

I’m an old-timer, I’m surprised that paying for shared hosting is now “self-hosting.” Nothing wrong with that, but that would never have been called self-hosting ten years ago.

I guess it’s like how “cooking from scratch” evolved. A cookbook from the nineteenth century might have said “1 hog” as an ingredient and instructed you to slaughter it. Now of course you buy hog pieces on foam trays.

diggan2• 2 years ago

Seems to me the term "self-hosting" tends to auto-adjust its position based on the other end. So if "not self-hosting" is hosting on a shared VPS, then self-hosting is hosting on a computer at home. But "not self-hosting" has now become "hosted in cloud" so self-hosting moved to "shared VPS" instead, as the other end moved.

Kind of makes sense, but kind of also makes historical texts more difficult to understand. In the year 2124, who know what "self-hosting" meant in 2054? I guess it's up to future software archeologists to figure out.

icedchai2• 2 years ago

Yes, the goalposts move. When I started w/the internet 30+ years ago, self hosting meant your own 56K leased line.

kayson2• 2 years ago

There are definitely plenty of people who would say that using a hosting provider doesn't count, even if you're deploying the software yourself.

The one generally accepted exception to this is network protection. You don't want to expose your home ip address to the outside world if you can help it, so a lot of people use tailscale, cloud flare tunnels, or a vps as a proxy.

Cyph0n2• 2 years ago

A VPS that proxies traffic over Tailscale is another neat option. I use this approach to serve self-hosted services that I want to be accessible over the internet.

stuckkeys• 2 years ago

Cloudflare?

hoosieree• 2 years ago

Yeah, my list of requirements for self-hosting starts with:

1. battery backup

That said, I'm not zealous about it. "Perfect is the enemy of good" and I like ecosystem diversity in general. Better to have a few dozen shared hosting providers than 2 or 3 monopolies.

eximius• 2 years ago

Every time this sentiment comes up I'm reminded that it's a spectrum.

m463• 2 years ago

I agree with you.

and so does the author, kind of...

"And so, here is a gentle introduction to self-hosting that is not "true self-hosting", but whatever. Sue me."

transpute7• 2 years ago

Sue me

Or read an HN thread on "true self-hosting", https://news.ycombinator.com/item?id=41440855#41460999

_heimdall• 2 years ago

I must be becoming an old timer too, I only really consider it self hosting if its on my own hardware.

In case that doesn't make me an old timer, I also actually have pork and home cured bacon in the freezer from hogs we raised and processed. "An old soul living in a new world" feels pretty fitting here.

apitman8• 2 years ago

There are other issues with the terminology as well. The self hosting community (centered at /r/selfhosted) has a very technical vibe. These people enjoy tinkering with computers. They're like kit car builders.

But there's a whole market of people who could benefit from self hosting, but shouldn't be required to understand all the details.

For example, you can get many of these benefits by using a managed service with your own domain. Things like data ownership, open source software, provider competition, etc.

I think we need a broader term. I've been using "indie hosting" lately.

simcop2387• 2 years ago

This is the kind of thing I've been watching unfold with some home "NAS" boxes over the past couple of years. It started much earlier but it's started to become more of a differentiating factor in some of the lines lately because the NAS side of things is basically entirely a solved problem for 99% of people, so the manufacturers (Synology, QNAP, Terramaster, U-Green, etc.) have been adding support for doing what looks a lot like turn-key installation of things like NextCloud, Plex, and a bunch of other services that the self-hosting community has been talking about for years.

I think one of the big drivers of it has been the serious increase in performance and capability of the low power embedded processors from Intel and AMD (and in the last year or so some ARM based ones), like supporting more than 2GB of ram and having multiple cores that can meaningfully do work even with a 15W TDP.

immibis2• 2 years ago

I call it self-hosting when it's on your server, and hosting at home, if we want to be specific that the server is at home.

nielsole• 2 years ago

"digital souvereignity" maybe?

fractalnetworks2• 2 years ago

We call it home-running. My company Fractal Networks is starting with "self-hosted" game servers (on Windows) to get the ball rolling. Check us out https://fractalhome.run.

dylan604• 2 years ago

Hosting from home was always subject to home ISP ToS limits on doing that very thing. When I self-hosted in the early days, it was still paying someone to mount my system in their rack and use their network. So whether that was hardware that I rented from them, built the box myself, or using a VM they provide, it's still the same amount of work to maintain it. That's still different from using Wix/Squarespace, geocities, or using a social media platform.

ang_cire5• 2 years ago

was always subject to home ISP ToS limits on doing that very thing

Every ISP prohibition on self-hosting that I have seen specifies commercial use, not just hosting services (since obviously that could technically prohibit tons of normal and authorized uses like co-op games).

bityard2• 2 years ago

I have always understood self-hosting to mean being in charge of your applications and data instead of delegating it to a company. An example might be, setting up Nextcloud instead of Dropbox. Or Taiga instead of Trello.

WHERE and HOW it is hosted, is less important to me. Because if you self-host your own tools, you can freely pick them up and move them to any hosting provider, a cloud provider, or a Raspberry Pi in your basement. Self-hosting FREES you from infra/vendor lock-in.

al_borland3• 2 years ago

But isn’t using a 3rd party web host giving up some of that control. Hopefully a reputable hosting company won’t shutdown at a moments notice, but could. Or if they go down, you’re stuck sitting there waiting for them to come back online with no access to your services.

Hosting from home has its own challenges, so I get why people would go to a hosting provider, but I do think some control is given up in the process.

hk13372• 2 years ago

I'm not sure what your classification of "old-timer" is with how it compares to me but I would think of myself as an old-timer as Gen X.

I feel like there's another term for what you're thinking of but I cannot come up with what it is.

Self-hosting definitely was locally hosted on your own hardware back when hosting providers like Linode, Digital Ocean, AWS, etc existed or were as customizable.

Even corporations "self-host" GitHub Enterprise or Gitlab when they set it up on AWS. Self-host just means you're not reliant on creator of the application to host it for you and manage the server.

There are certainly advantages and disadvantages to self-hosting on your own hardware, as there are to using a hosting provider.

user010102• 2 years ago

You reckon you could even use Github to archive some small things? If Github or GitLab suffers, then some parts of the internet will also have problems, correct? Legitimately asking, is there any way for Github to go around searching for "no-code" content through countless private repositories?

lelanthran• 2 years ago

I’m an old-timer, I’m surprised that paying for shared hosting is now “self-hosting.” Nothing wrong with that, but that would never have been called self-hosting ten years ago.

Depends, maybe? Was the speaker talking about hardware or software 10 years ago?

Because, when I was given the 'self-hosting' option by some SaaS vendor, it meant that I could host it on whatever I want to independent of the vendor, whether that is a rack in my bedroom or a DO droplet.

When I was given the 'self-hosting' option by some computer vendor (Dell, HP, Sun, etc), it meant that I can put the unit into a rack in my bedroom.

Context was always key; in my mind, nothing has changed.

ang_cire5• 2 years ago

If you are using a hosting provider, you are by definition not "self hosting", since you are in fact, not hosting (unless you happen to own the hosting provider company).

I actually self-host tools, and that involves having (in my case) a couple of rackmount servers in my spare bathroom, and an rPi5 with a 4x m.2 hat on my desk. Hell, even just running stuff on your own desktop/laptop is self-hosting.

But PaaS and SaaS are just as not-self-hosted as IaaS is. It's literally cloud hosting.

drpixie• 2 years ago

Yup - as they say, "the cloud" is just some else's computer.

It's not so hard to genuinely self-host. You just need a reasonable ISP who is willing to open your connection, and to be sensible about securing your systems.

mervz• 2 years ago

What if I rent physical server space at a colocation? Is that "self-hosted" by your definition? I'm curious where the imaginary line is drawn.

bankcust08385• 2 years ago

Renting space in a colo and running EBGP on leased dark fiber to HE is real self hosting. VPSes while more convenient are definitely nothing like running metal.

For a lot of stuff that doesn't need constant public network connectivity, I choose to run a home lab.

chris_armstrong• 2 years ago

It's different degrees of the same thing I would have thought.

If you're running your own box, you still depend on network infrastructure and uplink of a service provider, whereas a cloud infrastructure provider may go the other way and negotiate direct connections themselves.

Plenty of valuable lessons await for those who even just provision a virtual host inside AWS and configure the operating system and its software themselves. Other lessons for those who rack up their own servers and VNETs and install them in a data-centre provider instead of running them onsite.

There's only so much you can or should or want to do yourself, and its about finding the combination and degree that works for you and your goals.

j45• 2 years ago

Hosting software yourself is a step towards more.

nashashmi• 2 years ago

On-cloud self host is where you rent out a vm or full physical machine and then upload your stuff on there. Easy peasy lemon squeezy.

koziserek• 2 years ago

Indeed nowadays killing an animal is hidden behind industrial vail.

Unfortunately, it also normalized and further desensitivied us to the topic.

Though I'm quite happy to see that eating sentient beings gets out of fashion, at least in developed world.

from-nibly2• 2 years ago

Practically, it is foolishness, for what you save in money you lose in time and sanity.

Kubernetes gets a lot of side eyes in the self-hosted community. That's all of self hosting though. So why not go all in?

I've got 3 dell r720XDs running nixos with k3s in multi master mode. It runs rook/ceph for storage, and I've got like 12 hard drives in various sizes. My favorite party trick is yoinking a random hard drive out of the cluster while streaming videos. Does not care. Plug it back in and it's like nothing happened. I've still got tons of room and I keep finding useful things to host on it.

kayson2• 2 years ago

Plenty of people use k8s or k3s for self hosting. But for most, the added complexity doesn't buy enough for the trade-off to be worth it. Keep in mind most people have a single node, so docker does everything they need.

Personally, even with a 4 node setup (of tiny desktops; the hardware you have would easily cost me $200/mo in power bills), I use docker swarm. Old and unloved, but does everything I need for multi node deployment and orchestration with only a sliver more complexity than vanilla docker.

from-nibly2• 2 years ago

Yeah don't ask me about my power bill, It's definitely in the vanity realm. I have cheap power where I live so it's not anywhere near $200. Still too high though. One day I'll get solar to offset it.

Cyph0n2• 2 years ago

I just use NixOS as a VM and run services as containers directly. Self-plug: I wrote a tool that makes it easy to run Docker Compose projects on NixOS [1].

This way, I get the advantages of NixOS config, while also being able to run arbitrary applications that might not be available on nixpkgs.

As far as storage goes, I just use ZFS on the hypervisor (Proxmox) and expose that over NFS locally.

[1] https://github.com/aksiksi/compose2nix

jamwil2• 2 years ago

For a homelab k8s is way overkill. I do it because self-hosting is 1 part utility to 2 parts education.

pl4nty• 2 years ago

it's become much easier in the last few years. Talos + Flux + Renovate can be built in a day [0], and simplifies storage/backups/patching even for single-node clusters. There's also a great community, with services like kubesearch [1] providing templates for tons of apps

[0] https://github.com/onedr0p/cluster-template

[1] https://kubesearch.dev

ed_mercer• 2 years ago

Once you graduate from this guide, be sure to check /r/homelab and /r/homedatacenter ;)

meonkeys2• 2 years ago

And https://lemmy.world/c/selfhosted , and https://www.reddit.com/r/selfhosted/ . There are a few useful Matrix chat rooms related to self-hosting, too: #selfhosted:matrix.org , #self-hosted:jupiterbroadcasting.com , #steadfast:matrix.org

meonkeys2• 2 years ago

The author nails it here:

It is 2024, and I say it is time we revisited some of the fundamental joys of setting up our own systems.

Self-hosting really is joyful. It's that combination of learning, challenge, and utility.

+1 to Actual Budget

+1 to Changedetection.io

-1 for not mentioning threat modeling / security. The author uses HTTPS but leaves their websites open to the public internet. First-timers should host LAN-only or lock their stuff way down. I guess that's tricky with shared hosting without some kind of IP restriction or tunneling, though. No idea if uberspace offers something like that.

For folks getting past the initial stages of self-hosting, I'd really recommend something like Docker to run more and more different apps side by side. Bundled dependencies FTW. Shameless plug for my book, which covers the Docker method: https://selfhostbook.com

w10-1• 2 years ago

OK:

    This has not been a detailed step by step walkthrough 
    on how to do things, by design. You are meant to go and explore; 
    this is simply a way pointer to invigorate your curiosities

Sorry, but because I came looking for solutions, I found the invigoration aggravating, but then helpful in focusing my attention.

Scalable services and sites I can build, 10 different ways.

My enduring, blocking need is for dead-simple idiot-proof network management to safely poke a head out on public IP from home. And to make secure peer-to-peer connections. Somehow that process never converges on a solution in O(available) time.

</complaining>

transpute7• 2 years ago

dead-simple idiot-proof

Recent thread: https://news.ycombinator.com/item?id=41440855#41460999

network management to safely poke a head out on public IP from home

For remote access to private services, would Tailscale/Wireguard be an option? It can even use Apple TV as an exit node.

secure peer-to-peer connections

Which protocols would you consider secure for P2P use, e.g. which solutions have you tried previously which failed to converge?

jamwil2• 2 years ago

Tailscale is that.

transpute7• 2 years ago

"Why I self host my servers and what I've recently learned", 130+ comments, https://news.ycombinator.com/item?id=41440855

tonymet• 2 years ago

spoiler alert, the article isn't about self-hosting, it's about shared-hosting

crossroadsguy• 2 years ago

I loved the idea of PikaPods until I realised even if I use 10 small (no tiny) instances/services and that too just for me to be used really really rarely) I was getting into integral USD x 11 (or whatever the number is) cost. Can't blame them because it costs money to run things. But I would have rather preferred something that isn't that costly or doesn't go up in prices with number of services/apps used. I wish there was a cost effective solution for this self/web/app hosting.

johnklos2• 2 years ago

It's a good writeup, but I do take exception to this:

"Seriously, else-hosting is the practical option, let someone else worry about the reliability, concurrency, redundancy and availability of your systems."

Spend one time trying to get through a maze of automated phone answering systems, then try to ascertain whether the human, when you finally get them, even understands the issue, then wonder how much of what they're telling you is to just get you off the phone, all the time wondering if calling even really does anything, and you'll wonder whether it's better to blindly trust a company that likely doesn't have tech people we're allowed to talk to or to just do it ourselves.

At least when there's an issue with my things, I can address it. Although a bit of a tangent, I'd love to see a review of major hosting providers based on whether you can talk to a human, and whether said human knows anything at all about Internet stuff.

RajT88• 2 years ago

I'm with the other old timers.

If it's not your hardware running in a space you own or rent, you're not self-hosting.

Currently I have a little Micro-ITX box. But once upon a time I had a proper server rack with 6 U worth of servers, UPS, networking, etc. (Before I was married...)

Gentle Guide to Self-Hosting

Related Stories