HN user

jakeasmith

21 karma
Posts7
Comments14
View on HN

I've been running Ollama in my home lab for awhile now but I wanted to experiment with running something a little more "low level". I saw a ton of posts about Llama.cpp, which looked interesting, but there wasn't a lot specifically about running vLLM on Proxmox, so I thought I'd give it a try. Setting up vLLM in an LXC wasn’t necessarily difficult but, even after doing it several times, it was still tedious. These are the notes I’ve taken along the way, if only for my own reference next weekend.

Feel free to tell me what I got wrong :)

Ya know, that’s what I would’ve thought as well, but a year or two ago Chase Bank refused to take my loose change. They sent me away with the paper to roll the coins myself to bring back to them

I'm very disappointed to see that Slack has decided to go the way of every other messaging service and move away from decentralized and standardized protocols

If every major messaging service eventually does this maybe those decentralized and standardized protocols are the problem.