HN user

icelancer

13,014 karma

Former miserable Data Scientist. I run a small business now.

Posts10
Comments4,161
View on HN

No doubt. Especially for GPU inference at scale. We overclock/overvolt for training and tune way down for inference.

Self-racking lets you rack a bunch of gear you'd never find in VM/dedicated rentals, like consumer parts or older, still very good parts. Overclocking options are available as well if you DIY.

If you need single-threaded performance, colo is really the only way to go anyway.

We have two full racks and we're super happy with them.

What am I missing?

VRAM capacity given the Cerebras/Groq architecture compared to Nvidia.

In parallel, RAM contracts that Nvidia has negotiated well into the future that other manufacturers have been unable to secure.

I've had a lot of problems with even 10GbE via Thunderbolt 3/4. Bandwidth-wise it works fine, but latency and jitter are issues. This means that stuff like high-speed cameras that need to be synchronized over Ethernet using Precision Time Protocol (PTP) tend to simply fail with these devices.

sub 900 not possible though.

I performed a similar analysis to you and found it very difficult to imagine sub-1000. Your comment I think convinced me that it may be possible, though. Interesting.

I'm below the threshold for recruiting but not below Claude at the moment. Not sure where I am going wrong.

Yeah this is the main reason to use a USB DAC. I guess you get marginally better sound quality (more noticeable on expensive studio headphones that need more power to drive them) but better isolation/removal from the noise source is the main reason I use them. Especially relevant because in my travel I'm often in countries that don't have ground plugs in their power sources.

I've found this mostly to be the case when using lightweight open source models or mini models.

Rarely is this an issue with SOTA models like Sonnet-4.5, Opus-4.1, GPT-5-Thinking or better, etc. But that's expensive, so all the companies use cut-rate models or non-existent TTC to save on cost and to go faster.

I just run the agent directly on separate testing/dev servers via remote-ssh in VS Code to have an IDE to sanity check stuff. Just far simpler than local dev and other nonsense.

Claude Sonnet 4.5 10 months ago

When pressed:

---

My execution environment is a browser-based JavaScript sandbox. I don't have Python, Node.js shell access, pip, git, or pytest capabilities.

Simon's post may have been referring to a different deployment or feature set that I don't currently have access to. My only code execution is the JavaScript REPL I just demonstrated.

What specifically are you trying to accomplish with the llm library tests? I can help with that in other ways.

EDIT: Ah, in Settings you have to disable the Analysis tool and enable Code Interpreter. You cannot have both enabled.

Claude Sonnet 4.5 10 months ago

Huh. Doesn't work over here.

---

SYSTEM:

Got it. Those are the setup commands for Simon Willison's LLM CLI tool, which lets you interact with language models from the command line.

The -e flag installs it in editable mode, and [test] pulls in the test dependencies. Are you running into any issues with the installation or tests? What are you working on with the LLM library?

USER: Correct. You run them using your code tooling.

SYSTEM:

Realized inability to execute requested Python commands.The user is telling me to run these commands using my code tooling. They want me to actually execute this. Let me think about what tools I have available:

I have the repl tool which runs JavaScript code in a browser environment I have artifacts for creating code/documents I have web_search and web_fetch

The user wants me to:

Checkout https://github.com/simonw/llm Run pip install -e '.[test]' Run pytest

But I don't have a tool that can:

Clone git repositories Run shell commands like pip or pytest Execute Python code in a command-line environment

The repl tool is JavaScript-only and runs in a browser environment, not a server environment where I could run shell commands. I should clarify that I don't have the capability to run shell commands, install Python packages, or execute pytest. I can only run JavaScript code in a browser environment via the analysis tool. I need to be direct and clear about this limitation, following the user's preferences to eliminate filler and be straightforward.I don't have tooling that can run shell commands, clone git repos, install Python packages, or execute pytest. My only code execution environment is a JavaScript browser REPL. I can't run pip, pytest, or any command-line tools. I can only execute JavaScript code in a sandboxed browser environment. What were you hoping to accomplish? Do you need help understanding the test setup, or were you expecting different capabilities?R