HN user
You can configure it with just config files.
Database is optional, if you add one, you get advanced caching.
It's just a single executable.
wget https://downloads.getmaxim.ai/bifrost/v1.6.5/linux/amd64/bifrost-http
chmod +x bifrost-http
./bifrost-httpUnder $20 it's a flat $0.8 fee. Over $20 it's +5.5%
+ credit card charges (+1.5%)
It's more just the value for money. It doesn't really offer me anything valuable other than billing consolidation - and the markup is excessive for that use case.
If I was using the routing features it would make more sense - but they charge significantly more for that so the markup is really just for billing consolidation.
I do understand the reasoning for it, it still sucks as a user who is doing the right thing.
Yeah I looked at it, LiteLLM is functionally more mature.
The only reason I didn't go for it is I'm not a fan of Python dependency management, Bifrost is just a single executable that uses nearly no memory and is lightning fast.
For me, the only utility OpenRouter gives me is billing consolidation - I don't really need the routing capabilities because I use Bifrost for that.
As a router, it's not very feature rich. For example I restricted the available models to the ones I want to use however the `/models` endpoint still lists all the models, making my LLM client list the 200+ models available on the service (even though they will throw an error if I try to use them).
With Bifrost, I can also create model aliases with custom configuration - for example I can create a model alias `deepseek-v4-flash-nothink` which disables thinking. I can create `deepseek-v4-flash-caveman` which injects the caveman skill (to save tokens) etc.
Plus I can contribute to Bifrost, which I can't do with OpenRouter.
I love the Chinese models.
I use DeepSeek exclusively and now Kimi K3 offers a great planning assistant for more advanced coding tasks.
DeepSeek v4 Flash is extremely fast and is able to handle pretty much anything I've thrown at it (I use mostly Rust, PSQL, Angular and Terraform).
I self host Bifrost as my LLM gateway, though I wish LLM vendors would do monthly/daily automatic billing (like VPS providers do) rather than prepaid + auto-top up.
It's annoying maintaining a non-refundable minimum balance across vendors, I would rather be billed for my exact usage.
OpenRouter helps, but I don't really like it as a service and not a fan of the mark up.
Thanks :)
Being unemployed and wanting to make something, I started studying quantitative trading concepts and got into algorithmic trading.
I decided to build out an algorithmic trading platform using the tools I developed for myself.
It's written entirely in Rust but user algorithms are written in TypeScript. It uses a Cloudflare-workers inspired approach to run the user functions.
The server uses under a megabyte of ram to run and user functions also use a negligible amount of memory per invocation.
It's also super fast, with round trip latency of 3ms - well, at least it does when I use the proper server. I'm running it on my low cost server right now so latency is around 50ms.
I know no one will use it, but it's been very fun to make
Any chance you could share how you achieved that?
I've been using LLMs a lot and they have been a force multiplier, but I am not sure I could knock out a year's worth of work in 2 months. I must be doing something wrong
Despite acknowledging that LLMs improve my productivity and am happy with how they have positively affected my life - I'm a bit cynical about the trajectory of this "AI bubble" and am looking forward to it popping sooner rather than later.
I just want to get it over with so we can abandon the hype, recover and move onto a less erratic/volatile investment/business landscape.
Sick of the "AI coding interviews" and having to virtue signal that I think AI will usher in economic nirvana.
Was a little bummed that Micron's earnings didn't send the Q's into a spiral after the insanity of the KOSPI the other day.
Sorry that you're being blamed. People have a pretty negative sentiment towards Americans in general right now because we have been watching helplessly in horror as our best friend tears things down.
I have friends in blue states who didn't vote because it wouldn't have changed the outcome given the electoral college - it's unfair to blame them for not participating as they were never able to in the first place.
Not that democrats are angels. You guys are damned if you're blue / (extra) damned if you're red - given the state of both options.
I appreciate that you're doing what you can
Something I wonder about in this domain is if this is the fault of the language.
Game development is usually in C++ or C#. With C++, bugs are a nightmare to find. C# less so, but there are still memory leaks possible.
I've been writing Rust for a few years now and it has been absolutely phenomenal. I get the performance of C++ and the only thing I think about when developing is the logic itself.
There aren't any mature game engines that use Rust (Bevy is rapidly improving, but it's no Unreal/Unity) - though you can compile Rust to a dll so it can be consumed by Unreal/Unity.
Due to the language constraints of Rust, I have found that LLMs need to work _way_ less to figure out bugs and render out code. The compiler gives very specific error messages and if it compiles, it works.
This is a great point. In the past this is how I learned to architect software in general as you are never the only contributor and often must accommodate the contributions of contributors who are early in the careers. Good software architecture reduces the damage possible by low quality contributions.
I have had great success with contract driven development workflows in this regard.
LLMs being effectively very knowledgeable jr engineers, slot right into those patterns.
Another interesting thing is that Rust _really_ shines with LLMs. The compiler gives very helpful messages which the LLM bounces off very effectively. If it compiles, it works, you just need to double check the logic is correct.
By contrast, with something like Go, you'd need to be on the lookout for nil pointers, race conditions and so on. Node.js / TypeScript is less dangerous, but you still have memory leaks that aren't possible in Rust and lack pattern matching so must be vigilant about type discohesion.
I'd be fine if it cut both ways. It shouldn't be legal for works distributed by people under licenses that would prohibit derivative work whilst if I reverse engineered a Nintendo Switch 2 with an LLM I'd be sent to the underworld.
To be fair, most of their votes don't actually count.
And even with our westminster system, One nation has swept the right wing. Considering how many seats they will likely get in the next election, my high horse has shrunk a fair bit.
That's why experience + cost effective model is IMO the best combination.
Experience allows you to design the skeleton where the implementation details are often inconsequential. There are relatively few scenarios where an LLM would need more guidance to render an outcome, but even dumb local models can do that.
Building a simple UI component vs an efficient multi-threaded bidirectional socket implementation (both examples of things I did recently with DeepSeek flash).
Angular can only be written in one way so the UI was trivial. I know the architecture for the socket implementation and the trade offs for various approaches, so I sketch out the implementation and get DeepSeek to complete it (error handling, keepalive messages, timeouts, etc).
I don't think I would have saved much time if I just asked Fable to "make the socket implementation" but even if it got it right the first time, it would have only saved me a few minutes given that's how long it took to write any way.
Even in more conventional applications, Node.js / React CRUD applications - "write a graphql query for blah" "add an endpoint to run query" "add validation to endpoint" are all trivial for DeepSeek flash. In most of these cases, I have found you're constrained by context window size because these are rarely well architected applications.
If I spoke freely about how I feel about the US right now and the direct personal impact of policies implemented by the recent administration, I'd be denied entry into the country to visit friends later this year.
It is inconsequential if the US or China have my data, both will misuse it and I am powerless to protect myself from that fact.
Not using LLMs presents a bigger threat to my career than protecting my data.
I am using Flash and accessing the API directly via vscode insiders and occasionally Zed (it's buggy but I keep coming back to it because I want it to succeed).
Unless you need enterprise multi-model management, I don't see the point in OpenRouter as it just adds cost overhead and you can just self-host an open router alternative (LiteLLM, Bifrost, etc). Running an LLM gateway locally is kind of nice as it allows you to normalize your configurations against your internal gateway - but I haven't really needed to.
I wish it was as simple as `git clone <url> --lazy`
I had previously worked on a big tech monorepo that has gigabytes of history. It would take forever to clone or do operations on. I had a cheat sheet of git commands that would do things lazily but I forget them (which is the issue).
Seeing that R&D costs are the lion's share, I wonder if we are at a point where the focus can shift to improving the cost of inference.
Unless we are genuinely pushing to find AGI, at which point nothing matters, LLMs in their current form don't replace knowledge workers but are an effective force multiplier. How good is enough?
For instance, I pay about $1-2 a month for DeepSeek. It's not as sophisticated as Claude, but it still doubles my productivity as a SWE.
If Fable comes out and demands 50x the price of DeepSeek in order for Anthropic to make a profit on it, how much more productive would I be compared to my personal experience + DeepSeek? 3x? 50x?
Is it cost effective for a business to hire someone without SWE experience + Fable verses hiring someone with SWE experience and DeepSeek? When does R&D hit diminishing returns?
I use DeepSeek every day (via VSCode Insiders and Zed Editor). It's very affordable and, while it's slightly behind Claude (not sure how far behind Fable), it suits my working style well. I'm not using unsupervised multi-agent workflows and don't need a library of skills files - I'm writing most of the code and leaning on AI to help with mundane tasks - like;
- generating types for APIs
- generating boilerplate based on existing code
- improving existing code (adding error handling, timeouts, things like that)
- Writing SQL repository boilerplate / queries
- Creating implementations against hand written tests
- Helping me understand and implement APIs from third party libraries
- Writing documentation
I've spent like $2 in the last month and have used over 100 million tokens.
It's doubled my productivity and unlocked work that I could not have done before.
As an Australian, I'm not sure that I care about the safety of my data when it comes to LLMs. US companies already stole scores of data to train their models on and it's hard to imagine they suddenly grew some integrity. I'll care when regulators step in, until then it's out of my control so I'll just use the best price-to-productivity product available.
Another thing git isn't good at is massive codebases with long histories. IMO git should have a configuration option to pull commits lazily without the need for `--depth` and the `git fetch` dance that goes along with it.
Makes sense. I honestly haven't thought about it too much. I liked that I could move to Turso if I needed cloud storage given the free tier and that the API works with tokio nicely
For me, I have a use case that needs to support a few thousand users, probably a few hundred concurrently.
The combination of SQLite (libsql, a concurrent implementation of sqlite) and Rust means I can do so from a $2/m VPS and a single server instance.
Backups are done via a cron job that uploads to S3.
Does it pass the "Netflix scale" test? No
But it doesn't need to. I'm not profiting from the service and SQLite offers a path to scale if/when ready because... well it's just SQL and I can literally just swap `libsql::Connection` with `psql::Connection` in my repositories.
Plus upgrading from a $2/m VPS to a $10/month VPS quadripples the number of concurrent users I can support.
IMO, you can vertically scale extraordinary far with SQlite and an efficient server implementation.
I'd wager that 90% of forum websites, wordpress sites and online shops would be fine with SQLite.
I wouldn't underestimate people in the React ecosystem not to adopt insanity en mass.
I wrote Go professionally for years. I don't know how many hours I've spent debugging and reviewing nil pointer bugs and race conditions. Generics were sorely needed but the initial implementation was lacking.
I moved to Rust professionally 4 years ago and haven't looked back. Mutex<T> Option<T> Result<T, Err> are all phenomenal.
I've written everything from web backends, frontends (hurry up wasm, seriously), to Node.js and Python extensions.
Web backends use under 1mb of memory and can support hundreds of thousands of concurrent users on a $2/m VPS. Frontends can be beautifully multithreaded. Native extensions can dance between OS threads and multi-threaded runtimes.
When I review code I focus only on the logic, not sidetracked by reasoning about race conditions or anything. Great when you review the work of less experienced contributors.
The ultra strict compiler is extremely helpful with LLMs. You bounce back and forth until it compiles and, if it compiles, it's usually correct.
It's at the point where I can't really see a use case for another language - and yet, no one uses it! It's madness!
Shameless plug. I've been developing a web server library for Rust based on the ergonomics of the Golang standard library.
It has a router, middleware, and uses AsyncRead, AsyncWrite for the request/responses.
I use this for my production applications and have found it much easier to work with than Hyper or Axium.
https://github.com/alshdavid-public/uhttp/blob/main/examples...
The API is largely complete but under the hood I have a few things that need doing. Open to contributions so please feel free to help out
I wrote Go professionally for years. Moved to Rust and couldn't be happier. There are some annoying syntax quirks but they are minor.
After writing web services, GUI apps and terminal apps professionally in Rust, I honestly struggle to see a use case for other languages.
You gotta know how to write Rust (and general software arch) first. LLMs + Rust have been great for me.
"Write an SQL Repository with this interface"
Sweet - no need for SQLc or an ORM