HN user

valzam

575 karma
Posts3
Comments268
View on HN

I mean the easiest explanation would be that the model harness doesn't always take the most likely token but does top-k sampling or similar. temperatur just means that probabilities get more and more equalized, boosting the chance that an unlikely token gets picked. but even with temp 0 you could have 0.8 T1, 0.19 T2, ... and sometimes sample T2

One of the great drawbacks of "Cloud Native Computing" as it now exists is that it's really, really complicated. It is often more complicated than the old, non-cloud, sort of computing. In order to deploy your app outside of Minerva you now need to know something about k8s, or Cloud Formation, or Terraform.

Highly agree with this. I think it's very underappreciated in startups that if you want people to deploy a lot of small services you have to make that really super easy. I always thought that the value of things like Spark is that you can run "things" without having to worry about how they run. K8s is similar but much more complex. AWS Lambda is nice but also comes with a lot of baggage at scale. I always wanted to try something like Dapr, which seems to provide a very opinionated happy path for application development.

If this works `git clone me@github.com:me/mine.git release_01 && ln -s release_01 /var/www/me/mine/current` then your Docker builds should also be extremely quick. Where I have seen extremely slow docker builds is with Python services using ML libraries. But those I reallly don't want to be building on the production servers.

"ECS would have worked for 99% of these apps, if they even needed that."

I used to agree with that but is EKS really that much more complicated? Yes you pay for the k8s control plane but you gain tooling that is imho much easier to work with than IaC.

I always feel like I am taking crazy pills when I read these threads. The k8s API and manifests config feels like a create standardardized way to deploy containers. I wouldn't want to run a k8s cluster from scratch but EKS has been pretty straightforward to work with. Being able to use kind locally for testing is amazing and k9s is my new favourite infra monitoring tool.

Even if you just run on 2 nodes with k3s it seems worth it to me for the standardized tooling. Yes, it is not a $5 a month setup but frankly if what you host can be served by a single $5 a month VM I don't particularly care about your insights, they are irrelevant in a work context.

I don't disagree that very numerical tasks like revenue forecasting are not a good fit for LLMs. But neither did a lot of data scientist concerns themselves with such things (compared to business analysts and the like). Software to achieve this has been commoditized.

Do those use cases need LLMs? Probably not. but if good results can be had with a day of prompting (in addition to the stuff mentioned in the article, which you have to do anyway) and a smaller model like Haiku gives good results why would you build a classifer before you have literally millions of customers?

The LLM solution will be much more flexible because prompts can change more easily than training data and input tokens are cheap.

As others have mentioned, one big issue is that every company does these things differently and just because someone texts you a link doesn't mean it's phishing, even though it feels shady. In Australia I have had calls by immigration officers on supressed numbers that wanted PII over the phone without being able to tell me what the purpose of the call is.

The average person self hosts literally nothing, why would it be different for inference? Which benefits severely from economies of scale and efficient 24/7 utlization

AirPods Max 2 4 months ago

I think people are mainly confused because the AirPod Pros are quite competitively priced compared to other higher end offerings. The Max are so far off the market that it doesn't seem to make any sense and it seems unlikely that apple couldn't make up for lost margins with higher volume. Maybe they just literally can't/don't want to produce many of the Max and price them accordingly.

An Update on Heroku 6 months ago

I mean I know what it means but how can you corpo speak so hard that it's pretty much unclear what the change even is

Unifi Travel Router 7 months ago

Do you have a pixel? On Samsung you cannot share WiFi, Hotspot only works with mobile connections. I learners above that this is possible with pixel phones, makes me want to get one...

Go subtleties 9 months ago

I guess as a corollary, Go really rewards writing the dumbest code possible. No advanced type shenanigans, no overuse of interfaces, no complex composition of types. Then you will end up with a very fast, resource light system that just runs forever.

Go subtleties 9 months ago

Great list of why one can love and hate Go. I really did enjoy writing it but you never get the sense that you can be truly certain your code is robust because of subtle behaviour around nil.

And during downtime you could sell space on your train car. Maybe even have an app for it, like uber for trains. Or as commonly know, regular trains.