HN user

mustyoshi

159 karma
Posts0
Comments86
View on HN
No posts found.

I don't see how we'll ever get to widespread local LLM.

The power efficiency alone is a strong enough pressure to use centralized model providers.

My 3090 running 24b or 32b models is fun, but I know I'm paying way more per token in electricity, on top of lower quality tokens.

It's fun to run them locally, but for anything actually useful it's cheaper to just pay API prices currently.

I don't buy that it's because of the monopoly. TSMC has been starting a fab for close to 5 years.

It doesn't matter how many companies are in this market, it takes a real amount of time to add capacity.

Instead, we should make it illegal to discriminate based on criminal conviction history

Absolutely not. I'm not saying every crime should disqualify you from every job but convictions are really a government officialized account of your behavior. Knowing a person has trouble controlling their impulses leading to aggrevated assault or something very much tells you they won't be good for certain roles. As a business you are liable for what your employees do it's in both your interests and your customers interests not to create dangerous situations.

But realistically, I just had 2 flights last month, checking what model of aircraft I was on didn't even cross my mind. I survived both flights btw.

The framing of 100% of people getting a prescription as being bad seems weird to me. We live in the age of AI we don't need a doctor to tell us the side effects of medication anymore, people should be free to buy for themselves whatever they want to take.

Obviously involving insurance and (other people's money) makes a difference, but for direct to consumer, I don't think doctors should be gatekeepers any more.

Hyundai is partnering with Boston Dynamics to deploy 30k robots a year.

Amazon is looking to replace 600k employees over the next decade.

Why do you believe demand for humanoids isn't high?

Robotaxi has to be cheaper than a normal taxi to kill taxis. The margin of that company can't be that much more than a company like uber.

This just isn't true. If you're a woman, choosing a slightly more expensive robotaxi over a ride share where you might meet your end is a valid choice.

I politely disagree. I spent maybe 8 hours over a week rightsizing a handful of heavy deployments from a previous team and reduced their peak resource usage by implementing better scaling policies. Before the new scaling policy the service would scale out and new pods would remain idle and ultimately get terminated without ever responding to a request quite frequently.

The service dashboards already existed, all I had to do was a bit of load testing and read the graphs.

It's not too much extra work to make sure you're scaling efficiently.

Yeah this is the thing people miss a lot. 7,32b models work perfectly fine for a lot of things, and run on previously high end consumer hardware.

But we're still in the hype phase, people will come to their senses once the large model performance starts to plateau

Our $100M Series B 12 months ago

I'm not part of Oxide, but I think you're assuming everyone is okay with not controlling the hardware they run on.

There is plenty of addressable market where a public cloud or even a colocation of hardware doesn't make sense for whatever bespoke reason.

I don't think their target customer is startups, and that's okay. You likely aren't their target customer. But they've identified a customer profile, and want to provide a hardware + software experience that hasn't been available to that customer before.