HN user

stocknoob

1,083 karma
Posts0
Comments318
View on HN
No posts found.

My standard advice:

For any young programmers: live within your means, invest the difference, become independent, and work on what you enjoy. It’s the best (work related) gift you can give yourself. Skip the self promotion politics unless you enjoy it.

At this point, anyone calling AI a trend sounds like someone calling electricity a trend.

Believe it or not, there are paradigm shifts in technology every now and then. Not everything is 3D television.

Thank you for sharing. For any young programmers: live within your means, invest the difference, become independent, and work on what you enjoy. It’s the best (work related) gift you can give yourself.

How much VRAM and inference compute is required to run 3.1-70B vs 2-70B?

We aren’t trying to mindlessly consume the same VRAM as last year and hope costs magically drop. We are noticing that we can get last year’s mid-level performance on this year’s low-end model, leading to cost savings at that perf level. The same thing happens next year, leading to a drop in cost at any given perf level over time.

For training. Not for inference. GPU prices remained about the same, give or take.

See:

https://epoch.ai/blog/trends-in-gpu-price-performance

We don’t care about the absolute price, is the cost per flop or cost per GB decreasing over time with each new GPU?

—-

If it isn’t clear why inference costs at any given performance level will drop given the points above, unfortunately I can’t help you further.

Take a look at the latest Llama and Phi models. They get comparable MMLU performance for ~10% of the parameters. Not to mention the cost/flop and cost/gb for GPUs has dropped.

You aren’t trying to run an old 2023 model as is, you’re trying to match its capabilities. The old models just show what capabilities are possible.

You’ve subconsciously altered your search behavior to avoid categories that Google is horrible at.

Any product reviews will be SEO garbage (blogspam too 10 lists). Anything travel will be a page full of ads before organic results, if any. You just know to not even bother so you’re left with the queries that still work.

Imagine you’re running a physics simulation. With conservation of energy: if energy shows up somewhere, it went down somewhere else. We shouldn’t assume a balance changed on its own without any other effects in the system. If you start “losing net energy” you can see what’s going on.

I think Your Money or Your Life (Vicki Robbins, 1992) predates it and helped inspire Mr Money Moustache, but Collins’ book is great too.

For everyone reading this, pursuing FIRE will impact your life more than any other book. Set the ball in motion and wait. Soon you’ll have time to enjoy all these other great books.

My 2c if helpful: the solution was to question deeper assumptions.

Why are you at a desk? Because you have to work 8 hours.

Why? Because your job requires it.

Why do you have that job? Because you need income to live.

Is a job the only way to have income to live? (See: FI).

Ultimately the root cause is not being in control of your time. It takes time, but once solved the downstream problems solve themselves. You can work (exert effort on goals you care about) in a way that prioritizes your health.

Land is a shared, zero-sum resource: a parking lot is not a park.

Your software development methodology is your own. Why does someone else’s use of a tool deprive you of doing things the way you want?

Does a player piano rob you of playing music yourself? A car from walking? A wheelbarrow from working out? It’s up to you if you want to stop practicing!

Chess has become even more popular despite computers that can “rob us” of the joy. They’re even better practice partners.

The naysayers used to bother me, but then I realized it’s no skin off my back if they don’t want to become familiar with a transformative technology. Stay with the old tools, people are getting excited for no reason at all, everyone is just pretending to be more productive!

It reminds me of how blackberry users insisted physical keyboards were necessary and smartphone touchscreen users were deluded.