golly_ned

I expect the inference market to grow much faster than training costs

This was my assumption as well. It's also generally true of 'traditional' deep learning models that inference cost is expensive compared to training.

But the cost per token for inference has been very quickly dropping. I don't recall where, but I recall about ~50x down from GPT3, even as model complexity has increased. Even with agentic systems, there are lots of optimization opportunities. I'm less assured about claims like this.

https://web.archive.org/web/20260721001337/https://techcrunc...

I agree, and the audience also matters too. A writer speaking to writers at a writing workshop? "I'm a writer." A writer at an independent business conference? "I'm a content creator", as a means to relate to people making videos, blog posts, photographs, tweets, etc.

Some 'content creators' might also do: videos, blog posts, photographs, tweets, and a lot of other activity that drives the same business: getting user 'engagement' for sponsorship or advertising revenue.

I see that abstraction as the purpose of the word when viewing it from a 'business' context: when assessing businesses of the same kind, the actual thing that's being produced matters a lot less.

I'd suppose that blog writers and video producers have a lot of the same kinds of business problems: how people can discover [the thing they create] and spread awareness of it or make money off it.

In order to be able to discuss this category of business, it's useful for writers and video producers to be able to relate to each other by abstracting out the thing that actually makes them distinct and particular.

I see it less as devaluing that special thing and more as a lens to put on or take off when certain audiences are interested in the 'non-special' aspects of running businesses based on that special thing.

I agree. It's a generic word since it has a generic meaning.

I know there's a big bias against 'business speak' among engineers. But a lot of the time, it's just a matter of abstracting in a domain engineers typically don't work in. It's useful to describe from the 'business' point-of-view since terms like this can be used to analyze the similarities across these similar 'content creation' type businesses.

I'm trying to think of a similar case where to an outsider, the 'tech speak' jargon a software engineer would use seems useless to an outsider. Maybe things like 'framework', or 'deployment', or 'library', or web application instead of web site.

Turing's cathedral, on the invention and development of the earliest digital computers just before, during, and after WWII. Lots of excellent detail, descriptions of the personalities and relationships of many interesting people, like von Neumann.

Everyone I’ve met personally who attended RC was insufferable. Condescending to the point they couldn’t disguise their contempt for people who they thought knew less. So extreme to the point they simply refused to learn from other people. One even literally bullied a teammate to tears.

Why would you think they are talking about an entirely different page and topic?

I'd love to see the 'AI as personal tutor' approach. Even incorporating things like spaced repetition or the testing effect, or evaluating free-written responses.

A lot of potential that's currently unrealized. It takes a student to swim upstream to get there. The convenience of cognitive offloading is difficult to say no to. For evidence, I see it everywhere at work, including (at least in some cases) in my own work, for matters I don't care to invest effort in learning because it's a one-off.

The rates of AI use show it far exceeds the rate of good old-fashioned cheating, and not an equivalency between them.

So I am convinced AI will make the ~80% dumber, at least until there are excellent teaching products and changes to teaching practices that end up making that 'AI as a personal tutor' the norm. In the absence of the actual right answer -- actual people as personal tutors with qualified, well-paid teachers and right-sized classrooms -- an AI as personal tutor is extremely scalable and would allow productive 'struggle' learning.

My favorite is ignoring or downplaying a problem. 95% of apparent 'problems' are best ignored in favor of solving the worthwhile 5%.

Sometimes this means 'creating' a new problem by pointing to something unconsidered that's 10x more important.

When I was at AWS, they famously required an extensive "CoE", correction of errors, or post-mortem, in an instance of over-charging a customer $0.26.

The idea is that if we can make small billing mistakes like that, we can make large billing mistakes, and need to invest in the correctness of the systems powering billing.

I have great respect for the engineering culture within AWS during those times. I am glad to have left before seeing it degrade and decline.

Even if something wasn't written by ChatGPT, I'm supposing ChatGPT's influence on writing has been so strong that (1) any typical reader of text on the internet will have ingested a lot of ChatGPT writing habits in their own writing, and (2) that any reader of ChatGPT text is so habituated to those writing habits that even non-ChatGPT generated text appears similar to ChatGPT generated text.

Has anyone figured out why anyone would bother adopting the google 'open knowledge format'?

Normally I expect a set of tooling to be build on top of any open format. Value-adds and interoperability. Instead I just see a way to organize markdown files.

I'm from the ML platforms and systems domain.

I strongly recommend it if one's able. It's a bit more stable than a quickly evolving ML/DL/AI ecosystem or frontend ecosystem. The skills are more durable. It repays deep investment and knowledge.

It allows you to straddle both the distributed systems and services domain and the ML domain.

ML systems problems are extremely interesting since they require extremes of compute, storage, network, and latency, in very different parts of the model lifecycle. Its unique problem is the scarcity and cost of hardware accelerators.

I've worked eleven years in the space and rarely have had the desire to leave.

Pytorch is widely accepted as the de facto ML framework in both research and industry. TensorFlow comes second in industry. Jax is hardly used at all, but uses the same backend as TensorFlow.

Triton is a python-like language to define ML math operations that run efficiently on hardware accelerators like GPUs or TPUs. OpenAI open sourced it. If there's a particular math operation you have a unique need for in your model, and it hasn't already been implemented by some other library, and it's important for efficiency, you'd probably write it in triton these days. It'll be compiled to an intermediate representation, then to an efficient runtime.

The course linked deals with "MLSys", or "ml systems". That means using GPUs and other hardware accelerators efficiently to run ML math operations on one or more computers.

95% of working ML engineers will never need to write Triton, and will be more than satisfied with PyTorch. Many more ML engineers will, nevertheless, write Triton code, because it is interesting, fun, easy, and people are impressed when you tell them you did.

Hosting pytorch models efficiently is currently awkward, because there's no clear winner in the ecosystem. ONNX is a way of representing model graphs in a framework-agnostic way. Other systems can interpret ONNX graphs to do inference. So sometimes, when someone wants to host a pytorch model, they turn it into an ONNX model and run it with an efficient runtime on CPUs or GPUs.

Please read the article. You've ignored proof checkers.

I'd be thrilled with a second thread much less GPU.

You're right that "assumes" might be misleading. Maybe "implies" would be more correct.

The point the author's trying to make is that if we state in a paper something like "the LLM understands, believes, thinks, ..." then we're supposing an intelligence much like our conception of a human intelligence. It's a form of 'begging the question' -- assuming what you're trying to prove.

It is not quite a fair argument, just because we don't have a precise vocabulary around how to talk about the activity of LLMs that doesn't involve making these loose analogies. Except for philosophers and people engaging in this kind of "is it truly intelligent or no" conversation, being imprecise in this way doesn't necessary have any cost, but is just a convenient way to avoid developing a jargon.

Yes. There's nothing essentially new this latest round of AI has unturned that philosophers haven't turned over decades (or more ago). Nothing stopped philosophers supposing even a functionally perfect simulacrum of human intelligence, and getting technologically closer to it doesn't.

The real effect of the latest round of AI has been inducing software engineers to be pretend-philosophers as they're approaching this set of questions for the first time -- and are having a very hard time engaging given their enthusiasm for technology.

You're missing the satire.

And the 'argument', which is a funny way to recast the chinese room argument, which has also been discussed to death.

And you're also assuming any kind of position other than your own dogma -- that AI has Intelligence In Its Name and Humans Have Intelligence Therefore AI Has Human-Like Intelligence -- is based on some religious belief in the specialness of humans instead of pointing out where this analogy between Intelligence in its two senses breaks down.

Please tell me your comment is satirical too?

What is the truth about AI & climate change?

The article says that the company /may/ have been speaking in their capacity as Amazon employees and not citizens.

Even if they described themselves as working for Amazon, that doesn't mean they were posing as representatives of the company.

Further, even if this violates Amazon policy -- the policy itself violates Seattle law: https://library.municode.com/wa/seattle/codes/municipal_code...

I've even seen AI-generated roadmap documents that promise the world, are broadcasted to director+, who sign off without a thought.

Words and code are so cheap they're meaningless, while human focus, attention, and understanding are so expensive and overloaded.

Amodei is convinced he's abraham prefiguring AI's christ. Very different than Altman's cold power-seeking. You can always trust someone who's selfish, since they'll always do whatever's in their benefit at all times.

This is why 'agents' are the solution for these companies. Token spending goes through the roof. As long as a human is in the loop needing to read or review at human speed, that's a ceiling on how many tokens per user they can generate.

From the other end, I've seen this go wrong a couple ways:

When I'm doing it: I can go on way too long trying to consider way too much, when really, putting down some code and reading it and writing it myself would give me a better understanding.

When others are doing it: they can get very entrenched in a certain way of thinking, and are sure it's correct because of their AI conversations. Some context or data point was missing from their conversations with the AI.

Newsweek already found him back in 2014, remember?

They searched for everyone named SN and narrowed it down to the most likely candidate. Foolproof.

Seriously, the article says exactly the opposite.

Why do those experiences indicate the presence or non-presence of an afterlife?

This claim from Ayer -- how do we make the leap from these experiences existing to being evidence of a life after consciousness?

On the face of it, these experiences, on the assumption that the last one was veridical, are rather strong evidence that death does not put an end to consciousness