hatthew

The article has a good point and is correct. But it gets too far down into the weeds. For example, the paragraph about how priest conversions don't count as empire-building seems like the sort of argument one makes when desperately clutching at straws—a position that's entirely unnecessary given how reasonable the premise of the article is. It's building a straw man for a nonexistent opponent.

I read that part and I still don't like the article. I have several hundred hours in AoE2. Maybe I am unique, but it has never even occurred to me to call the collection of people and buildings I control an "empire". I think "it's just a game['s name]" is still a perfectly valid argument.

I would like the article better if it took an approach such as "Age of Empires is a bit of a misnomer, let's explore the actual definition of an empire," rather than its current combative approach.

For typical social media sites, engagement will pretty much always be proportional to both revenue and cost per user. Either revenue > cost, and the site is incentivized to increase engagement, or revenue < cost and the site dies. There is no middle ground where a site gets a healthy revenue that's greater than its costs, but increasing engagement won't increase revenue. The exceptions are niche sites that do things like fixed subscriptions, or cost money to create content but not to consume it (but even in these cases, increasing engagement probably still increases the chance users start/continue to be paid customers).

Presumably significantly less. But GP's point is that reducing advertising revenue doesn't remove the incentive to maximize engagement.

If it is to be believed—which I wouldn't count on—it sounds like the intention of the prompt was to keep the model focused, but the model's interpretation of it was to keep you focused.

Some guesses:

- A lot more direction-dependent.

- More maintenance related to keeping the right direction, maintaining more moving parts, etc.

- Less convenient form factor, more susceptible to wind, more unsightly.

- Less space-efficient, which matters for some applications like house rooftop installations.

Simply by the fact that you say computer engineering, you already went deeper than 99% of "computer people" in 2010

With a car, when I turn the steering wheel a quarter turn, the car immediately starts to turn at a known rate (I can't quantify it, but my muscle memory knows it). With an LLM, if I ask it to "turn the car", my "steering wheel" is natural language. I have to spend effort to figure out how to convey the rate at which I want the car to turn, and because the LLM has a noticeable delay, I also need to preemptively include some language anticipating how the expected rate should feel and what to do if it doesn't seem to feel right. And then the next time I say the same thing, the LLM decides to steer the car a bit differently anyways.

This matches my experience coding, and the expectations I (and many other) have been voicing for years about why AI is still a ways away from solving coding. The hard part of programming isn't pressing the keys to put code into a file, it's understanding the problem and coming up with a good, clean, extensible solution. AI is great at making ok solutions that technically work, but pretty bad at having a good vision for a solution. AI is still very useful for bouncing ideas off of and exploring to create a better understanding, but writing a good spec for an LLM to implement isn't much easier than just writing the code yourself.

I'm also curious, specifically about the cost of training vs inference, and comparing that to other industries that can have high R&D costs. My instinct says that open weights aren't feasible because of the obvious issue where there is no incentive to develop your own model rather than just taking someone else's model. However, I could see a scenario where a hardware company designs a model that is open weights but optimized strongly for their own proprietary hardware, cutting their costs of inference low enough to be competitive with a hypothetical other company that doesn't have any R&D expediture.

To me, hippopotomonstrosesquippedaliophobia feels less like vocabulary and more like trivia

It's a bad aphorism if you take it literally and don't think about any nuance, but it's the correct priority. In many cases each of the three steps can take longer than the last, and each step is useless if the previous step isn't possible. The lesson is supposed to be to focus on each step at a time, not to completely ignore the other steps.

If you're making a complicated webapp, use your favorite framework to make it functional, and then if it's functional and not already fast enough, look at the slowest parts and replace them with faster alternatives. It's not going to result in the most elegant solution, but in most cases it will be good enough. Better to have something that works than to spend an extra year reinventing the wheel.

I have a similar anxiety about emailing or messaging people directly. If I'm just posting a comment on HN or reddit, there's not much commitment and if someone responds but I don't feel like continuing the thread, it's fine to completely ignore it. But if I send something directly to someone, it feels very personal, and like I'm starting a conversation. Once I'm in a conversation I'm obligated to continue it until it reaches a conclusion, and while the anxiety of trying to write things in a personal way is bad, the anxiety of trying to force a conclusion to the conversation is worse.

This feels like less of a gamma issue and more of a color space and monitor calibration issue. Possibly affected by the fact that oklab didn't exist when this article was written and the general awareness of color spaces was lower.

As someone with no plans to live or vacation in the caribbean, I'm curious. Is there a specific notable reason, or is it just a combination of littler things (cost, convenience, politics, weather, etc.)?

Part of the premise of the article is blatantly wrong. Distillation prevention was always visible. The only invisible safeguard was against frontier model development like development of training pipelines. This doesn't change the general idea that invisible degradation is bad and has been reverted, but the article changes the framing of the original issue from "preventing accelerating AI in the future" to "preventing cheaper AI right now".

Our logic at the time was that the relatively fixed cost of figuring out the hardware and developing device-specific software was less than the variable cost-per-board delta of like $20.

Huh. I had a work project a decade ago where we were evaluating SBCs as drivers for kiosks. At that time, the prevailing wisdom was that the Pi was specifically not for industry, as its main advantage was the strong community to provide support for DIYers. Competitors like PINE64 and Orange Pi were the same/better specs at half the price.

genius

This misses the point of the discussion. Yes, we can understand what it is like to be a human with echolocation. However, we can't understand what it is like to be a bat.

Anyone not familiar with "JPL" will not be helped by spelling out "Jet Propulsion Laboratory", since it's a misnomer anyways.

TFA addresses this

Now, we all know that it can take a while to find a long sequence of digits in π, so for practical reasons, we should break the files up into smaller chunks that can be more readily found.

In this implementation, to maximise performance, we consider each individual byte of the file separately, and look it up in π.

I work on "AI" stuff. Not LLMs, but large neural nets that include transformers and are as big as the smaller LLMs of today. Half the prompts I give fit their category of examples like "building pretraining pipelines, distributed training infrastructure, or ML accelerator design." I generally don't trust AI and have been very slow to trust and adopt it, but recently I've been warming up to it as part of my coding workflow.

Now with this, it makes me wonder if I should step back? Should I try to get used to a non-claude model/harness? Should I go back to less AI in my workflow? Either way, it makes me less inclined to pay for tokens from claude.

In addition to what mbauman said, hair follicles and the hair itself are not single-cell. I can't immediately find the composition and average cell size, but even a long and thick strand of hair is less than 2 orders of magnitude larger than the largest neurons. I doubt any individual hair cell is very large.

Some back of the envelope math says this is true. A conservative estimate for the size of an alpha motor neuron axon is 10μm diameter and 1m long, which already puts it over an order of magnitude larger than the 4,000,000µm³ oocyte quoted in the article.

I'm kinda of surprised that so many here on HN were dismissive/unaware of the capabilities and potential in the DALL-E days and earlier. I feel like this is the sort of forum where most people would be both aware of advancements and aware of their potential.

My moment was GANs and GPT-2 back in 2019. I feel like that's where computer-generated media went from "obviously fake" to "sometimes can be mistaken as real." RLHF for LLMs and diffusion for image generation are both important improvements, but I feel like they aren't fundamental prerequisites for they type of stuff we have today. I think the main advancements since then are just marginal improvements, larger models/datasets, and better surrounding tooling.

I'd say that broadly speaking, a system understands things when it can interact with them "correctly". I agree a pocket calculator understands math, but I'd say a pocket translator understands grammar, not language as a whole. A wikipedia page does not interact with anything, so I'm not interested in pushing the definition that far. However, if the wikipedia page were to make recommendations for nuclear safety based on some context it receives as input (say via an integrated LLM), I'd be happy to argue that it understands [that part of] nuclear physics.

I don't think that LLMs as black boxes are fundamentally novel, I just think that their internal design is novel, and their generality and ability to give correct responses to complex topics is far beyond anything previously. For example I would argue that wolfram alpha has a poor understanding of language and a very good understanding of math. I would argue that LLMs have an excellent understanding of language and a mediocre understanding of math, but are able to temporarily increase their understanding of math through document retrieval and "thinking" (or whatever you want to call the process of iteratively generating tokens that build on each other to result in a final response).

I think most debates about LLMs understanding boil down to different definitions of the word "understand." For example, with the definition of "understand" that I typically use in my daily life, I would argue that in the chinese room, the system as a whole "understands" chinese.

The data on the graph is real, but the labels are made up for vibes.

Yeah the graph is more for vibes than an actual analogy. The yellow slice represents the top 1%, purple 90-99%, green 50-90%, and red bottom 50%. It would make a bit more sense if those slices were labeled "1 person", "9 people", "40 people", and "50 people" respectively.