Chirono

That’s usually not true due to caching. It may be true if you leave a large gap in between, but if you send “make it red” right after, then it’s purely incremental

Exactly. This number is so so much bigger than 10^100000 or however many grains of sand would fit, that dividing by that amount doesn’t really change it, certainly not enough to bring it down closer to 9,999,999sub10

The two other changes they mention have been widely adopted, and are included in at least some of the models they benchmark against. It seems they list them for completeness as changes to the original transformer architecture.

This is just an artefact of tokenisation though. The model simply isn’t ever shown the letters that make up words, unless they are spelled out explicitly. It sees tokens representing groups of words. This is a little like saying a human isn’t intelligent because they couldn’t answer your question that you asked in an ultrasonic wavelength. If you’d like to learn more this video is a great resource: https://youtu.be/zduSFxRajkE?si=LvpXbeSyJRFBJFuj

I used to use K professionally inside a hedge fund a few years back. Aside from the terrible user experience (if your code isn’t correct you will often just get ‘error’ or ‘not implemented’ with no further detail), if the performance really was as stellar as claimed, then there wouldn’t need to be a no benchmark clause in the license. It can be fast, if your data is in the right formats, but not crazy fast. And easy to beat if you can run your code on the GPU.

For anyone wondering what this does, it looks like it produces optimal configurations for belt balancers given a specified number of input and output belts. Belt balancers evenly distribute items between belts: https://wiki.factorio.com/Balancer_mechanics

In some cases you can directly test hypotheses like that, but more often than not, there isn’t a way to test without just trying.

Nice paper. I particularly like how they talk through the ideas they tried that didn’t work, and the process they used to land on the final results. A lot of ML papers present the finished result as if it appeared from nowhere without trial and error, perhaps with some ablations in the appendix and I wish more papers followed this one in talking about the dead ends along the way.

That’s only true for linearly ordered structures, but isn’t true for partially ordered ones.

For example, set inclusion. Two different sets can be neither greater than not smaller than each other. Sets ordered by inclusion form a partially ordered lattice.

Interesting! Do you have a link to that research?

From reading this book you’d have a very good grasp of the underlying theory, much more than many ML engineers. But you’d be missing out on the practical lessons, all the little tips and intuitions you need to be able to get systems working in practice. I think this just takes time and it’s as much an art as it is a science.

Wayve is developing their own autonomous vehicle system, so I think they are their own target audience here.

The other angle on ‘Corporate AI’ is when we’ll start to see product placement and adverts inside generated content. Create an image of coffee, and you’ll find Starbucks logos everywhere. Ask an LLM about a topic and see it work in an advert about a particular brand of beer. I’m sure people are working on this already, but I really hope it never happens.

It’s entirely static for now. I’m sure we’ll see techniques for dynamics objects eventually though.

Link seems to be dead?

What would you propose instead?

out of interest, is there anything specific you think humans will always be better at than AI?

Of course they’re different. But so what? That’s not exactly proof of anything, unless you’re suggestion biological neurons are the only configuration in the universe capable of thought? Maybe that’s true, but it seems unlikely to me.

The pressure of natural selection can lead to the phenomenon of consciousness. Why not the process of training llms? Perhaps developing the machine equivalent of consciousness helps that particular configuration of weights survive the otherwise destructive process of gradient descent.

This paper, and most other places i’ve seen it argued that language models can’t possibly be conscious, sentient, thinking etc, rely heavily on the idea that llms are ‘just’ doing statistical prediction of tokens.

I personally find this utterly unconvincing. For a start, I’m not entirely sure that’s not what I’m doing in typing out this message. My brain is ‘just’ chemistry, so clearly can’t have beliefs or be conscious, right?

But more relevant is the fact that llms like ChatGPT are only pre-trained on pure statistical generation, followed by further tuning through reinforcement learning. So ChatGPT is no longer simply doing pure statistical modelling, though of course the interface of calculating logits for the next token remains the same.

note: i’m not saying i think llms are conscious. I don’t think the question even makes much sense. I am saying all the arguments that i’ve seen for why they aren’t have been very unsatisfying.

Yes there are: https://www.volvotrucks.com/en-en/trucks/alternative-fuels/e...

Stretch the timelines out a bit. Say the asteroid is two years off (a fairly typical startup runway). I would much rather know the planning, decisions and execution of the one thing that could save my life were done by well rested and level headed individuals, not stressed out sleep deprived people more prone to missing details and making mistakes.

I still find it strange that housing is expected to be an appreciating asset class, rather than a depreciating one that requires continual investment to counteract wear and tear etc. I understand that this is largely by design, but it seems odd to me how much this is accepted as fundamental to properties, rather than something that’s been constructed.

That code is written in a very declarative, functional style. Haskell is a language that forces you to write code like that, so might be a worth a look if your goal is to write ‘pretty’ code.

However, I’d also add that becoming an “elegant” and “useful” programmer are often at odds with each other. It’s very easy to spend so much time trying to make your code pretty with the perfect abstractions that you never actually finish anything.

If your goal is to be useful and productive then learning by writing a lot of code in a lot of different languages, styles and code based might serve you better than focussing on beautiful source code. Though if you can do both, then please do!

I’d double check your working there because it certainly is true. In your example the standard deviation (2e29) is far bigger than the difference between the median and mean (5e28).

Are there any details on how this works anywhere? It kinda looks like a diffusion model, but using Gaussian blur instead of the Gaussian noise that dall-e etc use.

Yes, the DLR in London has goes overground and has run without drivers since the 80s.

To get an idea of comp for companies in London, levels.fyi is a good starting point. https://www.levels.fyi/Salaries/Software-Engineer/London/ Bottom line is that comp is way under what you would get for the same job in the US, but better than most other engineering jobs in London outside of certain finance roles. The market seems to have been going up a lot in the past couple of years though.

The interview process is pretty much the same as you get in the US. Lots of leetcode and system designs.

I think the surprise comes from how long it takes to get the answer, not that it's incorrect.

By comparison, gcc is able to reduce those two implementations to the same thing and prove them equal in a fraction of the time (https://godbolt.org/z/6oMEd8ebY). If it takes Symbolica a minute for the same tiny example, how does it work on real codebases?

Except it turns out to be "drop millions of allocations by changing an exponential recursive algorithm to a linear scan that does a different (and incorrect) thing". So it seems like the linked list part of it was incidental.

The paper in this post doesn't use neural networks at all. It also includes the time to train the model in the sorting time, which the paper you linked doesn't. NN-sort trains on historic data to sort future data, which requires the distribution to be roughly the same, whearas this algorithm learns online.