HN user
I've made a visual walk through the machinery inside a large language model: from raw text, to tokens, to vectors, to attention, to the next token.
If you have any comments/questions/remarks/improvements, let me know!
That sounds like otter nonsense...
Roy van Rijn here, the guy that got an honerable mention for being “first”. Ended up 7th or something.
If you’re interested in even more in-depth tips and tricks that were used in this contest, I highly recommend this three hour (!) long deep dive I did with Thomas Wuerthinger (the winner of 1BRC):
https://youtu.be/_w4-BqeeC0k?si=p4NsRFPe6Jtq7HvA
There is so much valuable content in that talk, some things that I didn’t even realize until long after the contest was over.
It’s probably the best and most detailed conference talk I’ve ever done.
Last January a challenge was posted online by Gunnar Morling: How fast can you parse a file with one billion rows of weather data using Java? Little did I know this deceivingly simple question would lead me down a path that taught me all about: parallelism, memory mapped files, SWAR techniques (SIMD as a register), bit twiddling, branchless code, mechanical sympathy, Graal native compilation and finally... I even turned to the dark side: using sun.misc.Unsafe. Join me in this deep dive where I'll explain all the code changes and tricks that took me from the reference implementation which processes the billion records in less than 4 minutes, to processing everything in under two seconds. Who knew Java could be this fast?
I've experienced the same thing! We laughed so hard.
https://twitter.com/royvanrijn/status/1214162400666103808
It is caused by an electric EMI spike from gas lift office chair. There is even an official Dell issue about it:
https://support.displaylink.com/knowledgebase/articles/73861...
These fully customizable developers are powered by the advanced GPT-4 model. Digital Developers™ offer a cost-effective alternative to traditional Java developers, with a host of exceptional capabilities.
Or: Why is Python still growing and Java falling behind?
This happened to us at work: https://twitter.com/royvanrijn/status/1214162400666103808
He also did a couple of recordings/talk on YouTube about how it all works and also does a live demo:
https://youtu.be/T4PJoAh4X1g?t=1977
It even works to align/overlay music, which is really cool:
Sure there is a lot of bad music in there, but the next top 40 song that tops the charts will likely also be in there, hidden away.
There are many ways to properly implement content-id, for example using fingerprinting (like Shazam does). These methods are able to detect matches even with a lot of noise and changes...
I hope they implement something like this, to make detecting duplicates and blocking "revenge porn" much easier and to have regression against re-uploading.
Explanation of Log4Shell, including demo of multiple RCE's and information leakage.
Some LEGENDARY eight-amulets:
eiGHt EIgHT eIght eighT eight EIgHT EIgHT eIGHt eiGhT eIgHT - 8 :888888881a7402c0c8ba04e6fa2cbbd4aa5111af3fb2dce8c9d90b06ab13ed38
EigHT EIGHT EIGhT EiGhT EighT EIGhT eIGhT eIGhT EIGHT eight - 8 :0888888885a04f80a17a45468c761c0e6a80b9b1309a408003b024a1b7ddfc3d
EighT EigHt EighT eiGht EIGHt eiGHt EIGhT eIght eight Eight - 8 :985a2888888889d774f2c083b44bd5a87f86974295f6178bf40da5ac2bcd9d6e
EigHt eigHt eighT eight EigHt eigHT eIgHT eight eiGHT eIgHT - 8 :06da1388888888b0fe865087da99839debe557e4a7d762d740d93d5eea6a683b
EIgHt eIght EigHT eigHt EiGhT EigHt Eight eight eIGht eiGHT - 8 :fe9143429488888888c375251de2493913a07ffccd0e104810153085d27dfb04
eiGHT eiGhT eIGHT eIGHt eigHT EiGht EIGht eiGhT EIGht eIGhT - 8 :91e10b15288c9888888884ca0082ce0ccfe4bfafcb13444bc36fcf30b19ca363
eiGHT eIGht eIgHt EIGht EIGHt EIgHT eiGht EiGHT eIGHT eiGHt - 8 :fff5d81258a222b88888888b258ce415a1ae1b12b14535c09b9dcb19d70d4328
EIghT EiGHT EIght EIGhT EigHt EIGHT eIghT eIGhT EighT EIGht - 8 :2b7de871ee1694892ddbf2937b688888888e99d37e5c920804424670a178a133
EIGHt EIght eigHT eiGHt eiGHT eIGHT EigHT EIGht eigHT eiGHt - 8 :51670c25d4dab2d891cd18beed868888888870850a92e42887c62bd750a16c5f
EIghT EIgHT eIgHT EIGHt eiGHT eight EigHt EIGHT eIght eIGHt - 8 :80b5c98d08eacad221f25bde009488888888b58dbb3aafaf617b0b364133e87b
EIgHT EIGhT EIGHt EigHT eIGhT EIght EIgHT EIGhT eIgHT EIGHt - 8 :cd0b0fd7ef75fc8a37001bc87183d326aeba888888885cc2c48dbf48d38a7bfb
eigHt eIGhT EighT eIgHT eigHT eIGHT EigHT EighT EIGht eigHt - 8 :8ecfe9ad597bad2e537beacce70464cb57c18f88888888fd105c8c31de75345e
eiGHT eIGHt EIGhT EIghT EIghT EiGHT eiGHT eIghT eIGht EIgHT - 8 :e88f20df9f85114be9e95f88daeb57c94be3564e4c8688888888c0fbbc7e4e8c
eigHT eIgHt EigHt EigHt EigHT EIGHt EiGht EigHT eIGHT EiGhT - 8 :bcab4227100078135599839a3d3b35c38b75e94e753e95a88888888a29616478
EIgHt EiGHT eIGhT eIght EIgHT EiGHt EiGHt eIGHT - 9 :6dcfd600d05e1249d6590aaee053793b792c60ff94513456b697888888888f67
Completely agree here, the entire 'solver' is a library there.
However, it does show how cool and versatile SAT solvers are for people who haven't used or seen them yet.
I wonder how much of that is programmed/automated and how much work is still being done manually...
Backtracking and eliminating numbers can be easily automated.
Regarding Conway's Law:
We have found that by changing our software/system architecture we have also inadvertently changed our organisation structure.
- Inverse Conway Law or just Roy's Law ;-)
Before we had four cross functional teams, working on a single application, everyone felt responsible for it, worked overtime to fix bugs etc, we had good communication between the teams.
But after we switched to microservices the teams became responsible for just a part of the system, their microservice(s). Whenever we had an outage, one team was left to fix it, the others just went home. They stopped talking to each other because they didn't share any code, no issues... they stopped having lunch together, some things got way worse in the organisation, all sparked by a 'simple' architectural change, moving to microservices.
And... its gone. Database error. :-(
Ah, just now noticed this: https://news.ycombinator.com/item?id=18802763
Given the maze on the website, what is the highest number you can get on a square when creating a maze?
This is a fun puzzle, I got up to 124:
Also, showing people only what they want to see leads to echo chambers
Not sure I agree with this. The internet made it possbile for the whole world to see information coming from all kinds of sources. Before, in history, you had to rely on local news/newspapers, or even less information. Much more bubbly than this modern age.
Again: I think the reason those bubbles appear have to do with society and the urge to belong to a group, Facebook is not to blame here. Facebook reflects people and society, not the other way around.
Facebook doesn't spread hatred, people spread hatred.
Sure, Facebook could crack down on the way people are using their service/their tool, but I don't think they are responsible for the actions of users.
What about:
- Broadband internet is spreading hatred
- Chrome Browser is spreading hatred
- ISPs are spreading hatred
(etc)
Don't blame a company, or the service they provide. A crackdown on Facebook will force the hating people to another tool or communication method. They'll probably move to a more encrypted method of communication, which will make finding the wrong-doings and the people much harder.
The only thing they could do is ask Facebook to cooperate and provide details to find the haters.
Most important: Don't just start refactorings, finish them (!!!)
I've used a couple of the refactoring approaches mentioned in the article over the last 10 years in our current project. The biggest mess comes from partially executed parallel changes. You go from one bad solution to one bad solution + a better solution, ending up with a bigger mess than you started with halfway through. And when that mess becomes visible/a problem again, nobody wants to finish the initial refactoring, they create another parallel change, not finishing that either, etc etc.
I think they've saturated the market...
The existing methods orElse(defaultValue) and orElseGet(supplier) and orElseThrow(exception) should be enough to cover everything get() does.
But the Java community should move towards ifPresent/map/filter/flatMap etc.
Of course it does, you actively choose to throw something! That is, in some cases, perfectly acceptable. All three methods tell you there is something else in case you don't have a value in the optional. get() doesn't.
How does orElse() cause new problems and/or break code? It is the same as get but forces you to think about the not available option...?
I think the options orElse/orElseGet and orElseThrow are enough to replace every get() method, and they'll force you to think about the missing scenario.
If you want to do something in case it is present, use ifPresent(lambda).
In case you want to return something, use orElse/orElseGet or orElseThrow, or just return the Optional itself.
Just a joke, needed to squeeze in the term 'microservices' for search results :-P
Also: I'm not really buying into the whole microservices movement, it is the latest silver bullet.
Quite a coincidence, I made a movie yesterday about neural networks and how Google's 'deep dreaming' works:
Watching the video, the only thing I see is robot abuse. #freeatlas!
The point is: An A5 booklet on A4 is the same as A4 booklet on A3, etc.