HN user
Shameless plug: https://huggingface.co/spaces/smolagents/ml-intern
It’s a simple harness around Opus, but with tight integration to Hugging Face infra, so the agent can read papers, test code and launch experiments
Hugging Face Buckets are pretty simple: https://huggingface.co/docs/huggingface_hub/en/guides/bucket...
Disclaimer: I work at HF
The analogy stems from the notion that neural nets are "grown" rather than "engineered". Chris Olah has an old, but good post with some specific examples: https://colah.github.io/notes/bio-analogies/
Thanks! I expect the book will remain relevant as long as the Transformers architecture does. That’s why we mostly focus on topics we think will stand the test of time, but let’s see how that plays out :)
In the specific case of SmolLM, it originates from the meme in this dataset https://huggingface.co/datasets/bigcode/the-stack-smol
Hi, Lewis here (one of the co-authors). Happy to answer any questions people have about the book :)
For those interested in playing with an implementation of these ideas, my colleagues at HF made some recipes here: https://github.com/huggingface/trl/blob/main/docs/source/lor...
“QED and the Men Who Made It” [1] might be close to what you’re after for quantum theory at least. Unlike other popular accounts, it gets quite technical and covers a lot of the historical dead ends that people had during the development of quantum field theory.
[1] https://press.princeton.edu/books/paperback/9780691033273/qe...
We instantiate this idea through Preference-prior Informed Linucb fOr adaptive rouTing (PILOT), a novel extension of LinUCB
Academics are pretty creative at naming their creations
Indeed we opted for offline methods like Anchored Preference Optimization as we found in the Open R1 project that doing multi-task RL on small models is quite a hassle to get right. With offline methods, you focus much more on dataset curation / generation, but that still provides faster iteration cycles for the model scale we’re dealing with!
The absolute best way of doing this is these days is likely through a vision based machine learning model, but that is an approach that is very far away from scaling to processing hundreds of gigabytes of PDF files off a single server with no GPU.
SmolDocling is pretty fast and the ONNX weights can be scaled to many CPUs: https://huggingface.co/ds4sd/SmolDocling-256M-preview
Not sure what time scale the author had in mind for processing GBs of PDFs, but the future might be closer than “very far away”
I gave the demo a spin and it’s pretty nice! One thing I noticed is that the avatar doesn’t seem to be aware of it’s surroundings- for example, I asked it why it was wearing a cowboy hat and it was adamant that it wasn’t wearing a hat at all :)
I expect language models to also get crazy good at mathematical theorem proving
Indeed, systems like AlphaProof / AlphaGeometry are already able to win a silver medal at the IMO, and the former relies on Lean for theorem verification [1]. On the open source side, I really like the ideas in LeanDojo [2], which use a form of RAG to assist the LLM with premise selection.
[1] https://deepmind.google/discover/blog/ai-solves-imo-problems...
Hello everyone, we just did a speed run with Argilla and KAIST AI to fine-tune the beefy new Mixtral model with some new techniques that came out recently. More details in the model card - enjoy!
The myth is also promoted in Chapter 3 of The Making of the Atomic Bomb by Richard Rhodes:
Plank had taught at Berlin since 1889. In 1900 he had proposed a revolutionary idea to explain a persistent problem in mechanical physics, the so-called ultraviolet catastrophe
Talks and discussion with the creators of Stable Diffusion and more :)
Spoud | Data Scientist / Senior Software Engineer / DevOps Engineer | Bern, Switzerland | ONSITE | http://spoud.io/jobs.html
Spoud is a Series A funded startup that is building a data market platform to enable enterprises to easily find, understand and access their data sources. We are looking for engineers and data scientists to help us develop the product and work on a wide range of projects with our customers.
If you like dynamic environments, solving challenging problems and are passionate about event-driven architecture, then you'll love working at Spoud.
We're more interested in a willingness and ability to learn than what you currently know. But in case you're interested, our tech stack is mainly Java, Kafka, Flink, NodeJS, GRPC, and Docker.
Apply via jobs[at]spoud[dot]io