legel

You are wrong.

Text tokens are high-dimensional vectors, not 8 bits per character. Every token has a deep embedding, e.g. 1024 float values per text token.

DeepSeek-OCR proved 10x+ compression from visual embedding of text, which was a groundbreaking result. [1]

Very cool to see OP's project hacking on this principle. It's still not lossless, as noted in the github, but is a promising research direction.

[1] https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSe...

For anyone in AI, this reads like someone starting a new company called Google

Very very warm nostalgic fuzzy feelings here

I've been using Google Workspace for over a decade. Historically, after removing a user from your organization, it was always easy to migrate their data (e.g. Google Drive folders) to your own user. Suddenly, I recently had a nightmare experience where I needed to remove users no longer in my company, but I was unable to save their very important data. Google intentionally removed the feature in order to promote their new "Archive" user feature, where "data is safe" and you pay for it at continued unnecessary, extortionist rates.

Separately, I just had another terrible interaction with my own data through Google Labs Flow. That is the site that is serving the latest Nano Banana image and Veo video generations. My takeaways on the quality and value and issues with those "world models" are for another time. Here I'm pointing out another unique "dark pattern" that product managers seem compelled to apply: "if you want us to save your data in a database, then you have to let us view that data and train models on it". It's ridiculous, either I can have my data automatically deleted and have sessions be completely "dumb", or I can submit my soul for eternity and "allow any reviewer to analyze".

Beware, founders and developers relying on Google. Don't be surprised if you wake up with your data held hostage. Don't be surprised when you realize your intellectual property can either be deleted or stolen, but not simply saved.

Thanks Google.

Once upon a time, the "stupid hazing ritual" made sense.

Now it means company is stupid.

One could use any number of LLMs on real-world problems.

Why are we still interviewing like its 1999?

I happen to have a background at this interface as well, as the founder of DeepEarth and Ecodash.ai. I can tell you that I would greatly value such experience in collaboration, although I am not currently hiring. While having such a specific interdisciplinary niche can feel limiting, I also see it as a potential superpower in excelling in a very important domain. I'll also add that machine learning and other modeling techniques are a great bridge between classical natural sciences and modern tech today, that I would look for in collaborators. More specifically from the earth sciences, "GeoAI" would be a key focus.

Just shared this myself. Very lovely!

Thanks for reporting these metrics and drawing the conclusion of an underlying breakthrough in search.

In his Nobel Prize winning speech, Demis Hassabis ends by discussing how he sees all of intelligence as a big tree-like search process.

https://youtube.com/watch?v=YtPaZsasmNA&t=1218

All of this read and written on a smartphone.

Reversion to the past is not preparation for the future.

Very cool!

The training objective is clever.

The 50+ filters at Ecodash.ai for 90,000 plants came from a custom RAG model on top of 800,000 raw web pages. Because LLM’s are expensive, chunking and semantic search for figuring out what to feed into the LLM for inference is a key part of the pipeline nobody talks about. I think what I did was: run all text through the cheapest OpenAI embeddings API… then, I recall that nearest neighbor vector search wasn’t enough to catch all relevant information, for a given query to be answered by an LLM. So, I remember generating a large number of diverse queries, which mean the same thing (e.g. “plant prefers full sun”, “plant thrives in direct sunlight”, “… requires at least 6 hours of light per day”, …) and then doing nearest neighbor vector search on all queries, and using the statistics to choose what to semantically feed into RAG.

I’m glad Ilya starts the talk with a photo of Quoc Le, who was the lead author of a 2012 paper on scaling neural nets that inspired me to go into deep learning at the time.

His comments are relatively humble and based on public prior work, but it’s clear he’s working on big things today and also has a big imagination.

I’ll also just say that at this point “the cat is out of the bag”, and probably it will be a new generation of leaders — let us all hope they are as humanitarian — who drive the future of AI.

Math comes from brains.

Obviously the article is challenging the view — scientific or not — that mitochondria are not living.

Side note: previously I was funded by NSF and NASA to study such questions from biophysics and astrobiology.

That said, this was a delightful read. I did not realize or conceive of mitochondria as, like bacteria in our bodies, independent living networks with unique genomes, evolution, and flows of information and energy.

Reading about the health benefits of “external mitochondria” made me think about when I hug my dog: are we exchanging mitochondria, perhaps?

Restricted Boltzmann Machines were a huge revolution in the field, warranting a publication in Science in 2006. If you want to know what the field looks like back then, here it is: https://www.cs.toronto.edu/~hinton/absps/science.pdf

I remember in 2012 for my MS thesis on Deep Neural Networks spending several pages on Boltzmann Machines and the physics-inspired theories of Geoffrey Hinton.

My undergraduate degree was in physics.

So, yes, I think this is an absolutely stunning award. The connections between statistical entropy (inspired by thermodynamics) and also of course from biophysics of human neural networks should not be lost here.

Anyways, congratulations to Geoffrey Hinton. And also, since physics is the language of physical systems, why not expand the definition of the field to include the "physics of intelligence"?

From their official explanation page (https://www.nobelprize.org/uploads/2024/09/advanced-physicsp...): "With ANNs the boundaries of physics are extended to host phenomena of life as well as computation."

Congrats to Jayesh and team! I was lucky to meet the founding CEO recently, and happy to let everyone know he's very friendly and of course super intelligent.

As a fellow deep learning modeler of Earth systems, I can also say that what they're doing really is 100% top notch. Congrats to the team and YC.

It was fascinating to see the counter-proposal to the Army Corps of Engineers for Miami's design of a downtown wall to deal with storm surges: https://dirt.asla.org/2022/09/12/uproar-causes-u-s-army-corp...

The counter proposal was indeed funded by the City of Miami, to point out how ridiculous it would be to have a 20 foot concrete wall around the city.

As a local resident, I loved seeing this sad 3D render in particular, which even has a graffiti on it nearly spelling "Berlin": https://i0.wp.com/dirt.asla.org/wp-content/uploads/2022/09/0...

In seriousness, it was really cool to see the counter proposal's "nature-based solution" which would design 39 acres of distributed barrier islands around the coastline, to block storm surge naturally.

As a technical engineer, I’ve learned the value of starting sentences with “basically”, even when I’m facing technical uncertainty. Basically, “basically” forces me to be simple.

Being trained to say words like “Alternatively”, “But…”, “Wait!”, “So,” … based on some metric of value in focusing / switching elsewhere / … is basically brilliant.

Which is why it is incredibly depressing that OpenAI will not publish the raw chain of thought.

“Therefore, after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users. We acknowledge this decision has disadvantages. We strive to partially make up for it by teaching the model to reproduce any useful ideas from the chain of thought in the answer. For the o1 model series we show a model-generated summary of the chain of thought.”

This haiku response is art.

https://www.cs.columbia.edu/~jebara/4771/handouts.html

I remember taking Andrew Ng’s (delightful) Coursera ML and believing I knew ML.

Then I took a Columbia ML graduate course IRL: it was like being hit by a train.

I love the questions the writer presents to effectively manage the psychological and personal dimensions of solo development:

1. How much time can I afford to spend on this without feeling bad?

2. What can obstruct me from finishing this within that time?

3. How am I going to avoid these obstacles?

I’ve been using the AI Overviews in Google Search for months now and overall find more utility in them.

I’ve developed an instinct for when to ignore them. Typically it is when “edge information” is involved: facts / data that are only very sparsely published in time / web space, perhaps by only one or a few web authors, perhaps only very recently.

As an AI developer for over a decade, I feel like if we’re calling this “slop”, I fear what these authors think of human error. I’m also doubtful that these people who are damning this project have ever in fact innovated or developed anything significant from scratch themselves.

Makes me want to start a company called Slop.

This brings up an interesting “risk control” that one of my tech investors personally implemented with his family, in case a audio/video version of him ever asks to do anything crazy: secret passwords, agreed upon in person.

Super lekker! (That means love love love)

As an American who lived in Amsterdam for 3 years, to learn from their horticultural and eco leadership, I can say this is perfectly typical of the Dutch. Smart, easy, sustainable, beautiful.

Love your username, the beginning of Unix Time — you’re like our Jesus

As I lead a group of AI/ML researchers to develop systems for automatically generating 3D model animation parameters for plants and flowers, I wanted to share a wonderful capture of several flowers blooming. Indeed, I believe our "metaverse" should look and feel so real -- but also, we're working on this problem to advise gardeners in selecting plants for their real world gardens, eventually to include 3D photorealistic simulations of these plants growing in A.I. recommended garden designs.

I once quit a big tech company triggered by a desk move.

All of the social disruption described here is true. I also felt an environmental gentrification: my “secret” window spot, where the data engineers were stuffed into, being hijacked by “more important” salespeople… I still remember the feeling when they walked over and surveyed my land, smiling in delight of their new gains.

All of this is to say: people belong to spaces, too.

And stylish and culturally engaging.

Loved the Zoolander “context window for ants?!”