HighFreqAsuka

Take a look at The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text (https://arxiv.org/pdf/2506.05209). They build a reasonable 7B parameter model using only open-licensed data.

Fruition is my entry recommendation. Every bar they make is good. Then Castronova, Goodnow Farms, Askinosie, Soma, and Dick Taylor. A good heuristic for evaluating a new brand is if the package tells you where the beans are from and if the ingredients list is “cocoa beans, sugar”.

EDIT: I realize you asked about bars not brands, but I’m in transit and brands was easier than individual bars. I’m a huge fan of the Askinosie orange bar in particular.

Here’s what we’ve found out: It’s not about the cocoa-beans, but about the way they are treated during the manufacturing process.

I eat a lot of high end single origin chocolate bars, and I simply don’t believe this. Two bars from the same brand at the same percentage of cocoa content using different beans taste completely different. In exactly the same way as wine or coffee. It’s one of the most interesting parts of eating good chocolate. I just don’t believe this approach will ever replace my chocolate consumption, but may have a shot at the larger market of bad chocolate bars.

We've collectively failed at this problem as a society.

We've 1. made it extremely difficult, even illegal, to build more denser housing. 2. devalued the status of electricians, plumbers, and carpenters leading to a shortage of people we need to build more homes. 3. made it much too easy to get very large mortgage loans, incentivizing people to leverage themselves much more than they should to purchase homes. Thus bidding up the price of homes as otherwise financially smart people are forced to play the game as well. 4. built software to collectively price fix rents, favoring higher rents over maximum occupancy.

All of this has turned housing into an asset class, in which a significant fraction of the average American's net worth is invested, and has led to huge inflows of investor money. The incentives to not fix any of this are very strong.

Did you not read the word “potentially”? Topological spaces are a more general case of spaces that contain the discrete case as a subset.

By having nearly no volume to begin with. Many US stocks are listed across many of the ~13 US exchanges, so what you're describing is already the status quo. According to RegNMS regulations, traders cannot trade through protected quotes, and so trading firms already have to be aware of the prices on all US exchanges.

But LSTE has so little volume, that you frankly forget they exist most of the time.

The point of that comment doesn't have anything to do with how ResNets actually work. You missed the actual point.

We don’t actually know why resnets work so well.

Yes actually we do. We know, from the literature, that very deep neural networks suffered from vanishing gradients in their early layers in the same way traditional RNNs did. We know that was the motivation for introducing skip connections which gives us a hypothesis we can test. We can measure, using the test I described, the differences in the size of gradients in the early layers with and without skip connections. We can do this across many different problems for additional statistical power. We can analyze the linear case and see that the repeated matmults should lead to small gradients if their singular values are small. To ignore all of this and say that well we don't have a general proof that satisfies a mathematician so i guess we just don't know is silly.

Yes, I have a very good point in fact. But the above comment purposely chooses not to argue with it, because it's easier to ignore it entirely and argue something else.

Empirically yes, I can consider a very deep fully-connected network, measure the gradients in each layer with and without skip connections, and compare. I can do this across multiple seeds and run a statistical test on the deltas.

No, there are many very mathematically inclined deep learning researchers. It's an empirical science because the mathematical tools we possess are not sufficient to describe the phenomena we observe and make predictions under one unified theory. Being an empirical science does not mean that the field is a "wild west". Deep learning models are subjectable to repeatable controlled experiments, from which you can improve your understanding of what will happen in most cases. Good practitioners know this.

Just read the section on ResNets (Section 1.5) and tell me if you think that's the best way to explain ResNets to literally anyone. Tell me if, from that description, you take away that the reason skip connections improve performance is that they improve gradient flow in very deep networks.

I agree with that, I think UDL uses the necessary amount of math to communicate the ideas correctly. That is obviously a good thing. What it does not do is pretend to be presenting a mathematical theory of deep learning. Basically UDL is exactly how I think current textbooks should be presented.

I've seen quite a few of these books attempting to explain deep learning from a mathematical perspective and it always surprises me. Deep learning is clearly an empirical science for the time being, and very little theoretical work that has been so impactful that I would think to include it in a book. Of the such books I've seen, this one seems like actively the worst one. A significant amount of space is dedicated to proving lemmas that provide no additional understanding and are only loosely related to deep learning. And a significant chunk of the code I see is just the plotting code, which I don't even understand why you'd include. I'm confident that very few people will ever read significant chunks of this.

I think the best textbooks are still Deep Learning by Goodfellow etal and the more modern Understanding Deep Learning (https://udlbook.github.io/udlbook/).

Looks like you left the submissions instructions for AISTATS on the last page of the PDF. Don't know if that was intentional but I'm guessing it wasn't.

however, it was not truly "independent",

Of course I don't mean to imply they operate in a vacuum. They can obviously see competitors raising their prices. I just mean to say they don't get into a room and actively decide on a price.

There's a theory, price-over-volume, that the sudden shift in demand and expectation of inflation gave companies room to explore a different point on the revenue curve, where they increase the price and simply sell less volume. Prior to the pandemic this was risky and people assumed they were near optimal already. During the pandemic a bunch of companies learned they could push on price, sell less, but still make more revenue. All companies did this independently and simultaneously so the usual competitive effects didn't kick in. And now we're at a new equilibrium that the few companies in each industry are happy with.

Just for clarity, the linked paper in the twitter thread is "An autonomous laboratory for the accelerated synthesis of novel materials" (https://www.nature.com/articles/s41586-023-06734-w) which does have two authors from DeepMind but seems to be mostly from material science researchers at UC Berkeley. This thread is not about the recent Nature paper "Scaling deep learning for materials discovery" (https://www.nature.com/articles/s41586-023-06735-9) from Deepmind which made news a few days ago.

Transformers have disadvantages too, and so LSTMs are still used in industry. But also it's not that hard to learn a couple new things every year.

Quite a lot of techniques in deep learning have stood the test of time at this point. Also new techniques are developed either depending on or trying to solved deficiencies in old techniques. For example Transformers were developed to solve vanishing gradients in LSTMs over long sequences and improve GPU utilization since LSTMs were inherently sequential in the time dimension.

Because he's the former president of YC. One of the powerful people you'd be pissing of by doing this is Sam Altman.

You'd be getting on the bad side of some of the most powerful people in tech and taking the helm of a company full of people who don't want you running it. It's not a surprise a lot of people don't want the role.

I think it's pretty clear by this point that the board consists of several people who are woefully ignorant about the way the world actually works.

These simply can't be the real reasons.

Whenever I run into a doctor like this I just assume they're willfully ignorant because they don't want to rock the boat. How can you not be curious about such a large part of your profession?! It's a pretty negative signal in a person in my opinion.

I've spent considerable time tracking down why X costs Y, explaining it to a doctor, only to have the reply be "woah, you should tell people!" No, you should tell people, that's part of your job! Mind blown at some of these people.

In computational geometry, Raimund Seidel wrote a paper that proves an upper bound theorem for polytopes in two sentences in the abstract. The rest of the paper just comments on the result.

https://www.sciencedirect.com/science/article/pii/0925772195...