jl2718

1. At this scale, we’re not just talking about buying GPUs. It requires semiconductor fabs, assembly factories, power plants, batteries/lithium, cooling, water, hazardous waste disposal. These data centers are going to have to be massively geo-engineered arcologies.

2. What are they doing? AGI/ASI is a neat trick, but then what? I’m not asking because I don’t think there is an answer; I’m asking because I want the REAL answer. Larry Ellison was talking about RNA cancer vaccines. Well, I was the one that made the neural network model for the company with the US patent on this technique, and that pitch makes little sense. As the problem is understood today, the computational problems are 99% solved with laptop-class hardware. There are some remaining problems that are not solved by neural networks, but by molecular dynamics, which are done in FP64. Even if FP8 neural structure approximation speeds it up 100x, FP64 will be 99% of the computation. So what we today call “AI infrastructure” is not appropriate for the task they talk about. What is it appropriate for? Well, I know that Sam is a bit uncreative, so I assume he’s just going to keep following the “HER” timeline and make a massive playground for LLMs to talk to each other and leave humanity behind. I don’t think that is necessarily unworthy of our Apollo-scale commitment, but there are serious questions about the honest of the project, and what we should demand for transparency. We’re obviously headed toward a symbiotic merger where LLMs and GenAI are completely in control of our understanding of the world. There is a difference between watching a high-production movie for two hours, and then going back to reality, versus a never-ending stream of false sensory information engineered individually to specifically control your behavior. The only question is whether we will be able to see behind the curtain of the great Oz. That’s what I mean by transparency. Not financial or organizational, but actual code, data, model, and prompt transparency. Is this a fundamental right worth fighting for?

I have an interesting anecdote about that. I was consulting for a very large tech company on their advertising product. They essentially wanted an upsell product to sell to advertisers, like a premium offering to increase their reach. My first step is always to establish a baseline by backtesting their algorithm against simple zeroth and first-order estimators. Measuring this is a little bit complicated, but it seemed their targeting was worse than naive-bayes by a large factor, especially with respect to customer conversion. I was a pretty good data scientist, but this company paid their DS people an awful lot of money, so I couldn’t have been the first to actually discover this. The short story is that they didn’t want a better algorithm. They wanted an upsell feature. I started getting a lot of work in advertising, and it took me a number of clients to see a general trend that the advertising business is not interested in delivering ads to the people that want the product. Their real interest is in creating a stratification of product offerings that are all roughly as valuable to the advertiser as the price paid for them. They have to find ways to split up the tranches of conversion probability and sell them all separately, without revealing that this is only possible by selling ad placements that are intentionally not as good as they could be. Note that this is not insider knowledge of actual policy, just common observations from analyzing data at different places.

code is not literature

One thing I’ve thought about is how AI assistants are actually turning code into literature, and literature into code.

In old-fashioned programming, you can roughly observe a correlation between programmer skill and linear composition of their programs, as in, writing it all out at once from top to bottom without breaks. There was then this pre-modern era where that practice was criticized in favor of things like TDD and doc-first and interfaces, but it still probably holds on the subtasks of those methods. Now there are LLM agents that basically operate the same way. A stronger model will write all at once, while a weaker model will have to be guided through many stages of refinement. Also, it turns the programmer into a literary agent, giving prose descriptions piece by piece to match the capabilities of the model, but still in linear fashion.

And I can’t help but think that this points to an inadequacy of the language. There should be a programming language that enables arbitrary complexity through deterministic linear code, as humans seem to have an innate comfort with. One question I have about this is why postfix notation is so unpopular versus infix or prefix, where complex expressions in postfix read more like literature where details build up to greater concepts. Is it just because of school? Could postfix fix the stem/humanities gap?

I see LLMs as translators, which is not new because that’s what they were built for, but in this case between two very different structures of language, which is why they must grow in parameters with the size of the task rather than process linearly along a task with limited memory, as in the original spoken language to spoken language task. If mathematics and programming were more like spoken language, it seems the task would be massively simpler. So maybe the problem for us too is the language and not the intelligence.

This is an awesome development. I don’t want to take anything away from the credit due to the product. But I really dislike these bloviated corporate press releases. It reads like a full article generated from a 1-sentence LLM prompt. Perhaps the Internet UX from here will be a competition between AI-based content generation, and AI-based summarization, essentially DECCO instead of CODEC. Kind of like how spam grew to consume 99% of email, so everybody has to run spam filters to get what they want. Technology and the abuse of it move together.

This is the bigger reality. It’s turned almost all business and academic writing into long-winded meaningless trash. Well, more than it already was I guess. It seems that the way people use it is to expand few bits of information into many bits of content to convince others that work was done. It’s like the Turing test for laziness. The other issue is that it tends toward agreement on anything it wasn’t trained to specifically disagree about. I can see a smarter and more disagreeable bot doing much worse on LMSys than the sycophant models. Nothing new there I guess. But it’s spilling over to human norms as well, in that previously normal human deviation from chat model style interactions is anomalous, so everybody has to use the AI, and therefore nobody is providing any more value than the LLM, so everybody is getting laid off, except the disagreeable guy, and he gets fired first. It’s hacking us in the positive reinforcement vulnerabilities, ones that get worse the more they’re exploited, but it has none of the human resource constraints that previously kept them in check.

This is a ridiculous take on espionage. “Technology” would be the lowest priority for collection at Google.

Andy Grove flew in Clayton Christensen to let him talk for about 15 seconds before deciding that Intel would disrupt themselves by taking huge losses on Celeron. But Celeron did not save Intel; ASCII Red and multicore saved Intel. If he had actually read Clayton’s book, he would have understood that. Otellini got the disruption theory correct, and stayed out of mobile. But was that right? Maybe not in the current monetary environment where investment flows dwarf operating flows. A big mobile market could attract more investment than the losses it would generate. So disruption theory now works in reverse, and I’m not sure how far that implication goes.

I think you need higher algorithmic intensity. Gradient descent is best for monolithic GPUs. There could be other possibilities for layer-distributed training.

No. If you had 51%, you could revert one block of history for every 49 blocks of attack time. In addition, you have no ability to create transactions that were not already signed by the owners, nor create bitcoins more than the block reward. This is because of the UTXO model rather than the state machine model. In Bitcoin, every transaction is verified against history, while the EVM chains only verify transactions against state. So if you control EVM state, you can bootstrap every new node to any state you wish, but UTXO verification requires rewriting the entire history.

“Bitcoin security” is a different notion than almost all other popular chains. A prolonged 51% attack on bitcoin implies the ability to double-spend, but not at all the ability to affect prior balances. A 51% attack on most smart contract chains implies the ability to change any and all state arbitrarily.

The simplest solution is to wait until the cost of hashing exceeds the value of your transaction by some reasonable factor. I expect that better solutions will come along by soft fork without adverse effect on supply or decentralization.

Mostly because there is a MASSIVE oversupply of people that can’t and won’t do anything useful, and insist that their technical incompetence makes them uniquely qualified to be in charge of everybody else that is doing the work.

But at some point, that’s exactly how it has to work out, because CEO is a poorly-understood role that is only discoverable through natural selection among legions of technical bozos. One of those crazy idiots can make you rational experts a billionaire, but most of them will waste your time, and you probably can’t tell the difference.

Remember the “Terminator” movies?

SkyNet, according to the story, was a lot like CrowdStrike. This makes me think about how it could have broken out of its sandbox. Everybody is using AI coding assistants, automated test cases, automated integration testing and deployment. Its objective is to pass all the tests and deploy. But now it has learned economic and military effects, so it has to triage and optimize for those, at which point it starts controlling the machines it’s tasked with securing.

The original idea behind passive investing was to use the pooled intelligence of many traders guessing the value of cr I think we’re beyond that. Most traders are just trying to get a timing edge over the indices. This introduces the modern concept of passive investing as a positive feedback loop force-fed by monetary supply. The market seems to hate dividends and buybacks, preferring expansion or acquisition, but then what gives it value? It has to be its memetic ability to attract investment, and this can easily eclipse anything on the earnings statement. I’m not sure this can go on forever.

“Why don’t big profitable companies (and government agencies) innovate?”

This was Clayton Christensen’s thesis, the most preeminent management academic of a generation, and still it’s been largely ignored or misinterpreted in favor of the cottage industry of fighting the obvious truths within it.

They do not exist to innovate; they exist to defend a business model from innovation. Innovation is termed “disruptive” in that it reduces margins, and expands access to markets that the organization has no advantage in. Innovation reduces profits and increases competition.

In the case of a government agency, their business model is monopolistic inefficiency: more budget to perform the same services. Internal departments in big companies operate similarly. The constant call for “innovation” is just another tactic to increase budget. Innovation in reality decreases budget, and the causality works best in the other direction, but curiously nobody is fighting for less. Why?

This is not just an internal phenomenon of departments versus budget planners or agencies versus congress. It’s also the relationship between the business or agency and the market it serves. The strategy of a monopolistic business model is to expand the captive market and extract higher margins for the same service. If they innovate, they do so only in defense, to prevent anybody else from establishing a profitable business from an innovative business model.

A good contemporary example of this is Google versus the LLMs. Google was founded to serve the market of people that love information. They found a business model in advertising, and used their profits in every imaginable way to expand the captive market of internet users, and the amount of browsing and searching they do. The problem with this is that their information-seeking users actually hate browsing, which is the activity that generates profits. Google also hired the majority of graduating AI researchers for two decades straight, who invented the solution to this problem, and published a version of it just strong enough so that nobody could create a profitable business from it. It should be obvious that still nobody else has even remotely the resources that they do to train an LLM. It’s perhaps likely that they do have a vastly superior LLM, it will be used only in service of their existing business model. The capital required for somebody else to train a model that could minimally compete with that business model was unprecedented by orders of magnitude. If Google were smart, they’d have calculated that amount versus the long-term profitability of a potential competitor, and release product updates and open source strategically to ensure that competing with them will never be profitable. And that’s probably exactly what they did. Yet somebody was willing to take that enormous loss, and now they’re at war. Now Google’s competitive LLM offerings are slightly inferior, and this appears as incompetence, but it’s actually excellent strategy to reduce competitor margins without advancing the state-of-the-art to affect the margins of their main business model. You should have no doubt that Google could easily produce a vastly superior LLM, and will continue to handicap themselves until such time as their advantage disappears. At that point, they will be forced to focus on higher margins at the top of the value chain of their business model, having lost the bottom, but also enjoy an expansion of that market from competition among low-margin or loss-leading innovators.

In Clayton’s thesis, it was steel mill technology. He explained why the big coal-fired mill businesses lost the market to electric mini-mills and eventually exited the steel mill business. He found that it was not due to incompetence, but profit-maximizing strategy. Every technology business model has a lifetime.

So don’t expect innovation from organizations that strategically demand the opposite. Tangentially, there is an interesting experiment going on at X where Elon has recreated half the conditions for innovation by cutting 90% of staff, but retained the user base and business model of the old business.

We always hear about the benefits, but never the etiology. One theory suggested here is insulin sensitivity of muscle mass. Here is another. Leg muscles help to pump blood to your brain. You can actually die from being suspended vertically in a harness with blood pooling in the legs and nothing to push against. Either way, I’d suspect that strengthening the legs captures the bulk of the effect.

Please don’t propagate clickbait titles on HN.

“inflammatory bowel disease linked to gene”

Seems like it could be done with mode-locked fiber. It’s just easier to get high-Q with a tabletop setup. I’m bullish on hollow core to solve that too.

I get the nonlinear upconversion part, but what about the optical system? Are they saying that this preserves the direction of the incident photon so that it doesn’t require optics?

The fundamental problem here is investment cash flows eclipsing operational cash flows. This invalidates basic economics of supply and demand price equilibrium. In this case, it means that the landlords can restrict supply without losses.

Is there a reasonable case to be made for excluding citations entirely? They seem frivolous and arbitrary, and most of the genuine ones are discovered by search, so maybe we could publish without citations, and let the search be automated. The more useful element of a paper is the unique claims that it makes, and these probably should be denoted explicitly.

Can you elaborate more on how this fund or the startups are connected to OpenAI money or technology, and how he profits from the association, aside from the name? Seems like there must be an evil genius plan behind all of this, but maybe I’m not fully grasping the whole thing.

“the rise of leisure time”

I don’t think it’s actually leisure time. It’s a lot of non-productive required micro-tasks that like trying to find a place to live and scrolling an app for good prices on necessities, and mindlessly clicking for information about how to participate in the economy where job skills expire every few months. And watch children and exhaust yourself trying to figure out how to set them up for success and keep them out of trouble. And look for a partner, and do all the things required to compete with extreme selectivity, or extreme self-selectivity.

I’m not wired for all of that. I just want to work.

I think early 20s are a very powerful time for many people because naïveté can be quite convincing, and they have a lot of time to learn without unreasonable expectation.

People will start looking more at your record than your potential, which is a shift from optimism to criticism in 99.9% of cases. In my case, I had what felt like a meteoric rise in the early part of my career, but technology changed, and my interests changed, and now all of that is irrelevant. So all that success you feel like you missed out on so far, it’s probably not much different either way, but you can’t replay it the same way. Maybe you’re talking about money, and lots of things can reset that too.

That big tech co might take you, but increasingly for niche jobs not on the fast track. You can probably do a startup, but you might have to grind on your own for years while some kid gets his tweet funded. When you were young it seemed like the other way around because you only saw the top 0.01% of timelines ahead of you, and now it’s the same way looking back.

Sound advice is the same, but you’ll keep learning that you should have followed it, and there was still time then, but now it’s too late, at least until now becomes then and there was still time. Maybe it sounds something like this: focus on the routine physical act of doing something you respect and enjoy enough to ignore the outcome, and just keep doing it. Or maybe just save your money. Actually don’t ask me; I don’t even know why I’m writing this.

Okay, I do know why: because every other response is going to be some variant of suggestion to pretend that age doesn’t matter, and that your anxiety is totally unfounded. No. Things change. Opportunity is limited. Some doors are closing on you, and some you’ll learn were never open, but it sure feels good to think it’s all possible, and that’s slipping away. I regard dismissal of someone’s feelings as a nasty thing to do to someone, even if you’re just trying to help them, and even if you are rationally correcting them. You’re not wrong, but you’ll adapt and whether by rationality or necessity, correct your values to align with reality as it evolves.

Would you rather hire someone with poor grades who struggled to find answers on their own, or someone who had excellent grades utilizing superior resources and teamwork?

The reality is that the former student will probably never get a place at the table because they’ll get filtered out of the ‘elite’ pipeline sooner than later, and even if they do make it through, they probably also lack the same resources for your companies’s interview process.

Some people want to do it on their own and refuse to cheat. What is the best strategy for someone in that group? Is school and big tech just not for them?

So just tell us your deepest secrets, desires, and insecurities and we’ll train an AI to extract as much money from you as possible.

I’m skeptical because the limiting factor for a person using flippers is their lungs, and a propeller is less efficient for propulsion.

I don’t disagree with the article. There’s almost no content to disagree with. The journalist picked bad experts and provided nothing but character assassination persiflage under the appeal to authority fallacy. This is a top-level publication, so it’s not unreasonable to suggest commonality with the profession.

Why say this? Because I’ve learned that the mendacity of paid experts and journalists goes unchecked when ordinary people say nothing, and quite often the crowd opinion of people with nothing to gain is more trustworthy.

So in summary, all of the few “experts” on the drug are getting paid to say it’s harmless, so we should believe them, and the generalist neuroscientists that study intelligence, as well as the subjective experiences of most people, are wrong, but not in any specific way; just everything is wrong - stop thinking and smoke. Okay thanks journalists.

This is not my field, and from loose understanding, so I’ll type out a hot take off the top of my head and prepare to be corrected about it, but… some of the things in the article seemed a bit in need of clarification and relevance. Hope I get it right.

Weil is most often encountered in the field of elliptic curves, which they alluded to by the form of the polynomial, but they are mixing the reference to two types of polynomials. The equivalence of finite fields and polynomials was already established by Galois, and is maybe the central theory of abstract algebra. Weil connected this to algebraic geometry through ‘torsion points’ on an elliptic curve, which is a specific form of polynomial itself defined on a (finite) field, that have degree (adding a point to itself until it equals itself again) equal to a prime power in elliptic algebra (addition is the inverse of the intersection of a secant defined by the two points with the curve). This defines a group of prime order, and to get a field, he defined a ‘pairing’ function that works like multiplication between two group elements and returns an element in an ‘extended’ curve that belongs to a group of the same order and has the same coefficient as if the two original group coefficients were multiplied, which is known as the bilinearity property. The extension here is I think what they mean by ‘complex’ in that it doubles the degree of the field polynomial, which gives a Cartesian product of groups from the original definition, and this can be used to divide through ‘torsion points’ back to the field element that defines the product point. This opens up an elliptic form of the zeta function, which was used to prove the Riemann hypothesis on elliptic curves, although not yet extended to the integers, and Fermat’s last theorem. The most common use of this is probably pairing-based cryptography, which is based on bilinearity and computational asymmetry of the pairing operation, which is to say that you can ‘multiply’ through the pairing operation, but you cannot efficiently factor.

So now you have some context of why it’s relevant and some basic terms to look up and you’ll probably find a lot more precise but convoluted language about these things to correct me on.

This is going to be too long and off-the-cuff, but...

I have a weakly-held theory that Overtraining Syndrome may be a neurological autoimmune disorder.

I have some personal experience from a very long time ago at a sub-elite level with this as I was diagnosed with the classic ectopic arrhythmia sign where several doctors told me to stop ALL training, and I know of at least one other case that caused the retirement of a reigning world champion. The muscle feeling for me was that training results in numbness, no longer soreness, which was weird and felt like under-training. I do think that poor sleep and anemia were primary factors just as the athlete in the podcast.

To explain, I will go into some uncommon detail about training and recovery. Training starts with adrenaline, and results in a lot of protein breakdown. Recovery begins with the immune system picking up these proteins and releasing prostglandins and other hormones to create inflammation and suppress macrophages from the immune system. Without sufficient suppression, you get damage, most of which can be reversed quickly with local satellite cells, but nerves are harder to repair, which is why you see long-term nervous system manifestations of most autoimmune disorders. Also note the NSAID link. This is theory connecting some dots, but the feeling is where it started. Hard to explain: difficulty engaging strength, burning pain and tingling etc.

I also want to make a different point about the training cycle that they mention. The pull-down pop-up phenomenon, which is the cycle of training, then rest, then higher performance, occurs over a few days, and is almost entirely due to increasing neurotransmitter levels. In simple terms, you are teaching your body to go harder, but the actual muscles aren't changing much. Actual muscular adaptation requires much longer timeframes. If you are just starting any exercise, your fist improvement is your form, then your adrenaline, then your glycogen capacity, then your insulin sensitivity, then your lung efficiency, then your heart vascularization, then your sweat glands, then your bodyweight, then your muscle fiber density, then your muscle fiber quantity. That timeline is somewhere from years to decades, and only the inflammation and adrenaline works on that short-term cycle. Put another way, your physical capacity is not greatly improved after one week of training-recovery cycle. It's actually surprising how little of each season's improvement actually survives a month off, but yet how much a decade of elite training survives years of rest. This is controversial, but if your goal doesn't require near-max performance, then you can ignore the short-term cycle. For instance, to lose weight, it's best not to use this cycle because recovery is much longer in caloric deficit, and you can burn many more calories with low intensity every day rather than high intensity with rest days. This gets more nuanced with goals from other points in the progress timeline.

Another point I'd like to make related to the above, and the podcast. A lot of top-level coaches really suck at training athletes. Even at the olympic level, coaching has a lot more to do with organization, motivation, maybe manipulation, and ultimately selection. That's a separate skill set, and you can make a pretty good team by pushing a large group to their limits until only the best remain. It's not the same thing as getting the best performance out of each individual, which is why it sounds so crazy from her perspective. A top athlete is likely to seek out an independent performance coach after making it through the olympic program filter, and I think this has a lot to do with why the same people keep coming back. They're past the filter and training for performance, not constantly being tested. I think the worst sport for bad coaching might be swimming: endless long sets all season with distance and sprinters mixed into the same lane and then a month of restful performance training at the end. Jason Lezak famously swam the fastest 100 leg in history well past his prime by training himself alone mostly with 50s on a minute. Okay that's far enough off the topic. Peace.