marsten

A lot of people believe in Musk and will invest in anything he wants to do. Is this rational? It depends on how it plays out.

I think we may be seeing a new type of capitalism that maybe Steve Jobs and Warren Buffet hinted at: A business empire built around the outsized ambitions of a single charismatic individual. The valuations of Tesla and SpaceX only make sense if you attach an enormous premium to Musk the individual.

Second-highest P/E is Tesla, currently at 97.

Apart from Palantir and Tesla, the other big companies are trading at what would historically be considered reasonable P/Es given their growth rates and profitability.

What's really changed in the last 20-30 years is the incredible profit generated by the tech industry, and the defensive moats the biggest companies have built.

Japan is a classic example. You can drop your wallet there and someone will send it to you with all the cash intact. It makes you realize how much overhead it causes when you need to guard against cheaters and thieves.

I was an undergrad at Caltech in the late 80s and likewise it never even occurred to us to cheat on take-home exams. Maybe things have changed.

People did plenty of collaboration on homework sets. Some of the harder ones were almost impossible unless you did, like those 20 page Phys98 homework sets...

Academic integrity committees at prestigious schools are horribly lax. They want these types of issues to go away quietly.

I have a friend who in college had another student take his test from the "complete" pile, erase my friend's name, and put on his own instead. It was only through blind luck that my friend figured it out. He, the TA, and the professor reported it – with smoking gun proof – but nothing happened.

The same laxness applies to academic research integrity. Universities rarely punish academics who are discovered to falsify data.

This is exactly right. Gone are the days when you could get a C+ average at Harvard and still land a good job or a spot in a prestigious law program – purely by virtue of having gone to Harvard.

Everyone is in competition now. Everyone has to prove their worth, all the time. It's more egalitarian but it also creates a lot of stress.

If you posit that Altman suffers from main character syndrome (as many CEOs do), then he likely believes that he alone can lead OpenAI to success. In this case, doing whatever it takes to get himself back into the job is by definition justified. It's obviously worth stepping on a few toes if the success of the company is at stake.

Anthropic asking hypothetical questions in an interview doesn't seem like a very good signal. Everybody knows what they're supposed to say. If they want an unfakeable signal they should make offers with no equity component.

CEOs are hired to run companies and make themselves, and their investors, wealthy. That is their prime directive and CEOs are the ultimate partisans.

If a CEO feels that bending the truth, or outright lying, will advance the prime directive – then that is what they will do. Applying adjectives like "honest" or "untrustworthy" to them is a category error. Altman will say whatever benefits OpenAI, full stop. Musk will say whatever benefits his interests, full stop.

CEOs can't be good or bad people in a moral sense, or have the best interests of society at heart. (Despite what they may try to convey.) Better to think of them as automatons carrying out well-defined, and ultimately simple, goals.

You cannot be sure that anyone other than yourself is conscious. It is only basic human empathy that allows people to believe that.

In Bayesian terms what makes it reasonable to ascribe consciousness to other people is that (a) other people have an origin that is objectively very similar to your own (genetic origin, embryonic development, birth, gradual acculturation, education, etc.), and (b) you have a firsthand experience of your OWN consciousness.

It would be remarkable if the very small differences (relatively speaking) between you and other people were enough to destroy the experience of consciousness.

Generalizing, the farther you stray from a "common origin story", the more a leap of faith consciousness becomes. Needless to say an LLM is quite a different thing than a human being.

Judging from online reactions to robot testing (e.g., engineers kicking the Spot robot "dog" to test its balance), we humans trigger on some fairly superficial cues when deciding how much to empathize. People express more sympathy for the robot dog than they do for the chicken they ate for lunch – despite the fact that the chicken has a far better claim to consciousness than the robot.

This then is how I interpret "do not anthropomorphize": We should try to ignore the superficial cues when judging the similarity of other beings to ourselves.

Bitcoin attempted to replace cash, but failed because the transaction costs are orders of magnitude too high. The high cost of zero-trust makes it a desirable medium of exchange only for criminals and scammers.

In an effort to make Bitcoin a reasonable medium of exchange, various businesses arose to act as intermediaries/market makers. But this violates the trust-free model – and many of those intermediaries have proven to be outright scams. It turns out that trusting an intermediary to handle your cryptographically untraceable asset is not a wise thing to do.

So that leaves Bitcoin in a similar category as gold. You're either a paranoid type for whom the high cost of holding and transacting the asset is a price you're willing to pay for an asset that could survive a global meltdown. OR you extend trust to various intermediaries (gold ETFs, bitcoin ETFs for example) and treat it as just another tradable financial asset.

Bitcoin is undeniably the cleverest way anyone ever became a billionaire. Nakamoto's sole contribution was posting an anonymous 9 page whitepaper to the internet and voila, today he (or she, or it) is worth $80+ billion.

I can see Apple's position: The Mac has been through four major ISAs and the value to most users of maintaining that entire stack indefinitely in the current macOS is nil.

A number of digital preservation standards are emerging (Wasm, MAME/MESS, EaaSI, Olive, ...) and I would like to see a legal requirement on OS vendors that platforms that are no longer supported must be made available to archivists.

More likely they would buy the assets of AI companies for pennies on the dollar. There could be a lot of H100s floating around at fire sale prices. Or they would acquire these companies for talent.

Google did this for several years in the early 2000s – snapping up talent and data center capacity from the casualties of the dotcom bust.

Unless they change their tack, Apple is unlikely to go head to head against Microsoft 365 or Google Workspace because they only target their own ecosystem (macOS/iOS). For MS and Google these Apple-owned platforms are not their primary user base.

In fairness to all concerned, the MacOS to MacOS X transition was brilliantly executed. These days we take VMs for granted, but back then it was a novel idea to run MacOS 8 as a process inside of MacOS X (the "blue box"). For most users it was seamless.

In the grand scheme it's good to invent things that replace human labor. It frees up people to do more interesting things. The goal should be to put everyone out of a job.

Agents playing the iterated prisoner's dilemma learn to cooperate. It's usually not a dominant strategy to be entirely sociopathic when other players are involved.

What hit me when I read Rama in the 1980s is how alien it all was. This is not Star Trek where the aliens speak English and look human-ish.

There's a lesson there for AI I think. We anthropomorphize AI in the media but perhaps the more realistic possibility is that AI is a fundamentally different type of intelligence that may never be fully human-like.

IMHO the EU is the best place to live if you're a rank-and-file worker, and the US is the best place to live if you're ambitious.

EU integration brings some advantages but it also becomes harder to experiment. Ideally you'd have a few member states vying to become the Shenzhen of Europe but that won't happen under EU integration.

Well said. Another factor that nobody in the EU likes to talk about is regulations like worker protections that make it hard to do layoffs. Such regulations are popular but they strongly favor large predictable companies over startups.

No economy has both: (1) a predictable investment and work environment, and (2) a vibrant technology sector. You make your choices and you live with them.

Yes, and housing is priced by competitive auction so if you drop out of the rat race and other people don't, you'll just get out-bid.

As we approach this pole running off to infinity, what bit of reality will intervene? An infinity in a model indicates you're missing some aspect of saturation or friction that will act to slow things down. Every exponential eventually becomes an s-curve.

Data center space? Electrical power? The amount of training data available? Society's capacity to accept rapid change?

Yes but the thing is, most people don't actually want realistic movement. They want to be Neo in The Matrix, not some average schlub that gets easily winded and jumps six inches high.

Lex Fridman's interview with Todd Howard goes into this in depth.

Intel is strategically important.

Fab capability on US soil is strategically important. Intel is one of many possible routes to that.

Intel should have spun out their fab in 2009-2010 when the signs were clear: Mobile was taking off, AMD spun out their fab, Intel had missed the boat on mobile CPUs, and Apple had acquired PA Semi and was investing heavily in custom silicon.

High-end fab is a volume game and that was the time frame when Intel was still process competitive and could have competed for Apple's business (and Nvidia's, ...). But that would never happen as a division of Intel, nobody wants to send their designs to a competitor.

Biological weapons are probably the more worrisome case for AI. The equipment is less exotic than for nuclear weapon development, and more obtainable by everyday people.

I attribute the curriculum shift to something slightly different, which is the changing perception of CS as a career.

When I was in college in the late 1980s, CS was not perceived as the moneymaking career it is today. Accordingly the kids who went into CS were typically the nerds and hackers who truly loved the field.

Many kids now perceive CS as a safe, lucrative career option akin to becoming a doctor or lawyer. It attracts many students who are smart but perhaps not as intrinsically excited about the field. The universities adjusted their curricula to what these students care about: Less beautiful theory, and more practical training.

A similar thing happened in statistics. At one time it was hardcore stats nerds. Now "data science" has brought a ton more people into the field and the teaching methods have changed dramatically.

The problem isn't that time blindness is a fake issue.

The problem is that many people incorrectly self-diagnose as suffering from conditions like time blindness. Which they do for a variety of reasons: To externalize accountability for why they're late, to feel special, and so on.

A comparison is the large number of people who claim "gluten sensitivity" and maintain special diets. Now there are serious medical conditions like celiac disease that require one to avoid gluten. But the vast majority of self-diagnosed "gluten sensitives" do not have such conditions. Researchers conclude that for many of them there is no physical basis for their self-diagnosis.

Among other things this phenomenon makes it harder for people with actual conditions to be taken seriously, because there are so many impostors.

Over the years, the magic was never lost on me. However, I can never see LLMs as more than a "token prediction machine".

The "mere token prediction machine" criticism, like Pearl's "deep learning amounts to just curve fitting", is true but it also misses the point. AI in the end turns a mirror on humanity and will force us to accept that intelligence and consciousness can emerge from some pretty simple building blocks. That in some deep sense, all we are is curve fitting.

It reminds me of the lines from T.S. Eliot, “...And the end of all our exploring, Will be to arrive where we started, And know the place for the first time."

You raise a good point that this isn't a low marginal cost business like software, telecom, or (most of) the web. Efficiency will be a big advantage for companies that can achieve it, in part because it will let them scale to new AI use cases.

With the race to get new models out the door, I doubt any of these companies have done much to optimize cost so far. Google is a partial exception – they began developing the TPU ten years ago and the rest of their infrastructure has been optimized over the years to serve computationally expensive products (search, gmail, youtube, etc.).

Interesting that they converged on a memory/network architecture similar to a rack of GPUs.

- 152 cores per chip, equivalent to ~128 CUDA cores per SM

- per-chip SRAM (20 MB) equivalent to SM high-speed shared memory

- per-board DRAM (96 GB across 48 chips) equivalent to GPU global memory

- boards networked together with something akin to NVLink

I wonder if they use HBM for the DRAM, or do anything like coalescing memory accesses.