gmt2027

By "capitalist," I'm referring to investors whose primary contribution is capital, not making a political statement about capitalism itself.

Capital is crucial when tools and infrastructure are expensive. Consider publishing: pre-internet, starting a newspaper required massive investment in printing presses, materials, staff, and distribution networks. The web reduced these costs dramatically, allowing established media to cut expenses and focus on content creation. However, this also opened the door for bloggers and digital news startups to compete effectively without the traditional capital requirements. Many legacy media companies are losing this battle.

Unless AI systems remain prohibitively expensive (which seems unlikely given current trends), large corporations will face a similar disruption. When the tools of production become accessible to individuals and small teams, the traditional advantage of having deep pockets diminishes significantly.

The typical AI economic discussion always focuses on job loss, but that's only half the story. We won't just have corporations firing everyone while AI does all the work - who would buy their products then?

The disruption goes both ways. When AI slashes production costs by 10-100x, what's the value proposition of traditional capital? If you don't need to organize large teams or manage complex operations, the advantage of "being a capitalist" diminishes rapidly.

I'm betting on the rise of independents and small teams. The idea that your local doctor or carpenter needs VC funding or an IPO was always ridiculous. Large corps primarily exist to organize labor and reduce transaction costs.

The interesting question: when both executives and frontline workers have access to the same AI tools, who wins? The manager with an MBA or the person with practical skills and domain expertise? My money's on the latter.

AI is an existential threat to tech companies not software engineers.

In many domains, the scope and complexity of software systems goes beyond the ability of a single software engineer to manage. A coordination layer becomes necessary when the number of engineers required goes beyond a threshold (say 5 or so). When the development effort must be coordinated over extended periods (say several months or years), mechanisms to raise capital and manage risk become necessary. These functions are why companies exist.

Consider that a massive increase in software engineer productivity will make coordination unnecessary for many kinds of software. In the market that opens up, companies with expensive executives, middle management and coordination inefficiencies will not be competitive. Smaller shops with a solo engineer or a team of less than 5 will outcompete larger players because their costs will be significantly lower. Massive one-size-fits-all products will be harder to justify when a small dev shop can quickly build or customise software for the unique requirements of a business or niche.

Before the CEOs stop needing engineers, engineers will stop needing CEOs and managers to coordinate their efforts and raise capital.

If AI increases the productivity of a single engineer between 10-100x over the next decade, there will be a seismic shift in the industry and the tech giants will not walk away unscathed.

There are coordination costs to organising large amounts of labour. Costs that scale non-linearly as massive inefficiencies are introduced. This ability to scale, provide capital and defer profitability is a moat for big tech and the silicon valley model.

If a team of 10 engineers become as productive as a team of 100-1000 today, they will get serious leverage to build products and start companies in domains and niches that are not currently profitable because the middle managers, C-Suite, offices and lawyers are expensive coordination overhead. It is also easier to assemble a team of 10 exceptional and motivated partners than 1000 employees and managers.

Another way to think about it is what happens when every engineer can marshal the AI equivalent of $10-100m dollars of labour?

My optimistic take is that the profession will reach maturity when we become aware of the shift in the balance of power. There will be more solo engineers and we will see the emergence of software practices like the ones doctors, lawyers and accountants operate.

You're thinking of Grok, the model from xAI. This Groq is the inference hardware company with a cloud service.

The government is getting rid of tax exemptions for non-domiciled individuals who up to this point did not need to pay tax on foreign source income that was not brought into the UK. The economy overall is in a dire state, Labour are projected to win the election this week in a landslide. The rich are worried about further tax raids from a left wing government.

In the UK, the loss of subsidised childcare and tax free allowances creates an effective marginal tax rate of over 100% for a parent earning £100,000-£125,000.

Already sold. Just waiting for updates on production-readiness and Android support.

This isn't quite fair. The problem is that JavaScript expertise is still volatile. It is a use it or lose it skill. It quickly becomes alien unless you stay on the threadmill and keep up with the developments, patterns and tooling. I admit syntax is not actually difficult to relearn but patterns take time to acquire.

I've been programming for over 20 years and the languages and frameworks I favour have a sticky quality. These are the ones I can put away for years, return, get back into it and pick up new features in a few hours at most. The cognitive load of getting back into them are low.

I picked up django around 2011 and moved on to other things including JavaScript at the time. When I needed to build a Django project in 2016, there were some improvements and new features but the syntax, tooling and patterns were the same. It was possible to just jump right back in after a quick skim of the docs. It is possible to develop mastery of this stack, set it down for a few years and return to it with familiarity and continue on that journey.

The JavaScript ecosystem in that period however has pulled off a massive amount of transformation. I learned the basics of jQuery, yui, knockout, emberjs, backbone, coffeescript, ecmascript X, knockout, baconjs, rxjs, react with classes, react with hooks, redux, typescript, nextjs, react server components, react native, graphql, assemblyscript and on and on. It has been impossible for me to develop any deep mastery of anything in the JavaScript space. This does not even include the insane tooling with vague overlapping responsibilities. The closest I have found to reasonable stability in this area has been clojurescript with reagent.

I am constantly stuck in tutorial mode when I need to dive back into the frontend. I recently upgraded a 4 year old Django codebase in 2 hours. I am ripping out all of the react code because it is just not worth the effort any more. The tooling, dependencies and patterns are now outdated. HTMX has that simplicity and elegance that will keep the cognitive load manageable while I focus on the real problems.

This is exactly right. The lakehouse is a custom data warehouse you can build out of these cloud primitives to suit the specific data needs of an organisation. Think of it as a database scaled up by several orders of magnitude. Everything from storage costs to latency can be optimised as design choices. The common core in this architecture is data held in standard file formats such as parquet, delta tables, avro etc.

One of the most powerful ways to integrate LLMs with existing systems is constrained generation. Libraries such as outlines[1] and instructor[2] allow structural specification of the expected outputs as regex patterns, simple types, jsonschema or pydantic models. Llama.cpp supports bnf grammars.

These outputs often consume significantly fewer tokens than chat or text completions.

[1] https://github.com/outlines-dev/outlines

[2] https://github.com/jxnl/instructor

https://nitter.net/timsneath/status/1727852099407622163

There are well described links between intelligence and information theory. Intelligence is connected to prediction and compression as measures of understanding.

Intelligence has nothing specific to do with The Universe as we known it. Any universe will do, a simulation, images or a set of possible tokens. The universe is every possible input. The training set is a sampling drawn from the universe. LLMs compress this sampling and learn the processes and patterns behind it so well that they can predict what should come next without any direct experience of our world.

All machine learning models and neural networks are pure functions. Arguing that no function can have intelligence as a property is equivalent to claiming that artificial intelligence is impossible.

We have an algorithm and computational hardware that will tune a universal function approximator to fit any dataset with emergent intelligence as it discovers abstractions, patterns, features and hierarchies.

So far, we have not yet found hard limits that cannot be overcome by scaling the number of model parameters, increasing the size and quality of training data or, very infrequently, adopting a new architecture.

The number of model parameters required to achieve a defined level of intelligence is a function of the architecture and training data. The important question is, what is N, the number of model parameters at which we cross an intelligence threshold and it becomes theoretically possible to solve mathematics problems at a research level for an optimal architecture that we may not yet have discovered. Our understanding does not extend to the level where we can predict N but I doubt that anyone still believes that it is infinity after seeing what GPT4 can do.

This claim here is essentially a discovery that N may be much closer to where we are with today's largest models. Researchers at the absolute frontier are more likely to be able to gauge how close they are to a breakthrough of that magnitude from how quickly they are blowing past less impressive milestones like grade school math.

My intuition is that we are in a suboptimal part of the search space and it is theoretically possible to achieve GPT4 level intelligence with a model that is orders of magnitude smaller. This could happen when we figure out how to separate the reasoning from the factual knowledge encoded in the model.

A strategic play is to release the models - the nuclear option. Go full OpenAI. Even if Microsoft has an exclusive license, leaks and breaches happen.

The board should have led with this as an implied threat behind closed doors and given both Altman and Nadella a face-saving public retreat. Altman goes off to start GPTSoft with equity, a license to GPT4 and access to Microsoft's cloud compute. Everyone gets to call this a win. Public humiliation was a mistake.

https://nitter.net/jeremyphoward/status/1725712220955586899

GPT4 is good at sketching out stories like this.

https://chat.openai.com/share/3143a927-a283-4abd-aae3-1f0542...

Absolutely agree with this. Real skin in the game would require CEOs and executives to face ruin in the event of failure.

Pilots have skin in the game. There is rarely any doubt that their priority is landing that plane safely. Now imagine a world where airline pilots are able to safely eject with a golden parachute and minimal damage to their professional reputation. What are the chances of achieving the same safety record?

Are you saying that the threat of civil legal action is an effective deterrent against anonymous actors that conspire to overthrow governments?

I do not believe the line is so clear cut for a recognisable face. Brands seek out these celebrities and pay a premium for the aura. The endorsement is implicit.

Consider that the relationship works both ways, the celebrity's own brand is tarnished by involvement with a product hit by scandal. See the celebrities that promoted FTX as an example. It is in their own interest to do some due diligence.

Conversely, the ads would be dropped if the celebrity actor became toxic. If Kelly Rowland made Kanye-style comments about Nazis or Jews, the ad would be buried. Are Kevin Spacey ads still airing?

The entire celebrity/influencer marketing system intentionally exploits a failure mode of human cognitive bias for profit. Transactional endorsements are morally bankrupt. A family member or friend that blindly recommended a product or service for pay would lose respect for that breach of trust. Celebrities that people respect should be held to the same or higher standards.

Consider this, if John Carmack regularly did ads playing a generic software engineer at BigCo and it turned out he didn't even know what the products were, would this community hold him in the same high regard?

Any celebrity should familiarise themselves with the companies and products they endorse. If all they can manage is to do so at a superficial level, they should seriously consider declining. It is just amazing that society rewards such a lack of integrity.

150 is quite generous if you ask me.

Flexbox Froggy is great. Thanks for the recommendation.

There's an argument against service layers in Django. The recommendation is to use instance and manager methods.

https://www.b-list.org/weblog/2020/mar/16/no-service/

I'm really struggling. All of the industry is now driven by greed. The transition to agile micromanagement happened when we became a means to an end - making money. Whatever happened to building cool things and making enough money while doing it to stay in business?

There's an undeniable existential crisis brewing within the tech industry. It's concerning to see how the pursuit of monetary gain seems to have eclipsed other, arguably more essential, objectives. Life's raison d'etre, after all, is not about amassing a surplus of glucose or oxygen.

It's staggering how numerous high-profile tech firms are sitting on gargantuan cash reserves, with seemingly no coherent strategy on how to allocate these resources. In the meantime, employees bear the brunt through layoffs, while innovative competition gets steamrolled under the guise of threats.

I'm of the belief that our technological endeavors should fundamentally aim to enhance the human experience in a long-lasting, sustainable manner. A deliberate and well-orchestrated transition to more efficient and equitable ways of operation should be the industry's primary focus, not just an afterthought.

US investors are not doing this out of generosity. European elites built up their wealth with land holdings and real estate over centuries. In most cases, the value of these assets are unknown. There is little benefit to be gained from disposing of that wealth to speculate on the latest fad and facing taxes.

The Duke of Westminster for example was the youngest billionaire in the UK with an estimated net worth of £10bn at 35. His family's land includes 100s of acres of the most valuable real estate in London. Their ancestor was granted these lands by William the Conqueror in 1066.

Text is an instance of language. Think of it as the difference between the python language and a large collection of python programs. The language describes syntactic and semantic rules, the collection is a sampling of possible programs that encodes a significant amount of information about the world. You could learn a lot about the laws of nature, the internet, even human society and laws by examining all the python programs ever written.

An extreme version of the same idea is the difference between understanding DNA vs the genome of every individual organism that has lived on earth. The species record encodes a ton of information about the laws of nature, the composition and history of our planet. You could deduce physical laws and constants from looking at this information, wars and natural disasters, economic performance, historical natural boundaries, the industrial revolution and a lot more.

I completely agree about the colonial mindset and an inherent bias towards self-perpetuating elitism in the UK.

Christian Bale, Damian Lewis, Millie Bobbie Brown, Idris Elba, Henry Cavill, Tom Holland, Dominic West are all actors people are surprised to learn are British.