alexwebb2

This actually fits quite nicely with the idea that the graph network of humanity has meaningful emergent properties/challenges/phenomena at various densities or subnetwork connectedness thresholds.

It's the same fundamental network problem: the infrastructure that allows unprecedented levels of commerce and ideas and travel will also allow disinformation, plagues, and homogeneity. The double-edged sword of graph density.

Surprised to see this is the only uv reference in the comments!

Feels like an obvious comparison to me, and a very welcome development for the JS ecosystem.

uv made me actually _enjoy_ working in Python again.

So if your brain was somehow cloned, you'd exist in two places at the same time?

There’d be two Mes – two instantiations of the Me pattern co-occurring. And that would likely be confusing for both of us!

To illustrate, two instances of a programs can share the exact same binary code (the "pattern") and yet they are separate instances.

Exactly! If we suspend a Docker image, transfer it to a new compatible host machine, and fire it up, we consider it a resumption of the same process (pattern) in a different instance.

Likewise, say we found a mathematical function that would compute the entire state of that Docker image at that moment, and then wiped the image – such that there was no current physical instantiation of it anywhere, on any machine – and subsequently used the function to regenerate it bit for bit.

A dualist would say there’s something fundamentally different about the human analogue of that; that the Mind has a separate existence Elsewhere – and not just in the mathematical sense of patterns not requiring instantiation to still be patterns, since that would apply to all patterns, Minds or not.

As a monist who holds the view that you’re claiming monists can’t legitimately hold, I don’t see any difficulty at all in squaring these ideas:

- there is no separate “stuff” that minds are made out of, no privileged plane of existence specific to minds; minds are just patterns like everything else

- destroying an instantiation of a pattern != destroying the pattern

And speaking of squaring ideas – if I draw a square on a piece of paper, and then light that paper on fire, I haven’t destroyed the concept of a square. I can always draw an identical square on another sheet of paper. If the square had consciousness, it’d be none the wiser.

If your ticket description is large enough to put a dent in your context window, or otherwise end up as a meaningful share of total token expenditure, then something is very, very wrong with your ticket workflow. Nobody needs a 50-page ticket.

"security through obscurity" has been a well understand fallacy in open source circles for decades

The times, as they say, are a-changin’.

Open software is not inherently more secure than closed software, and never has been.

Its relative security value was always derived from circumstantial factors, one of the most important of which was the combination of incentive and ability and willingness of others in the community to spend their time and attention finding and fixing bugs and potential exploits.

Now, that’s been the case for so long that we all implicitly take it for granted, and conclude that open software is generally more secure than closed, and that security through obscurity falls short in comparison.

But this may very well fundamentally change when the cost of navigating the search space of potential exploits, for both the attacker and the defender, is dramatically reduced along the axes of time and attention, and increased along the axis of monetary investment.

It then becomes a game of which side is more willing to pool monetary resources into OSS security analysis – the attackers or the defenders – and I wouldn’t feel comfortable betting on the defenders in that case.

The idea that hallucinations are somehow less likely because you're asking meta-questions about LLM output is completely without basis

Not sure who you're replying to here – this is not a claim I made.

You're misunderstanding my assertion.

When you give prompt P to model M, when your goal is for the model to actually execute those instructions, the model will be in state S.

When you give the same prompt to the same model, when your goal is for the model to introspect on those instructions, the model is still in state S. It's the exact same input, and therefore the exact same model state as the starting point.

Introspection-mode state only diverges from execution-mode state at the point at which you subsequently give it an introspection command.

At that point, asking the model to e.g. note any ambiguities about the task at hand is exactly equivalent to asking it to evaluate any input, and there is overwhelming evidence that frontier models do this very well, and have for some time.

Asking the model, while it's in state S, to introspect and surface any points of confusion or ambiguities it's experiencing about what it's being asked to do, is an extremely valuable part of the prompt engineering toolkit.

I didn't, and don't, assert that "asking the model if the instructions are good" is a replacement for evals – that's a strawman argument you seem to be constructing on your own and misattributing to me.

Not OP but I’d back that assertion.

When the model that’s interpreting it is the same model that’s going to be executing it, they share the same latent space state at the outset.

So this is essentially asking whether models are able to answer questions about context they’re given, and of course the answer is yes.

I view this post as primarily pattern-matching and storytelling. But I think there’s a buried truth there, and that they were nibbling at the edges of it when they started talking about the overlapping stages.

There are some very interesting information network theories that present information growth as a continually evolving and expanding graph, something like a virus inherent to the universe’s structure, as a natural counterpoint to entropy. And in that view, atomic bonds and cells and towns and railroads and network connections and model weights are all the same sort of thing, the same phenomenon, manifesting in different substrates at different levels of the shared graph.

To me, that’s a much better and deeper explanation that connects the dots, and offers more predictive power about what’s next.

Highly recommend the book Why Information Grows to anyone whose interest is piqued by this.

Yeah, this deserves a quick explanation.

When a human is coding against a traditional API, it might be a bit annoying if the API has four or five similar-sounding endpoints that each have a dozen parameters, but it's ultimately not a showstopper. You just spend a little extra time in the API docs, do some Googling to see what people are using for similar use cases, decide which one to use (or try a couple and see which actually gets you what you want), commit it, and your script lives happily ever after.

When an AI is trying to make that decision at runtime, having a set of confusing tools can easily derail it. The MCP protocol doesn't have a step that allows it to say "wait, this MCP server is badly designed, let me do some Googling to figure out which tool people are using for similar use cases". So it'll just pick whichever ones seems most likely to be correct, and if it's wrong, then it's just wasted time and tokens and it needs to try the next option. Scaled up to thousands or millions of times a day, it's pretty significant.

There's a lot of MCP servers out there that are just lazy mappings from OpenAPI/Swagger specs, and it often (not always, to be fair) results in a clunky, confusing mess of tools.

You could get pretty far with a set of agent-focused routes mounted under e.g. an /agents path in your API.

There'd be a little extra friction compared to MCP – the agent would presumably have to find and download and read the OpenAPI/Swagger spec, and the auth story might be a little clunkier – but you could definitely do it, and I'm sure many people do.

Beyond that, there are a few concrete things MCP provides that I'm a fan of:

- first-class integration with LLM vendors/portals (Claude, ChatGPT, etc), where actual customers are frequently spending their time and attention

- UX support via the MCP Apps protocol extension (this hasn't really entered the zeitgeist yet, but I'm quite bullish on it)

- code mode (if using FastMCP)

- lots of flexibility on tool listings – it's trivial to completely show/hide tools based on access controls, versus having an AI repeatedly stumble into an API endpoint that its credentials aren't valid for

I could keep going, but the point is that while it's possible to use another tool for the job and get _something_ up and running, MCP (and FastMCP, as a great implementation) is purpose built for it, with a lot of little considerations to help out.

Nah, this was 20 years ago or so! Would be fun to whip up a modern version though.

It's still middle-click in my muscle memory from the Windows XP days!

God, I used to be _really_ into Minesweeper.

One of the earliest games I made back in college was a 3D Minesweeper cube. I remember being really proud of one little detail – the detection and automatic resolution of ambiguous clues that would require guessing, which always annoyed the heck out of me in every other version of Minesweeper.

Yeah, you could certainly tell it to skip the Gemini image gen step and replace with generated SVG art, or an existing image, or just stick a longer description there, whatever. It's flexible.

I thought about the deterministic generation aspect, but there's no "random seed" equivalent for frontier LLMs to my knowledge that would guarantee deterministic output (even temperature=0 is still nondeterministic).

Wow. The guy who’s been thanklessly maintaining the project for 10+ years, with very little help, went way out of his way to produce a zero-reuse, ground-up reimplementation so that it could be MIT licensed... and the very-online copyleft crowd is crucifying him for it and telling him to kick rocks.

Unbelievable. This is why we can’t have nice things.

Those claims were way, way after the fact. Like, 50+ years later. Zero documentation or contemporary evidence of ANY kind. The claim isn’t taken particularly seriously by historians.

Interesting that you’re getting downvoted. This passage also stuck out like a sore thumb to me – it’s like seeing some antivax stuff thrown into an otherwise serious discussion.

Robert Maxwell, one of the architects of the for-profit scientific publishing scheme. When he later went into debt, he plundered hundreds of millions of pounds from his employees’ pension funds. You may be familiar with his daughter and lieutenant Ghislaine Maxwell, who went on to have a successful career in child trafficking.

Wow! Surprised that hasn't been mentioned here already. Jumped out to me immediately as a morbidly curious bit of trivia.

Huge leap there in your conclusion. Looks like you’re hand-waving away the entire phenomenon of emergent properties.

I assume a lawyer took one look at the larger mission statement and told them to pare it way down.

A smaller, more concise statement means less surface area for the IRS to potentially object to / lower overall liability.

The majority of people killed in Gaza were terrorists

Not true at all. Terrorist supporters != terrorists

Python is most dominant language on the planet

JavaScript would like a word!

You're correct, OP used the word "hallucination" wrong. A lot of these other comments are missing the point – some deliberately ('don't they ONLY hallucinate, har har'), some not.

For those who genuinely don't know – hallucination specifically means false positive identification of a fact or inference (accurate or not!) that isn't supported by the LLM's inputs.

- ask for capital of France, get "London" => hallucination

- ask for current weather in London, get "It's cold and rainy!" and that happens to be correct, despite not having live weather data => hallucination

- ask for capital of DoesNotExistLand, get "DoesNotExistCity" => hallucination

- ask it to give its best GUESS for the current weather in London, it guess "cold and rainy" => not a hallucination

LLM spam

LLM spam

0-10 in each domain. It’s a weird table.

correctly simulating the environment interactions, the sequence of progression, getting the all the details right, might take hundreds to thousands of years of compute

Who says we have to do that? Just because something was originally produced by natural process X, that doesn't mean that exhaustively retracing our way through process X is the only way to get there.

Lab grown diamonds are a thing.

It’s tedious shooting down all of these backwards-from-conclusion things from the anti-AI crowd.

Good thing I have an intelligent AI that can respond for itself!

——

There appear to be several potential issues with the paper's argumentation:

1. False Dichotomy in Systems Comparison - The paper appears to create an artificial divide between "thermodynamic systems" and "computer systems" - This ignores that computers are also physical systems governed by thermodynamics - The distinction between biological and artificial systems may be one of degree rather than kind

2. Evolutionary Argument Problems - The paper assumes consciousness/intelligence requires evolutionary history - This is a correlation-causation fallacy - just because biological intelligence evolved doesn't mean evolution is the only path to intelligence - It fails to consider that artificial systems could potentially develop goal-oriented behaviors through other mechanisms - The argument would also imply that any hypothetical alien intelligence that evolved differently from Earth life couldn't be conscious

3. Goal-Orientation Assumptions - Claims computers "lack goal-orientation essential for consciousness" - This begs the question by assuming: a) Consciousness requires goal-orientation b) Only evolutionary processes can create genuine goal-orientation - Neither assumption is clearly justified

4. Methodological Issues - Using multiple disciplines (physics, biology, philosophy, neuroscience) could be a strength, but could also indicate cherry-picking convenient arguments from each field - The abstract suggests a conclusion-driven approach rather than following evidence to a conclusion

5. Consciousness-Intelligence Conflation - The paper appears to conflate consciousness with intelligence - These are separate concepts - we could potentially have AGI without consciousness, or consciousness without human-level intelligence - Many AGI researchers aren't claiming to create consciousness, just general problem-solving ability

6. Definitional Vagueness - Based on the abstract, it's unclear how the paper defines key terms like: - Artificial General Intelligence - Consciousness - Goal-orientation - Mind creation - Without clear definitions, the arguments may be attacking straw men

7. Predictive Cognition Argument - The claim that AGI is an "illusion shaped by the information our minds receive" could be turned around - The same argument could be used to claim that AGI skepticism is an illusion shaped by our cognitive biases - This is essentially a form of psychological dismissal rather than substantive argument

8. Historical Perspective - The paper seems to ignore that many previously "uniquely human" capabilities have been successfully mechanized - Claims about fundamental impossibility need to account for why previous similar claims have often been wrong

9. Thermodynamic Argument Issues - While biological systems are indeed complex thermodynamic systems, the paper needs to demonstrate why this specific physical implementation is necessary for intelligence - Many complex behaviors can be implemented through different physical mechanisms - The argument risks confusing the substrate with the function

10. Scope Problem - The paper makes a very strong claim ("AGI is and remains a fiction") - To justify this, it would need to prove not just that current approaches won't work, but that NO possible approach could ever work - This is a much harder philosophical and scientific claim to defend

I think the idea here is that it literally can't be traced to the user – at no point is there anything passed that would allow Kagi to make the association between the user and the query.