HN user
“Man, what are you talking about? Me in chains? You may fetter my leg but my will, not even Zeus himself can overpower.”
― Epictetus
Factorization is key here. It separates dataset-level structure from observation-level computation so the model doesn't waste capacity rediscovering structure.
I've been arguing the same for code generation. LLMs flatten parse trees into token sequences, then burn compute reconstructing hierarchy as hidden states. Graph transformers could be a good solution for both: https://manidoraisamy.com/ai-mother-tongue.html
From the paper: "sleep insufficiency was significantly associated with lower life expectancy when controlling for traditional predictors of mortality, with only smoking displaying a stronger association."
Both the US and China rely on uranium-based fission, but they seem to be diverging on their next bet. China is exploring thorium based fission, while US is leaning toward fusion [1]
[1] Trump Media’s merger with the fusion company TAE - https://www.globenewswire.com/news-release/2025/12/18/320754...
University of Florida scientists use light to perform AI image convolutions with near-zero energy, making AI 10–100× more power-efficient. [1]
Tsinghua scientists use light to do AI math extremely fast (12.5 GHz, trillionths of a second), enabling real-time decisions like trading or robotics. [1]
[1] https://scitechdaily.com/this-chip-computes-with-light-break...
Hi HN,
Ever since "Attention Is All You Need", I've been reading research papers directly instead of waiting for tech news coverage. My information supply chain has evolved from news sites as explainer to following experts on Twitter to ChatGPT these days. I'm experimenting with one more step: what if the papers themselves were memes?
For example, mapping AlexNet's 50-year journey to the Pirates of the Caribbean sinking ship scene [1]. Or using Sheldon's milking stool argument to explain transformer architecture [2]. The absurdity seems to make the concepts more memorable. Each meme has a quiz to dig deeper into the paper.
What do you think? Is humor a legitimate tool for learning about research papers, or does it undermine the seriousness of the work?
Me too, but it gets harder during winter. High intensity yoga in this paper seems interesting. I’m planning to try that over the next few months.
Three years ago, when we started making profit as a bootstrapped startup, I was stunned by how little money I could reinvest in my own company compared to a funded competitor. We paid ourselves 20% of the profit and paid 40% in taxes and reinvested the rest i.e. 40% into our business. Meanwhile, a VC-backed competitor could show losses and invest 100% of the revenue plus the $10 million or $50 million they raised from investors. In this essay, I explain you how screwed up the incentives are for bootstrapped companies and a solution to fix this:
Think Stripe Atlas for bootstrapped companies, a service that incorporates you in countries with favorable tax treatment for reinvestment, where you only pay taxes on founder distributions, not on profits you reinvest.
Hi HN,
OP here. In my previous post [1], I argued that code generation is the kingpin behind reasoning models. The bottleneck is LLMs generate code lossily due to tokenization fragmentation and treating code like natural language instead of structured graphs. In this post I propose:
1. Parsing user prompts into input graphs (using controlled English like ACE)
2. Parsing code into output graphs (AST trees)
3. Using graph transformers to map input graphs → output graphs
Core hypothesis: eliminating tokenization fragmentation ("ladlen" → ["lad", "len"]) and preserving tree structure could improve FrontierMath accuracy from 26% to 35-40%. No benchmarks yet. Just theory and a plan to test the improvement.
I've built compilers, not transformers, so would love technical feedback on:
- Is tokenization & linear structure really the bottleneck in code generation, or am I missing bigger issues?
- Is 35-40% improvement plausible, or overly optimistic?
- For those working on graph transformers: what approaches look promising?
Thanks in advance!
[1] Previous post - https://manidoraisamy.com/reasoning-not-ai.html
HN thread - https://news.ycombinator.com/item?id=45683113
Personally, I’m subscribing to multiple AI services in the hope of increasing my productivity. I don’t think I’ve ever subscribed to this many SaaS products before. My expectation is if these tools are taking a share of my spending, its coming from services I would otherwise hire contractors or employees for.
Ah, got it! "Advanced settings" is hiding below "Browse connectors". I thought its available only for some select companies and wanted to find it out here. Thanks!
I’ve updated my article (replaced GPT-5 with ChatGPT-5 in this section):
When you ask the latest model, ChatGPT-5 to multiply two large numbers, it doesn't calculate. It generates Python code, executes it in a sandbox, and returns the result. Unlike ChatGPT-3, which at least attempted arithmetic internally (and often failed), ChatGPT-5 delegates computation to external tools. [1]
And added this note:
[1] There are 2 ways to multiply numbers in GPT-5:
- Python mode, which uses python sandbox as mentioned above
- No tool mode, which uses internal reasoning
Python mode is approximately 2x more accurate than no tool mode in FrontierMath (26.3% vs 13.5% accuracy on expert level math). Python mode is also 4x to 10x more cost effective than no tool mode. The GPT-5 API uses no-tool mode by default (tools must be explicitly enabled in API calls), while ChatGPT UI likely uses Python mode by default since Advanced Data Analysis is enabled by default for all Plus, Team, and Enterprise subscribers. This creates a significant cost optimization for OpenAI in the consumer product, while API users bear the full cost of inefficient reasoning unless they manually configure tool use.
---
Thanks again for flagging the inaccuracy, Simon! If you think any part of this update still misrepresents the model behavior, I’d love your input.
Yes, my understanding is:
- finding patterns in data is memorization
- finding patterns in metadata is intelligence
- finding patterns in meta-metadata is invention
For example, if you ask someone to hang a painting in an art gallery 12 feet from the floor using a 13-foot ladder:
- a worker will use the safety rule of staying 5 feet away from the wall. This is what GPT-3 does. [1]
- an engineer will apply the Pythagorean theorem. This is what o3 does.
- Pythagoras, seeing it for the first time, will derive the theorem. GPT-5 is nowhere close to that.
This climbing up the ladder of abstraction existed even before LLMs. DeepMind's AlphaGo learned from human games. But AlphaGo Zero and AlphaZero trained entirely through self-play and began uncovering new strategies across Go, chess, and shogi. So whether it's code, a game, or pseudocode, they're all metadata operating at the same level of abstraction.
[1] The Nature of Intelligence is Meta - https://manidoraisamy.com/intelligence-is-meta.html
wow, tool use seems to reduce total tokens by ~4 to 10× and cost by orders of magnitude. I wonder what the accuracy difference would be. I'm going to try multiplying larger and larger numbers to see how accuracy compares between tool use and pure reasoning.
Exactly. Both the theater script and code are metadata that manipulates entities: characters in a play or variables in memory. There's definitely abstract-level understanding emerging: that's why models can be trained on python, but write code in java. That could be instructions like pseudo-code or the hidden document/theater script you mentioned. That capability jump from GPT3 to o1 is real. But my point is: pure metadata manipulation has hit a ceiling or is moving at a crawling pace since o1. The breakthrough applications (like agentic AI) still depend on the underlying model's ability to generate accurate code. When that capability plateaus, all the clever orchestration on top of it plateaus too.
That's interesting, though I wonder if what's "baked into the weights" is closer to intuition than reasoning. Once reasoning traces are distilled into weights, the model stops thinking through problems and starts pattern-matching answers. That feels more like a stochastic parrot with intuition than an analytical reasoner.
OP here. Thanks for the thoughtful reply. Curious if you’ve measured o1’s accuracy and token cost with tool use enabled vs disabled? Wondering if python sandbox gives higher accuracy and lower cost, since internal reasoning chains are longer and pricier.
[Edit] Its probably premature to argue without the above data, but if we assume tool use gives ~100% accuracy and reasoning-only ~90%, then that 10% gap might represent the loss in the probabilistic model: either from functional ambiguity in the model itself or symbolic ambiguity from tokenization?
Hi HN, OP here. I'd appreciate feedback from folks with deep model knowledge on a few technical claims in the essay. I want to make sure I'm getting the fundamentals right.
1. On o1's arithmetic handling: I claim that when o1 multiplies large numbers, it generates Python code rather than calculating internally. I don't have full transparency into o1's internals. Is this accurate?
2. On model stagnation: I argue that fundamental model capabilities (especially code generation) have plateaued, and that tool orchestration is masking this. Do folks with hands-on experience building/evaluating models agree?
3. On alternative architectures: I suggest graph transformers that preserve semantic meaning at the word level as one possible path forward. For those working on novel architectures - what approaches look promising? Are graph-based architectures, sparse attention, or hybrid systems actually being pursued seriously in research labs?
Would love to know your thoughts!
Signatures are intent proof, not biometric data. Google Docs and Formesign (with Google Forms) add unique identifier next to the signature when generating the signed document to verify signer intent and prevent copying.
Yeah, Google Sheets is a surprisingly powerful interface for business teams, especially when they’re the ones curating the data. Curious to see how Promptrepo fits into your workflow. Happy to help if you explore finetuning on top of your existing setup.
There's some truth to that. Early stage products often start out rough and part of the journey is finding early users who believe in the potential enough to stick around while we improve.
That said, in our case, the switch away from monthly plans wasn't just about churn. We actually got shut down by GoDaddy due to phishing abuse [1], which forced us to rethink our approach. We've since added a flexible trial extension to avoid punishing genuine users, but I’m open to feedback and willing to change if enough people feel strongly.
Depends on who’s using the tool. Developers might be fine coding a form into their app, but an HR person needs a form builder. Similarly, the data for training models usually lives with domain experts, but they don’t have a tool to actually do the training. That’s why, in this use case, a simple interface makes sense, IMO.
Don’t want to sound like a shill, but Google doesn’t use data from your Google Sheets to train its models or for advertising. By default, your data stays private and is protected under Google Workspace’s privacy policies.
I understand your concern. We’ve had cases where short-term users abused our product for phishing, which led us to remove the monthly plan initially. To address that without locking out genuine users, we’ve added an option to extend the trial for up to 3 months before choosing a monthly or yearly plan. Feel free to message me if you’d like more time.
Yes, it does. Updated the doc. Thanks for pointing it out! As long as you have the logprobs, it should work:
const jsonOutput = JSON.parse(response.choices[0].message.content);
const result = calculateConfidenceScores(jsonOutput, response.choices[0].logprobs.content);
Yes, it does. It recursively walks through the JSON structure, calculating a confidence score for each individual field — whether it’s a top-level key, nested inside objects, or part of an array of objects. Each leaf field gets a {value, score} pair, and parent objects get an aggregated score based on the confidence of their children.
Hi everyone, I built @promptrepo/score because we’re no longer using generative AI just for suggestions — we’re making decisions with it. But generative AI is probabilistic, not the deterministic systems we’re used to. So when AI makes decisions, we need to know how confident it is, and how much we can trust each field in the output.
This tool looks simple — it just converts OpenAI’s logprobs into field-level confidence scores — but that changes how you use AI in production. It lets you mark low-confidence fields, send them for human review, or retry with better grounding. In high-volume systems, you can also track low-confidence patterns to improve prompts or fine-tune with better data. Its a lightweight npm and has no dependencies, so its easy to integrate it into your AI workflows. Would love to hear your thoughts!
Not yet. @promptrepo/score relies on token-level logprobs, which OpenAI exposes for their models. But, Anthropic’s Claude currently doesn’t expose token-level confidence (like logprobs) in their API. So, we can’t support Claude until they do. We’d love to add support if/when Claude exposes this capability.