HN user
To be fair, TurboQuant is also a purely theoretical paper. I just wanted to get this out and share.
TurboQuant is not a purely theoretical paper. Section 4 "Experiments" (page 15) [0] has a bunch of figure based on actual GPU computations.
Yeah, for me Claude identified the phrase "this holds with probability 1 over random weight matrices since the null space has dimension"
Treating trained weights as random for the purpose of a proof is immediately discrediting for a paper to me.
the ratio remains approximately 914x over TurboQuant, with compression improving rather than degrading as context length grows.
This line from the abstract got me really suspicious. Obviously a compression scheme that incorporates the entire sequence shouldn't get worse compared to a per element one as the length increases.
It is important to note that this paper is PURELY theoretical. I couldn't find much meat on the bone from a quick skim.
The single author, Gregory Magarshak, has only published one paper on arxiv before and appears to be a professor of business / music. I don't plan to give it more of a read hoping for something of value.
The grid is not all energy use. To get the numbers on an even playing field you need to compensate for that only ~40% of energy goes through the grid.
And that leaves a 6:1 ratio assuming projections run true. It very well might be possible to get efficiency wins from the transportation sector that outweigh growth in AI.
Also, I'm not sure about your math. 4% would be 4% of the whole like in a pie chart, not 4% of the remainder after removing one slice. 4% AI, 30% transportation, 66% other. I don't know where that 40% is from.
40% is for energy use in the US in the form of electricity. It was a rough number that I pulled from my memory. It is roughly right though. Check https://www.eia.gov/energyexplained/us-energy-facts/
AI is not currently 4% of the energy market of the US. Only the grid. I should have been more clear about the ALL ENERGY vs GRID distinction.
I think it might be more emissions-efficient at generating value than AI by a factor exceeding the 7.5x energy use. Moving rocks from (place with rocks) to (place that needs rocks) continues to be just an insanely good thing for humanity.
I really made no statement on the value of doing things. Transportation is obviously very valuable. I just wanted a more fact based conversation.
Compare that to ~30% of all energy use for transportation. So approximately 40%*4% = 1.6% vs 30%. I find your correction to be more wrong that the initial statement.
And most of that new capacity will be natural gas. That increase would basically whipe out the reduction in CO2 emissions the USA has had since 2018.
Emissions in 2018 were ~5250M metric ton and in 2024 it was 4750M. That is a reduction of 10% total emissions. Without going into calculations of green electricity and such, its still safe to say AI using 10% of the grid would not completely wipe out the reduction.
[0] https://www.statista.com/statistics/183943/us-carbon-dioxide...
I agree that the movements look quite robotic (though not as much as you might expect), but I don't think any movies have depicted robots moving like that. A much more common depiction is moving only a single joint at a time.
Sharp, unsure movements, a lot of hesitation, ...
I like these particular descriptors. Another I would add is holding poses unnaturally still. While waiting for the ball, the robot holds its racket extremely consistently relative to its body even while sharply turning.
You would not. You don't normally post lots of comments. The occasional return after a long period of inactivity is not in itself suspicious.
The d suffix makes it not compile under clang. The PRs seem like mostly small changes that are clear improvements.
The criteria were laid out in 2019 [0]. It was less clear then.
If you are a "rustacean" and feel that Rust already meets the preconditions listed above, and that SQLite should be recoded in Rust, then you are welcomed and encouraged to contact the SQLite developers privately and argue your case.
It seems like the criteria are less of things the SQLite developers are claiming Rust can't do and more that they are non-negotiable properties that need to be considered before even bringing the idea of a rust version to the team.
I think it is at least arguable that Rust does not meet the requirements. And they did explicitly invite private argument if you feel differently.
0: https://web.archive.org/web/20190423143433/https://sqlite.or...
Based on Table 1: This method is actually worse than generating a random number (0-100% independent of the program) and testing if it is less than 98.8%. That would achieve a better detection rate without increasing the false positive rate.
It doesn't seem worth it to try to follow the math to see if there is something interesting.
I have seen many people try to use Claude Code and get LOTS of bugs. Show me any > 10k project you have made with it and I will put the effort in to find one bug free of charge.
Which ones of those have been achieved in your opinion?
I think the arbitrary proofs from mathematical literature is probably the most solved one. Research into IMO problems, and Lean formalization work have been pretty successful.
Then, probably reading a novel and answering questions is the next most successful.
Reliably constructing 10k bug free lines is probably the least successful. AI tends to produce more bugs than human programmers and I have yet to meet a programmer who can reliably produce less than 1 bug per 10k lines.
You did not actually address the core of my points at all.
It isn't a case of ratio it is a fundamentally different method of working hence my point of not needing all human literary output do the the equivalent of an LLM.
You can make ratios of anything. I agree that human cognition is different than LLM cognition, though I would think of it more like a phase difference than fundamentally different phenomena. Think liquid water vs steam, the density (a ratio) is vastly different and they have different harder to describe properties (surface tension, filling volume, incompressible vs compressible).
Humans provide the connections, the reasoning the thought the insights and the subsequent correlations THEN we humans try to make a good pattern matcher/ guesser (the LLM) to match those.
Yes, humans provide the training data and benchmarks for measuring LLM improvement. Somehow meaning about the world has to get trained on to have any understanding. However, humans talking about patterns in number is not how the LLMs learned this. It is very much from just seeing lots of examples and deducing (during training not inference) the pattern. The fact that a general pattern is embedded in the weights implies that some general understand of many things are baked into the model.
This common retort: most humans also makes mistakes, or most humans also do x, y, z means nothing.
It is not a retort, but some argument towards what "understanding" means. From what you have said, my guess of your definition makes "understanding" what humans do and computers are incapable of (by definition). If LLMs could out compete humans in all professional tasks, I think it would be hard to say they understand nothing. Humans are a worthwhile point of comparison and human exceptionalism can only really hold up until being surpassed.
I would also point out that some humans DO understand the properties of numbers I was referring to. In fact, I figured it out in second grade while doing lots of extra multiplication problems as punishment for being a brat.
My digital thermometer uses an algorithm to determine the temperature. ... The paper will not be thinking if that is done.
I did not say "All algorithms are thinking". The stronger version of what I was saying is "Some algorithms can think." You simply have asserted the opposite with no reasoning.
In fact at the extreme end this anthropomorphising has led to exacerbating mental health conditions and unfortunately has even led to humans killing themselves.
I do concede that anthropomorphizing can be problematic, especially if you do not have a background in CS and ML to understand beneath the hood. However, you completely skipped past my rather specific explanation of how it can be useful. On HN in particular, I do expect people to bring enough technical understanding to the table to not just treat LLMs as people.
The simplest example being that LLM's somehow function in a similar fashion to human brains. They categorically do not. I do not have most all of human literary output in my head and yet I can coherently write this sentence.
The ratio of cognition to knowledge is much higher in humans that LLMs. That is for sure. It is improving in LLMs, particularly small distillations of large models.
A lot of where the discussion gets hung up on is just words. I just used "knowledge" to mean ability to recall and recite a wide range of fasts. And "cognition" to mean the ability to generalize, notice novel patterns and execute algorithms.
They don't actually understand anything about what they output. It's just text.
In the case of number multiplication, a bunch of papers have shown that the correct algorithm for the first and last digits of the number are embedded into the model weights. I think that counts as "understanding"; most humans I have talked to do not have that understanding of numbers.
It's just an algorithm.
I am surprised so many in the HN community have so quickly taken to assuming as fact that LLM's think or reason. Even anthropomorphising LLM's to this end.
I don't think something being an algorithm means it can't reason, know or understand. I can come up with perfectly rigorous definitions of those words that wouldn't be objectionable to almost anyone from 2010, but would be passed by current LLMs.
I have found anthropomorphizing LLMs to be a reasonably practical way to leverage the human skill of empathy to predict LLM performance. Treating them solely as text predictors doesn't offer any similar prediction; it is simply too complex to fit into a human mind. Paying a lot of attention to benchmarks, papers, and personal experimentation can give you enough data to make predictions from data, but it is limited to current models, is a lot of work, and isn't much more accurate than anthropomorphization.
I would hope god can do better than 40% on a test. If you select experts from the relevant fields humans, they together would get a passing grade (70%) at least. A group of 20 humans is not godlike.
I disagree with the framing in 2.1 a lot.
> Models look god-tier on paper:
> they pass exams
> solve benchmark coding tasks
> reach crazy scores on reasoning evals
I would not say that any model shows a "crazy" score on ARC-AGI.
I broadly have seen incremental improvements in benchmarks since 2020, mostly at a level I would believe to be below average human reasoning, but above average human knowledge. No one would call GPT-3 godlike and it is quite similar to modern models in benchmarks; it is not a difference of like 1% vs 90%. I think most people would consider gpt-3 to be closer to opus 4.5 than opus 4.5 is to a human.
In this paper both the diffusion and the auto-regressive models are transformers with O(n^2) performance for long sequences. They share the "Exact KV Cache" for committed tokens.
Diffusion just allows you to spend more compute at the same time so you don't redundantly access the same memory. It can only improve speed beyond the memory bandwidth limit by committing multiple tokens each pass.
Other linear models like Mamba get away from O(n^2) effects, but type of neural architecture is orthogonal to the method of generation.
Its not clear to me this is an actual problem. I just actually googled "how to fix a Git conflict" and not a single one has multiple paragraphs describing what things are.
The first result [0] pretty much immediately drops into what commands to run. If that result is part of the problem, I fully disagree it is a problem.
[0] https://docs.github.com/en/pull-requests/collaborating-with-...
Do you actually disagree with the "minutiae was always borderline irrelevant" part or that it comes along with "making somebody money"? I pretty strongly agree with the original quote including the "possibly with software" part.
Minutiae such as tabs vs spaces and other formatting changes are pretty clearly "borderline irrelevant" and code formatters have largely solved programmers arguing about them. Exactly how to best factor your code into functions and classes is also a commonly argued but "borderline irrelevant." Arguments about "clean code" are a good example of this.
Broadly, the skills I see that LLMs make useless to have honed are the the minutiae that were already "borderline irrelevant." Knowing how to make your code performant, knowing how to make good apis that can be stable long term, in general having good taste for architecture is still very useful. In fact it is more useful now.
There is a long history of people thinking humans are special and better than animals / technology. For animals, people actually thought animals can't feel pain and did not even consider the ways in which they might be cognitively ahead of humans. Technology often follows the path from "working, but worse than a manual alternative" to "significantly better than any previous alternative" despite naysayers saying that beating the manual alternative is literally impossible.
LLMs are different from humans, but they also reason and make mistakes in the most human way of any technology I am aware of. Asking yourself the question "how would a human respond to this prompt if they had to type it out without ever going back to edit it?" seems very effective to me. Sometimes thinking about LLMs (as a model / with a focus on how they are trained) explains behavior, but the anthropomorphism seems like it is more effective at actually predicting behavior.
The first iteration vectorized with numpy is the best solution imho. The only additional optimization is using modulo 9 to give you a sum of digits mod 9; that should filter out approximately 1/9th of numbers. The digit summing is the slow part so reducing the number of values there results in a large speedup. Numpy can do that filter pretty fast as `arr = arr[arr%9==3]`
With that optimization its about 3 times faster, and all of the none numpy solutions are slower than the numpy one. In python it almost never makes sense to try to manually iterate for speed.
What "O(1)-state shuffle" could you possibly be talking about? It takes `O(nlogn)` space to store a permutation of list of length n. Any smaller and some permutations will be unrepresentable. I am very aware of this because shuffling a deck of cards correctly on a computer requires at least 200 random bits.
If the requirements are softer than "n random permutations", there might be a lot of potential solutions. It is very easy to come up with "n permutations" if you have no requirements on the randomness of them. Pick the lowest `k` such that `n < k!`, permute the first k elements leaving the rest in place, and now you have n distinct permutations storeable in `O(log(n)` (still not O(1) but close).
I know this is not really your point, but misusing `O(1)` is a huge pet peeve of mine.
Advising the opposite to humans does not make sense. 13 words is already tiny to convince someone. The choices I was thinking were restricting LLM word count and increasing human word count. The goal is specifically to make them more comparable.
The given study does not show any strength of humans over LLMs. Both goal metrics (truthful and deceptive) are better for LLMs than humans. If you are misinterpreting my advice as general advice for people not under the study's conditions, I would want to see the results of the proposed rerun before suggesting that.
However, if length of text is legitimately convincing regardless of content, I don't know why humans should avoid using that. If LLMs end up more convincing to humans than other humans simply because humans are too prideful to make their arguments longer, that seems like the worst possible future.
I'm not sure what effect you think I want. The suggestion was just to increase the "interestingness" of the study. It seems to be like the main difference between LLM and human shown was length of response. Controlling for that variable and rerunning the experiment would help show other differences.
I do think its distinctly possible that LLMs will be much less convincing due to increased hallucinations at a low word count. I also think that may have less of an effect for dishonest suggestions. Simply stating a lie confidently is relatively effective.
I would prefer advising humans to increase length rather than restricting LLMs because of the cited effects.
Sure. I am not contesting that LLMs are more persuasive in this context. That basic result comes through very clearly in the paper. Its not as clear how relevant this is to other situations though. I think its quite likely that humans given the instruction to increase word count might outperform LLMs. People are very unlikely to have practiced the specific task of giving advice on multiple choice tests whereas LLMs have likely gotten RLHF training which likely helps in this situation.
I always try to pick out as many tidbits as possible from papers that might be applicable in other situations. I think the main difference of word count may be overshadowing other insights that may be more relevant to longer form argumentation.
Based on the data in table 3, I would attribute most of the difference to length of advice. LLMs average word count (29.4) is more than double human word count (13.25). Most other measures do not have a significant ratio. "Difficult word count" would be the only other with a ratio higher than 2, but that is inherited from total word count.
I think it would be difficult to truly convince me to answer differently in a test with 14 words where 30 would have enough space to actually convey an argument.
I would be very interested to see the test rerun while limiting LLM response length or encouraging long responses from humans.
The study is testing a very specific type of "recognizing shapes"; which the title of the article calls "geometric regularity". The "background stimuli" are shapes that crows would be expected to be able to distinguish, and are used to train the crows on the task. Whereas the "probe stimuli" are the actual experiment.
As a sibling indicated baboons can not distinguish these shapes easily. Additionally, rather than a binary "crows can recognize shapes" the study shows how well crows process the shapes. One of the graphs in the paper, but not the article shows that two different crows have a similarly hard time with the rhombus.
In other studies, this same test was applied to humans to find that it is a fairly innate skill rather than developed by doing geometry in school.
I would agree the "don't blunder" and "punish opponents blunders" are harder than endgame knowledge. However, knowing the basics of endgames is actually important to closing out games. Specifically, knowing KQvK, KRvK, and the "ladder technique" is important.
Without any tactics "take free pieces" probably only gets you to around 1200 (chess.com), but if it includes knight and pawn forks, skewers, pins, and discovered attacks it can get you to 1500. Playing perfectly every game is hard though. I would recommend having more chess knowledge than just those 3 rules before you really try for 1500.
This seems like a very shallow way of thinking. "Losing all respect for the person" implies that you think this is NEVER an appropriate way to address someone. Phrasing a disagreement of opinion as a question of reasoning is often the best course of action.
In particular if a choice has been made and going back to reverse it has significant costs, it is important to not say anything like "We should not be doing this" or "You made a mistake." Unless there is a good of action to reverse course that is simply being rude for no reason. Even in the case where there is a good way to reverse a decision, I would rather ask for the reasoning that led to the decision than strongly state the decision is wrong. If I am working with someone I respect at all, I must entertain the thought that I am wrong and they made the right decision with good reasoning.
What would you say to a superior who made a decision that you disagree with, but don't think is worth reversing? My best guess is either nothing or something that more strongly asserts your belief, but I can't think of any better option than phrasing it as a question.