foundry27

I like this idea. This might be one of the more effective social pressures available for getting inference providers to fix long-standing issues. AWS Bedrock, for example, has crippling defects in its serving stack for Kimi’s K2 and K2.5 models that cause 20%-30% of attempts to emit tool calls to instead silently end the conversation (with no token output). That makes AWS effectively irrelevant as a serious inference provider for Kimi, and conveniently pushes users onto Bedrock’s significantly more expensive Anthropic models for comparable performance on agentic tasks.

I’m not sure if this is just an “on mobile” thing, but I can’t find any reference to ISO 27001 or SOC2 at that datacentres URL. Taking your word for it being there previously, this seems like a major red flag! Faking these certs is no joke, and silently removing references to that after being called out would be even more of a bad look.

@ybceo you seemed to represent this org based on your previous comments, is the parent commenter missing something here?

I get it, really I do. But do the HOAs really need financial enforcement mechanisms intended to seriously harm people, and to punish them as judge, jury and executioner? A HOA’s legal job is to maintain the common-interest property and enforce the CC&Rs. It is not a HOA’s job to extract enormous sums of money out of its members, even annoying ones. The right lever to pull to get some rich person partying at 4am and trashing the place (for example) to stop is for the HOA to file for a court injunction after repeated violations; once a judge orders “no loud music 10 pm - 7 am”, the next 4 am party will become contempt of court, which is a problem for the cops, not the HOA. Hell, 4 a.m. noise is a municipal nuisance and probably a crime; people should be calling the cops every time it happens. Individual members could even sue the owner in small-claims court for private nuisance, where judges can issue even more injunctions or award damages. All this to say, you don’t need to take people’s money to get them to stop doing bad stuff. But you do need to take people’s money to get rich, and to hurt people. This new legislation should be deeply concerning to people interested in the latter, and IMO shouldn’t really be a concern to people interested in the former.

Model cards, for the people interested in the guts: https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7...

In my mind, I’m comparing the model architecture they describe to what the leading open-weights models (Deepseek, Qwen, GLM, Kimi) have been doing. Honestly, it just seems “ok” at a technical level:

- both models use standard Grouped-Query Attention (64 query heads, 8 KV heads). The card talks about how they’ve used an older optimization from GPT3, which is alternating between banded window (sparse, 128 tokens) and fully dense attention patterns. It uses RoPE extended with YaRN (for a 131K context window). So they haven’t been taking advantage of the special-sauce Multi-head Latent Attention from Deepseek, or any of the other similar improvements over GQA.

- both models are standard MoE transformers. The 120B model (116.8B total, 5.1B active) uses 128 experts with Top-4 routing. They’re using some kind of Gated SwiGLU activation, which the card talks about as being "unconventional" because of to clamping and whatever residual connections that implies. Again, not using any of Deepseek’s “shared experts” (for general patterns) + “routed experts” (for specialization) architectural improvements, Qwen’s load-balancing strategies, etc.

- the most interesting thing IMO is probably their quantization solution. They did something to quantize >90% of the model parameters to the MXFP4 format (4.25 bits/parameter) to let the 120B model to fit on a single 80GB GPU, which is pretty cool. But we’ve also got Unsloth with their famous 1.58bit quants :)

All this to say, it seems like even though the training they did for their agentic behavior and reasoning is undoubtedly very good, they’re keeping their actual technical advancements “in their pocket”.

It’s easy to be fooled, myself included it seems :)

For context, here’s a handful of the ChatGPT cues I see.

- “wasn’t just my backup—it was my clean room for open‑source development” - “wasn’t standard AWS incompetence; this was something else entirely” - “you’re not being targeted; you’re being algorithmically categorized” - “isn’t a system failure; the architecture and promises are sound” - “This isn’t just about my account. It’s about what happens when […]” - “This wasn’t my production infrastructure […] it was my launch pad for updating other infrastructure” - “The cloud isn’t your friend. It’s a business”

I counted about THIRTY em-dashes, which any frequent generative AI user would understand to be a major tell. It’s got an average word count in each sentence of around ~11 (try to write with only 11 words in each sentence, and you’ll see why this is silly), and much of the article consists of brief, punchy sentences separated by periods or question marks, which is the classic ChatGPT prose style. For crying out loud, it even has a table with quippy one-word cell contents at the end of the article like what ChatGPT generates 9/10 times when asked for a comparison of two things.

It’s just disappointing. The author is undermining his own credibility for what would otherwise be a very real problem, and again, his real writing style when you read his actual written work is great.

Editorial comment: It’s a bit weird to see AI-written (at least partially; you can see the usual em-dashes, it’s-not-X-it’s-Y) blog posts like this detract from an author’s true writing style, which in this case I found significantly more pleasant to read. Read his first ever post, and compare it to this one and many of the other recent posts: https://www.seuros.com/blog/noflylist-how-noflylist-got-clea...

I’m not much of a conspiracy theorist, but I could imagine a blog post almost identical to this one being generated in response to a prompt like “write a first-person narrative about: a cloud provider abruptly deleting a decade-old account and all associated data without warning. Include a plot twist”.

I literally cannot tell if this story is something that really happened or not. It scares me a little, because if this was a real problem and I was in the author’s shoes, I would want people to believe me.

I started doing some experimentation with this new Deep Think agent, and after five prompts I reached my daily usage limit. For $250 USD/mo that’s what you’ll be getting folks.

It’s just bizarrely uncompetitive with o3-pro and Grok 4 Heavy. Anecdotally (from my experience) this was the one feature that enthusiasts in the AI community were interested in to justify the exorbitant price of Google’s Ultra subscription. I find it astonishing that the same company providing free usage of their top models to everybody via AI Studio is nickel-and-diming their actual customers like that.

Performance-wise. So far, I couldn’t even tell. I provided it with a challenging organizational problem that my business was facing, with the relevant context, and it proposed a lucid and well-thought-out solution that was consistent with our internal discussions on the matter. But o3 came to an equally effective conclusion for a fraction of the cost, even if it was less “cohesive” of a report. I guess I’ll have to wait until tomorrow to learn more.

I couldn’t even make it past the onboarding! There didn’t seem to be an option for the single most common pay schedule, bi-weekly

Did you just use a LLM to write this reply?

I find it amusingly ironic how one comment under yours is pointing out that there’s a mistake in the model output, and the other comment under yours trusts that it’s correct but says that it isn’t “real reasoning” anyways because it knows the algorithm. There’s probably something about moving goalposts to be said here

What kind of focus do biopharma companies put on their stock prices? If a company like the one you described had a great treatment option that could genuinely help people and was raking in money by the boatload, is that “enough” for them as a “winning” business strategy regardless of how outside investors might perceive it?

I find the situation the big LLM players find themselves in quite ironic. Sam Altman promised (edit: under duress, from a twitter poll gone wrong) to release an open source model at the level of o3-mini to catch up to the perceived OSS supremacy of Deepseek/Qwen. Now Qwen3’s release makes a model that’s “only” equivalent to o3-mini effectively dead on arrival, both socially and economically.

I just tried the same puzzle in o3 using the same image input, but tweaked the prompt to say “don’t use the search tool”. Very similar results!

It spent the first few minutes analyzing the image and cross-checking various slices of the image to make sure it understood the problem. Then it spent the next 6-7 minutes trying to work through various angles to the problem analytically. It decided this was likely a mate-in-two (part of the training data?), but went down the path that the key to solving the problem would be to convert the position to something more easily solvable first. At that point it started trying to pip install all sorts of chess-related packages, and when it couldn’t get that to work it started writing a simple chess solver in Python by hand (which didn’t work either). At one point it thought the script had found a mate-in-six that turned out to be due to a script bug, but I found it impressive that it didn’t just trust the script’s output - instead it analyzed the proposed solution and determined the nature of the bug in the script that caused it. Then it gave up and tried analyzing a bit more for five more minutes, at which point the thinking got cut off and displayed an internal error.

15 minutes total, didn’t solve the problem, but fascinating! There were several points where if the model were more “intelligent”, I absolutely could see it reasoning it out following the same steps.

tl;dr for anyone who may be put off by the article length:

OP built an arena allocator in Go using unsafe to speed allocator operations up, especially for cases when you're allocating a bunch of stuff that you know lives and dies together. The main issue they ran into is that Go's GC needs to know the layout of your data (specifically, where pointers are) to work correctly, and if you just allocate raw bytes with unsafe.Pointer, the GC might mistakenly free things pointed to from your arena because it can't see those pointers properly. But to make it work even with pointers (as long as they point to other stuff in the same arena), you keep the whole arena alive if any part of it is still referenced. That means (1) keeping a slice (chunks) pointing to all the big memory blocks the arena got from the system, and (2) using reflect.StructOf to create new types for these blocks that include an extra pointer field at the end (pointing back to the Arena). So if the GC finds any pointer into a chunk, it’ll also find the back-pointer, therefore mark the arena as alive, and therefore keep the chunks slice alive. Then they get into a bunch of really interesting optimizations to remove various internal checks and and write barriers using funky techniques you might not've seen before

I believe this is out of date. There’s a very explicit opt in/out slider for permitting training on conversations that doesn’t seem to affect conversation history retention.

It’s always a touch ironic when AI-generated replies such as this one are submitted under posts about AI. Maybe that’s secretly the the self-reflection feedback loop we need for AGI :)

I’ve been using xonsh as my daily driver for a few years now, and it’s a massive productivity booster!

Broadly speaking I’ve found that most of the reported compatibility and usability concerns in their GitHub issues have boiled down to user error, rather than any kind of a defect with the shell itself. That’s not to say there aren’t any issues, but they’re few and far between, and it’s more than solid enough for regular use. It isn’t bash, and you shouldn’t expect to execute a bash script with xonsh or use bash idioms (even though some compatibility layers exist for e.g. translating your ~/.bashrc and sourcing envvars from bash scripts).

Barbour is criminally underrated as a physics author. He’s published a lot of interesting ideas regarding the role of time, or lack thereof, in modern theories! (The End of Time, and its treatment of Causality as a direct substitute for time in any future theory of everything, was very fun)

If that’s how it’s being advertised, and that’s the reason people are giving it a shot based on that advertising, then I certainly do! And so, I imagine, did the people who have left feedback so far!

I get that sliding in references to a passion project on top-scoring articles might seem like an easy way to give the project exposure, but commenting the same thing over and over comes off as a bit boorish. And just plugging the URL isn’t really contributing anything to the discussions IMO. Why not show us something your tool explained or summarized from the articles that isn’t obvious from a cursory read? Citing the tool as the source for something cool wouldn’t be nearly as in-your-face.

Aye, there’s the kicker. The correct configuration of hardware resources to run and multiplex large models is just as much of a trade secret as model weights themselves when it comes to non-hobbyist usage, and I wouldn’t be surprised if optimal setups are in many ways deliberately obfuscated or hidden to keep a competitive advantage

Edit: outside the HPC community specifically, I mean

I think the approximate reciprocal approach is interesting here. The doc mentions multiplying the dividend by ~1.00025 in the math to avoid FP error so you don’t end up off-by-one after truncation, but I think this hack is still incomplete! On some inputs (like 255, or other unlucky divisors near powers of two), you might get borderline rounding behaviour that flips a bit of the final integer. It’s easy to forget that single-precision floats don’t line up neatly with every 8bit integer ratio in real code, and a single off-by-one can break pixel ops or feed subtle bugs into a bigger pipeline.

I suspect a hybrid scheme like using approximate reciprocals for most values but punting to scalar for unlucky ones could handle these corner cases without killing performance. That’d be interesting to benchmark

I love that this addresses that probabilistic confirmation is fundamentally intransitive in Bayesian epistemology, and bumps in probability don’t necessarily propagate through multiple probabilistic links (Evidence -> BBU and BBU -> BB doesn’t mean Evidence -> BB).

Really well put-together and careful main line of argumentation IMO

When employees can’t see how their day-to-day actions influence the outcome of bonus decisions, they lose engagement, and it undermines your credibility. At the same time, when individual KPIs pit people against each other (e.g. the dev team is rewarded when they deliver on schedule, while ops is rewarded for cost savings that might slow dev productivity), it breeds conflict and silo mentality. Which sucks for all involved!

The “best” solutions boil down to economics IMO. In a smaller org you can probably get away with directly tying a % of the bonus to overall company health, which is as authentic and controllable as it gets if everyone’s trying to make the business a success in an interdisciplinary way. In a larger org you might need to measure proxy metrics that are correlated with that overall company health function but more closely aligned with the core missions of each department. Pick out a handful of KPIs per department, but weighted so no single number disproportionately influences the outcome. Smaller numbers of carefully chosen, stable metrics that the team can influence build trust over time.

Like other comments mentioned, this is just an AI-generated rehash of the underlying reporting that actually took place on this issue (see the “it isn't just X - it’s Y” and “- Bold Point Title: Short point contents”). Is there any way to change this link?

Weirdly enough, a few minutes ago I was using o1 via ChatGPT and it started consistently repeating its complete chain of thought back to me for every question I asked, with a 1-1 mapping to the little “thought process” summaries ChatGPT provides for o1’s answers. My system prompt does say something to the effect of “explain your reasoning”, but my understanding was that the model was trained to never output those details even when requested.

First off, congratulations folks! It’s never easy getting a new product off the ground, and I wish you the best of luck. So please don’t take this as anything other than genuine constructive criticism as a potential customer: generating tests to increase coverage is a misunderstanding of the point of collecting code coverage metrics, and businesses that depend on getting verification activities right will know this when they evaluate your product.

A high-quality test passes when the functionality of the software under test is consistent with the design intent of that software. If the software doesn’t do the Right Thing, the test must fail. It’s why TDD is effective: you’re essentially specifying the intent and then implementing code against it, like a self-verifying requirements specification. When we look at Qodo tests in the GitHub MRs you’ve linked, it’s argued that a high-quality test is defined as one that:

1. Executes successfully

2. Passes all assertions

3. Increases overall code coverage

4. Tests previously uncovered behaviors (as specified in the LLM prompt)

So, given source code for a project as input, a hypothetical “perfect AI” built into Qodo that always writes a high-quality test would (naturally!) never fail to write a passing test for that code; the semantics of the code would be perfectly encoded in the test. If the code had a defect, it follows logically that optimizing the quality of your AI for the metrics Qodo is aiming for will actually LOWER the probability of finding that defect! The generated test would have successfully managed to validate the code against itself, enshrining defective behavior as correct. It’s easy to say that higher code coverage is good, more maintainable, etc., but this outcome is actually the exact opposite of maintainable and actively undermines confidence in the code under test and the ability to refactor.

There are better ways to do this, and you’ve got competitors who are already well on the way to doing them using a diverse range of inputs besides code. It boils down to answering two questions:

1. Can a technique be applied so that a LLM, with or without explicit specifications and understanding of developer intentions, will reliably reconstruct the intended behavior of code?

2. Can a technique be applied so that tests generated by a LLM truly verify the specific behaviors the LLM was prompted to test, as opposed to writing a valid test but not the one that was asked for?

Why was this being downvoted? It’s first-party evidence that substantiates the claims of the parent comment, and adds interesting historical context from major industry players

Bigger discussion from two days ago: https://news.ycombinator.com/item?id=42259184

My understanding was that the metric for LMArena is that one answer is “better” than another, for a deliberately 100% subjective definition of better.

My experience has been that typical LLMs will have more “preamble” to what they say, easing the reader (and priming themselves autoregressively) into answers with some relevant introduction of the subject, sometimes justifying the rationale and implications behind things. But for o1, that transient period and the underlying reasoning behind things is part of OpenAI’s special sauce, and they deliberately and aggressively take steps to hide it from users.

o1 will get correct answers to hard problems more often than other models (look at the math/coding/hard subsections on the leaderboard, where anecdotal experiences aside, it is #1), and there’s a strong correlation between correctness and a high score in those domains because getting code or math “right” matters more than the justification or explanation. But in more general domains where there isn’t necessarily an objective right or wrong, I know the vibe matters a lot more to me, and that’s something o1 struggles with.