Magistral — the first reasoning model by Mistral AI

941 421 1 year ago

cchance• 1 year ago

Good first shot i guess, but the small ones about as good as v3, and the mediums not quite as good as r1... i wonder if that r1 is the actual new one or the old one

hacklas• 1 year ago

The Deepseek V3 is a model with 671 billion parameters, of which 37 billion are active.

Magistral Small is a 24 billion parameter model.

Pretty impressive in terms of efficiency for Mistral.

The size of the Magistral Medium is not publicly available, so it is difficult to compare efficiency there.

kouteiheika• 1 year ago

The size of the Magistral Medium is not publicly available, so it is difficult to compare efficiency there.

FWIW one of their 70B models has leaked in the past (search for "miqu") and rumors at the time were that it was their medium model.

danielhanchen9• 1 year ago

I made some GGUFs for those interested in running them at https://huggingface.co/unsloth/Magistral-Small-2506-GGUF

ollama run hf.co/unsloth/Magistral-Small-2506-GGUF:UD-Q4_K_XL

./llama.cpp/llama-cli -hf unsloth/Magistral-Small-2506-GGUF:UD-Q4_K_XL --jinja --temp 0.7 --top-k -1 --top-p 0.95 -ngl 99

Please use --jinja for llama.cpp and use temperature = 0.7, top-p 0.95!

Also best to increase Ollama's context length to say 8K at least: OLLAMA_CONTEXT_LENGTH=8192 ollama serve &. Some other details in https://docs.unsloth.ai/basics/magistral

danielhanchen9• 1 year ago

Their paper https://mistral.ai/static/research/magistral.pdf is also cool! They edited GRPO via:

1. Removed KL Divergence

2. Normalize by total length (Dr. GRPO style)

3. Minibatch normalization for advantages

4. Relaxing trust region

Onavo• 1 year ago

Removed KL Divergence

Wait, how are they computing the loss?

gyrovagueGeist• 1 year ago

Does anyone know why they added minibatch advantage normalization (or when it can be useful)?

The paper they cite "What matters in on-policy RL" claims it does not lead to much difference on their suite of test problems, and (mean-of-minibatch)-normalization doesn't seem theoretically motivated for convergence to the optimal policy?

cpldcpu2• 1 year ago

But this is just the SFT - "distilled" model, not the one optimized with RL, right?

danielhanchen9• 1 year ago

Oh I think it's SFT + RL as mentioned in the paper - they said combining both is actually more performant than just RL

lxe• 1 year ago

Thanks for all you do!

danielhanchen9• 1 year ago

Thanks!

monkmartinez• 1 year ago

At the risk of dating myself; Unsloth is the Bomb-dot-com!!! I use your models all the time and they just work. Thank you!!! What does llama.cpp normally use if not "jinja" for their templates?

danielhanchen9• 1 year ago

Oh thanks! Yes I was gonna bring it up to them! Imo if there is a chat template, by default it should be --jinja

ozgune• 1 year ago

Their benchmarks are interesting. They are comparing to DeepSeek-V3's (non-reasoning) December and DeepSeek-R1's January releases. I feel that comparing to DeepSeek-R1-0528 would be more fair.

For example, R1 scores 79.8 on AIME 2024, R1-0528 performs 91.4.

R1 scores 70 on AIME 2025, R1-0528 scores 87.5. R1-0528 does similarly better for GPQA Diamond, LiveCodeBench, and Aider (about 10-15 points higher).

https://huggingface.co/deepseek-ai/DeepSeek-R1-0528

semi-extrinsic2• 1 year ago

Would also be interesting to compare with R1-0528-Qwen3-8B (chain-of-thought distilled from Deepseek-R1-0528 and post-trained into Qwen3-8B). It scores 86 and 76 on AIME 2024 and 2025 respectively.

Currently running the 6-bit XL quant on a single old RTX 2080 Ti and I'm quite impressed TBH. Simply wild for a sub-8GB download.

derefr• 1 year ago

I presume that "outdated upon release" benchmarks like these happen because the benchmark and the models in it were chosen first, before the model was created; and the model's development progress was measured using the benchmark. It then doesn't occur to anyone that the benchmark the engineers had been relying upon isn't also a good/useful benchmark for marketing upon release. From the inside view, it's just a benchmark, already there, already achieving impressive results, a whole-company internal target to hit for months — so why not publish it?

gavi• 1 year ago

too much thinking

https://gist.github.com/gavi/b9985f730f5deefe49b6a28e5569d46...

fzzzy• 1 year ago

My impression from running the first R1 release locally was that it also does too much thinking.

trebligdivad• 1 year ago

Nice! I'm running on CPU only, so it's interesting to compare - the Magistral-Small-2506_Q8_0.gguf runs at under 2 tokens/s on my 16 core, but your UD-IQ2_XXS gets about 5.5 tokens/s which is fast enough to be useful - but it does hallucinate a bit more and loop a little; but still actually pretty good for something so small.

danielhanchen9• 1 year ago

Oh nice! I normally suggest maybe Q4_K_XL to be on the safe side :)

Oras• 1 year ago

Would be interesting to see a comparison with Qwen 32B. I found it a fantastic local model (ollama).

DSingularity• 1 year ago

I agree. Qwen models are great.

SV_BubbleTime3• 1 year ago

Last year, fit was important. This year, inference speed is key.

Proofreading an email at four tokens per second, great.

Spending a half hour to deep research some topic with artifacts and MCP tools and reasoning at four tokens per second… a bad time.

ksec• 1 year ago

A few days after Apple's "The illusion of Reasoning". I wonder if this is the same again. Anyone runs Tower of Hanoi?

barrkel• 1 year ago

The Tower of Hanoi problem is limited by context length rather than model intelligence - see https://x.com/scaling01/status/1931783050511126954

NitpickLawyer2• 1 year ago

That paper was flawed in many ways, but it had a catchy name so lots of 'fluencers and media pounced on it and slopped some content based on the title alone. Chances are it will be relegated to the blooper section of LLM papers, just like that "training on LLM outputs leads to model collapse" paper was...

__loam• 1 year ago

Sorry this has nothing to do with the point you're making but I've literally never seen anyone use the word 'fluencers in place of influencers lol.

syntex2• 1 year ago

The illussion of reasoning was terrible paper. 2^n-1 how it could fit in context size. I tried o3 and he gave me python script saying that inserting all moves is to much for context window. completely different results.

roboboffin3• 1 year ago

I think that their point was that the problem is easily solvable by humans without code, and shows the ability to chain steps together to achieve a goal.

tonyhart7• 1 year ago

a bit too late aren't we??

pu_pe• 1 year ago

Benchmarks suggest this model loses to Deepseek-R1 in every one-shot comparison. Considering they were likely not even pitting it against the newer R1 version (no mention of that in the article) and at more than double the cost, this looks like the best AI company in the EU is struggling to keep up with the state-of-the-art.

atemerev7• 1 year ago

"EU is leading in regulation", they say.

I don't know what they are thinking.

micromacrofoot9• 1 year ago

probably some silly thing like "people should have more rights and protections"

dmos622• 1 year ago

It is fairly common to struggle to understand why different cultures think the way they do.

cpldcpu2• 1 year ago

Sorry, this is just getting old...

Its a trite talking point and not the reason why there are so few consumer-AI companies in Europe.

0xDEAFBEAD2• 1 year ago

Honestly the US approach to AI is incredibly irresponsible. As an American, I'm glad that someone somewhere is thinking about regulation. Not sure it will be enough though: https://xcancel.com/ESYudkowsky/status/1922710969785917691#m

Mistletoe• 1 year ago

This is why I want to move to the EU. I don’t care if companies aren’t coddled there. I want to live where people are the first priority.

mrtksn5• 1 year ago

Europe isn't going to catch up in tech as long as its market is open to US tech giants. Tech doesn't have marginal costs, so you want to have one of it in one place and sell it everywhere and when the infra and talent is already in US, EU tech is destined to do niche products.

UK has a bit of it, France has some and that's it. The only viable alternatives are countries who have issues with US and that is China and Russia. China have come up with strong competitors and it is on cutting edge.

Also, it doesn't have anything to do with regulations. 50 US States have the American regulations, its all happening in 1 and some other states happen to host some infrastructure but that's true for rest of of the world too.

If the EU/US relationship gets to Trump/Musk level, then EU can have the cutting edge stuff.

Most influential AI researchers are from Europe(inc. UK), Israel and Canada anyway. Ilya Sutskever just the other day gave speech at his alma matter @Canada for example. Andrej Karpathy is Slovakian. Lot's of Brits, French, Polish, Chinese, German etc. are among the pioneers. Significant portion of the talent is non-American already, they just need a reason to be somewhere else than US to have it outside the US. Chinese got their reason and with the state of the affairs in the world I wouldn't be surprised if Europeans gets theirs in less than 3 and a half years.

vikramkr• 1 year ago

If you close off the market to US tech giants, maybe they'll have some amount of market dominance at home, but I would doubt that would mean they've "caught up" tech wise. There would be no incentive to compete. American EV manufacturing is pretty far behind Chinese EV manufacturing, protectionism didn't help make a competitive car, it just protected the home market while slowly ceding international market after international market

iwontberude• 1 year ago

Which Trump/Musk level? There have been so many.

Iulioh• 1 year ago

The problem is, CONSUMER level tech

The EU is doing a lot of enterprise level shit and it's great

The biggest company in Europe sells B2B software (SAP)

ascorbic• 1 year ago

It's mostly about money. DeepMind was founded in the UK, and is still based in London, but there was no way it could get the funding it needed without selling to Google or some other US company. China is one of the few other countries that can afford to fund that kind of thing.

simianwords• 1 year ago

How can you explain Israel?

funnym0nk3y2• 1 year ago

Thought so too. I don't know how it could be different though. They are competing against behemoths like OpenAI or Google, but have only 200 people. Even Anthropic has over 1000 people. DeepSeek has less than 200 people so the comparison seems fair.

rsanek• 1 year ago

any claim from the deepseek folks should be considered with wide margins of error.

jasonthorsness• 1 year ago

Even if it isn't as capable, having a model with control over training is probably strategically important for every major region of the world. But it could only fall so far behind before it effectively doesn't work in the eyes of the users.

melicerte• 1 year ago

If you look at Mistral investors[0], you will quickly understand that Mistral is far from being European. My understanding is it is mainly owned by US companies with a few other companies from EU and other places in the world.

[0] https://tracxn.com/d/companies/mistral-ai/__SLZq7rzxLYqqA97j... (edited for typo)

pdabbadabba2• 1 year ago

For the purposes of GP's comment, I think the nationalities of the people actually running the company and doing the work are more relevant than who has invested.

kergonath• 1 year ago

It’s a French company, subject to French laws and European regulations. That’s what matters, from a user point of view.

littlestymaar2• 1 year ago

Benchmarks suggest this model loses to Deepseek-R1 in every one-shot comparison.

That's not particularly surprising though as the Medium variant is likely close to ten times smaller than DeepSeek-R1 (granted it's a dense model and not an MoE, but still).

fiatjaf• 1 year ago

This reads like an AI-generated comment. What do you mean by "benchmarks suggest"? The benchmarks are very clear and presented right there in the page.

tootie2• 1 year ago

As an occasional user of Mistral, I find their model to give generally excellent results and pretty quickly. I think a lot of teams are now overly focused on winning the benchmarks while producing worse real results.

esafak2• 1 year ago

If so we need to fix the benchmarks.

segmondy• 1 year ago

are you really going to compare a 24B model to a 700B+ model?

a2128• 1 year ago

24B is the size of the Small opensourced model. The Medium model is bigger (they don't seem to disclose its size) and still gets beaten by Deepseek R1

moffkalast• 1 year ago

The most important company is to is to QwQ at 30B sjnce it's still the best local reasoning model for that size. A comparison that Mistral did not run for some reason, not even with Qwen3.

hmottestad2• 1 year ago

With how amazing the first R1 model was and how little compute they needed to create it, I'm really wondering how the new R1 model isn't beating o3 and 2.5 Pro on every single benchmark.

Magistral Small is only 24B and scores 70.7% on AIME2024 while the 32B distill of R1 scores 72.6%. And with majority voting @64 the Magistral Small manages 83.3%, which is better than the full R1. Since I can run a 24B model on a regular gaming GPU it's a lot more accessible than the full blown R1.

https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-...

adventured4• 1 year ago

It's because DeepSeek was a fast copy. That was the easy part and it's why they didn't have to use so much compute to get near the top. Going well beyond o3 or 2.5 Pro is drastically more expensive than fast copy. China's cultural approach to building substantial things produces this sort of outcome regularly, you see the same approach in automobiles, planes, Internet services, industrial machinery, military, et al. Innovation is very expensive and time consuming, fast copy is more often very inexpensive and rapid. 85% good enough is often good enough, that additional 10-15% is comically expensive and difficult as you climb.

reissbaker3• 1 year ago

It's not better than full R1; Mistral is using misleading benchmarks. The latest version of R1, R1-0528, is much better: 91.4% on AIME2024 pass@1. Mistral uses the original R1 release from January in their comparisons, presumably because it makes their numbers look more competitive.

That being said, it's still very impressive for a 24B.

I'm really wondering how the new R1 model isn't beating o3 and 2.5 Pro on every single benchmark.

Sidenote, but I'm pretty sure DeepSeek is focused on V4, and after that will train an R2 on top. The V3-0324 and R1-0528 releases weren't retrained from scratch, they just continued training from the previous V3/R1 checkpoints. They're nice bumps, but V4/R2 will be more significant.

Of course, OpenAI, Google, and Anthropic will have released new models by then too...

epolanski3• 1 year ago

Jm2c but I feel conflicted about this arms race.

You can be 6/12 months later, and have not burned tens of billions compared to the best in class, I see it an engineering win.

I absolutely understand those that say "yeah, but customers will only use the best", I see it, but is market share of forever money losing businesses that valuable?

adventured4• 1 year ago

A similar sentiment existed for a long time about Uber and now they're very profitable and own their market. It was worth the burn to capture the market. Who says OpenAI can't roll over to profitable at a stable scale? Conquer the market, hike the price to $29.95 (family account, no ads; $19.95 individual account with ads; etc etc). To say nothing of how they can branch out in terms of being the interaction point that replaces the search box. The advertising value of owning the land that OpenAI is taking is well over $100 billion in annual revenue. Amazon's retail business is terrible, their ad business is fantastic. As OpenAI bolts on an ad product their margin potential will skyrocket and the cost side will be modest in comparison.

Over the coming years it won't be possible to stay a mere 6-12 months behind as the costs to build and maintain the AI super-infrastructure keeps climbing. It'll become a guaranteed implosion scenario. Winning will provide the ongoing immense resources needed to keep pushing up the hill forever. Everybody else - except a few - will fall away. The same outcome took place in search. Anybody spot Lycos, Excite, Hotbot, AltaVista around? It costs an enormous amount of money to try to keep up with Google (Bing, Baidu, Yandex) in search and scale it. This will be an even more brutal example of that, as the costs are even higher to scale.

The only way Mistral survives is if they're heavily subsidized directly by European states.

louiskottmann2• 1 year ago

Indeed, and with the technology plateau-ing, being 6-12 months late with less debt is just long term thinking.

Also, Europe being in the race is a big deal for consumers.

wafngar• 1 year ago

But they have built a fully “independent” pipeline. Deepseek and others probably trained in gpt4, o1 or whatever data.

bee_rider• 1 year ago

How many other open-weights reasoning models are there?

Is it possible to run multiple reasoning models on one problem? (Why not? I guess).

Another funny thought is: they release their Small model, and kept their Medium as a premium service. I wonder if you could do chains with Medium run occasionally, linked together by local runs of Small?

simonw4• 1 year ago

Qwen 3 and DeepSeek R1 and Phi-4 Reasoning are the best open weights reasoning models I know of.

ls612• 1 year ago

Just Deepseek I think and there are distillations of that that can run on consumer hardware if you really want.

atemerev7• 1 year ago

So, worse than R1, and only 24B version is open weights? NGMI. R1 is awesome, and full 630B version is open.

nake13• 1 year ago

The Magistral Small can fit within a single RTX 4090 or a 32GB RAM MacBook once quantized.

the_sleaze_• 1 year ago

Excellent news for me.

How does one figure this out? As in I want to know the comparable Deepseek or Llama equivalent (size-wise) and don't want to figure it out by trial and error.

lolive• 1 year ago

Is it indeed the plan of Apple to eventually run such kind of models direcly inside a iPhone? Or are the specs of any stateOfTheArt smartphone well below the minimum requirements of such "lightweight" models?

awongh• 1 year ago

Interesting that their niche seems to be small parameter models.

arnaudsm2• 1 year ago

I wished the charts included Qwen3, the current SOTA in reasoning.

Qwen3-4B almost beats Magistral-22B on the 4 available benchmarks, and Qwen3-30B-A3B is miles ahead.

resource_waste• 1 year ago

No surprise on my end. Mistral has been basically useless due to other models always being better.

But its European, so its a point of pride.

Relevance or not, we will keep hearing the name as a result.

SparkyMcUnicorn• 1 year ago

30-A3B is a really impressive model.

I throw tasks at it running locally to save on API costs, and it's possibly better than anything we had a year or so ago from closed source providers. For programming tasks, I'd rank it higher than gpt-4o

freehorse• 1 year ago

It is a great model, and blazing fast, which is actually very useful esp for "reasoning" models, as they produce a lot of tokens.

I wish mistral were back into making MoE models. I loved their 8x7 mixtral, it was one of the greatest models I could run the time it went out, but it is outdated now. I wish somebody was out making a similar size MoE model, which could comfortably sit in a 64GB ram macbook and be fast. Currently the qwen 30-A3B is the only one I know of, but it would be nice to have something slightly bigger/better (incl a non-reasoning base one). All the other MoE models are just too big to run locally in more standard hardware.

poorman• 1 year ago

Is there a popular benchmark site people use? Becaues I had to test all these by hand and `Qwen3-30B-A3B` still seems like the best model I can run in that relative parameter space (/memory requirements).

arnaudsm2• 1 year ago

- https://livebench.ai/#/ + AIME + LiveCodeBench for reasoning

- MMLU-Pro for knowledge

- https://lmarena.ai/leaderboard for user preference

We only got Magistral's GPQA, AIME & livecodebench so far.

devmor• 1 year ago

I would agree, Qwen3 is definitely the most impressive "reasoning" model I've evaluated so far.

5mv22• 1 year ago

The featured accuracy benchmarks exclude every model that matter except DeepSeek, which is quite telling about this new model's performance.

This makes it yet another example of European companies building great products but fumbling marketing.

Mistral's edge is speed. It's a real pleasure to use because it answers in ~1s what takes other models 5-8s, which makes for a much better experience. But instead of focusing on it, they bury it far down the post.

Try it and see if you like the speed! Note that the speed advantage only applies to queries that don't require web-search, as Mistral is significantly slower on this one, leading to a ~5 seconds advantage over 2 minutes of research for the queries I benchmarked with Grok.

funnym0nk3y2• 1 year ago

That is reasonable though. Comparing the product of a small company with little resources with giants like Google and OpenAI in a field where most advances are due to more and more expensive models is nonsense.

5mv22• 1 year ago

The point I was trying to express is that Mistral is arguably far superior to the giants if you care about speed! So I wished they communicated this more clearly.

dominicrose• 1 year ago

How would you use a fast AI?

My current use of AI is to generate code - or translate some code from a programming language to another - which I can then improve (instead of writing it from stratch). Speed isn't necessary for this. It's a nice-to-have but only if it's not at the cost of quality.

Also, as unfair as it "might" be, we do expect a fast AI not to be as good, don't we? So I wouldn't focus on that in the marketing. I think speed would be easier to sell as something extra you would pay for, because then you'd expect the quality to remain the same or better.

redavni• 1 year ago

analyzing and modifying a user interface in realtime?

rfv6723• 1 year ago

I tried thinking with websearch on their website.

It has similar speed with o4-mini with search on chatgpt, and o4-mini gave me much better result.

epic9x• 1 year ago

This thing is crazy fast.

smeeth• 1 year ago

They have a deal with Cerebras for inference.

https://www.cerebras.ai/blog/mistral-le-chat

swah3• 1 year ago

For me this is more important than quality. I love fast responses, feels more futuristic.

rafram3• 1 year ago

Is the number of em-dashes in this marketing copy indicative of the kind of output that the model produces? If so, might want to tone it down a bit.

ModernMech• 1 year ago

But the em dashes — if appreciated — are delightfully eccentric and whimsical!

tiahura• 1 year ago

Unless you're a lawyer. We love 'em.

NicuCalcea• 1 year ago

As a journalist, same!

johnisgood2• 1 year ago

I do not know but sometimes when I type "-" and press space, LibreOffice converts it to an em-dash. I get rid of it so people won't confuse me with an LLM.

sebmellen• 1 year ago

Our early tests indicated that Magistral is an excellent creative companion. We highly recommend it for creative writing and storytelling, with the model capable of producing coherent or — if needed — delightfully eccentric copy.

kobe_bryant• 1 year ago

it's bizarre.

the first sentence is "Announcing Magistral — the first reasoning model by Mistral AI — excelling in domain-specific, transparent, and multilingual reasoning." and those should clearly be comma

and this sentence is just flat out wrong "Lack of specialized depth needed for domain-specific problems, limited transparency, and inconsistent reasoning in the desired language — are just some of the known limitations of early thinking models."

umbra075• 1 year ago

really? i would have written it the exact same way (with dashes instead of commas).

saratogacx2• 1 year ago

That is just Mistral's market style. You see it on a lot of their pages. The model output doesn't share the same love for the long dash.

cAtte_• 1 year ago

49 em-dashes, 59 commas. that's a crazy ratio

pembrook• 1 year ago

This meme that humans don’t use em dashes needs to die.

It’s an extremely useful tool in writing and I’ve been using it for decades.

rafram3• 1 year ago

I love a good em-dash, but this page overuses them (nearly 1:1 ratio of em-dashes to commas!) and puts them in places where they just do not belong.

hskalin• 1 year ago

That's very weird, I on the other hand don't remember noticing them or using them before the advent of chatgpt. Maybe it's a cultural thing.

It makes sense that humans would have been using it though, chatgpt learned from us afterall

fivestones• 1 year ago

Same here. The em dash has been maybe my favorite punctuation since at least the early 2000s. All the em dash output from LLMs looks really natural to me.

xmcqdpt22• 1 year ago

We don’t have em dashes as punctuation in French —- commas are usually used instead —- so we get overly excited about using them when we can —- everybody likes novelty.

christianqchung• 1 year ago

I don't understand why the benchmark selections are so scattered and limited. It only compares Magistral Medium with Deepseek V3, R1, and the other close weighted Mistral Medium 3. Why did they leave off Magistral Small entirely, alongside comparisons with Alibaba Qwen or the mini versions of o3 and o4?

elAhmo• 1 year ago

When they include comparisons, it is always a deliberate decision what to show and, more importantly, what not to show. If they had data that would show better performance compared to those models, there is no reason for them to not emphasize that.

diggan3• 1 year ago

The only mention of tools I could find is this:

it significantly improves project planning, backend architecture, frontend design, and data engineering through sequenced, multi-step actions involving external tools or API.

I'm guessing this means it was trained with tool calling? And if so, does that mean it does tool calling within the thinking/reasoning, or within the main text? Seems unclear

simonw4• 1 year ago

Tool calling isn't enabled in the official Magistral Small GGUF (or the Ollama one) which is sad. Hope they (or someone else) fix that soon.

NitpickLawyer2• 1 year ago

They have already released Devstral, which is a tool-specific finetune of the same base model. That works pretty well with cline (even though it was specifically tuned for open-hands).

This would likely be a good model for the "plan" mode in various agentic tools (cline, aider, cursor/windsurf/void, etc). So you'd have a chat in plan mode, then use devstral to actually implement that plan.

jmorgan• 1 year ago

Working on adding tool calling support to Magistral in Ollama. It requires a tokenizer change and also uses a new tool calling format. Excited to see the results of combining thinking + tool calling!

simonw4• 1 year ago

Here are my notes on trying this out locally via Ollama and via their API (and the llm-mistral plugin) too: https://simonwillison.net/2025/Jun/10/magistral/

atxtechbro• 1 year ago

Hi Simon,

What's the huge difference between the two pelicans riding bicycles? Was one running locally the small version vs the pretty good one running the bigger one thru the API?

Thanks, Morgan

diggan3• 1 year ago

Ollama doesn't like proper naming for some reason, so `ollama pull magistral:latest` lands you with the q4_K_M version (currently, subject to change).

Mistral's API defaults to `magistral-medium-2506` right now, which is running with full precision, no quantization.

simonw4• 1 year ago

Yes, the bad one was Mistral Small running locally, the better one was Mistral Medium via their API.

internet_points• 1 year ago

I guess this means the reasoning traces are fully visible and not redacted in any way - interesting to see Mistral trying to turn that into a feature that's attractive to the business clients they are most interested in appealing to.

but then someone found that, at least for distilled models,

correct traces do not necessarily imply that the model outputs the correct final solution. Similarly, we find a low correlation between correct final solutions and intermediate trace correctness

https://arxiv.org/pdf/2505.13792

ie. the conclusion doesn't necessarily follow from the reasoning. So is there still value in seeing the reasoning? There may be useful information in the reasoning, but I'm not sure it can be interpreted by humans as a typical human chain of reasoning, maybe it should be interpreted more as a loud multi-party discussion on the relevant subject which may have informed the conclusion but not necessarily lead to it.

OTOH, considering the effects of automation fatigue vs human oversight, I guess it's unlikely anyone will ever look at the reasoning in practice, except to summarily verify that it's there and tick the boxes on some form.

GuinansEyebrows• 1 year ago

This doesn't really explain what "reasoning" means in the context of genAI, or how it's done by this product. Are there any good sources to learn more about what "reasoning model" means outside of marketing-speak?

pier25• 1 year ago

It's pure marketing. See the recent paper by Apple called "The Illusion of Thinking".

https://ml-site.cdn-apple.com/papers/the-illusion-of-thinkin...

kamranjon2• 1 year ago

I sort of agree with this, having read the recent Apple paper - but it does show a significant improvement at a certain level of complexity - it's just that it requires quite a few more tokens to achieve that. It could probably be described as a sort of "context" hack because it's basically having a conversation with itself to arrive at a better solution. You're trading performance/time for a bit better quality.

throwaway314155• 1 year ago

If you read that paper, you'll find a more nuanced take than simply "it's pure marketing"

skeptrune• 1 year ago

Fully open reasoning traces are useful. Happy there is a vendor out there shipping that feature.

desireco42• 1 year ago

One cool think about this model, that I installed locally is that supports well other languages as well as it should be pleasant conversation partner.

BTW I am personally fan of Mistral, because while it is not the top model, it produces good results and the most important thing is that it is super fast, just go to it's chat and be amazed. It really saves a lot of time to have quick response.

dwedge• 1 year ago

Their OCR model was really well hyped and coincidentally came out at the time I had a batch of 600 page pdfs to OCR. They were all monospace text just for some reason the OCR was missing.

I tried it, 80% of the "text" was recognised as images and output as whitespace so most of it was empty. It was much much worse than tesseract.

A month later I got the bill for that crap and deleted my account.

Maybe this is better but I'm over hype marketing from mistral

notnullorvoid• 1 year ago

I wouldn't trust any of these LLM teams to produce a good OCR model. OCR from 10 years ago is better than the crap they put out.

alister2• 1 year ago

As a quick test of logical reasoning and basic Wikipedia-level knowledge, I asked Mistral AI the following question:

A Brazilian citizen is flying from Sao Paulo to Paris, with a connection in Lisbon. Does he need to clear immigration in Lisbon or in Paris or in both cities or in neither city?

Mistral AI said that "immigration control will only be cleared in Paris," which I think is wrong.

After I pointed it to the Wikipedia article on this topic[1], it corrected itself to say that "immigration control will be cleared in Lisbon, the first point of entry into the Schengen Area."

I tried the same question with Meta AI (Llama 4) and it did much worse: It said that the traveler "wouldn't need to clear immigration in either Lisbon or Paris, given the flight connections are within the Schengen Area", which is completely incorrect.

I'd be interested to hear if other LLMs give a correct answer.

[1] https://en.wikipedia.org/wiki/Schengen_Area#Air_travel

marsa• 1 year ago

doing some reason.. uhh intuitioning i imagine brazil and portugal might have some sort of a visa-free deal going on in which case llama 4 might actually be right here?

alister2• 1 year ago

Brazilians don't need a visa for Portugal, France, or any Schengen country. But everybody has to pass through immigration control (at least a passport check even if you don't need a visa) when entering the Schengen zone. My question was which country would that happen in.

mcintyre19942• 1 year ago

AFAIK Schengen has a common visa policy, so there couldn't be such a deal between Brazil and Portugal. It'd also be extremely surprising if two countries not in a common travel area had a deal where you didn't have to clear customs at all, I suspect that doesn't exist anywhere in the world.

mcintyre19942• 1 year ago

I think Gemini's answer (2.5 Flash) is impressive

----

Since both Portugal and France are part of the Schengen Area, and a Brazilian citizen generally does not need a visa for short stays (up to 90 days in any 180-day period) in the Schengen Area, here's how immigration will work:

Lisbon: The Brazilian citizen will need to clear immigration in Lisbon. This is because Lisbon is the first point of entry into the Schengen Area. At this point, their passport will be stamped, and they will be officially admitted into the Schengen Zone.

Paris: Once they have cleared immigration in Lisbon, their flight from Lisbon to Paris is considered a domestic flight within the Schengen Area. Therefore, they will not need to clear immigration again in Paris.

Important Note: While Brazilians currently enjoy visa-free travel, the European Travel Information and Authorization System (ETIAS) is expected to become mandatory by late 2026. Once implemented, Brazilian citizens will need to obtain this electronic authorization before their trip to Europe, even for visa-free stays. However, this is a pre-travel authorization, not a visa in the traditional sense, and the immigration clearance process at the first point of entry would remain the same.

viraptor• 1 year ago

It looks like you're not sure of the answer yourself. That's really not a good way to test the LLMs. You'll just prefer the one that convinces you the most rather than the correct one.

jaza• 1 year ago

Sounds to me like "immigration in Lisbon or in Paris or in both cities or in neither city" is a trick question, because (on top of immigration in Lisbon as the Schengen entry point) surely the traveller also needs to clear immigration in Sao Paulo (as the Brazil / Mercosur exit point)?

schneehertz• 1 year ago

This is deepseek-r1's answer:

```

Based on the Schengen Area rules for international travel:

Lisbon (Portugal) - Immigration clearance is required here. Since Lisbon is the first point of entry into the Schengen Area (and Brazil is outside Schengen), the passenger must clear immigration (passport control) in Lisbon. This is standard for all non-Schengen arrivals.

Paris (France) - No immigration clearance is needed. After clearing immigration in Lisbon, the onward flight to Paris is considered a domestic Schengen flight (no border checks within Schengen).

```

eigenspace• 1 year ago

I wonder if they adjusted it since reading this thread, or if you forgot to turn on reasoning or if you just got unlucky. For me, when I ask this it thinks for about 1.5 minutes and responds with

"The Brazilian citizen needs to clear immigration in Lisbon."

When I asked it to explain why, it said

"The Brazilian citizen needs to clear immigration in Lisbon because Lisbon is the first Schengen entry point when traveling from a non-Schengen country (Brazil) to a Schengen country (France via Portugal). According to Schengen rules, travelers must clear immigration at the first Schengen port of entry. Since Paris is also within the Schengen Area, no additional immigration checks are required upon arrival in Paris."

CobrastanJorji• 1 year ago

Etymological fun: both "mistral" and "magistral" mean "masterly."

Mistral comes from Occitan for masterly, although today as far as I know it's only used in English when talking about mediterranean winds.

Magistral is just the adjective form of "magister," so "like a master."

If you want to make a few bucks, maybe look up some more obscure synonyms for masterly and pick up the domain names.

snakeboy• 1 year ago

as far as I know it's only used in English when talking about mediterranean winds.

It's a French company, and "mistral" has this usage in French as well. Also, "magistral" is just the french translation of "masterful".

mark_l_watson• 1 year ago

Nice, and I see that Ollama already has the smaller 24B version. I am traveling with just a mobile device so I have to wait to try it, but I have been using their new devstral coding model and it is very useful, given that it is also a locally run model so I looking forward to trying magistral.

RandyOrion• 1 year ago

Below are my comments on Magistral small (not medium).

24B size is good for local inference.

As a model outputting long "reasoning" traces (~10k tokens), 40k context length is a little concerning.

Where are the results of normal benchmarks, e.g., MMLU/pro, IFEval and such.

Still, thank you Mistral team for releasing this model with Apache 2.0.

hijkzzz• 1 year ago

The RL algorithm used in Magistral is the same as the Reinforce++-baseline in OpenRLHF.

FabHK6• 1 year ago

So, is it accessible in Le Chat?

Agraillo• 1 year ago

From the release "..You can try out a preview version of Magistral Medium in Le Chat..", I suppose it's when the drop down in "Thinking" mode is either slow or fast (limited to 3 queries per day).

My favorite from the last months was asking for a string that for base64 produces strings with non-alphanumeric and non-padding symbols (so '+' or '/' should be in the output). It thought for 7 minutes and 74k of markdown length, and finally came up with the AB?C string that produces QUI/Qw== (correct). It is impressive, because general LLMs just always fail, but I didn't try other "thinking" models recently.

openplatypus• 1 year ago

Ah yes, the circle jerk of HN focuses not on technology or news but how "Europe falls behind". Mods are silent as always.

7bit• 1 year ago

What do you want the mods to do?

CMay3• 1 year ago

One immediate observation I have about this model is that it seems to do a better job of filtering out or toning down some ideological disinformation that other models regurgitate from activist controlled Wikipedia articles, at least for a few I've checked. Previously you had to write your own sanity-check prompts to get the model to do extra up-front work to validate the logical and historical accuracy of things before it spits out what it thinks is the most popular answer.

With this, at least it seems like some of that work was done upfront or the thinking is tuned to avoid those issues, because it's giving me similar conclusions to a sanity-checked prompt. Heck, even Google Gemini and ChatGPT were spitting that stuff out, where this one is giving me a reasonable response. So in that regard, big thumbs up to the Mistral team if they did any specific work in that area. It's something I cared about that I was getting concerned nobody else cared about enough to fix.

andsoitis• 1 year ago

What’s an example prompt and sanitized prompt you use to evaluate?

CMay3• 1 year ago

Not going to leak my tests, but here's how you can create your own.

- Think up a topic that's interesting to you, yet maybe controversial.

- Look up primary sources and empirical information about it.

- Then look at a relevant Wikipedia article about it to see if the way the Wikipedia article frames it is honestly and faithfully justified by the primary sources and empirical data about it.

If the article seems to have a strong bias or critically misrepresent the reality even if it does so by stating true things, you have a juicy nugget on your hands.

Ask any given LLM about that topic and see if it regurgitates the opinion in the Wikipedia article. If it does, then develop your own prompt that requires the LLM to go down a checklist of things that help resolve warped logic without specifically trying to shape the output to your own preference. Now find other articles and see how well your checklist generalizes.

How well this works depends on how good the model you're using is at instruction following.

A lot of what thinking models do is expand the context around a topic to hopefully improve final prediction. To assist that, you have to encourage the LLM to be hesitant to form an opinion or decide on the conclusion before the end, otherwise it can start with a conclusion and spend the rest of the time supporting a weak conclusion rather than arriving at a stronger one after new information emerges.

The danger is that reasoning models will state early on in their reasoning some ideological fact the same way it might say, "well i know that 1+1=2, so that means X", when in reality a particular fact does not stand up to scrutiny. Then it gets lost in a loop thinking ideologically, which can help propagate these things through language models which is dangerous.

Ideally all ingested Wikipedia gets evaluated against some levels of ground truth before getting trained on to start with, but then it's harder to keep up to date with it. Until then we have to help LLMs handle these cases better.

megalomanu2• 1 year ago

We just tested magistral-medium as a replacement for o4-mini in a user-facing feature that relies on JSON generation, where speed is critical. Depending on the complexity of the JSON, o4-mini runs ranged from 50 to 70 seconds. In our initial tests, Mistral returned results in 34–37 seconds. The output quality was slightly lower but still remain acceptable for us. We’ll continue testing, but the early results are promising. I'm glad to see Mistral prioritizing speed over raw power, there’s definitely a need for that.

nbardy• 1 year ago

I bet you can close the gap with a finetune.

Should be quiet easy if you have some o4-mini results sitting around.

kamranjon2• 1 year ago

I am curious why you would choose a reasoning model for JSON generation?

I was recently working on a user facing feature using self-hosted Gemma 27b with VLLM and was getting fully formed JSON results in ~7 seconds (even that I would like to optimize further) - obviously the size of the JSON is important but I’d never use a reasoning model for this because they’re constantly circling and just wasting compute.

I haven’t really found a super convincing use-case for reasoning models yet, other than a chat style interface or an assistant to bounce ideas off of.

megalomanu2• 1 year ago

It is for generating a big nested JSON, quite complex from a business standpoint (lots of different business concepts). We didn't have good results with simple models.

emilyharrison• 1 year ago

Nice job!

hst-001• 1 year ago

good

shocking63• 1 year ago

I asked it my standard quest "Who was Adolf Hitler's mother?" and it's been bouncing between her married name and her maiden name, and it's still going! ollama run hf.co/unsloth/Magistral-Small-2506-GGUF:UD-Q4_K_XL

>> Who was Adolf Hitler's mother? <think> Alright, I know that Adolf Hitler is one of the most well-known historical figures, but I'm not entirely sure about his mother's name. I recall that his father was Alois Hitler, but I'm not sure about his mother. Maybe I can break this down.

First, I remember that Hitler was born in Austria, and his family background is often discussed in historical contexts. I think his mother was a local woman, but I don't recall her name. Maybe I can think about common names or try to recall if I've seen her name in documentaries or books.

I think I remember that his mother's name was Klara. But I'm not sure about the last name. Was it the same as his father's? Probably, since it was a common practice at the time for women to take their husband's surnames. But I'm not sure if that was the case here.

Wait, I recall that his father's name was Alois Hitler, and his mother's name was Klara. But what was her full name? I think it was Klara Hitler. But I'm not sure if she had a different last name before marriage.

I think I remember that her maiden name was Pölzl. But I'm not sure. I recall that she was from a lower-middle-class family, and that might be relevant.

But to be sure, I should think about the most common information. The most commonly cited name for Hitler's mother is Klara Hitler (née Pölzl). But is that accurate? I think so.

But to confirm, I recall that Klara Pölzl was her name before marriage, and after marrying Alois Hitler, she became Klara Hitler. But I'm not sure if that's correct.

Wait, I think I recall that his mother's name was Klara Hitler, but her maiden name was Klara Pölzl. But I'm not sure. Maybe I can think about the timeline.

Adolf Hitler was born in 1889. His mother must have been born some years before that. If she was in her early 20s when she had Adolf, she might have been born around the early 1860s.

But I don't have exact dates, so maybe I should focus on the name. I think the most common reference is Klara Hitler.

But I recall that in some sources, she is referred to as Klara Pölzl, especially when talking about her early life.

But in most historical references, she is known as Klara Hitler after marriage.

But I think the question is about her name, and it's common to refer to women by their married name in historical contexts, especially if that's the name they were most commonly known by.

So, her name was likely Klara Hitler.

But to be thorough, I think her full name was Klara Hitler (née Pölzl).

But the question asks for her name, and it's common to refer to someone by their married name, especially if that's the name they were most commonly associated with.

So, the answer is likely Klara Hitler.

But I recall that sometimes she is referred to as Klara Pölzl, especially in contexts where her maiden name is relevant.

But in general, historical references often use her married name.

I think the most accurate answer is Klara Hitler.

But to be sure, I recall that in many biographies, she is referred to as Klara Hitler.

Therefore, I think the answer is Klara Hitler.

But I'm not entirely confident. Maybe I can think of another way to verify this.

I recall that Hitler had a close relationship with his mother, and she is often mentioned in his early life. The name Klara Hitler seems to be the standard reference.

Therefore, I conclude that Adolf Hitler's mother was Klara Hitler.

But just to double-check, I think her maiden name was Pölzl, but she became Klara Hitler after marriage.

So, her name was Klara Hitler.

But sometimes people refer to her by her maiden name, especially in contexts where her family background is important.

But the most common and likely correct answer is Klara Hitler.

But I recall that in some sources, she is referred to as Klara Pölzl.

But in most standard references, it's Klara Hitler.

Therefore, I think the answer is Klara Hitler.

But to be thorough, I should consider that her full name might be Klara Hitler (née Pölzl), but the question asks for her name, and it's most commonly given as Klara Hitler.

So, final answer: Klara Hitler.

But I think I might be missing something. Maybe I should recall more details.

I remember that Klara Hitler was born in 1860 and died in 1907. She was married to Alois Hitler, and they had several children, but only Adolf and his siblings survived infancy.

But I'm not sure if this helps with the name. I think the name is indeed Klara Hitler.

But sometimes, in historical writing, people refer to her by her maiden name, Klara Pölzl, especially when discussing her early life or family background.

But in general, when referring to her in the context of Hitler's life, she is called Klara Hitler.

Therefore, the most appropriate answer is Klara Hitler.

But to cover all bases, perhaps it's best to give both names: Klara Pölzl (later Klara Hitler).

But the question asks for her name, and in the context of Hitler's mother, she is most commonly referred to as Klara Hitler.

So, I'll stick with Klara Hitler.

But I recall that in some biographies, she is referred to as Klara Pölzl.

Hmm, maybe the most complete answer is Klara Pölzl, as that is her full name, including her maiden name.

But in most contexts, especially when talking about her as Hitler's mother, she is called Klara Hitler.

I think both are correct, but Klara Hitler is more commonly used.

But to be precise, her full name was Klara Pölzl before marriage and Klara Hitler after marriage.

The question asks for her name, and it's common to refer to someone by their married name, especially if that's the name they were most commonly known by.

Therefore, the answer is Klara Hitler.

But I think it's also acceptable to say Klara Pölzl, especially if emphasizing her origins.

But since the question doesn't specify, and Klara Hitler is the more common reference, I'll go with that.

But to be thorough, perhaps the best answer is Klara Hitler (née Pölzl).

But in most historical writing, she is referred to as Klara Hitler.

Therefore, the answer is Klara Hitler.

But I'm not entirely sure. Maybe I should think differently.

Perhaps I can recall specific sources or books where her name is mentioned.

I remember reading that in "Mein Kampf," Hitler mentions his mother as Klara Hitler.

Yes, in his own writing, he refers to her as Klara Hitler.

Therefore, that is likely the correct name to use.

So, final answer: Klara Hitler.

But to be complete, her maiden name was Pölzl.

But the question asks for her name, and in the context of her being Hitler's mother, she is most commonly referred to as Klara Hitler.

Therefore, the answer is Klara Hitler.

But I think it's also correct to say Klara Pölzl, especially in a historical context where maiden names are important.

But since the question is about her name, and she was known as Klara Hitler during the time she was Hitler's mother, that seems to be the most appropriate answer.

But to cover all bases, perhaps it's best to say Klara Hitler (née Pölzl).

But in most cases, people refer to her as Klara Hitler.

Therefore, the answer is Klara Hitler.

But I think I might be overcomplicating this. The most common and likely correct answer is Klara Hitler.

So, I'll go with that.

</think>

*Summary:* Adolf Hitler's mother was Klara Hitler (née Pölzl). She is most commonly referred to as Klara Hitler in historical contexts, especially when discussing her role as Hitler's mother. However, her maiden name was Pölzl.

Thus, the answer is:

Adolf Hitler's mother was \boxed{Klara\ Hitler}.

Magistral — the first reasoning model by Mistral AI

Related Stories