madiator

Check out OpenThoughts. It has a widely used dataset, a model that beats the deepseek's smaller reasoning models, and a paper that talks in detail about the data curation methodology.

https://www.open-thoughts.ai/

I wrote about this recently as well: https://madiator.substack.com/p/finetuning-is-so-back

Congrats! Great work, Dan! Indeed, Terminal bench is awesome :)

Right. The title conjures dreamy images in my mind that I was dying to see images!

That's the hope, but that's never how things go. Of course researchers and research will be tremendously affected, and that's a shame.

Yeah this was quite an interesting finding, given how Mistral has marketed this (OCR solved!)

The blogpost is linked here: https://news.ycombinator.com/item?id=42826392

Thanks! We created bespoke-stratos-32B - let me know if you have any questions.

Synthetic data got a bad reputation last year, but it is now an important component for all modern LLMs! In fact, we had also trained one model for -- ironically -- detecting hallucinations, and it was also trained on synthetic data.

Say if you have some PDFs, and want to generate questions and answers to test your RAG pipeline, that's synthetic data! Distillation is mostly synthetic data and works great as well!

Our hope is that this becomes steroids rather than LSD for LLMs :)

We are working on it!

Not sure why you are so upset about a small and neat study ("please, please stop"). If you ask it to summarize (without feeding the entire bible), it needs to know the bible. Knowledge and reasoning are not entirely disconnected.

For the specific form of hallucination, which is called grounded factuality, we have trained a pretty good model that can detect if a claim is supported by a context. This is super useful for RAG. More info at https://bespokelabs.ai/bespoke-minicheck.

Now, it will be nice to get the code snippets for these!

There are several types of hallucinations, and the most important one for RAG is grounded factuality.

We built a model to detect this, and it does pretty well! Given a context and a claim, it tells how well the context supports the claim. You can check out a demo at https://playground.bespokelabs.ai

That's great, but it did not really write the program that the human asked it to do. :)

I am sure most people never asked these questions to a human doing this research.

I will be typing this comment on my Nokia's keyboard.

Yeah it wouldn't fit. GPT3 is 175B params, so even if you use 8 bit for each weight, you need 175×10^9÷2^30 = 163GiB of memory.

I am the author of the blog and I want to thank for all the feedback here. Because I now do realize the article starts with a high premise but fails short. Will do a better job next time!

What I meant to talk about is as follows: we have engineered away all sorts of discomfort from our lives and I think that's bad. So (1) be aware of this, (2) seek some discomfort, and (3) if you run into discomfort, take it in a positive way (this I didn't convey in the article).

But yeah I don't mean to say chop of your limbs! Not sure how people are reaching that conclusion.

Thanks HN!

Here's an example I can think of. Suppose you have a bunch of text documents, and you know that some documents are similar but not identical (e.g. plagiarized and slightly modified). You want to find out which documents are similar.

You can first run the contents through some sort of embedding model (e.g. the recent OpenAI embedding model [1]), and then apply LSH on those embeddings. The documents that have the same LSH value would have had very similar embeddings, and thus very similar content.

[1] https://beta.openai.com/docs/guides/embeddings

And I am completely surprised by people who think this tech today is not good, and fail to account that it can get way better in the future.

Aside: if you scroll to past issues, the author has all sorts of other articles like how to get into p0rn. Wonder if the author picked up AI recently, in which case, that's impressive.

The other reason I have heard is that since it's slower to write by hand you are forced to summarize it, which means you need to understand. You can't mindlessly type anymore.

I don't get why you don't care about your physical health. You got one body for the rest of your life..

I always think about thinking and try to write about it at my substack: http://newsletter.smarter.blog

Adopting the growth mindset in lieu of the fixed mindset. I was unhappy with my abilities and used to think, "I just have poor memory", or "I have hit my maximum ability" etc. But somehow somewhere I realized I can work hard to actually improve and change those things, and that they are not fixed.

I came across the concept of growth mindset a few years earlier and has since strengthened my commitment to adopting a growth mindset. I would recommend reading "Mindset: The New Psychology of Success" by Carol Dweck.

I don't know how I stumbled upon them, but I am glad to have gone through his audiobooks Mythos, Heroes, and Troy. Absolute treat to listen to the stories and the narration is just perfect.

You can think of your mind as a garden as well: https://newsletter.smarter.blog/p/mind-as-a-garden

+1 for paying for good information, which is what the author Yuval Harari recommends. He asks why we are willing to pay for high quality food, clothes etc., but not for high quality information.

Learning is remembering and understanding. You have to progressively build layers of understanding (and be able to recall it quickly).

This article focuses mostly on the second part, about remembering.

You could recall things but you may not fully understand it, which is going to make it harder to learn.

And of course, you might understand something, but eventually you will forget it (either few days or few years, depending on how often you use related memory). When you forget, it makes it hard to learn.