stilesja

It's a future evaluation for me, as I'm keeping the app with a "Data Not Collected" App Store label for now. Once I have some visibility to stats of how people answer each question and at which confidence level I expect it to reveal bad/poorly worded questions that made it through my pipeline.

Also on my to do list is an upgrade similar to my de-duplication logic. Currently I pass in my approved set of questions as a list of topics and questions to avoid. It works well especially when combined with the more deterministic comparison of generated questions vs approved question by the separate model. The upgrade would be to pass in the questions REJECTED by me. These are question which made it through ever automated gauntlet of models and code, yet still ultimately were sent to the waste bin by human review.

I'm imagining that by including the list of examples of what NOT to do, it could inform the initial Generation to be smarter. I currently have a limited list of a hand picked "bad question shapes" where I provide the question and why its bad kept statically in my prompt. But I've realized sorting through thousands of questions, I've probably built implicit data through rejection that I may not actually recognize myself which the model may pick up on.

I have a process generating trivia questions for my app and by passing in my existing dataset each time as a cached exclusion list, the output tokens it save more than pay for it.

Guardrails can and do help but they don't prevent hallucinations entirely. I generate trivia questions for my app. Questions are true/false and I tell it you must include an explanation of why the question is true for false. I also tell it to give me supporting links which validate its claim. I've had it make up explanations and invent plausible looking URLs. But I wrote a checker that pulls the content of the URL and asks a different model if the URL supports the claim. LLM's are much better at something like that than pulling a trivia question out of thin air. Reviewing diffs can be much the same. It isn't having to write the code but just evaluate if it matches what the commit comments claim it does. You are right, though. Without good comments on PR's and commits (i.e. context) the diff review is going to be a lot of assumptions made by the AI which is where hallucinations get more common.

I hear you! I read the article and thought the same thing because I've actually built a surprisingly similar pipeline myself, but it culminates with Human review. I've got a trivia app (Hedge Calibrated Trivia) whose whole content pipeline is built on a similar architecture of models evaluating models. I didn't want the same trivia that every other app has and I naively imagined that it would be easy to ask an LLM to give me questions. Turns out when you ask an LLM for trivia questions, the FIRST thing it gives you are the same questions every other app asks!

After iterating into many of the same problems OP describes I settled into a similar pipeline. I generate questions with Gemini 3.5 flash and review with Claude. The different vendor generation and validation eliminates training holes in a single model solution. This is True/False trivia so I ask for an equal mix of True/False questions and also ask for alternate phrasing of the question to the opposite variant. Each statement must provide an explanation and sources which back up the claim. I pass in my existing dataset as questions to avoid, but found the model then rewording the questions or category shopping the questions into related categories in an attempt to re-use them. That led to using Claude Sonnet to check for duplicate "themes" and similar topics. Claude also acts as a critic of the question and the explanation. Sometimes Gemini's "thoughts" leak into the explanation. The critic catches these leaks and also catches "weak" questions (basically shapes that I have identified via examples as lazy or projecting the answer in the question). Once the comparative models have stripped out anything obviously bad, my server curls the source URLs and uses Sonnet to verify that the fetched source supports the claim. Any URL that doesn't resolve throws up flags, and I found Gemini was particularly bad at inventing source URLs for sports trivia. Updated prompting directed it to prefer Wiki links over its fabricated deep links into league websites, which helped immensely. Still one of the biggest tells in review is that the URLs don't resolve or don't confirm the statement.

But review is the entire point here. I have to review them all. A question that lands in the app and is wrong reflects terribly. So while I have a trivia app on the App Store, the real complex App is my private admin app that lets me sit on the couch swiping through each and every question sorted by the model's confidence in its own quality and correctness (weakest first). No question makes it into my bank without me reading, checking all the indicators that came from the models and potentially even rewriting the question to make it better. It all happens in the admin app and exists entirely to prevent hallucinations from reaching real users. Early on I was maybe accepting 1 in 3 questions generated, but I'm probably closer to 2/3rds now. Of course not every dismissed question is wrong. The quality checks I've put in the pipeline are probably giving me 80% questions that are correct and sourced, but sometimes they just aren't what I'm looking for or don't meet my standard.

The best thing I ever did was make an app to review these questions though. Sitting on the couch, kick off a question generation, come back 20 minutes later with 50+ questions in the queue. Sort through them and pull out the 30 best when I have the time. That's the real anti-hallucinations pipeline.