Is the future of fact-checking automated?

https://www.poynter.org/fact-checking/2022/is-the-future-of-fact-checking-automated/

by insightcheck • 4 years ago

3 2 4 years ago

hunglee2• 4 years ago

interesting to see technological solutions to misinformation. I wonder though whether misinformation actually is technical problem rather than a manifestation political conflict. Hence, easy to see automated fact checking being weaponised, as manual fact checking already is

insightcheck• 4 years ago

I'm concerned about how this technology will change journalism, especially because it has momentum—the article reports that the tool is in use by a UK government office and two news agencies, and has funding by Google.org. The article lists three tools: the first categorizes claims (okay), the second compares claims to conclusions of previous fact-checks, and the third compares statistical figures to official figures.

"The second tool addresses the problem of how to increase the reach of existing fact checks. This tool cross-references claims identified from the claim detection tool with existing fact checks done by Full Fact. [...] “Anyone who is involved in fact-checking knows that a fact check can take [a lot of time ...] “But once all of that effort and work has been invested, the outcome is one fact check.""

It's plausible that this can train journalists (especially under deadline) to automatically trust previous fact checks without independently verifying them (which is a good thing, and not a waste of time like the interviewee may be implying). High-profile stories by well-funded newspapers that are fact-checked get correction notices all the time. Deferring to previous fact-checks without verifying them independently can cause these mistakes to perpetuate. Even correct fact-checks in one context may have nuance that should be considered when applied to a new context, and these nuances can be missed when relying on an automated tool.

"The third tool, stats-checker, enables fact-checkers to rapidly cross reference live claims about statistics with official available statistics. For now, this tool is limited to statistics surrounding inflation and unemployment."

There can be value for economic statistics, but if applications expand to other contexts, it's important to note that official figures can be egregiously wrong. New York State underwent a scandal with nursing home figures involving an FBI investigation where the state's attorney general "released a report finding that Governor Andrew Cuomo had understated the toll of COVID-19-related deaths in state nursing homes by as much as 50 percent" [0] [1]. I get that ideally, fact-checkers will use it as an 'assistive tool' instead of blindly accepting the results. But in practice, many people do blindly trust the recommendations of software all the time (especially under time pressure). I've seen it with people trusting questionable recommendations for writing edits on Grammarly or bad directions on Google Maps.

Good training can mitigate blind trust in recommendations, but because it's not guaranteed that good training can be available across all news outlets. At the least, maybe very visible warning signs in the software that suggestions are meant to be assistive instead of definitive would be useful (but then again, users can eventually ignore these over time).

[0] https://www.nytimes.com/2021/03/04/nyregion/cuomo-nursing-ho...

[1] https://en.wikipedia.org/wiki/New_York_COVID-19_nursing_home...

Is the future of fact-checking automated?

Related Stories