Overdiagnosis can be a problem. On the flip side, I wonder if adding the time dimension to the data (i.e., you could realistically have scans from every few weeks over the course of years) could significantly change that.
Instead of looking at a single snapshot of a person, you're now looking at trends over time. We probably don't have the analytical tools to effectively evaluate medical imaging with that time dimension at such scale (because I assume it would be rare for someone to get MRIs so frequently), but maybe with more data and study, we'll be able to more definitively distinguish benign quirks from real concerns.
Rather than a human comparing a couple of scans five years apart, you're talking about computationally identifying outlying regions in the data (a motion picture of the entire body) that are trending towards areas of concern.