Here's an example that illustrates a shortcoming about how medicine has historically been practiced and how it might change with the advent of AI.
A person very familiar with me was having an interaction at a review board level at Stanford. They had this rare illness that they were treating and someone had the "bright idea" of suggesting of saying ... "hey, why don't we look at the ten other times we treated this exact illness and see what worked!" Everyone was delighted with this novel idea (discussion circa 2021). My person was a bit disgusted as this is simple feedback loop style improvement and WTF! they should be doing this all the time to get probabilistic style suggestions for many treatments. I know it happens within certain healthcare systems (e.g., Kaiser had full EMR back in early 2000s and saw right away that VIOXX was killing people. So, they stop prescribing it. citation: Kaiser panel member paraphrase at a healthcare conference in 2010). If you just observe the healthcare system, you can see that the healthcare systems and most EMRs don't typically capture the feedback loop (i.e, when's the last time a doctor followed up and said "did you feel better after the last treatment?" or measures the result.) AI itself can't solve this as it doesn't have access to the data feedback loop. However, maybe AI's within the EMR will help "suggest" evidence based treatments. I could go on and on, but as a math guy, I've often been shocked at the non-evidence based assertions some doctors make. My conclusion is that if you're not "in the fairway", they are typically just guessing.