we run screen-driven agents against web forms in production and the failure mode that took us longest to find wasn't navigation, it was commits that don't commit. a react controlled select can render the right value after a click while the framework's internal state never updated, so every pixel says done and the submitted payload says null. vision-only verification passes because the screen genuinely looks correct.
curious how you handle that class without DOM access. screenshot-after-action catches missing UI feedback, but when the UI itself is lying about form state the only reliable tells we found were downstream: the confirmation page, an outbound request, an email arriving. do your verification events ever consume anything besides pixels, or do you lean on the human approval gates for the risky commits?