Ask HN: How to visualize UI interactions?

Hello everyone! My request is kind of difficult to explain, so please bear with me.

The problem: Using an Android device and an app, I record the screen. What I get is data (in JSON format) with coordinates of any UI elements (buttons, texts etc).

What I want to do is simulate and visualize the interactions from this data. For this, I need a mockup/layout tool that is able to render a (mobile) UI from a data source (xml, json, whatever), with a timeline to allow changing views. Optimally, this should work by reading from an input file, without manual input.

Imagine using any library (e.g. Qt) to render a GUI, but this GUI needs to be manipulated and changed automatically according to a specific timeline.

I am looking for any free software, framework and open-source project (for desktop or web) that can help with that. Has anyone ever heard or seen something like this before?

Thanks in advance!