HNGlance

Top Best New Show

OmniParser V2 – A simple screen parsing tool towards pure vision based GUI agent

https://github.com/microsoft/OmniParser

by punnerud • 1 year ago

64 4 1 year ago

GI

Related Stories

Loading related stories...

Comments

4

icodar• 1 year ago

This is not the intended use but it good working on parsing document layout from image.

NewUser76312• 1 year ago

Very cool work. Accurate GUI text and element parsing is exactly the kind of input that LLMs need to be effective agents.

rgovostes• 1 year ago

The OS has additional information including how different graphics layers are composited, and what accessibility metadata is attached to interface elements. It ought to be useful to exploit this to do better than screenshot parsing.

nighthawk454• 1 year ago

One ponders the connections with the Recall feature

Source preview

github.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms