What's Google Chrome's Screen AI?

https://news.ycombinator.com/item?id=39701917

by itvision • 2 years ago

6 3 2 years ago

news.ycombinator.com

I don't want any fancy AI features from my web browser yet I've recently discovered a huge library/extension in my Google Chrome's profile directory:

    $ du -hs screen_ai
    255M screen_ai

which contains libchromescreenai.so weighing 263,110,752 bytes.

What is it? Is it possible to disable it for good? I can imagine Google will now regularly push its updates which I do not want either.

itvision• 2 years ago

From https://issues.chromium.org/issues/40810109 and https://chromium-review.googlesource.com/c/chromium/src/+/33...

    Add Screen AI service for accessibility annotations.

    To improve the accessibility tree based on visual data, a snapshot of
    the screen is sent to a local machine learning library. The library will
    provide annotations for the snapshot and the annotations will be used
    to update the accessibility tree.
    This CL adds a Screen AI sand-boxed service to load the library, and
    general infrastructure to pass the images from renderer to the library
    and getting back and applying the annotations.
    The actual details on how the annotations are processed and used will be
    added in subsequent CLs.

    See more in go/chrome-screen-ai.

    This change is behind kScreenAI flag and disabled by default.

I still have no clue. Annotations for screenshots?

nolist_policy• 2 years ago

Sounds like a11y without the apps having to explicitly implement it.

"accessibility tree" is generally the interface trough which apps provide a11y information to the OS.

bertman• 2 years ago

  DESCRIPTION='ScreenAI is a binary to provide AI based models to improve 
  assistive technologies. The binary is written in C++ and is currently used by 
  ReadAnything and PdfOcr services on Chrome OS.'

from https://source.chromium.org/chromiumos/_/chromium/chromiumos...

What's Google Chrome's Screen AI?

Related Stories