Chrome experimental AI features

https://blog.google/products/chrome/google-chrome-generative-ai-features-january-2024/

by mleroy • 2 years ago

106 127 2 years ago

blog.google

seydor• 2 years ago

what i want is 'read aloud', like MS Edge

TOMDM2• 2 years ago

Edge is honestly slowly turning into a better Chrome, and the better parts aren't even the LLM craze.

If they keep it up, this might actually threaten Googles browser dominance.

On the AI end though Microsoft aggresively pursues support for other AI providers (Mistral and Lamma both being on Azure API's now), Google tying themselves to Gemini seems to be tying themselves to the best they can do while Microsoft seems to be accruing the best they can get.

lxgr3• 2 years ago

Serious question: What do you like about Edge?

I’ve tried it a couple of times (when Bing AI chat was still Edge-only) and was extremely put off by all the coupons, rewards, and other distractions.

Solvency• 2 years ago

It's absolutely wild that this isn't the first thing anyone would make there. ChatGPTs talk mode is so good, I'd kill for the ability to listen to longform articles at varying speeds/voices.

lulzx• 2 years ago

Again features nobody cares about

izolate2• 2 years ago

The first two are seemingly of questionable utility, but the 3rd feature (Help me write) is actually quite interesting.

As of late, most of my public written responses (bar HN) have had some sort of collaboration with ChatGPT, and I've often wondered about a native browser integration. For those of us who struggle with communication, this is an exciting prospect!

wolverine8762• 2 years ago

I suck at plenty of things and when there is software (or another tool) to help me, I am happy to use it. Software that helps with communication, at least on this level, is a new frontier and therefore, as always, people feel a lot of uncertainty.

As someone who seriously utilizes this particular tool, what do you think of those issues? For example, do you feel like the result has your own voice? Your own specific, precise thoughts? Does it help or hurt growth in communication skill? How do those things play out in real application of the technology and what is the best way to use it?

Incidentally, communication is a strong point for me and therefore ChatGPT doesn't benefit me much in that respect. I hate to think that my skill has lost most of its value, but working in technology, I can hardly complain when it happens to me: Are communication skills even needed now, or how has that need changed?

izolate2• 2 years ago

The uncertainty you pointed out does seem to explain the downvotes I received.

For me personally, communication is arduous. I struggle daily to articulate what I want to say in a logical and efficient manner, let alone in a graceful or artistic one. I've noticed my vocabulary and communication skill has regressed as I get older, despite efforts to improve it.

Overall, ChatGPT has helped tremendously. I never had a written communication style that I was proud of, so I'm happy to assume its more generic tone of voice. If language fulfills its primary purpose, to get a point across, that's enough for me. Any kind of inherent artistic integrity is above my pay grade, so to speak.

summerlight5• 2 years ago

I see this may be a good replacement for general autofill features as well. Seemingly simple autofill tasks like filling e-mail, address, name, country etc... fields never "just works" for almost all sites since this relies on correctness of the target page implementation and devs usually never care of it. Large language models should be better on this task.

worksonmine5• 2 years ago

You don't need an LLM to figure out what a certain field is. It's like choosing the 18 wheeler to drive your kids to school...

smith70182• 2 years ago

I'm on the other end of the spectrum; I'm pretty worried that that feature will make it really easy to automate bot comments in-browser. It's sad that the internet as we knew it 3 years ago is gone forever and changes like this to push some team's OKRs means I won't be able to trust more online comments.

rockemsockem9• 2 years ago

The Internet 3 years ago? Was that Internet that great?

jabroni_salad• 2 years ago

Ironically, the message "I'm interested in this place - do you allow dogs?" is a piece of decent business writing on its own and way better than anything I saw while trying to sublet a room. I would rather see AI suggest phrases like that rather than their proposed answer.

Now that I think about it, a Clippy that interviews you about needs and follows your browsing session to highlight stuff you like / don't like and propose questions to ask would be pretty sweet.

burkaman2• 2 years ago

Yeah that suggestion is terrible. Really not looking forward to seeing this kind of writing everywhere.

bogtog• 2 years ago

Is there any good AI autocomplete tool out there? The only LLM tool I like currently is GitHub copilot. I basically want copilot for gmail + MS word, but every product I've found wants me to prompt an AI.

kccqzy2• 2 years ago

Gmail already has content-based autocomplete. The feature is called Smart Compose: https://support.google.com/mail/answer/9116836?hl=en

I also find the UX much better than typical LLMs that require you to write a prompt first; it simply suggests continuations of your sentences that you can accept or ignore, without requiring you to switch your mind between writing for your intended email recipient and writing a prompt for the LLM.

kccqzy2• 2 years ago

I find it the exact opposite. Writing is joyful to me, whether I'm writing a short one-paragraph comment on HN or writing a thousand-word essay. I do not like the current crop of "Help me write" features because they take away the joy.

smusamashah2• 2 years ago

I have a chrome extension (https://github.com/SMUsamaShah/LookupChatGPT) where i just added in-place text replacement option. Right click your selected text and chose your own prompt to refine the text your way.

int_19h3• 2 years ago

I don't know. The screenshot that they use to showcase it makes me feel the opposite - it took a perfectly clear and concise question and dressed it up in a lot of unnecessary verbiage that the person on the other end will now have to unpack to get to the gist of it.

Or they could use an LLM to translate it back to clear and concise, I suppose. But then what is it even for?

cute_boi• 2 years ago

I don't want another microsoft edge where I have to press hundred buttons just to open website.

mvdtnz• 2 years ago

What are you talking about?

Hoefner2• 2 years ago

I hope the Chrome devs will add some features like those in Arc browser, such as summarizing content while hovering over a link and pressing Shift.

lukan• 2 years ago

That would be useful, but also probably quite expensive, if every chrome user use this feature?

nolist_policy2• 2 years ago

Not if inferencing happens locally, e.g. with Gemini Nano.

summerlight5• 2 years ago

Should be doable with a local model, but there might be some trade-off here. I expect it to roll out to Pixel users first where Google has a better control.

jklinger410• 2 years ago

I hope the Arc Browser team makes their Chromium based project as cross-platform as Chromium itself.

rockemsockem9• 2 years ago

It's coming to Windows really soon! I'm not really holding my breath for Linux support though :/

summerlight5• 2 years ago

As a tab hoarder, I remember there were some attempts to implement rule-based tab organizer (using features like tab name, url, etc...) but most of them were only marginally useful for my case.

I wondered if generative models make any differences here so just tried it and a bit disappointed, it's consistently returning an error with a message "Tab groups suggestions are currently unavailable". It's just launched and the team might be experiencing lots of pages, perhaps I should try this again later.

punkspider• 2 years ago

How did you use it? I'm trying with Chrome Beta but it doesn't seem to have the option to organize tabs.

I looked in Experiments and it's not there either.

It's updated to the latest version.

summerlight5• 2 years ago

https://support.google.com/chrome/answer/14519765?visit_id=6...

AFAIK, this is geo limited to the US, at least for now.

Al_Ptr3• 2 years ago

Have you tried ToChunkA Smart TabS extension(https://chromewebstore.google.com/detail/tochunka-smart-tabs...)?

It organizes tabs and bookmarks by content similarity and does little more. Designed for tab hoarders in mind. :)

nolist_policy2• 2 years ago

Meanwhile I keep refreshing the Google Bard changelog site, waiting for Gemini Pro in Europe.

esha_manideep• 2 years ago

Having a feature to summarise the website would be a much better than the first two. As I constantly find myself wishing for such a feature :/

renewiltord• 2 years ago

The Arc browser I use has that feature on hover.

jsf01• 2 years ago

I’ve been working on a project [1] to do just that from within a Chrome extension. The idea was that as an extension, it could make use of the context menu and feel more like a native feature of the browser. I’m always hesitant to link to my things from comments but in this case I think it’s a perfect fit for what you’re describing.

[1] https://smudge.ai

rockemsockem9• 2 years ago

I suspect Google is afraid of getting sued by more publishers :/

amf12• 2 years ago

The linked post links to another blog which is about this.

worksonmine5• 2 years ago

Anyone else have AI fatigue yet? I haven't even tried GPT and I'm already sick of AI this LLM that.

danielbln• 2 years ago

You haven't tried what would only be called science fiction 4 years ago but are tired of it? The hype machine is grating, but do try a Gen AI model. I use it for code, for ideation, for various NLP tasks. It's at the very least moderately useful in various tasks, and extremely useful in some.

edit: tone

worksonmine5• 2 years ago

I get what it does, I've seen demos and it is no doubt impressive, but I have no personal use and don't see the point of giving them any data. I'm not limited in my current workflow and prefer official docs over the LLM interpretation of those same docs. The real satire are all the comments on any problem asking "have you tried GPT", more annoying than the Rust community.

What do you use it for that you think would benefit me?

rldjbpin• 2 years ago

you are not alone, and ironically i work in this field lol

mattigames• 2 years ago

The only AI feature I'm interested its an AI that lets me remove ads automatically, but Google by definition its not interested in such feature.

rockooooo• 2 years ago

AI theme generation really seems like a solution looking for a problem

voidhorse• 2 years ago

The vast majority of AI development right now fits the solution looking for a problem mold. People are pushing hard for the adoption of LLMs in areas where the existing solutions are not only more predictable, but require equivalent or less effort to using an LLM.

At some point the hype will die down and we'll find out where these tools actually fit, but yeah right now it's madness.

duxup• 2 years ago

It is a neat "it can do that" kinda thing but I also wondered when I need that.

Having said that chrome customization has always kinda bit me in the butt eventually when something changes and looks odd now and ... I just tend to avoid it altogether now.

chankstein385• 2 years ago

Yeah honestly I just want my browser to be dark and not spam me with stuff when I open a new tab. I don't need some fancy picture.

yanis_t• 2 years ago

Here are some better ideas from the top of my head:

- summarise an article

- find information on a given topic (free-form input text)

- full voice control ("click that link", "read that article", "find this")

- auto-submit a captcha

marricks• 2 years ago

Seriously, their first example seems useless to most people. Naming a tab group??? That doesn't take any time, little thought, and who does that regularly?

Summarizing an article seems like something everyone else can do OK. It's a huge avenue for bias (maybe that's why it's reasonably elided) but at least it's actually useful.

summerlight5• 2 years ago

It's not just naming, but grouping as well. If you have 200 tabs and don't want to spend too much time on it, this can be very helpful.

Hoefner2• 2 years ago

Maybe they are just testing AI with less used features first

chankstein385• 2 years ago

This is the only feature I'm excited about. I perpetually have 100+ tabs opened and have tried tab groups but eventually things get disorganized again. The ability to automatically group similar tabs, assuming it works, is going to be game changing.

kuhewa3• 2 years ago

Seriously, their first example seems useless to most people. Naming a tab group??? That doesn't take any time, little thought, and who does that regularly?

Funny, naming things, whether variables or groups of things is the main reason I use LLMs to date. Add in grouping as well and that handles something that puts me under a lot of cognitive load, because I can never shake the feeling I have ot yet manually grouped things optimally.

rschiavone• 2 years ago

"auto-submit a captcha"

we have come full circle

crummy2• 2 years ago

well, they're too hard for me to solve on my own

rubslopes• 2 years ago

Well, we have been training them to do this for a while...

lawlessone• 2 years ago

Watch an ad so i don't have to

ehsankia3• 2 years ago

1. This is actually a feature on Pixel through the assistant [0], surprised it's not on Chrome itself

2. That doesn't really seem like a Chrome feature? Belongs more on Bard.

3. That seems like a Google Assistant feature too, some of that actually may work on a pixel phone, though might be nice to have on desktop too.

4. Will never happen. Google themselves have a captcha product so defeats the point.

[0] https://support.google.com/assistant/answer/14163109?hl=en

mderazon• 2 years ago

I was excited to test it on my pixel 7 just to find out it's only available on Pixel 8. Why?....

jtolmar• 2 years ago

I think for #2, they meant like AI-powered control-F / find in page.

Which is actually the first non-novelty AI tool I've heard someone pitch that actually sounded like a good idea. Way more visible failure mode than summarizing.

kuhewa3• 2 years ago

1. This is actually a feature on Pixel through the assistant [0], surprised it's not on Chrome itself

It is only on the Pixel 8, not the previous models and their mid-range $ variants so they aren't giving it away for free just yet

bane• 2 years ago

Take this tab collection, build a model or a RAG or whatever around them:

- Let me chat with a bot that knows the information from the collection

- Use the information to generate a summary

- Let me guide it in generating a well sourced article

Build a knowledge graph from the web

- Trace a source of information back to the originating point to help eliminate derivative blog spam

- Help moderate media bias and challenge echo chambers

Automatically recognize spam, scams, etc.

Let me describe something I need in text, return back links to shopping sites that sell that thing, if nobody has it, generate a 3d model, or more formal description of it and supply me with connections to let me farm it out to an additive manufacture, one-off makerspace place or something.

rezonant• 2 years ago

- Post resulting article on your spam blog

- The circle continues

everdrive• 2 years ago

I don't want any AI features in anything. Yet another reason to avoid Chrome. I have Chrome installed solely to watch netflix on Linux.

ugh123• 2 years ago

Love the textarea integration. I wish Chrome could do a better job of saving "drafts" and/or backing up text somehow. Adding long content is a constant worry for me to lose it somehow to an error or accidentally closed tab.

ehsankia3• 2 years ago

Absolutely. For the longest time I had this extension called "Comment Save" which saved anything you typed in a textarea. It doesn't seem to exist anymore, and I haven't been able to find a good replacement. I would also much rather have it in the browser than giving full permission to some third party.

The back button in Chrome sometimes help but I still lose long messages all the time.

darkhorse222• 2 years ago

The last thing I want is further telemetry from Chrome to Google. I'm so glad I switched to Firefox. This should be an API, not a browser integration.

crummy2• 2 years ago

an API to group your browser tabs..?

hoten• 2 years ago

Maybe they meant an extension API, not an external service.

Al_Ptr3• 2 years ago

There is an extension for Firefox that does it.

https://addons.mozilla.org/uk/firefox/addon/tochunka-smart-t...

Communitivity• 2 years ago

I had a thought while reading this, and I don't know if this would be the case but...

If it works by you hover over a link and Google gets the content in the browser behind the scenes and sends it to the mothership, where it's summarized and the summary then sent back to you to be displayed by the browser, then you may be accessing the linked page using your stored credentials, which give Google access to content they wouldn't otherwise have access to.

lxgr3• 2 years ago

The same is true for translations in most browsers, right? At least I'm not aware of any browser that does it client-side/offline.

Edit: I stand corrected, Firefox does it offline! Thank you, Firefox team, this is awesome and I'll likely be using it more often now :)

michaelmrose• 2 years ago

Firefox does it offline

burkaman2• 2 years ago

Firefox does it locally: https://www.mozilla.org/en-US/firefox/features/translate/

asadotzler• 2 years ago

Firefox does local LLM based translation. I kicked that project off in 2022 and it shipped last year.

host0• 2 years ago

Unlike other browsers that rely on cloud services, Firefox keeps your data safe on your device. There's no privacy risk of sending text to third parties for analysis because translation happens on your device, not externally.

https://support.mozilla.org/en-US/kb/website-translation

comprev• 2 years ago

Sounds like a sneaky way to add your personal social media feed into their AI training data.

Edit: the suggestion that translation functionality already does this is valid though perhaps this expands the scope to data in the users default language?

smusamashah2• 2 years ago

I have a chrome extension [1] which lets you re-write your selected text, or look-it up via ChatGPT using your own custom prompts. Gives you more control on what kind of suggestions or answers you want basically.

Won't help with rearranging/grouping tabs, but can definitely help rephrase text in input fields or looking up info.

[1] https://github.com/SMUsamaShah/LookupChatGPT

phillipcarter• 2 years ago

I really wish the first part of this article had an explicit "Here is how you get started" section. I just about missed it because it's a paragraph that links to a support article. If they want people to actually use this stuff, why not make turning it on front and center?

butz• 2 years ago

Soon you won't need to browse the web at all, Chrome will do everything for you: watch youtube ads, click on sponsored links, write positive reviews for restaurants buying ads from adsense, and write negative reviews for ones not advertising with google, fight with edge which browser is the default one. On the bright side, you will be able to enjoy more time offline.

bradgessler• 2 years ago

I look forward to AI blocking ads after it gets tired of clicking through all of them.

delta_p_delta_x• 2 years ago

AI blocking ads after it gets tired of clicking through all of them

The future is (almost) now: SponsorBlock[1] skips embedded ads in YouTube videos, based on crowdfunded timestamps.

[1] https://github.com/ajayyy/SponsorBlock

disqard• 2 years ago

Your comment reminded me of Douglas Adams' Electric Monk:

"The Electric Monk was a labour-saving device, like a dishwasher or a video recorder. Dishwashers washed tedious dishes for you, thus saving you the bother of washing them yourself, video recorders watched tedious television for you, thus saving you the bother of looking at it yourself; Electric Monks believed things for you, thus saving you what was becoming an increasingly onerous task, that of believing all the things the world expected you to believe."

shipit1999• 2 years ago

I switched to Brave. Slowly trying to get the number of Google products I use down to zero.

gr__or• 2 years ago

Kind of amazing how unable to deliver Google seems to be here. Looking at Arc, a new player, and the kind of AI features they came up with, this here looks more like features developed by McKinsey rather than by someone with domain knowledge.

ncann• 2 years ago

The only "AI feature" I use in Chrome is the live caption one for French (which requires Chrome Canary). I use it to get automatic live caption while listening to French podcast since I'm learning the language. It's buggy as hell though, so if anyone has a suggestion on a replacement that would be much appreciated!

littlekey• 2 years ago

Anyone else think the whole idea of "tab management" tools is crazy? The way to manage tabs is to close them.

pixelbath• 2 years ago

And if you'd like to read it later, bookmark it! You can organize bookmarks into groups, give them custom names, and they don't require the browser to be constantly hoarding multiple gigabytes of RAM.

eviks2• 2 years ago

Neither do the tabs, autosleep/unload exists, you know

And bookmark organization is worse since it's a separate UI breaking your mental connection to the physical layout that you spend most of the time with

layer8• 2 years ago

Or filtering and searching. Anything you can have a large number of should have an obvious filtering & searching UI. After how many decades of UI are we still so behind on basic usability design.

rockemsockem9• 2 years ago

This is a feature in Chrome today.

eviks2• 2 years ago

If you don't limit yourself to only the closing one part of management, you might see the reason behind the craze

dougb5• 2 years ago

I wonder if this "Help me write" will give different suggestions from the Google Docs "Help me write" feature, or from the dozens of other help-me-write features that are cropping up these days within text-oriented webapps (e.g. Notion).

Imnimo• 2 years ago

This feels like a VP at Google told the Chrome team they needed more AI.

albert180• 2 years ago

Another reason to use Firefox

Chrome experimental AI features

Related Stories