Gemini API File Search is now multimodal

https://blog.google/innovation-and-ai/technology/developers-tools/expanded-gemini-api-file-search-multimodal-rag/

by gmays • 2 months ago

156 46 2 months ago

This might be great and all but I am still miffed at how simple search on AI Studio is. You can only search the titles of your conversations and nothing inside them. On top of that they messed with the scrolling so Ctrl+F doesn't work reliably.

greesil• 2 months ago

Too bad they can't just easily vibe code new features.

bloqs• 2 months ago

Yeah, what happened to no more SWE

telotortium• 2 months ago

Ironically, they probably could if they used Codex or Claude Code. Those harnesses and models are good enough to do that these days (since late last year, and getting better since then). However, it seems that DeepMind and no one else at Google has access to either of these.

stingraycharles2• 2 months ago

Yeah, it’s surprising, Claude Desktop has had project files since decades which are chunked/indexed and automatically injected into your context based on the topic.

You’d think this would be fairly obvious for Google to do, but it’s probably an organizational problem rather than a technical one.

pants24• 2 months ago

It's incredible how far behind Gemini has gotten, both the product and the model. Even the ChatGPT plugin for Google Sheets blows away the native Gemini integration.

Everyone thought Google was pulling ahead with Gemini 3. For a minute there they had the best language model, image model, AND video model in the world. But it's like they decided to pull over for a nap while OpenAI and Anthropic flew by.

wilj• 2 months ago

I just cancelled my Gemini subscription yesterday. I have a big private fork of OpenCode, and I did it the wrong way to start with, so I couldn't pull from upstream.

So I put together a plan for refactoring it, step by step, with tests, etc. After literally 8 solid days of fighting with Gemini 3 Pro, I still couldn't pull it off.

I gave GPT 5.5 a chance with the same prompt, plans, and repo. I'm not sure how long it took, but when I checked in on it a few hours later it was done. All tests passed, everything exactly how I'd asked, and better (it made some improvements).

comboy• 2 months ago

3.1-pro is still very capable, and API is at competitive price vs e.g. Anthropic, they just can't seem to figure out RLHF and harness. It needs a lot of guiding, it tends to be lazy and poorly sticking to instructions by default.

It just feels like many google products really, they are capable of really amazing things, it's just that nobody there seem to care. I would guess they are likely optimizing more for internal use than their vast userbase.

thefounder• 2 months ago

I never felt Gemini was ever better than the OpenAI or Anthropic. I think it’s more on par with open source models than the top 2

diegoperini• 2 months ago

I have the opposite experience where Gemini (even the flash models) has the only useful model for my reverse engineering related use case. My hunch is Google utilizes its free access to entire Google search indices to train itself from niche non-English speaking community websites, much frequently and in a "relevant" manner, which in the end gives these models the most up to date info for this particular kind of work. Every other model is just either 10 years outdated with their answers or simply hallucinates like waaaay crazy.

riddlemethat• 2 months ago

It’s still the best option for uptime, document analysis (on a cost basis), and Google is less likely to experience a significant cybersecurity breach than a less established company. They’ll be fine as long as they stay in the game even if they never have a Ferrari again plenty of people buy Toyota.

jmathai• 2 months ago

My non technical wife knows both ChatGPT and Anthropic (admittedly, because of me) but doesn’t know Gemini. This is amazing to me.

Surely she has seen Gemini in Google search but even her use of that is plummeting.

Google has so much revenue that they’ll be around for a long time. But I feel they are fumbling the opportunity with AI. Even in corporate, where we have Gemini. The conversation is fully around Claude. No one talks about Gemini.

bachmeier3• 2 months ago

Maybe they've decided they don't want to play the same game as OpenAI and Anthropic? They're much better positioned for the high volume AI work that's likely to be where the money is made, with calls to APIs doing routine things for all the businesses of the world. They're also the only big US player that has an open model that you can build on. I don't think vibe coding or the most cutting edge capabilities are what will determine profit from AI.

qingcharles• 2 months ago

I've come across a few weird search issues like this with Google lately. Entire company built on the best search engine ever created; can't do search properly in their apps.

sega_sai• 2 months ago

The search in Gemini app in the browser is so embarrassingly bad that I get an impression that nobody of importance in Google must be using it otherwise they would have fixed long ago.

varispeed• 2 months ago

I am more miffed that you cannot delete conversations.

buggyipadmettoo• 2 months ago

I built a solution for this (in 20 mins with the help of Gemini!).

1. use Google App Scripts to duplicate all the convos saved in your google drive to another folder, and append each ".txt". Mine runs every day.

2. enable API in google cloud console, install google-api-python-client on desktop,then created a workflow in Alfred to search from desktop.

Now I can search my AI Studio convos + content just as fast as any other file on my desktop : )

trilogic• 2 months ago

Good to have a choice between clouds and local use.

How much would you pay to have this yours forever, running locally, GDPR and HIPaa compliant, without the headache of privacy or subscriptions.

That´s what we offer with HugstonOne and we did it before Google. Multimodal, Lighting fast RAG, terabytes not kilobytes only :)

All you need is a 32gb ram laptop and HugstonOne, not a rocket science.

FirstPoint• 2 months ago

It’s a striking irony that the world's leader in search is receiving so much heat for poor search functionality and UX within its own flagship AI products

WarmWash• 2 months ago

One of Googles core problems is internal silos of talent. The search team has likely never interacted with the Gemini app team or perhaps even the Gemini app.

For all intents and purposes Google Gemini is a totally separate company from Google search.

kyrra• 2 months ago

Google has ~190k employees. You can't have total collaboration across that many people.

Teams will cross collaborate, but they have to be for specific projects with specific people.

lousken2• 2 months ago

Haven't touched gemini api since they did not support having a $ limit per api key. Is it possible now?

jwithington• 2 months ago

Yes https://ai.google.dev/gemini-api/docs/billing#spend-caps

lousken2• 2 months ago

Finally!

algoth1• 2 months ago

With a 10min delay, via aistudio

ecommerceguy• 2 months ago

My free trial ends this week, which i'm obviously canceling.

ninjagoo• 2 months ago

Is anyone tech-savvy going to actually let any tool with this backend run on their personal PCs?

Any app with this behind the scenes is a non-starter for me.

And anyone think that all those folks ditching Win11 will be going for or recommending any app built on this?

thawab• 2 months ago

Tried multiple times to use the api file search and it’s complex to setup. Ended up going a different approach.

noashavit• 2 months ago

The race for unstructured data continues. It feels like everyone is trying to crack unstructured data extraction with the underlying goal of ultimately using AI to classify and tag insights from unstructured data to create a structured data/graphs for agents to consume and traverse.

Gemini API File Search is now multimodal

Related Stories