Why Can't Gmail Search?

http://designbygravity.wordpress.com/2009/08/24/why-cant-gmail-search/

by cschanck • 17 years ago

131 85 17 years ago

designbygravity.wordpress.com

tvon3• 17 years ago

The problem seems to be substring searching, which I guess isn't something I've ever tried to use.

I tend to think of the gmail search as being fairly powerful and much faster than my usual mail client (Mail.app), it's one of the few reasons I ever use the gmail web interface.

If anyone is curious, the docs are here http://mail.google.com/support/bin/answer.py?hl=en&answe... (I had never bothered to look them up until now).

einarvollset• 17 years ago

I once spoke to someone in the know about this, and the main reason is that it's quite expensive for them to do any reasonable job with stemming, etc.

I know that sounds weird (but it's google, they're omnipotent!), but it makes sense: It's worth their while to stem content they crawl and index off the web, cause everybody could in theory access any given page. However, with email, the only person who'll ever benefit is the recipient.

anurag2• 17 years ago

> However, with email, the only person who'll ever benefit is the recipient.

I'm not sure that makes sense - if they add stemming, all users of GMail benefit. Going by your explanation, it wouldn't make sense to add any expensive features to GMail, because the only person who would ever benefit from them is the single user.

antipaganda• 17 years ago

I think you misunderstand the nature of stemming; the point is that each and every user's inbox would have to be processed individually, and apparently Google doesn't think the overhead is worth the results.

bd2• 17 years ago

They could use Gears for some more in-depth local indexing, then it would be you bearing extra computational and storage costs.

bradgessler• 17 years ago

That's a really interesting notion. Have any web applications moved indexing to the client-side before with gears and/or js?

IsaacSchlueter• 17 years ago

You could probably build a gadget that does this, yes?

zngtk4• 17 years ago

It would at least make sense to deal with plurals. I can't tell you the number of times I've searched for something using an s (or not) at the end, and failed to find what I was looking for, only to remember later that I should try to add or remove an s from my search term.

Freaky• 17 years ago

If it's expensive, then why not reflect that by providing it to people who pay for their accounts?

flatline• 17 years ago

It seems like they've worked on mitigating this on the web with suggestions for alternate spellings and by displaying related searches. While you can see stemming at work in many Google searches, I'm pretty sure they don't build extensive substring indices on the web end either. For example, I've had searches where a substring returns 0 results and the exact phrase returns a handful.

DannoHung• 17 years ago

Perhaps it would be an option that they could sell to people?

mistermann3• 17 years ago

Exactly....I have always wondered why there is no google premium...I would gladly pay $100/year to have a collection of certain domains easily removed from my google searches, or pick from a list of my favorite sites for a "site:" search, etc etc.

_pius• 17 years ago

Finally, someone else who feels this way! I've never understood why people praise the search in GMail so much.

jsonscripter3• 17 years ago

Gmail search is fast. On outlook, a desktop mail client, it can take up to an hour to search for a simple term, in my experience.

shiranaihito• 17 years ago

It may be fast, but what you want is results.

timcederman• 17 years ago

Have you used Outlook 2007 with the built-in search indexer? Incredibly quick.

discojesus4• 17 years ago

if I may quote some scripture:

"Gerald Weinberg tells the story of a programmer who was flown to Detroit to help debug a troubled program. The programmer worked with the team who had developed the program and concluded after several days that the situation was hopeless.

On the flight home, he mulled over the situation and realized what the problem was. By the end of the flight, he had an outline for the new code. He tested the code for several days and was about to return to Detroit when he got a telegram saying that the project was canceled because the program was impossible to write. He headed back to Detroit anyway and convinced the executives that the project could be completed.

Then he had to convince the project's original programmers. They listened to his presentation, and when he'd finished, the creator of the old system asked, "And how long does your program take?"

"That varies, but about ten seconds per input."

"Aha! But my program takes only one second per input." The veteran leaned back, satisfied that he'd stumped the upstart. The other programmers seemed to agree, but the new programmer wasn't intimidated."

"Yes, but your program doesn't work. If mine doesn't have to work, I can make it run instantly."

- _Code Complete_, pp.595-596

rufo• 17 years ago

An hour, really? I don't use Outlook, so I don't know if that's right - but I suppose that's why the various Outlook search plugins are so popular.

I haven't had a problem with Gmail search as described, but I would not classify it as fast. Often I'll type in a simple query and have it spend 10-20 seconds before it returns results. If I perform the same query on the same dataset in Spotlight in Mac OS X, it starts to return results instantly with the search completing within 5-10.

mattmaroon• 17 years ago

My outlook box is pretty large and it never takes more than 3 or 4 seconds. It's not the greatest search either, but it does handle substrings nicely and automatically. It works most of the time (though I'd still be much happier if it worked all of the time.)

dasil003• 17 years ago

Probably because most other email search sucks even worse. Frankly I think a lot of peoples threshold for saying something "sucks" is too low. Come on, it could be better, but it doesn't totally suck.

Reminds me of Louis C K's bit on Conan: http://videogum.com/archives/late-night/the-videogum-louis-c...

mistermann3• 17 years ago

In my estimation, not being able to do substring searches in emails, sucks.

plusbryan2• 17 years ago

I love Gmail search because I've seen the difference - have you ever tried searching email in Mobile Me? It's like they ignore your query and search a random string.

FooBarWidget• 17 years ago

Gmail's search works well. Except when you need substring search or fuzzy search, in which case it absolutely sucks.

Most of the things in Gmail are done very well but non-exact search really needs improvement.

imp• 17 years ago

I think I hit the same problem occasionally, but I can usually keep trying different keywords until I find one that works. If he only tried 4 keywords like he writes, then there was probably some word in the email that would have been at least semi-unique. Maybe "earbud" or "order" or something. Then browsing or date-based filtering can usually do the rest. Kind of a pain, but not enough for me to ditch GMail completely.

mildweed• 17 years ago

I hope they build more robust searching into their Wave client.

euroclydon• 17 years ago

That's really clever how he auto-forwards all his gmail messages to yahoo. I wish I had done something like that form the get go. Too late now. Here is a Lifehacker article explaining how to use automate gmail backups with sendmail and cygwin:

http://lifehacker.com/235207/geek-to-live--back-up-gmail-wit...

ramoq3• 17 years ago

I actually heard that it's quite costly to index emails for gmail (not from a gmail source, just random web chatter). It makese sense. Most emails are not important, they are just huge amounts of random chatter. I'd imagine indexing emails (full-text) properly would require some effort. The gmail team is probably on a budget :)

But then again, if they can index the web. Why not email.

plusbryan2• 17 years ago

I'm no expert, but reMail does it on my freakin iPhone - I'm guessing they could do even better on a cluster...

ramoq3• 17 years ago

Agreed, Google could probably do it better. But imagine maintaining separate indexes for each and every gmail acct and constantly updating those indexes. reMail used to do something similar to that.

ajb• 17 years ago

I have hit this problem too. If you subscribe to the git email list, but only want to look at discussion, not patches, it ought to be possible to filter out the patches with 'subject:patch'. But this doesn't completely work, because quite often the patches have patchv2 patchv3 etc, which doesn't match.

The curious thing is that google groups is even more painful; where you would think it would be more worth having the indexes, because more people are going to search the same data.

sfk• 17 years ago

I think Google found that they can make more money by directing Usenet searches to crappy archive sites full of Google ads.

Which is a shame, since 5 years ago Usenet search was absolutely wonderful.

mblakele• 17 years ago

I find it easier to let an external service index my list subscriptions:

http://markmail.org/search/?q=list%3Aorg.kernel.vger.git#que...

(or http://git.markmail.org/search/?q=#query:type%3Adevelopment if you want to use the list-specific site).

sundae79• 17 years ago

Not only that, I also found that if you search by sender name and the sender name is never in any of the subject of any of your emails, it doesnt find it.

tvon3• 17 years ago

use "from:example.com" or similar... important part being the "from:" bit

gecko2• 17 years ago

Speaking of which: you and I, as developers, understand search axes, and use them intuitively. But they're utterly opaque for someone who hasn't encountered that interface before--and given that Google itself doesn't support search axes in its main product, I don't think they're an interface someone's likely to learn elsewhere, either. If you're unaware of search axes, you have to click, "Show search options," which is in an extremely tiny font, next to a button that very notably says "Search the Web"--not search my inbox. Furthermore, once you actually discover that, and use it, you still don't know about the axes, because that search box doesn't use them. The only place where you can discover search axes is by clicking on a label, which results in "label:foo" being shown in the search box--but that's actually a lie, because the output if you dump that into the search field, versus if you click on a label name, don't match if you have more than about 30 messages in that label. Go try it.

So, yeah. Axes are great. And they're completely unintuitive and impossible to discover in Gmail unless you read the help, which no user ever does. So you've done a great job finding a part of Gmail's search interface that's at least as broken as the underlying implementation.

shrikant• 17 years ago

This is just not true.. at least it doesn't seem to be, for me. Do a lot of people face this??

moldenke• 17 years ago

At least it's better than Google Groups search.

sahaj• 17 years ago

just an observation: lot of attacks on gmail today/recently.

justinhj• 17 years ago

Good points there. When I search in google I know I can be sloppy with my typing because 90% of the time it's quicker to get it wrong and click the "did you mean?" results, rather than edit my input.

That feature's absence is very obvious when struggling to find data in gmail.

Personally I use subject line tags for stuff I want to filter on. (Like 'music' 'todo' 'idea'), and when I store something in gmail I want to remember I make sure the key words I will search for are very obvious (and easy to spell).

periferral• 17 years ago

it seems to me that this problem isnt just limited to gmail. other than google search itself, I think search in other google products (android included) is lacking when it comes to what I presume to be basic functionality.

dpcan2• 17 years ago

His experiment fails.

He didn't apply the same inputs to both systems, therefore his findings must be discarded.

He changed it up, and had he STARTED at Yahoo and performed the exact same searches, he would yield the same results, only reversed, and his blog post would be about why he dumped Yahoo instead.

Here's my case:

1) He searched Google for "Zags" which is NOT in "Zaggs". So, no results.

2) Then he goes to Yahoo and searches for "Zag" which IS in "Zaggs" - AH HA, he gets a result (of course he does!)

3) He RETURNS to Google and searches for "Zagg" which IS in "Zaggs", and boom, results. Surprised? Me neither.

I guess my point is, I don't have enough karma to have down arrows next to this post, so I'm going to write a cranky comment about this article.

beza1e1• 17 years ago

He did search for "zag" in Gmail. Right after "zbuds" and before "headset".

dpcan2• 17 years ago

GAH! All that frustration over nothing. Thank you.

nkurz• 17 years ago

3) He RETURNS to Google and searches for "Zagg" which IS in "Zaggs", and boom, results. Surprised? Me neither.

You might want to hold off on your cranky comment. Would you be surprised if the search for 'Zagg' did NOT turn up the results for 'Zaggs'? Because for better or worse, that's what actually happens.

GMail search does not perform stemming (like removing that final 's') and also does not allow for substring searches. So in fact, a search for 'Zagg' will return nothing. While this isn't a fatal flaw, it is a drag.

ramoq3• 17 years ago

Oh yeah, the gap in gmail's search ability does hint at possible startup ideas to solve this problem. However I wonder why some great startups have passed on directly competing in this sphere (reMail being a good example)

calcnerd2562• 17 years ago

If lack of substring search is bad, how much worse is the fact that it sometimes doesn't even return all the results for a correct query. I use labels a lot, and I often use boolean searches to search for exactly which Venn intersection I need. A search for "in:inbox is:unread -label:Triangle" (without the quotes) in my gmail turns up messages labeled Triangle. I have similar other problems and cannot trust gmail's search.

calcnerd2562• 17 years ago

My solution was to use fetchmail, mb2md, and fgrep on my local server. Now I feel like I can almost trust the setup I have.

andyf• 17 years ago

At least gmail filters out spam better than any other mail client and yes it includes Yahoo Mail.

zandorg• 17 years ago

Typical of someone to glibly say 'where's regex matching?' when it's very advanced, and it's possible Google simply don't have the cpu for making (and searching) a search tree for many megabytes of mail.

I don't know how Yahoo do it, but this guy should at least present a solution to the problem. It reminds me of that Alexei Sayle sketch where he says 'I blame the council' (which in the UK is the local authority and handles all sorts of things in a town or city), and at the end, wanting someone to blame for blaming the council? He blames the council!

Jem• 17 years ago

I did a search for a mail I was sure I had in gmail a few weeks ago, and it returned 0 results. I came to the conclusion that I'd simply deleted the mail.

Now I'm wondering if it wasn't me at all.

cubedice• 17 years ago

What's interesting is that after reading this article I've realized I always expected e-mail search to be bad. Why? Possibly because, when another entity holds a large portion of my social information, it feigning ignorance about deep technical knowledge of my personal life (which is likely better than my own understanding) makes me feel good.

It's less 'big brother'y, which may explain why poor e-mail search doesn't bother me. It's as though its waiting for me to get the answer first.

snprbob862• 17 years ago

I'm going to keep my fingers crossed for "search as you type" across all Google properties. This has got to happen...someday...right?

slater• 17 years ago

I'd suggest he tries using Lotus Notes' built-in "search". If there was ever a more useless function, it's LN's pathetic attempt at searching for stuff. It'll find stuff to every search term you enter! Just nothing that a) contains that search term, and b) resembles anything you wanted to find.

Why Can't Gmail Search?

Related Stories