Google Handwrite

http://techcrunch.com/2012/07/26/search-in-cursive-google-now-lets-you-hand-write-search-queries-on-phones-and-tablets/

by sama • 14 years ago

143 76 14 years ago

techcrunch.com

realmickey• 14 years ago

When is Google's/Android's competitor to OneNote going to be available?

stcredzero4• 14 years ago

Android/iOS/Metro -- whichever one has a properly executed answer to OneNote is going to succeed like gangbusters.

(Putting it that way, it speaks to serious dysfunction at Microsoft if it's not them, and indeed, there's a good chance that it may not be them.)

rhplus• 14 years ago

You're right that note-taking will be one of the killer apps for the next generation of tablets. OneNote for Metro ("OneNote MX") is already available in beta, so I'm guessing Microsoft is chasing that:

http://apps.microsoft.com/webpdp/en-US/app/onenote-mx/f02238...

http://www.winsupersite.com/article/office-2013-beta2/office...

ericdykstra4• 14 years ago

Fun feature. Would be killer with support for non-latin based language. Imagine walking down a street in Japan, seeing a restaurant, and wanting to search for reviews. Oh no! It's written in kanji and you have a hard enough time reading basic words, much less names. Open up Google on your phone, write the kanji and add the japanese word for review behind it, and off you go.

Great potential, anyway.

positr0n• 14 years ago

The video shows a Japanese woman doing just that :)

Permit• 14 years ago

Judging from the commercial it does. Although I won't embarrass myself by guessing the language.

http://www.youtube.com/watch?v=uyeJXKfAcpc

ericdykstra4• 14 years ago

Just tested myself. You need to change the browser default language to make it work, but it does a very good job!

wheels• 14 years ago

That was Japanese (Hiragana, one of the glyph forms that Japanese uses).

T-R• 14 years ago

It was Japanese - it was the Kanji for sakura (桜) - hence the cherry trees in the background.

minwcnt5• 14 years ago

Having to handwrite the Kanji seems like an extra step though - it'd be easier to simply take a photo.

ken• 14 years ago

It's not easier if you're standing in a forest of 桜 and there's no paper and pencil around, as in the demonstration video. :-)

And even if you do have it written, it may be easier for the user, but it's harder for the computer. The order and direction of strokes is standardized, and getting this information as the user writes it is tremendously valuable.

I've played with handwriting recognition algorithms for kanji before, and if you assume the stroke order and direction are correct, you can make a surprisingly good algorithm with very little code. Detecting a character from an image is much harder.

The iOS handwriting recognition, for one, takes this into account. iOS officially only has hanzi (Chinese) handwriting recognition, but it usually works OK for Japanese, too, as long as it's a character that both languages have. But when you run into one of the characters where Chinese and Japanese have standardized on a different stroke order, like 王, it'll never recognize it when written using the Japanese stroke order, no matter how nicely you write it.

w1ntermute7• 14 years ago

You can already do this on Android in any text field of your choice using the "7notes with mazec" keyboard[0]. The kanji recognition is nothing short of miraculous[1]. It's a little expensive, so check out the free 10-day trial[2] if you need any further convincing.

0: https://play.google.com/store/apps/details?id=com.sevenknowl...

1: http://i.imgur.com/rfHTu.png

2: https://play.google.com/store/apps/details?id=com.sevenknowl...

ericdykstra4• 14 years ago

Thank you for the recommendation. I haven't checked this solution out yet, but it looks fantastic! I have never seen an app with this kind of kanji recognition.

californian• 14 years ago

The Google Translate app for Android does this. I was recently in Japan, and that app was invaluable. It's also pretty decent at Japanese speech to English translation.

a3_nm• 14 years ago

For this use case, I'd rather take a picture of the restaurant name rather than handwriting the kanji.

6ren• 14 years ago

Using the camera would remove a step - have you seen word lens? http://questvisual.com/us/ (though it doesn't seem to support kanji).

jenius4• 14 years ago

Sometimes opening a keyboard and typing is just too efficient. Sometimes you need a technology that will slow your life down and make it more prone to errors. Google handwrite: why type when you can awkwardly scribble?

bokonon2• 14 years ago

I agree with you, but at the same time I think it's great that somebody is at least trying to innovate the tools we have for interfacing with technology. I know handwriting recognition isn't anything new, but the idea that the keyboard is all we will ever need can only contribute to the stagnation of advancement.

I'm still waiting for the day I can telepathically control all my devices.

stcredzero4• 14 years ago

There could be a lot done with eyeball tracking. We already have setups that can accurately determine where you will be looking a fraction of a second in the future.

Eyeball tracking combined with speech or handwriting translation could be quite powerful. The eyeball tracking could provide a lot of contextual information to make speech/handwriting recognition more accurate.

TheSOB88• 14 years ago

Because touch-screen typing, and seeing the phone's wrong answers about what you meant EVERY time, isn't incredibly frustrating. ;)

ElliotH• 14 years ago

If you hate onscreen keyboards, try Swiftkey, it genuinely is an amazing thing. It's practically psychic.

eavc• 14 years ago

I can see handwriting on a small space being faster than typing on a small space if the technology for recognition were good enough.

raldi2• 14 years ago

Why speculate when we can race?

jenius4• 14 years ago

Is this a serious comment or are you trolling? Don't tell me you think hand writing is faster than typing...

agscala• 14 years ago

Basically google wants more data to improve their handwriting recognition for something else down the line. Smart move

SubZero• 14 years ago

Now THAT would be amazing. Incredibly smart move. What better way to create your own handwriting recognition program than to have all 4 billion people field testing it for you.

SoftwareMaven• 14 years ago

They've already done that. That was the entire purpose of Google 411 (except voice instead of hand writing).

abraham• 14 years ago

Not for something down the line though, for the current Drive search indexing.

https://support.google.com/drive/bin/answer.py?hl=en&ans... https://support.google.com/drive/bin/answer.py?hl=en&ans...

dag112• 14 years ago

This is pretty cool, and works really well! For me, though, Jelly Bean voice search from my lock screen is still the quickest.

setheron• 14 years ago

My company has forced pinpad lock screen on my jelly bean.

Can't seem to find the voice search from there :(

dag112• 14 years ago

I use pattern unlock. But I installed WidgetLocket to give me a primary lockscreen where I was able to make one of the swipe positions a Google Now shortcut, and another position a Google Now Voice Search shortcut, in addition to the two existing positions of a Camera shortcut and the Unlock shortcut.

ramanujam• 14 years ago

Draw a heart and it'll spell it out. Nice!

Splines• 14 years ago

It also handles diacritics too, which is handy.

grantjgordon• 14 years ago

This works remarkably well. I'm shocked.

RandallBrown• 14 years ago

yeah, I've done handwriting with a stylus on windows and mac before and had much worse results than I'm getting with my finger on this. Very impressive.

pragmatic• 14 years ago

Note: This doesn't seem to work on an Opera mobile browser. Seems to work ok in the default android browser.

MatthewPhillips• 14 years ago

Google doesn't release mobile products for any browser except WebKit. They specifically whitelist.

mikek2• 14 years ago

I wonder if, like (800) GOOG-411, the primary reason Google is doing this is to gather data.

mikek2• 14 years ago

Why am I being downvoted for this?

mikeevans• 14 years ago

I'm guessing because everything Google does is to gather data.

dannyr• 14 years ago

I guess people think it's a snarky remark.

My guess is you read Steven Levy's book (or at least an excerpt).

I read it so I get what you're talking about.

jemfinch• 14 years ago

Because it's terribly obvious.

jonursenbach• 14 years ago

Everything Google does is to gather data.

deelowe• 14 years ago

Well, their mission is to organize all of the world's information. So, yea. I think this is probably about gathering data.

mtgx• 14 years ago

I wonder, do latest versions of Android have support for 3rd party Wacom styluses? Although I'm not sure Wacom even sells 3rd party styluses for capacitive tablets that are as accurate as Samsung's S-pen or those old Windows tablets.

The reason I'm asking is because I'd like to buy say a Nexus tablet to use with a stylus, but want the same level of accuracy as Samsung's S-pen or better, without having to buy Samsung's Galaxy Note devices. Do those need a special panel as well to have that kind of accuracy, do they only require that Android has the necessary API's and the support for those styluses?

swombat• 14 years ago

TechCrunch appears to be mostly unreadable on the iPad, thanks to their fancy template reloading/redirecting... Smart.

dredmorbius2• 14 years ago

Mostly unreadable on many platforms, along with their affiliate sites, especially with JS disabled.

jonknee• 14 years ago

In case anyone is interested, I watched the network requests go by to see how it works. Google is doing the detection server side (no surprise there, but it was smooth enough that I had to check) and receives the drawing area (writing_area_width, writing_area_height), the "ink" which is an array of points and any already existing characters in the search field. The return data is the best guesses for the input. Curious that multiple options are returned, I'm not sure how those are used. Another AJAX request is made for Google Instant.

dredmorbius2• 14 years ago

As anyone who mastered PalmOS's grafitti can attest, sometimes writing on a PDA / smartphone is a lot more usable than typing, especially on a soft keyboard.

Input remains the weak point to many or most of these devices.

gbog• 14 years ago

The beta version of Swype is really a major improvement. I'say I can write faster on my phone than on my pc, under certain conditions. The much slower part is copyediting, but it is mostly because sites like hn break the flow with some fancy textarea tweaks.

wasd• 14 years ago

I wasn't able to the setting pages on Chrome while I was signed in. Am I blind or was this a UI oversight?

tonfa• 14 years ago

On the bottom of the search homepage.

darkstalker• 14 years ago

Can this be activated on desktop?

tsahyt• 14 years ago

From a technical point of view this is remarkable. Seems like the recognition works really well, and that's great.

However, from a usability point of view this is probably awful. I've had an app on my phone for a while now that doesn't do handwriting recognition but just stores the handwritten stuff as images. Using that without a proper pen-like stylus is a pain. Writing with your fingers just doesn't really work. Fingertips are just not accurate enough for this kind of movement, which is precisely why we guide a pen with multiple fingers instead of just one.

Additionally it's much slower than just typing it out, even on the sucky on-screen-keyboard that you're provided with these days.

So it's slow and surprisingly hard to use. And frankly, I don't see styluses making their big comeback next year. People are just too addicted to touching things with their fingers.

dazbradbury• 14 years ago

OK, so Google want to improve their handwriting recognition. Fine. But from a users point of view this just tells me one thing:

- Using an on-screen keyboard is such a pain for people that Google changed their HOMEPAGE to accommodate this.

Why does this irritate me?

1) None of the latest generation of smartphones are coming out with a hardware keyboard. The best are always keyboardless.

2) Stylus-based input, which is the other extremely fast form of text entry, has all but died out (I know some products still exist - but it was essentially dead when capacitive screens won out).

Hence, we've gone backwards from 10 years ago. It was faster for me to send a text/email from my phone 10 years ago then it is today. That is hugely disappointing.

asto• 14 years ago

Yes, this exactly. I desperately want to move from my Blackberry to an android phone but all the best phones seem keyboard-less. Typing on a touch screen is a major annoyance for anything more than typing a search string or sms.

hessenwolf• 14 years ago

Go horizontal and use two thumbs. You get pretty quick. There are likely other techniques.

ivanilla• 14 years ago

It doesn't work on my iPhone with iOS 4. Anyone else having problem getting it to work?

danyork• 14 years ago

I'll note that despite the Techcrunch headline about Google Handwrite accepting cursive handwriting, it doesn't really do so. In fact, Google's help pages specifically say you should use block printing versus cursive. I tried some cursive and the recognition didn't work all that well. However, the block printing was generally quite accurate.

I wrote about some of my views on this at: http://www.disruptiveconversations.com/2012/07/google-now-le...

I actually see a great value to it for times when I'm walking around a conference with my iPad and want to do a quick Google search. I can't really use voice (i.e. Siri) and I can't use two hands to type. This handwriting could be an interesting option.

Google Handwrite

Related Stories