Introducing Word Lens

by jf • 16 years ago

1297 281 16 years ago

acangiano• 16 years ago

Whoever created this deserves the boatload of money they are going to make. It has the potential of radically change the experience abroad for people traveling in foreign countries.

storborg• 16 years ago

"Any sufficiently advanced technology is indistinguishable from magic."

mechanical_fish2• 16 years ago

If this thing can be made to do Chinese or Japanese... even if it does so haltingly, with mistakes...

Raise the price. For the love of god. Raise the price. This is much, much more valuable than five bucks. There have been times in life where I'd gladly have paid a dollar per minute for this. And I'm a cheapskate.

btilly• 16 years ago

Raise the price. For the love of god. Raise the price.

Absolutely, read http://bit.ly/bdHKmm to understand why. (Sorry for the shortened URL, the URL recognition software here doesn't understand URLs with apostrophes in here, so it actually is necessary.)

Something like this should not be $5.

anigbrowl2• 16 years ago

Yes. No Android yet but this seems worth $10 to me. I'd do Chinese first. There's a big market for that in California; many Asian families have a language gap between grandparents or parents born abroad and a younger generation born here. Basic communication is easy but things slow down fast if conversation turns to anything complex. Being able to print some words neatly by hand in English and instantly get Cantonese would sell like crazy - I have 5 in-laws for whom it would be a no-brainer.

dholowiski5• 16 years ago

If I was traveling to japan or china, I would gladly pay $50 per language to translate. That's probably too high for an in-app purchase but I think $29 would be a no-brainer for languages that don't use English characters.

Cells• 16 years ago

There already exists an impressive Chinese real-time OCR dictionary for the iPhone called Pleco: http://www.youtube.com/watch?v=x7VTo0656Rc

The dictionary is free and the OCR is available as an in app purchase for $15.

I'm just saying this because it is not trivial to adapt an OCR engine to non-latin scripts, because the image analysis techniques are rather different.

avree• 16 years ago

I actually know the guy who made this, and he has a very convincing Korean version of it working.

jamii3• 16 years ago

Absolutely. I travel pretty much nonstop and I would pay dearly to be able to translate signs on arrival into a new country. Things like 'ATM' and 'Toilet' are crucial.

relix• 16 years ago

That's very, very impressive. Traveling would be so much easier with just this app, amazing! Let's hope they'll support Asian languages in the future, for round-the-world coverage.

michaelleland• 16 years ago

And when they get Swedish, I'm in!

The awesome thing is that once the platform is there, they could easily open it up to other languages and get them rolling too--though they might have to really think before diving into languages with a different character set, such as Japanese.

maukdaddy• 16 years ago

Another vote for Swedish. I'd pay a lot more than 9.99 too ;)

plusbryan• 16 years ago

This app makes me want to travel, right now, so I can just try out the app. Amazing.

xlorm• 16 years ago

Do most mobile translation apps not require internet connections? That part surprised me. I had assumed most apps do the work on a server.

jf3• 16 years ago

This app does all the work client-side in (30-60 frames per second) real time.

From what I understand, this is the only app that does that.

keiferski• 16 years ago

Very impressive. I'm curious to see how it performs in the wild though -- can it translate signs outdoors just as well as in a studio? I really hope so.

jf3• 16 years ago

It performs very nicely in the wild. One of the nice things about the "real time" aspect of this app is that you can nudge the phone around until you get a good translation.

ugh5• 16 years ago

What about smaller type (and sometimes weird fonts) in somewhat dim environments – I’m thinking of menus, that seems like a typical situation where translation is necessary for tourists.

solutionyogi8• 16 years ago

Joel, are you associated with the App Team OR just another geek impressed with their technology?

keyle• 16 years ago

That is really awesome. I can't wait for them to add more languages. And it's cheap... Anyone travelling would pay for it.

zacharyvoase• 16 years ago

If Smartphone + Wikipedia + Google = Hitchhiker's Guide to the Galaxy, then this is the Babel fish.

And I’m sure that it will also, by effectively removing all barriers to communication between different cultures and races, cause more and bloodier wars than anything else in the history of creation.

Groxx2• 16 years ago

Ultimately leading to faster-than-light travel by harnessing the speed of bad news.

See what you've done?!

dholowiski5• 16 years ago

All it needs is audio out to a bluetooth headset...

johndeweese7• 16 years ago

(hola, i'm john deweese, one of the creators) good to hear the early positive response, and we look forward to your insightful feedback (support@questvisual.com) -- mention this site so we pay some extra attention.

i hope this app will be nice and disruptive, and we'll be looking carefully at what people expect and how they actually use it. it's a platform, and we're really excited about the directions it opens up.

thanks for the link, cheers!

Sephr• 16 years ago

Do you plan on creating an Android version anytime soon?

mey• 16 years ago

An android or SDK version that could be used with a heads up display system would be impressive. If you can do more then Spanish, you should consider talking to the US Military.

GeneralMaximus• 16 years ago

For the love of FSM, do it! I speak for a vast majority of Android users here on HN when I say I'd buy this even if it cost five times what it does now.

macrael• 16 years ago

Definitely put up a non-flash version of your promo video on your website; your target audience won't be able to watch it, as is!

bootload• 16 years ago

"... Definitely put up a non-flash version of your promo video on your website ..."

Agree I can't read it, pretty simple request.

Raphael2• 16 years ago

iOS can't play YouTube videos?

tassl• 16 years ago

From the technological point of view I can just say wow! Congratulations, this is a great app.

Aside from that, and as a Spanish native speaker, the translated text sounds really weird to me. Even without knowing English I would be able to understand the translated text (at least the example that you gave) but it can be improved. But this is the only con that I see to your application. Probably you're already working on that, but I thought that you might like this input.

Great idea!!

jonursenbach2• 16 years ago

Glad to see this finally out there in the wild and to have demoed it a while back at my house. Grats!

middlegeek• 16 years ago

Please port it to the Android platform!

DanielN3• 16 years ago

Have you guy's explored at all how difficult mandarin translations will be? If it is possible this would be a huge market for you.

elblanco2• 16 years ago

spot on! I can usually muddle through Spanish and French signs when I travel, but I'm terrified of traveling to China or Japan where the signs are quite a bit more impenetrable.

darren_2• 16 years ago

Depending on how their OCR works, Mandarin/Japanese->English(/spanish/&c) could be a _lot_ more difficult (thousands of characters to distinguish) and in the case of Japanese the translation a lot trickier (completely different grammar, Mandarin's not so bad though).

This could just be a case of needing to optimize and train their recognizer to deal with a much larger set of possible characters, or it could require implementing kanji-specific OCR techniques like attempting to decompose the characters into their constituent strokes, and recognize based on classification of those strokes (orientation, position, direction).

kmfrk3• 16 years ago

You should register wordlensapp.com or something of the like. I'm not sure hosting nothing but the app promotion on a site with a completely different URL is beneficial.

With such a product, you should perhaps brand yourself more on your product than your company name. :)

Just throwing it out there.

staunch3• 16 years ago

For future reference: never mention registering a domain in a public forum. Some douchebag will always register it and try to sell it to you.

In this case someone has already registered wordlensapp.com. We can only hope it's the Quest Visual people.

pchristensen2• 16 years ago

Since you mentioned this, I registered wordlens.co and set it to redirect to their website. I emailed the info and asked them if they want it. That's how awesome I think this app is - I'd rather spend my own money to make sure they get a good domain.

otaviogood or johndeweese, if you're reading this, my email is in my profile.

scg• 16 years ago

How long was this app in development?

tortilla• 16 years ago

TechCrunch article says 2 1/2 years.

http://techcrunch.com/2010/12/16/world-lens-translates-words...

revorad3• 16 years ago

Please, do NOT sell this to Yahoo.

Or anyone for that matter. Make a business, be king of the world. You deserve it.

chime• 16 years ago

You guys have made Doctor Who's Tardis translation-in-the-brain come one step closer to being real. I just tried it and it works perfectly. In my minimal experience, it works great on medium sized text so you may want to advice users to get closer/further from the text if your app can't find anything to translate (i.e. OCR fails).

random42• 16 years ago

Awesome stuff. I assume you guys have the patent on this technology, if not, please get it ASAP.

iskander2• 16 years ago

Have you guys yet tried adding grammar models to the translator? Can a full-language bigram model run in real time?

darren_2• 16 years ago

As a language learner I'd love the ability to capture and export (in some fashion) the pre-translated text, as text not an image, to import into SRS learning software. This especially applies to the Japanese/Mandarin support that I hope you guys are working on.

nathanlrivera• 16 years ago

1. The logo seems similar to the United Nations logo. What was your inspiration for the logo? 2. In the video, who is holding the camera? And who is holding the signs? 3. Did you use an ad agency to make the video, or did you put it together yourselves?

The first card says "welcome to the future" which I thought was perfect :)

otaviogood2• 16 years ago

The designer who made the logo said he was going for a passport look. My brother made the video. He also did the music in the video. He's pretty good with that stuff.

pclark• 16 years ago

This looks so magical and incredible I can't believe it'll work reliably in the real world. If it does, wow.

trotsky• 16 years ago

You win.

The_Igor• 16 years ago

This is impressive! I could have really used something like this on my trip to Japan!

timerickson• 16 years ago

I have witnessed the future.

cpr• 16 years ago

The promo video's gotta be a fake--figuring out the fonts used & fitting in the same space with perfect color, etc.? Nope.

Would like to see what it really does.

johndeweese7• 16 years ago

you'll have to download and try it :)

staunch3• 16 years ago

You will admit that those sample cards are hand picked to be well suited to the application right? High contrast, quality printing, large fonts, etc.

exogen• 16 years ago

The color matching is easy. It's not using the same fonts, and pretty obviously stretching the text to take up the approximate amount of line space - also easy. Don't be too quick to judge! :)

veb• 16 years ago

This has to be the first real iPhone application that's actually impressed me. Something that can definitely change lives.

kmfrk3• 16 years ago

To quote Fry "Shut up and take my money!".

What is the ETA on French and German?

ianferrel• 16 years ago

I downloaded it immediately. Making it free to try and selling the dictionaries is a great sales model. The demo mode does a mirror-image reversal of the word to show you that it can detect words.

I was using a 4th gen iPod Touch, so lower-res camera and no flash.

Unfortunately, I was not able to get it to work as well as I'd hoped. I tested physical copies of a few things:

1. Mac OS X Snow Leopard cd case. Black sans-serif text on white background. This worked the best, with the word-reversal consistently getting "Snow" and "Leopard" reversed with little flicker. "Mac" seemed to go in and out. OS was rarely replaced, and usually along with "Mac" as though it were part of the same word.

2. C++ Programming language book cover (http://pixhost.info/pictures/631454). Dark blue serif text on white. This almost never worked. When it did recognize letters, the recognition shifted so much that the word was a constantly moving jumble.

3. Throat Coat tea bag. White serif text on Red. At any point in time, about 50% of the words were recognized and reversed.

You can take a snapshot, and each word it recognizes is highlighted, which is pretty cool.

I would definitely buy something like this that handled non-romance languages to English.

ugh5• 16 years ago

The current iPod touch has only a very low res, low quality camera (I think VGA or maybe a bit better, in any case less than 1 MP) without even autofocus. It would be nice if someone could try it with an iPhone to see if it fares better.

eml• 16 years ago

Right. Wikipedia says: 0.7MP fixed-focus camera with HD video capture (720p at 30fps) with 960 x 720 resolution still images

I believe the iPhone 3GS/4's autofocus makes a huge difference in clarity for close-ups.

dirtyaura• 16 years ago

For me it seemed to work better in the translation mode. The reversing mode flickered forth and back, but with English-to-Spanish seemed to be more stable.

codybrown• 16 years ago

Can I install this in my contact lens?

revorad3• 16 years ago

This is one of those times you have to admit it's not ALL about the execution. Good ideas do matter. And they are rare.

staunch3• 16 years ago

And I see the opposite. Anyone could have had this idea, but very few could execute on it. In fact all that will determine whether this is a big success or not is whether the thing really works or not (execution).

colanderman• 16 years ago

No kidding... I was just sitting down to research open-source Java OCR software to do this exact thing on Android. My dreams are dashed but I'm glad someone did it...

revorad3• 16 years ago

A lot of people must have thought of making OCR iPhone apps. But the idea of using OCR to do real-time language translation superposed on the original background image? That's genius. I hadn't thought of it. Did you?

At least in my limited experience, deciding what to work on is really important.

Acknowledging great ideas doesn't take away anything from their superb execution.

rbreve• 16 years ago

Of course it's execution, you need to be able to write a very good and fast algorithm, not everyone can do it.

DanielN3• 16 years ago

I disagree. There is a comment higher up suggesting that this app could change the way people live their lives. I read this and it occurred to me that this was likely true. With the caveats if they market it correctly, if they handle competition correctly, if they expand and the proper pace, etc.

The creators will likely see a payoff from this whether or not they do any of these things correctly. But the magnitude of that payoff is entirely dependent on those ifs and the difference between executing correctly and not is humongous.

bl4k3• 16 years ago

This idea has been around for decades. It is a prototypical futurist application. Difference is that these guys nailed it.

jonursenbach2• 16 years ago

A friend of my girlfriend is part of the team that made this and he gave me a demo of this 5 or 6 months ago and even back then when it was in an early alpha form I was absolutely blown away by this thing.

Seeing this thing in action in real life is life changing.

jamiequint• 16 years ago

I agree. I saw a demo at Epicenter cafe randomly by this guy about 6 months ago. Its one of the only demos I've ever seen where I felt like 'holy shit I've just seen the future'.

bigmac• 16 years ago

So, predictions on how long until Google buys these guys? I'd guess within the next 7 days.

Raphael2• 16 years ago

Apple could outbid.

d2viant• 16 years ago

The US military could outbid both of them.

solutionyogi8• 16 years ago

Wow. Just Wow.

I had to put the App to test.

Printed some simple English words and tried it out.

Original paper:

http://imgur.com/EYsko

What the App shows in REALTIME:

http://imgur.com/FhaSW

Comparison with Google Translate:

http://imgur.com/GOd94.jpg

[EDIT: Note, Translating Muerto Fin to English in Google Translate, does result in 'Dead End'. Can any Spanish reader clarify why the original Google Translate chose to translate 'DEAD END' to 'callejón sin sali'.]

[EDIT: So let me get this straight. Their program, running on iPhone [256MB RAM, 600 MHz ARM CPU], can take a live image, perform OCR, translate, create another image with the translated words. And all of this happens real time? Wow.]

I think it's really hard to get a 'Wow' reaction from HN crowd. And you have hit a home run! As someone else said in this thread, you are going to make boatload of money. Congrats!

zzleeper2• 16 years ago

Because "dead end" means "callejon sin salida". That's the correct translation (callejon = road, sin salida = w/out exit).

On the other hand, "muerto fin" has NO meaning in spanish, (and that's what you get when you translate word for word without context).

solutionyogi8• 16 years ago

Thank you. I think context is extremely important and very difficult to get right. I remember reading one article by Google engineer where they discuss how they keep improving their analysis of search queries.

E.g.

[Search Term] -> [Interpretation]

New York -> New York city

New York Times -> The news paper

New York Times Square -> Famous tourist spot in NYC.

There were other examples in the article. I will try to find that article and link it up.

I think Google should buy these guys and provide them with their knowledge of 'context'. Google Translate team and these guys should talk right away.

StavrosK2• 16 years ago

"Muerto fin" means "an ending that has died", no?

gus_massa3• 16 years ago

(Native Spanish speaker here.)

First: This app is amazing!

But I think that this app is doing almost a word by word translations, without a list of usual expressions or grammatical analysis.

For example, the correct translation of 'Dead End' is 'callejón sin salida', that means literally something like 'street without exit'. Translating word by word 'Dead End' you obtain 'Muerto Fin' that is unintelligible.

Another example from the video is the translation of

  'Lengua boliviana con una salsa picante de anchoas'.

The app translates this as

  'Tongue Bolivian with a sauce spicy of anchovies',

But the correct translations is something like

  'Bolivian Tongue with a spicy sauce of anchovies',

because in a translation between English and Spanish you must reverse the order of the adjectives and nouns.

johndeweese7• 16 years ago

yep, this is a word-for-word translation, because it's fast and it gets the point across. we're working to improve translation quality and finesse, but it's a much harder problem to understand grammar. so, we hope it gives the general meaning, and you can learn to piece it apart.

bioinformatics• 16 years ago

And in the last case, from English to Spanish, it puts direction as a masculine substantive, while in Spanish is feminine.

ygd• 16 years ago

(2 years of classroom Spanish here.)

I like how "Make right turn" becomes "Make correct turn".

narag• 16 years ago

It becomes "make correct shift" (as in "I work in the night shift"). The word in Spanish would be "giro". The natural translation would be "gira a la derecha".

omaranto5• 16 years ago

"make right turn" is apparently hard! Word Lens translates it to "make correct turn" but where "turn" is in the sense of taking turns not in the sense of turning. Google Translate gives "to the right which in turn".

frisco4• 16 years ago

These guys are going to have some hard decisions to make in the next few months. This has such incredible potential I'd be really sad to see it get bought and languish as a side-feature at Google. Run as a business, rather than a free technology project liable to be side-lined on whatever whim, and really developed for the next 5 years, think about what it could be! I mean, just look at what it is today on day one!

The offers will come, that much is certain; I can only hope they decide to go for it and turn this into something great.

51Cards2• 16 years ago

I doubt they will be bought by Google, Apple is probably already writing cheques. Apple has been getting the pants beat off them by Google in mobile translation for awhile now. If there is a patent behind this then Apple will grab this for their stable just to keep it off the Android platform. It would be a huge 'one-up' for them. (which makes me sad as I strongly prefer Android devices) In time you'll see it licensed to other platforms as a new cash cow.

This should also be viewed as a prime example that it is not always better to push complex processing into the cloud. This would not be possible trying to push to a server somewhere real-time.

Kudos to this team.

frisco4• 16 years ago

Fine, substitute "Apple" for "Google" in my post. My point remains.

ugh5• 16 years ago

I don’t think the translation is the amazing part – as one of the creators said here, at the moment it mostly does word-for-word translation. Google’s statistical machine translation is mightily impressive technology, I’m not sure whether you could even theoretically put it on a phone at the moment.

You would ideally want the OCR and text replacement together with Google’s translation algorithms, which, besides requiring a network connection at the moment, would also introduce way too much lag that would kill the experience.

(Word-for-word translation is a perfect compromise in the meantime.)

jamii3• 16 years ago

> just to keep it off the Android platform

Oh god, I hadn't thought of that. I just naturally assumed this would be coming to Android soon.

iandanforth• 16 years ago

Here's an interesting thought. This app is so good it could influence Apple hardware.

While I agree this is a killer app for the iOS platform, it doesn't yet have the polish that Jobs would demand for a live demo. A good deal of that is the hardware contraints of the system.

I think if you had invented a dedicated device that did nothing but this app it would sell like hotcakes, and we've heard in this forum that people are considering buying iPod touches just to give them out this app to family.

Thus if you wanted to move the needle from 'wow' to 'insanely great' it would take hardware level engineering to optimize the device, which I don't think it outside the realm of possibility.

Remember it was killer apps like quake, unreal, and crisis that moved the graphics card industry for years. This app has the potential for even broader appeal, and its successors will require even more horsepower.

rabble• 16 years ago

Totally awesome, but there's a big problem. Most of the time when you're in need of translation of text you find like signs, it's because you're in another country. That means data is very expensive, because your fancy iphone is locked and roaming.

It's awesome, many people may download it, but it will get very little use. :-\ They should probably charge like $5 or $10 for it, make money on people WANTING to use it, not on them actually using it.

Unless i'm mistaken and it's doing the image processing all locally, in that case, they've got a smash hit.

ugh5• 16 years ago

You don’t need a network connection.

Quote from the iTunes Store: “NO NETWORK required - results appear immediately on your video screen when you need it, anywhere in the world.”

frisco4• 16 years ago

Did you watch the video? Apparently it's all client-side.

BRadmin• 16 years ago

A slide in the video specifically says: "You need only the phone and world lens. Without connection to internet or costs of network."

milkshakes• 16 years ago

Yes, they're processing it all locally. 0:26 in the video :)

solutionyogi8• 16 years ago

It said in App description that it does not need network connectivity. To verify this claim, I put my iPhone in 'Airplane' mode and tried the sample paper again. And I can confirm that it works without any network connectivity.

Also, the App is free but you have to pay for each language to language translation separately. I think it's a smart move. I may buy 'Spanish to English' for 4.99 if I am going to Mexico. If I end up in France 6 months down the line, I will buy 'French to English' at 4.99. I won't worry about spending 4.99$ but they have successfully extracted 9.98 from me. This is awesome.

gce• 16 years ago

Really beautiful and executed beautifully.

Just one small piece of feedback on the "commercial." It's a bit confusing for the first 10-15 seconds or so. I was too busy looking at the iPhone screen and not the billboards. Can I recommend an animated version with v/o?

Great work!

Edit: One more piece of feedback. The icon isn't nearly cool enough for how cool this app is. My prediction is that you guys are going to make a boatload of money, hire a designer sooner rather than later to spiff anything up.

stevelosh2• 16 years ago

Wow.

I downloaded the app, bought the Spanish -> English pack, and tried it with some simple phrases in a big TextEdit window on my monitor. It flickered a bit, but I expected that from a monitor. It got them right.

Big deal, common phrases are easy -- I could just buy a phrasebook. Then I tried a random phrase from the Spanish version of "Dive into Python":

    Una función, como cualquier otra cosa en Python, es un objeto.

It flickered like before from the monitor, but it was easy to read the translation:

   A function, as any other thing in Python, is a object.

Perfect? No. Usable? Absolutely.

As soon as a French -> English pack is released I'll buy it, even if it's $100.

This is the kind of thing that would make it possible for me to move to Montreal. I love that city, but don't know French. I could learn, but it would take time and I'd be lost like a baby gazelle on the Serengeti while I learned.

This app could ease the process of moving to an entirely different country. That's amazing.

Hello, future, it's nice to see you.

istvanp• 16 years ago

I live in Montreal and you are right that you would need to understand some French but only to read signalization, some store names and public advertisement. Signalization of course is important if you are driving but an app like this won't help you (you are holding the wheel I hope). Besides that, almost any store or institution has bilingual staff (and when they don't most French speakers will understand basic English). Don't let that hold you back... unless you hate snow and cold winters ;)

ramchip• 16 years ago

Seconding this. I know many people who speak only English and are doing just fine here. Almost everybody who works at a store is bilingual. It's still worth it to learn French though, if only to get the francophones' respect. I often hear Quebecois friends complain about people in Montreal not doing an effort to learn French, but they'll respect people who put some effort into it.

nihilocrat• 16 years ago

I live in Montréal too and I'm chiming in that it's easy to live here with only English, 25% of the city is natively anglophone. Some people in the company I work at (also immigrants) see no reason to learn French, which the quebecois probably don't want to hear.

It is easy to get the hang of signage too; lots of French words are spelt almost identically in English, and the language is much easier to read than it is to listen to. You should get the hang of it quickly.

On the other hand, if you have to get any kind of job that involves speaking with customers ... you are shit out of luck.

johnyzee• 16 years ago

Combine the technology with something like http://www.gizmag.com/vuzix-wrap-920ar-augmented-reality-gla... and you could walk down the street and read everything in your own language.

cavorite• 16 years ago

The product seems amazing, but they are somehow cheating in the video.

The signs (in Spanish) have grammar mistakes, but they are automatically translated to correct English sentences. I have a few examples:

- The third sign says "Lo traduce el texto", but that sentence doesn't make sense. It should be "Traduce el texto" or even "Se traduce el texto".

- 'Ropas opcional' sounds strange to me, although it may be accepted in some countries.. It should be something like 'Ropas opcionales'.

I tried both examples in Google, Bing and Babel Fish and the results were OK, so I don't think that Word Lens translator is very accurate.

    "Mr. Babbage, if you put into the machine wrong figures, will 
    the right answers come out?" ... I am not able rightly to 
    apprehend the kind of confusion of ideas that could provoke 
    such a question."

ThomPete2• 16 years ago

It's called marketing and frankly I don't think it takes anything away from the app.

JeremyBanks• 16 years ago

Sure it does. I'd be impressed with this app doing word-for-word translation if that's what I expected. If On the other hand, if the examples uses grammatically incorrect source material to give the impression that it can do more than it can, I feel irritated at being misled, and I'm much less likely to support the product.

foresterh3• 16 years ago

Even though I have no use for it, I bought the English to Spanish to try it out. I'm blown away and these guys deserve to take away Facebook guy's man of the year award... amazing...

jorgem2• 16 years ago

AUDIO OUT for the blind.

EDIT: People with limited vision are often considered blind, so they could use this app.

foresterh3• 16 years ago

Not to sound harsh, but I don't think the blind would find a lot of use for this app...

jorgem2• 16 years ago

You don't have to be fully blind to be blind. Let's just say "limited vision."

alextgordon• 16 years ago

Try reading a book with your eyes closed.

iskander2• 16 years ago

I know many blind people who use iphones, and they'd love a tool to help them read street signs, cereal boxes, etc...

Blind people are patient by necessity, I'm sure there are many who wouldn't mind waving around their phone scanning for text.

maxawaytoolong• 16 years ago

If you're blind, how can you point the camera at text in the first place? (honest question, no snark)

chaosmachine• 16 years ago

Kurzweil has been selling devices that do what you're suggesting for several years.

Here's one from 2006:

http://www.youtube.com/watch?v=TOwDzDnd9NY

Here's a modern version on a cheap cell phone:

http://www.youtube.com/watch?v=iKCLjC5chuM

zzleeper2• 16 years ago

I agree that it's impressive, but the flash video felt like cheating.

For instance, no spanish speaker would EVER say "LO TRADUCE EL TEXTO" (@ 0:20), or "ROPAS OPCIONAL" (@ 0:50). They just picked some words in spanish that made sense when translated, but my guess is that average translation would be much more awful

sirrocco• 16 years ago

As others have said it doesn't really matter if the translation is awful. Given a context, and a poor translation into my native language - I will understand what it says.

Unless it translates `apple` as `chair` - I'll understand what the meaning is.

sportsTAKES• 16 years ago

So cool! Cannot wait for more languages...

Holy smokes, when is the Chinese version going to be available?

Let me know if you need help with a Portuguese version, seriously.

Agree with some of the other comments that their logo and site design could use a boost.

Killer idea, love it.

EwanG• 16 years ago

So am I honestly the only person who's first thought on seeing this is... ooh, give me a Japanese pack and I don't have to wait for scanlations anymore?

stevenrace• 16 years ago

Quite the understated title - a feat well worth hyperbole.

Can you speak upon the origins of the software?

Did this stem about from other projects/research (Edu, darpa, lone disillusioned coder..)?

I presume one could use dictionaries of things other than plaintext? Say symbols, objects, patterns? (for signage and 'custom' use)

Are you opposed to this being 'opensourced' at any time?

mrphoebs• 16 years ago

Simply Brilliant

sliverstorm• 16 years ago

Do you see yourselves extending this to languages based on non-Latin characters? This could be an amazing, amazing tool for Chinese characters- both for foreigners and natives.

johndeweese7• 16 years ago

absolutely, though some character sets such as chinese are a few orders of magnitude more complex. stretch goal, we're working on it :)

johndeweese7• 16 years ago

it's funny to think about how the advantages of compact encoding influenced the progress of technology. would like to think that we're getting past this by way of Unicode, but this is an analysis problem, so character 'visual encoding' rears its head again.

of course, people are fantastically good at it, if trained in school. looking at /other/ people's scripts though, well, here we are. :)

mleonhard• 16 years ago

The Pleco Chinese-English dictionary iPhone app does realtime lookups using the camera. It's more of a study tool than a realtime translation tool, but still really cool:

http://www.engadget.com/2010/11/24/pleco-chinese-dictionary-...

ericflo• 16 years ago

I just caught myself thinking: "I hope they got a software patent on this, they deserve it." But I usually hate software patents unequivocally. So, yet again, a real-world use case reminds me that nothing is so black and white.

ams61102• 16 years ago

There does seem to be legitimate novelty here.

dholowiski5• 16 years ago

Yeah if anyone deserves to make buckets of money patenting software it's the developers of this app.

andywood• 16 years ago

As a programmer, sometimes I think "this could be patentable, because the implementation is not immediately obvious to me." Maybe the non-programmers at the USPTO apply the same standard.

whatever_dude2• 16 years ago

The problem is that it is a clever combination of techniques that already exist.

* OCR, to identify the text * Removing the background of the text, filling it with surrounding colors * Translating words * Placing new words on the same area, using the same rectangle

It's very, very clever, but it's not something that should be controlled by one person or company. Should any combination of existing techniques will also be patentable? Where do we draw the line?

They were the first and they'll have a head start and make a lot of money. Now, be exclusive? I don't think so.

catch232• 16 years ago

As a friend who knows octavio, they've tried various "OCR" techniques from typical journal papers and in general those types of algorithms were mostly for static images. They struggled for 2 years to find the best techniques and I think they final product is great.

imack• 16 years ago

I hope they got a software patent if only to help protect themselves against trolls who are filing patents for this right now.

CamperBob5• 16 years ago

The problem with this reasoning is obvious: given that "idea patents" are now commonly awarded, these guys are much more likely to be sued by a troll who has already acquired a patent on the idea, than they are to be able to defend their IP by threatening a patent suit against someone else. An eye for an eye leaves the whole world blind.

Patents are no defense against patent trolls, because (by definition) the patent trolls don't actually produce anything. There is no defense against patent trolls, except lobbying to narrow the scope and duration of technology patents.

ig1• 16 years ago

I hope you guys are on AWS or something, because your server is going to get a lot of traffic over the next few days...

elblanco2• 16 years ago

What's next, universal translators that also make everybody's lips move correctly for the language we are hearing? Perhaps Star Trek got it correct?

Amazing...Android app next?

atomical• 16 years ago

How many people are on the team that created this?

dannyr• 16 years ago

Two guys: John and Otavio.

mattdeboard3• 16 years ago

My.

God.

nerfhammer• 16 years ago

I have two new heroes.

glhaynes• 16 years ago

Saw this comment on TechCrunch, sums it up perfectly:

When I was a kid, I would have bet all the money I would ever earn that we'd have flying cars before the ability to do something like this with a phone. Amazing. Absolutely amazing.

gursikh• 16 years ago

I'd love to see this technology applied to augmented reading / writing (i.e. look up any word/person/place on a printed page).

ebaysucks3• 16 years ago

This is the first time in the last few years that I have been WOWed by new technology.

Huge congrats, this really is magic to me. If you asked me yesterday I'd have told you it's impossible to do.

bobf2• 16 years ago

I work in the translation industry, and the interesting part of this to me is the implementation. Generating the resulting translation image in real time is a unique idea that is probably hard to make work well. OCR on a mobile device without a network connection is not as difficult as you might think (read: it can fairly easily be done on Android with tesseract-ocr, at least). Also, instead of the typical rule-based or statistical machine translation you get on the web, think of translating word-by-word as simply substituting words from a key/value dictionary list.

Having said that, I'm always glad to see interesting translation being developed and getting such a positive reaction on HN. Congratulations to Word Lens on launching!

brianbreslin• 16 years ago

I would pay $20 for japanese, chinese, korean, russian, arabic, and hindi language packs. seriously. if i was traveling around the world, I would HAPPILY drop $100 into this app.

alain94040• 16 years ago

At first the demo didn't work at all for me. Nothing would happen at all. Then I tried to click on a few buttons, and text started dancing left and right, with no apparent reason.

Then I got better at it. It seemed to reverse the words and letters more or less fine.

Then I spent the $5 on the product for Spanish (could be handy). I'll happily spend another $5 on a French version.

And I know how to impress everyone tomorrow!

pekinb• 16 years ago

Can someone at questvisual say more about the platform? I have some ideas I'd like to try.

johndeweese7• 16 years ago

we wrote everything ourselves. what would you like to try?

stcredzero• 16 years ago

This might just be the first commercially successful mainstream application of augmented reality.

alex_c• 16 years ago

It's easy to feel down when reading the news, but then something like this comes along and... the future's going to be so cool! :)

I don't care if it even only works 20% of the time right now. The rest is elbow grease and faster hardware - it'll get there.

It's not every day that you consciously realize - hey, the world just changed today (for the better!).

aresant2• 16 years ago

The walls are coming down so fast it's making my head spin - job well done!

I immediately sent the link to a dozen people, about half wrote back "How much is it?" and "How do I download it?"

Since QuestVisual.com is clearly going to sweep the entirety of the internets by this time next week a few friendly conversion suggestions:

a) Change your button to "Try Word Lens FREE - Click Here"

b) The logo under your button looks like another button, just get rid of it.

Dropped you an email if you'd like the Conversion Voodoo team to bang out a new page tomorrow AM gratis that's ready for installation by noon PST :).

SimonPStevens• 16 years ago

Totally agree. If you want people to follow that link you need a much much stronger call to action.

webjunkie• 16 years ago

omg.. please no "click here"

aresant2• 16 years ago

Please explain?

senorprogrammer• 16 years ago

Another observation about the site: it's very Facebook-unfriendly at the moment. Try sharing the URL on Facebook and you'll see that the site lacks any text or images that FB can readily pick up for display and thus only your URL is displayed. This app looks amazing, but that does make it a bit hard to tell people about on Fb.

ebaysucks3• 16 years ago

Why the name Word Lens? I had to come back to this thread to remember the name after talking to my brother about it.

I think "Babylon App" would be a perfect name for this app. Just my 2 cents.

bobf2• 16 years ago

Babylon is already the name of a translation software company (for >13 years, see babylon.com).

muon• 16 years ago

They could have stopped at just translating and printing it on the screen, but they show it the original form, that is the difference between doing things and doing the things beautifully.

I am spellbound.

ebaysucks3• 16 years ago

I agree - showing the original form is a big part of the WOW factor.

mattdeboard3• 16 years ago

Yeah, I just ran around showing all my coworkers, even the non-tech-savvy ones. Each was blown away. Gladly paid $10 for this.

m0nastic• 16 years ago

That was the fastest $10 I've ever spent in the App Store. I'll echo most people's sentiments, this is seriously cool (although I must look like an idiot walking around my apartment pointing my phone at everything).

dholowiski5• 16 years ago

I showed this app to my 60 year old semi-technophobic parents and they immediately understood it. They are ready to buy an iPod touch to use with this app for their next cruise. My mom guessed the cost of a language at $50 and was shocked to hear the actual price.

Screw zuckerberg, The developers of this app should be on the cover of time magazine.

tomjen3• 16 years ago

And that is what you call a killer app.

solutionyogi8• 16 years ago

Another cool real life example:

http://twitpic.com/3gnduy

ghshephard• 16 years ago

Cute undocumented (that i could find) Easter egg on the Reverse Words feature - It also 'corrects' the spelling. heh. (Discovered when it insisted the reverse of "Shephard" was drehpehS)

jodrellblank4• 16 years ago

I guess that's an accidental part of word based OCR rather than a deliberate spellcheck, I noticed it correct the name "Keysource" to "ecruosyeR"

anonymoushn2• 16 years ago

I will pay you $500 for this when it can read Chinese and Japanese.

knieveltech• 16 years ago

I. Am. Totally. Floored. Just wow. The last time I checked in on assisted reality apps for the iPhone there where cheesy laser tag apps and stuff to hang tag clouds in random locations. Interesting I guess but nothing to produce a visceral response. If I where you your shoes I'd see about buying a wheelbarrow for the money that's coming your way.

kevinelliott• 16 years ago

Wow, so Douglas Adams' babelfish is real, only it's an app for iOS. Fantastic. You've got an exciting venture on your hands!

koski• 16 years ago

This might be a game changer.

The product is so cool and it seems to work so well, I have to admid that I am a bit jealous. I would have been proud to be able to say "I did is".

To the executing team responsible of the application: be proud what you did. Great work.

I am so happy for these guys. You believed it's doable and you did it. Congratulations once again.

I would love to read the "project diary". How long did it take to, what were the unexpected problems, etc.

happyrichpinoy• 16 years ago

The Japanese to English pack can't come soon enough. Manga comics here I come!

brendano• 16 years ago

Awesome.

One note, it'll be hard to get really good translation quality without a network connection, because good models require tons of CPU and memory -- Google Translate is far better than word-for-word because it uses such things.

oscardelben• 16 years ago

This is the first time I regret not having an iphone (I have an ipod touch without camera). Good job guys!

bambax2• 16 years ago

Looks extraordinary by the HN comments; unfortunately the site is down right now... ;-(

La rançon du succès, I guess.

binbasti• 16 years ago

I'd pay at least $50 for a Chinese-English language pack. And even just 50% accuracy would be ok.

forkrulassail• 16 years ago

Seems dead.

Forbidden

You don't have permission to access / on this server.

Additionally, a 403 Forbidden error was encountered while trying to use an ErrorDocument to handle the request.

Apache/2.2.16 (Unix) mod_ssl/2.2.16 OpenSSL/0.9.7a mod_fcgid/2.3.5 Phusion_Passenger/2.2.15 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 Server at questvisual.com Port 80

prs• 16 years ago

See http://www.youtube.com/user/QuestVisual for QuestVisual YouTube channel and explanatory video.

http://itunes.apple.com/us/app/word-lens/id383463868 for direct iTunes link.

bl4k3• 16 years ago

You should probably disable some of those Apache modules and trim it up a bit

seldo• 16 years ago

One day tourists are going to buy "holiday glasses". They'll walk around Paris, stop at a nice restaurant, put on their glasses, and read the whole menu in English. Then they'll order by pointing. Hell, they can do this with their phone right now, the glasses just make it easier.

JabavuAdams• 16 years ago

Head-mounted displays have been around for so long in a crappy form. I can't wait until Oakley or Apple finally catch the HMD bug and make something nice. Maybe this kind of app will be the push.

covercash• 16 years ago

I'd like an English to English add on so I can get dictionary definitions of words using the 'pause' feature. This would also make visiting URLs in print extremely easy!

bambax2• 16 years ago

John Deweese, "one of the creators" says here that "it's a platform".

Here's one thing they could do: open the platform to distribute dictionaries from third parties.

Questvisual could still sell "basic" or "standard" dictionaries for each language pair, but they would also sell competing dictionaries, that could either try to address the problem from a different angle (phrase translation vs. word translation), or be specialized dictionaries: legal, medical, etc.

They would take a cut, of course, and they would create a market that they would curate. Great wins for everyone!

bl4k3• 16 years ago

Awesome. This should help me find my way around Redwood City

KeithMajhor• 16 years ago

Hacker Newsers usually downvote brief congratulatory comments. Seeing "you win" take home over 20 points is indicative of just how exciting this is.

bajsejohannes• 16 years ago

This is mindblowing! I hope you make it for Android as well.

A minor thing about the webpage: If I want to share it on facebook (and I do!), it does not come up with any sort of summary or images, like it normally does for links. Now, I don't know exactly how it gets this information, but I would definitely find out if I were you.

yurylifshits2• 16 years ago

Re-posting here from my blog http://yurylifshits.com/lessons-learned-from-the-launch-of-w...

    The lessons I learned today from Word Lens

Everything is secondary to compelling usecase. Word Lens has strange company name (QuestVisual), no logo, crappy landing page, and no PR preparation. But its darn cool! And that's what matters.

Assembly innovation is really cool. Every piece of Word Lens was here, but nobody made a perfect combination before today. Academic researchers will never do another Word Lens, as they are overfocused on novelty and hate just-assemble-the-pieces work.

Clever freemium business model. Word Lens is free, but you have to buy language packs. "erase words" and "reverse words" are free demo modes to prove that the app really works. Note, that you can even turn it into subscription model with dictionary updates.

BlendBack is the heart of this invention. Word Lens goes like this: (1) detect and recognize characters, (2) translate, (3) produce text in similar colors and shape and blend it back to the picture. The last step is the most innovative and can be used beyond Word Lens. E.g. one can do "Bar Code replacer". Turn your phone on any barcode and see some picture there. Can be used as a cheap replacement for road signs and ads.

No connection required. This is extremely important. 3G is unstable. WiFi is not everywhere. 4G has not really arrived yet. When you travel, your carrier can not cover you perfectly. I can see more and more essential apps that will not require connection. "Yelp in a box" anyone?

Global appeal. This is not another geek's app. It is mom and pop's app. It is an app for every country and and every village. We need to spend more time outside Silicon Valley to find needs like this one.

Science fiction inspiration. Part of the reason for press craziness is that Word Lens matched the science fiction story (Babel fish from "The Hitchhiker's Guide to the Galaxy"). We love seeing SF concepts turning into reality. Let's reread old classic and implement all other concepts from there :)

    What I would do next

Obvious: add more language packs. Asian languages can be a bit of a challenge (recognition is harder).

Brainstorm pricing. A lot of options are available: different price for the first month, bundle prices for several languages, one-time price discount (a la 23andme), subscription model, enterprise package.

Put on hold all talks to investors and potential acquirers.

Immediately start working on versions for other platforms (Android, Blackberry, Nokia, WinMo). Hire another person to do just that.

Run a contest: iPhone for the best "Word Lens in the wild" video.

ndunn2• 16 years ago

You didn't learn any lessons about.. how well it works? I had very poor success in my apartment, but I'd like to try it outside.

wallflower• 16 years ago

> Immediately start working on versions for other platforms (Android, Blackberry, Nokia, WinMo)

I disagree. I strongly believe this is close to the 1st successful iteration of a killer app for mobile devices (the Babel Fish). One that other platforms might not have for a while. I would be surprised if Apple has not already reached out to invite them to One Infinite Loop to talk about their future plans (not acquisition, but Android).

yurylifshits2• 16 years ago

That's can be the best deal: Apple invests, Word Lens keeps it iPhone only.

omaranto5• 16 years ago

I just read a bunch of reviews in the app store and there are plenty one star reviews that say something like "what a rip off! This isn't free at all! You need to buy the language packs to do anything", a lot of those reviews even add that if it just cost $10 instead of falsely claiming to be free it would sell like crazy. So maybe the freemium model is a little too clever.

oozcitak• 16 years ago

This is amazing and has great potential. Imagine word-lens for conferences: record the presentation -> translate the slides with Word Lens -> publish the translated slides live to the conference audience. There are hundreds of ways of making money from this technology, the iOS app should only be the beginning.

redfloatplane• 16 years ago

That's truly fantastic. I don't have much use for Spanish->English, but I'm going to buy it anyway for the sheer "wow" factor. Congratulations.

toumhi• 16 years ago

Am I the only one to be surprised by the overwhelming reaction of the HN crowd? Doing word by word translation is not so exciting from a technology point of view, is it because all the other apps doing translation on a phone are crap? I believe there are libraries that already store language translations, right? Seriously wondering why everybody is jumping and shouting how great and disruptive this is.

kelnos• 16 years ago

I think it's more the speed, and the fact that it's not just doing the word-by-word translation, but it's doing OCR first, and, as a nice touch, it at least makes a vague effort to visually look similar to the original text (well, color at least)... and it does all this without using a network connection at all.

The 'traditional' method of firing up something like the Google Translate app, manually (and potentially painfully, if it's a language that uses symbols you aren't used to typing in on a regular basis) typing in the text, and maybe waiting for a server in the cloud to spit back response... is just crap compared to something like this. Even if Word Lens doesn't do proper grammatical and idiomatic translation (yet?), it still seems like it'd be super useful.

jodrellblank4• 16 years ago

You should be the only one being surprised. This isn't just a nice app, it's a killer app.

People will buy smartphones/iTouches/tablets for this purpose. Alone.

This is a simple to use, cheap app. on an existing popular platform which makes a significant improvement on a task a lot of people want to do. You're standing there saying "What's all the fuss about Chrome and FireFox? I can browse the internet with IE6 and have done for years!".

This reaction is future shock, this is a surprisingly strong wave in the flow of future technologies and it's shaking people up.

CamperBob5• 16 years ago

Gestalt is everything.

kilian• 16 years ago

This app is too cool, but it seems a little over eager. Here's me pointing the app at a mosaic: http://desmond.yfrog.com/Himg612/scaled.php?tn=0&server=...

mattdeboard3• 16 years ago

Cool hack, and I noticed that as well (on other non-word objects). However I'd rather it be over- than undereager or slow.

samatman• 16 years ago

Overeager is good, since in the typical use case, you'll be pointing it at words.

The potential for amusing pareidolias is definitely there. Maybe a tumblr for 'found translations'?

uast23• 16 years ago

Is that the coolest thing I have seen recently ?

_b8r0• 16 years ago

Just bought it, this is indeed impressive stuff. I bought both English -> Spanish and the other way around. Not being a big speaker of Spanish I wasn't sure how good the translation was but the Spanish to English is good enough for my travels next year.

tomelders• 16 years ago

I give up. I will never make anything this amazing. What's the point now I've seen this.

brosephius• 16 years ago

for the sake of being contrarian, why all the freaking out? it's cool and all, but it's OCR + translation, two things computers are already pretty good at (not to mention word lens seems to be doing word-for-word translation, not context-sensitive).

yeah doing it so fast on a mobile device is impressive, but am I alone in thinking this isn't going to change the world?

atlbeer• 16 years ago

Yep.

You are looking at a product from it's technical merits only.

From a product perspective, I can travel to any country in the world and read the signs on the shop, read menus, or even learn the language since I have an always available translator to help.

It makes the world smaller and more accessible. Those things always change the world

blhack3• 16 years ago

It's doing a lot more than OCR + translation. It's doing OCR + translation very very quickly, on a very limited platform, then re-injected the translated form of the words back into the stream in real time.

If I could point my macbook at something and get the same thing, then I might be a bit fascinated...the fact that I can do this with something that fits in my pocket is what is so amazing.

mcantor• 16 years ago

I would pay $50 to have this on my Android device.

In a heartbeat.

mrchess• 16 years ago

I'm reading comments on this app on other blog sites and people say things like "translations suck", "not worth your time".

Times like this remind me how ridiculous consumer expectations can be. People still don't get that product development takes time and energy -- they just like to criticize with their elitist expectations only because they own an iPhone.

But yes, this app is great. Keep it up!

27182818284• 16 years ago

It is amazing that you can get a piece of software for free (with language packs for an extra low price)

This is something the US military would have paid millions and millions for in the past. Now, you see people say, "I'd even pay fifteen whole dollars for that app"

haha, oh wow.

jodrellblank4• 16 years ago

Wordlens, wow and thank you!

(Here's me desiring this a couple of years ago:

http://news.ycombinator.com/item?id=569558 )

atlei• 16 years ago

Very cool implementation :-)

So what they have is

- an OCR function that reads a few words from an (easy) image

- a (simple) translator that translates the words into a different language word for word

- and then paints the translated words back using the same font/size

...continuously and quickly !

Don't get me wrong, this is cool and all, but it would be much more useful with a single snapshot that translates better, instead of focusing on doing it "realtime" (but the video wouldn't be as cool :-) )

The idea is nice, so I expect Evernote and Google to implement this ASAP ;-)

Good luck !!!

foresterh3• 16 years ago

The blanking words out feature is going to make photoshopping fake products much easier...

zbailey• 16 years ago

I'm very surprised people have not mentioned this app does not work very well (if at all) when the phone is in any orientation other than straight up and down.

Why is it not possible for the developers to use the orientation sensor/gyroscope to accomodate for this? Many times it's easier to fit a bunch of text on the camera in landscape mode than the alternative. It's also a lot more natural for me to hold my phone in landscape mode when using the camera.

Just my $0.02 and congrats on all the hubbub. Can't wait for more languages :)

WiseWeasel2• 16 years ago

You can hit the 'lock' button to switch the orientation to landscape.

CamperBob5• 16 years ago

On an iPad, but not on an iPhone.

Groxx2• 16 years ago

Just saw this on the news, actually. Very much sorry, but I didn't catch the channel / show - wasn't close enough to the TV, and it wasn't mine.

Seems like the attention for this has gone way beyond HN. Congratulations!

solutionyogi8• 16 years ago

I saw it on CNN when I was visiting my dentist.

browngeek• 16 years ago

This is going to do to traveling what the iPhone did to smartphones.

code_duck• 16 years ago

Wow, this is really cool, obviously.

Not sure if it's the lighting or the text I'm trying, but it does have a tendency to make words dance around insanely. The execution could be improved upon a bit.

I can certainly see a lot of potential for this. Let me know when it's embedded in my augmented reality eyeglasses!

blhack3• 16 years ago

I'm sure I'm too late to the game to have anybody care about this, but here is how you can tell this is something cool:

I just went to the local bar with my roomate and, despite it being pretty busy, I told one of the bartenders "Hey, want to see something cool?" and pointed my phone at their menu. Now, almost immediately after I said this, I realized it was kindof rude (they were busy), but she looked anyway...then she proceeded to yell at all the other bartenders to come over and check it out...and the guy sitting next to me spent about a minute going over the menu (translating from english to spanish) wowed as well.

You're not going to get that with angry birds.

Awesome.

EGreg• 16 years ago

Wow is this the greatest # of points EVAR?

zentechen• 16 years ago

How can I put it on my glasses?

Introducing Word Lens

Related Stories