HN user

arnoldoMuller

20 karma

I am a data scientist / entrepeneur. My startup provides a fast similarity search engine available: http://simmachines.com

Similarity search is NoSQL 2.0, a key value storage where key is allowed to have mismatches and this similarity is defined with a distance function. Therefore you can store and retrieve audio, video and many other things.

Startup: http://simmachines.com Twitter: @amuller

Posts5
Comments19
View on HN

I went as a Monbusho scholar in the period 2004 - 2009 and this is not the case anymore. Nowadays you spend a lot of time filling forms and getting access to basic services like cellphones or Internet. They do give you around ¥160K these days...

You can solve complex things with k-nearest neighbor as long as you use an appropriate distance function. This is the beauty, a distance function abstracts the complexity. I tackled a tricky biology problem by applying a cascade of similarity filters. Check the presentation I gave at the Clojure/conj 2011: http://prezi.com/zaaoq6pjrl2z/clojure-conj-final/

My startup provides a very fast similarity engine (in a DB of 100 million objects I can find similar objects in under 20 millisec. with one CPU) in case you worry about scalability. URL: http://simmachines.com

There was a talk around where they were using two replacements at most(or two hash tables), and two hash functions. At insert time you add the object to the first hash table, if there is an object in the bucket, you then try with the second hash table. If you follow this approach a very large percentage of the buckets is going to be full. The remaining objects set (those that cannot be inserted in the first two hash tables because the buckets are full) is so small that you can keep them in a list or in a "standard" hash table. At some point I had a secondary storage implementation of this approach and it was good indeed.

When I was building OBsearch.net I used this book extensively. I grant you that some explanations are not so detailed. On the other hand, the explanations of the most relevant techniques are quite good. Check for instance the section that explains LSH or the explanation about Navarro's SAT.

If you count the number of people killed by wars in Europe or North America in the 20th century, Latin America is the place to be. Latin America has numerous social problems but the number of conflicts (and their magnitude) between countries has been traditionally very small.