PredictionIO – open source machine learning for predictive features

https://hacks.mozilla.org/2014/04/introducing-predictionio/

by rnyman • 12 years ago

108 32 12 years ago

hacks.mozilla.org

wheaties2• 12 years ago

Sounds fantastic right up until I read requires MongoDb. Thankfully it's open source and we can fix that.

berto992• 12 years ago

Why is mongo a problem?

lukasm• 12 years ago

I heard about two cases when MongoDB failed doing The Most Important Thing - storing data. No one really care about autosharding, no migrations etc. if you can't store the data. Due to some replication issue data was inconsistent.

ironchef• 12 years ago

A number of people have been bit by issues in mongo in the past such as: the approach it had taken to write locking, that it has silently discarded writes in certain cases, the charge that it uses inflated storage on disk, and the performance characteristics when the working set does not fit into memory. I'm sure there are more but when it arrived it had great marketing as was touted as the greatest thing since sliced bread. Unfortunately, some people ended up with horrendous sandwiches and remember the awfulness of said sandwiches.

micro_cam2• 12 years ago

For me it is a red flag in terms of scalability as lots of our data sets won't fit in mongo backed by a 1-2 TB disk even if they take up < 100 GB in the original format (usually binary/compressed genetic data).

It also uses a ton of ram and performance really suffers when the data won't fit in ram so it isn't a great choice if you are trying to push the limits of what your machines can do.

They are only using it to store models and whatever "behavioral data" is but models for things like random forests can be really big and you want to be able to write/read trees from separate machines etc.

I wonder why they chose to use mongo vs local disk or HDFS which they already require.

duaneb3• 12 years ago

Unless there's something about Mongo that means it's perfect for machine learning (unlikely), the last thing I want to maintain is yet another database because they didn't offer any choice.

ing33k• 12 years ago

and what you replace it with ?

smhchan4• 12 years ago

What would be the better alternatives, in your opinion?

dszeto2• 12 years ago

We can fix it together too if you prefer. Any suggestions on database choice? We are open to discussions and feedback.

wheaties2• 12 years ago

Yeah, I've been looking at PredictionIO. I'll happily contribute. I'll reach out to you guys.

lefrancaiz• 12 years ago

It also comes with a Vagrant box if you're looking to just try it out with your dev environment. I've tried it personally and had a good experience with it. I spot checked the recommendations and they felt really good.

http://docs.prediction.io/current/installation/install-predi...

duaneb3• 12 years ago

I wish it didn't depend on MongoDB. Is there a good reason for not using a more general purpose database, like SQL or any of the billion key value stores out there?

smhchan4• 12 years ago

MongoDB is used as the datastore for unstructured data, e.g. item attributes and user attributes. It's also used as a cache for prediction results, so queries like geospatial search can be performed.

There is no specific reason reason to stick with MongoDB only. It just happens to be the database the team has picked for the first implementation. It is very likely to support other databases in the coming future given the strong community demand.

dszeto2• 12 years ago

We picked MongoDB a while ago because of its built in geospatial indexing support. Would be happy to add support of other databases. Any strong candidates on top of your head?

yaph• 12 years ago

PostgreSQL / PostGIS

todazar• 12 years ago

RethinkDB!

saosebastiao• 12 years ago

Both Postgres and SQLite have extensive geospatial support (PostGIS and SpatiaLite respectively), including indexing and boatloads of useful geospatial functions. And in my experience, both of them are far more advanced than Mongo's geospatial support.

bduerst2• 12 years ago

I know it's a pain to get data into it, but how about Big Query?

duaneb3• 12 years ago

I haven't worked with geospatial databases as much (so verify this with someone who has), but Postgres is a good bet for an alternative to mongo in this scenario.

dminor• 12 years ago

One of the really nice things about PredictionIO is that it comes with a dozen different recommendation algorithms out of the box, and lets you simulate their results with your data. This makes it much easier to decide which is the right one to use.

jey• 12 years ago

How did these guys get distribution on an official Mozilla blog despite being a third-party project?

idlewan• 12 years ago

It's not new for hacks.mozilla.org to display write-ups about external projects that relate to the web.

rnyman• 12 years ago

I'm Robert, the Editor of Mozilla Hacks. We publish articles about anything regarding to the Open Web and is open source that we believe developers can learn and get inspired from.

More information at https://hacks.mozilla.org/about/

PredictionIO – open source machine learning for predictive features

Related Stories