stonecupi

Big data needs serious collaboration and metadata tools in a bad way. This tool looks promising but big data piece still in dev: www.veroanalytics.com

No, when they were building it out the notion of a data lake wasnt popularized. Its just naturally occurred. I think just distinguished the two as raw data and dwh data.

The most successful one I'm familiar with is the one employed at Netflix (I worked there on BI). However, they did not abandon the concept of a data warehouse, they just enhanced it with the data lake. Probably about 80% of ad hoc analytics come out the data lake while standard reporting needs are covered by a Teradata dwh. When ad hoc queries become regular needs they build aggregates in the data lake and move it to teradata or redshift where it can be sliced and diced along various dimensions.

I see this same strategy being attempted at some non-tech companies as well. Too early to say whether it will succeed.

Looking forward to feedback from the hacker community.