tshiran

This is great. Starflake reflections are so much nicer than old fashion cubes.

Arrow has really taken off over the last year. I'm seeing it in tons of different projects.

I would argue that the fact that Arrow has been integrated into so many projects over the last year is proof that a separate project made sense. Dremio, PySpark, Pandas, MapD, NVIDIA GPU Data Frame, Graphistry, Turbodbc, ...

We considered starting a company in SF a couple years ago. I'm glad that we decided to instead start it in Mountain View (http://www.dremio.com). We're able to attract very talented people from San Jose, Milpitas, Redwood City, etc. where rents are a bit more reasonable.

This is cool

Great article, this is the exact process I've observed multiple times.

Unless there's a really good reason to not split it equal, I think it's better if both founders have the same amount of equity. Otherwise you could create unnecessary tension which is the last thing you want at that early stage.

Nice

Another analogy that might help is strongly typed vs. loosely typed languages. There are pros and cons to both Java and Python, and both are widely used.

I think there's a tradeoff here. It's like MongoDB - hundreds of thousands downloads per month because it enables people to build and deploy applications faster. For many cases a relational database is still better though.

The nice attribute of Drill is that it works on schema-less data as well as data with strong schemas. The user can make the tradeoff between agility and 'safety'.

It depends what you mean by layout. If you add/remove/change the fields in the records, Drill can handle that. It can also handle a situation where you move from one format (say JSON) to another format (Parquet) and have mixed formats in a directory.

Drill is described as schema-free, but the data doesn't have to lack schema. Some data (Parquet, Avro) has schema, while some data (JSON, HBase, MongoDB) doesn't (ie, each record could have different fields). Drill is designed to allow queries on any data. Also, Drill leverages the structure that's embedded in the data without requiring IT to redundantly define schemas in a centralized schema repository. In other words, if there's a schema embedded in the Parquet file, why require the user to then go and maintain the same schema, explicitly, in Hive metastore?