HN user
Cofounder at Holistics.io, data modeling & self-service BI platform - https://www.holistics.io
Creator of dbdiagram.io, dbdocs.io
https://twitter.com/iamhuy
Congrats on the launch!
I think here's a few players in this space (dev-friendly BI tool) already: - Holistics.io - Lightdash - Hashboard
These tools all allow analysts to use both/either a local/cloud IDE to write analytics logic, and check in to Git version control.
How do you plan to differentiate with them?
While it looks really nice, hard to imagine it's quicker to hand draw this than reverse it out of your existing schema
They have an option to take your SQL dump text and convert it back to DBML which is the DSL they use to render.
If you're new to the BI space, this book should give you a pretty good high-level overview: https://www.holistics.io/books/setup-analytics/
If you’re using Chartio and looking to migrate, here are a few alternatives you can consider. These are mostly cloud-based tool that is SQL-native and sends your written SQL queries to your SQL data-warehouse.
1/ Redash: write SQL gets charts, similar to Chartio (although a bit less polished as it was developed by a small team). Open-source or Cloud-based. Downside is they recently get acquired by Databricks, so I’m not sure how that will affect their product roadmap. Support for Google Sheets as well.
2/ Superset: also write SQL gets charts. They’re both open-source (Apache Incubated), and being supported by a commercial company behind (Preset). Their visualization library looks pretty diverse too.
3/ Metabase: This is also based on SQL. The difference is that they put a simple modeling layer in between your DW and SQL, so that they provide a simple ‘ask question’ interface for business users to get data, it’s quite simple though so if business users want more sophisticated questions they’ll need to resort back to SQL. Their UI is pretty sleek. Both open source and cloud-based (pricing starts from $100) with a commercial company behind.
4/ Holistics: Similar to Metabase, where they provide a modeling layer and self-service capability to non-technical layer. Difference is their modeling layer seems more sophisticated than that of Metabase. They also support the T in ELT as part of modeling. Support for Google Sheets as well. Their UI looks well-polished. Cloud-based only, with pricing starting from $100. They’re also working on Git version-controlling your SQL/modeling code so that it’s easier to track change/audit, this might help prevent vendor lock-in and ease future migration.
5/ Mode: Write SQL gets charts (similar to Redash/Superset). They’re a lot more polished than Redash, with a focus towards doing Data Science (native Python/R support, Notebook, etc). Their Free plan supports up to 5 users, but beyond that you will need to talk to their sales people to get quoted pricing. I heard it can get over $1k/month based with additional user pricing on top.
6/ Looker: probably best in-class in term of SQL-based data modeling, but very expensive.
7/ There are also other tools which I don’t think fit in the SQL-native narrative, like AWS Quicksight, Google Data Studio, PowerBI, Tableau and Qlik.
P/s: These 2 posts give a fairly good framework on how to look/evaluate different BI tools. This is a pretty crowded space with many different paradigms baked in over many years, so it might be a bit confusing for new people looking into the field
- https://www.holistics.io/books/setup-analytics/navigating-th...
- https://www.holistics.io/books/setup-analytics/the-arc-of-ad...
There are so many such solutions, that It depends on your use cases.
If you're looking for high-level dashboarding/monitoring that connects directly with your sources (REST, 3rd-party apps like Facebook, Adwords, etc), then tools in the realm of Klipfolio, Databox, Grow.com will suit you well. If you're looking for visualization tools that connect to your data-warehouse/SQL database, then BI tools like Holistics, Metabase, Redash, etc will likely work for you. And of course there's tools that can support both kinds of sources, like PowerBI and Google Data Studio.
P/s: This post does a good job at laying down a framework for evaluating these sort of BI tools: https://www.holistics.io/books/setup-analytics/navigating-th...
For those who're curious, the tool to build the database diagrams is dbdiagram.io
I counted 6.
Fivetran, dbt, Preset (Superset/Airflow), Sisu, Imply and Databricks.
Though, as someone who's in this space a while, I think they did a decently fair job at articulating the 'modern' data infrastructure landscape.
I think you have a point, but there are more nuiances than that.
There are typically 2 types of data to collect: Transactional data and behavioural data.
Most transactional data, due to their important nature, are already generated and captured by the production applications. Since the logic is coded by application engineer, it's usually hard to get this data wrong. These data are then ETL-ed (or EL-ed) over to a DW, as described by the article.
For behavioural data, this is where your statement will most apply to. This is where tools like Snowplow, Posthog, Segment, etc come in to set up the proper event data collection engine. This is also where it's important to "collect data properly", as these kinds of event data changes structure fast, and hard to keep track over time. I'd admit this space (data collection management) is still nascent, with only tools like iterative.ly on the market.
For those who're interested in learning more about the history and evolution of data infrastructure/BI - basically why and how it has come to this stage - check out this short guidebook [1] that my colleagues and I put together a few months back.
It goes into details how much relevance the practices of the past (OLAP, Kimball's modeling) has with the current changes in by the cloud era (MPP, cheap storage/compute, etc). Chapter 4 will be most interesting for HN audience: It walks through the different waves of data adoption ever since BI was invented in the 60-70s.
I know of three (all OSS)
* https://www.posthog.com/ (more mixpanel alternative, can auto-track events too)
* https://snowplowanalytics.com/
* https://rudderstack.com/ (explicitly say it's Segment alternative)
There's also dbdiagram.io that focuses exclusively on database diagrams. Nice thing about it is you can just write simple DSL code and they render the diagrams automatically. The downside is that they don't support collaborative editing yet.
You can check out Holistics.io - It allows you to bring MongoDB data into a SQL DW, and visualize them in a single tool.
https://www.holistics.io/blog/build-reporting-analytics-mong...
It has a free version you can possibly fit yourself into.
This might help: https://www.holistics.io/features/scheduled-email-reporting/
It can generate a PDF with some data and charts. You can also schedule emails to share these PDFs with customers. Adding a customer filter and you'll get something like Mail Merge / Crystal Reports.
p/s: I work here.
You should check out Holistics.io if you're disappointed with all 3 options. A balance between drag-and-drop for business users and SQL for data analysts, this is done through a SQL-based data modeling layer.
Disclaimer: I work at Holistics.
Hey folks,
Have you ever tried building a simple analytics stack for your company, yet not sure what’s the best way to start with “just the right amount of effort” that can still scale it up later when you grow? And when you went googling you soon got confused by the conflicting advice on the internet: ETL or ELT, data warehouse or data lake, SQL or OLAP cube, etc?
That’s a common problem that I saw when speaking to our customers. I soon realized that books about data analytics are quite outdated given the current pace of development in the space.
There isn’t an up-to-date, high-level “map” that gives readers a proper framework of thinking about the analytics landscape. Much of the materials out there are step-by-step how-to based on a pre-selected set of tools/technologies.
We decided to write that “map” in form of this short book - “The Analytics Setup Guidebook”. The book aims to give readers better clarity over the confusing analytics landscape.
It covers:
* How does a simple, yet scalable analytics stack look like, and how you can build one cheaply and quickly
* ETL vs ELT, what’s the big deal? And why ELT is better than ETL nowadays.
* Why data modeling is important in analytics, Kimball data modeling and how it’s relevant in modern cloud systems
* Why the market of BI tools is so confusing nowadays
* How company’s BI adoption will predictably evolve over time
The book is intended as a comprehensive intro for data people, but data veterans might find chapter 3 and chapter 4 useful — we spoke about how Kimball Data Modeling fits in with modern cloud infrastructure (MPP), and how BI has evolved over time in an organization.
The best part about the book is it’s short, well-illustrated and can be finished within 3 hours of reading!
Here’s a direct, ungated link to the book if you don’t want to give your email: https://cdn.holistics.io/guidebook/the-analytics-stack-guide...
We’ve received some encouraging feedback, but love to hear you guys’ feedback on the book.
Regards -Huy
You’re right, and also right that we might be biased. We actually not trying to say that with the modeling layer the CEO can remove her reliance on the DA completely, that would be foolish. We think she can only reduce that reliance down when it only comes to getting access to data. We are not talking about complicated analysis that requires a proper data mindset.
We used iPad, Apple Pencil and Paper app :)
Yes.. we did get a number of early feedback about this exact topic of data quality. But we eventually decided to cover it in another separate sidebar to the book. It’s also not a simple topic to cover. And the book is supposed to “give you enough to be dangerous”.
Thank you for sharing your thoughts! Yes the concept of data modeling and self-service analytics are interesting, yet few people fully grasp it. Chapter 3.1 is probably one of my most favorites.
This is exactly why we started out writing this book. We actually spoke with a lot of customers, and a fair share of them sharing the same thing: They found a lot of how-to on the web, but none of which is comprehensive, and goes deep into the "why" and the "history" of BI.
I'm one of the authors of the book. Yes you're right. The book outline the transition from the "old world of BI" to the "new world of BI". If you read through the book, you'll see the biases clearly stated there:
- ELT over ETL
- Data Modeling is crucial as part of BI workflow
- Cloud DW over on-premise DW (but this depends on your org's requirements)
- SQL reporting (Redash, Metabase, Looker, Holistics) over non-SQL reporting (Tableau, Qlik)
OP here. This blog post follows the previous post "The Rise and Fall of OLAP Cube" which was shared on HN a few days back. We realized a lot of people mistook the 2 terms OLAP and OLAP Cube. So this post is aimed to clarify that.
Can you elaborate more on what you meAn by “creating clean data models”?
Interesting perspective. What do you say to someone who's been using OLAP cube for their entire BI implementation? What would be the transition plan to adopting MPP databases?
You might want to check out Holistics.io, allowing your non-technical to explore data without writing SQL.
It's an alternative to Looker but more affordable, and more powerful than Metabase.
Postgres, Pipedrive, Zendesk -> BigQuery, then BI using Holistics.io
Holistics handles both ETL, transformation and self-service visualiation (Looker alternative), all in 1 tool.
It might. But we hope it'll get more popular than Microsoft .NET DBML
It doesn't yet.
Yes we do. SQLite is in the roadmap.
Although it’s database-agnostic, we do realize that a lot of the use cases is about generating database-specific SQL based on DBML code. So we’re working on introducing dialects to the syntax so that it closely resembles specific DB better (e.g this DBML is of MySQL dialect).
That’s a great idea. We’ll certainly consider it in the near future.