HN user

vgt

2,349 karma
Posts112
Comments414
View on HN
duckdb.org 2y ago

Benchmarking Ourselves over Time at DuckDB

vgt
4pts1
duckdb.org 2y ago

Vector Similarity Search in DuckDB

vgt
21pts0
motherduck.com 2y ago

Build Sub-Second Data Applications with Motherduck's WASM SDK

vgt
3pts0
motherduck.com 2y ago

Differential storage: A key building block for a DuckDB-based data warehouse

vgt
98pts11
motherduck.com 2y ago

The Column Explorer: A Birds Eye View of Your Data

vgt
26pts1
motherduck.com 2y ago

An open source DuckDB text to SQL LLM

vgt
131pts30
motherduck.com 2y ago

Perf Is Not Enough

vgt
29pts1
motherduck.com 2y ago

Just Released: Hybrid Query Processing Paper at CIDR 2024

vgt
1pts0
duckdb.org 2y ago

Updates to the H2O.ai db-benchmark

vgt
192pts87
motherduck.com 2y ago

Making PySpark Code Faster with DuckDB

vgt
2pts0
news.ycombinator.com 3y ago

Show HN: MotherDuck – a serverless data analytics platform powered by DuckDB

vgt
17pts2
motherduck.com 3y ago

Why DuckDB Slaps

vgt
2pts1
techcrunch.com 3y ago

DuckDB-based MotherDuck valued at $175M

vgt
14pts0
www.firebolt.io 4y ago

About Those Data Warehouse Performance Wars

vgt
1pts0
cloud.google.com 6y ago

Autonomous Re-Clustering in Google BigQuery

vgt
1pts0
github.com 7y ago

ZetaSQL – Open Sourcing BigQuery's SQL

vgt
3pts0
cloud.google.com 7y ago

The BigQuery sandbox

vgt
1pts0
cloud.google.com 7y ago

Geospatial types and queries in BigQuery

vgt
2pts0
pantheon.corp.google.com 8y ago

NCAA Basketball Public Dataset

vgt
1pts0
cloud.google.com 8y ago

Data Definition Language in Google BigQuery

vgt
1pts0
cloud.google.com 8y ago

Google Cloud Dataprep Hits Public Beta

vgt
3pts0
analytics.googleblog.com 8y ago

Google Analytics data is now streamed to BigQuery

vgt
3pts0
cloudplatform.googleblog.com 8y ago

New German GCP Region – Open Now

vgt
3pts0
cloud.google.com 8y ago

Show HN: Creating an object detection application using TensorFlow

vgt
2pts0
venturebeat.com 8y ago

Google cuts prices on cloud SSDs by up to 63%

vgt
5pts0
in.3wks.com.au 9y ago

Google not Amazon. Make fantastic savings in a server-less world

vgt
25pts23
syslog.ravelin.com 9y ago

There are very few suitable use cases for DynamoDB

vgt
232pts140
cloud.google.com 9y ago

Speeding up queries with HyperLogLog++

vgt
2pts0
cloud.google.com 9y ago

Introducing the Video Intelligence API

vgt
1pts0
cloud.google.com 9y ago

In-Memory Shuffle in Google Cloud Dataflow

vgt
2pts0

Since we at MotherDuck are running a serverless cloud Data Warehouse on top of DuckDB, we've observed the massive leaps in performance over time ourselves. Less exciting but perhaps more important are the improvements in stability and semantics recent DuckDB versions brought out.

Shameless plug - if you love DuckDB but don't want to manage it yourself, or if you need a supported 3rd party ecosystem, sharing/IAM, a fully-fledged beautiful web UI - all the things beyond DuckDB that are important to adapt DuckDB as a solution - I encourage you to look at MotherDuck. You can sign up in seconds (about 17 seconds, I timed it), you get a 30 day Free Trial without having to put in a credit card. We also offer a perpetual Free Tier and a Startup Program for qualifying startups.

(co-founder at MotherDuck)

Co-founder and head of produck at MotherDuck here - would love to chat. We're running DuckDB in a serverless fashion, so you're only paying for what you consume.

Feel free to reach out to tino@motherduck.com.

DuckDB 1.0.0 2 years ago

You should give us at MotherDuck a try for this use case.

(Co-founder)

Shameless plug - MotherDuck[0] is a serverless managed DuckDB data warehouse with some interesting properties:

- Managed storage with zero-copy clone (and upcoming time travel)

- Secure Sharing

- Hybrid Mode that allows folks to combine client (WASM, UI, CLI, etc) with cloud data

- Approved and supported ecosystem of third party vendors

- Since we're running DuckDB in production, we're working very closely with the DuckDB team to improve both our service and open source DDB in terms of reliability, semantics, and capabilities

[0]https://motherduck.com/

(co-founder at MotherDuck)

I agree. I think full text-to-results (even bypassing text-to-SQL) is akin to L5 self-driving cars. It's easy to get to some reasonable level, say 90%. But to get to the point, where folks can fully trust the system and don't need a steering wheel (or know SQL) may take decades.

We at MotherDuck took an incremental approach. We launched something more akin to lane-assist. We're calling it FixIt - an in-flow visual aid to help you identify and fix errors in your SQL [0].

I think there's gobs of opportunities to improve the analytics experiences without resorting to "L5 self-driving" (e.g. full text-to-results)

[0] https://motherduck.com/blog/introducing-fixit-ai-sql-error-f...

Love these! We do want to deliver more features like FixIt! [0]

What's really exciting is what you can do with DuckDB, MotherDuck, and WASM. A powerful in-browser storage and execution engine tethered to a central serverless data warehouse using hybrid mode [1] opens the doors for unprecedented experiences. Imagine the possibilities if you have metadata, data, query logic, or even LLMs in the client 0ms away from the user and on user's own hardware.

So we're doing this in our UI of course, but we also released a WASM SDK so that developers can take advantage of this new architecture in their own apps! [2]

[0]https://motherduck.com/blog/introducing-fixit-ai-sql-error-f...

[1]https://motherduck.com/docs/architecture-and-capabilities

[2]https://github.com/motherduckdb/wasm-client

Perseverance of global rule of law. If Ukraine goes, we go back to 19th century. Any two-bit dictator wannabe will want to invade their neighbor for some sweet sweet territory.

Global economy relies on this stability, and US benefits the most from the stability of the global economy.

We also didn't "spend 75 billion dollars". Very little actual cash went to Ukraine from the US. We gave them old stockpiles, which we were going to decommission anyway, and paid maintenance for. We also paid US companies to increase production (of ammunition for example), and given that the Russia conflict is increasing demands for ammunition, increasing supply is not a bad thing.

This is such a lazy talking point permeating conservative spheres. And it quite likely coming straight from Russia psy-ops.

Hi, head of Produck at MotherDuck here.

Yes, we are indeed a good use case for this. For one, we built a fully-fledged managed storage system on top of DuckDB, with better performance and caching and the like. Two, we're going to be pretty good at reading from S3 because we've optimized that path. Three, our storage has sharing/IAM and is about to have things like zero-copy clone and time travel.

Happy to answer any Qs.

DuckDB 0.9 has fixed most of OOM scenarios by both using far less RAM and by spilling to disk better.

(Head of Produck at MotherDuck and a huge fan of DuckDB)

We at MotherDuck are working closely with the DuckDB folks to deliver a serverless analytics service powered by DuckDB [0]. Currently in open Beta and driving towards GA. Our users are certainly recognizing how fast DuckDB is in the cloud.

One of the reasons we exist is because DuckDB is meant to be a single-player database. MotherDuck is doing tons of heavy-lifting to turn it into a true multi-player data warehouse, so things like IAM/sharing, persistence, time travel, administration, the ecosystem and so forth.

What's magical about MotherDuck is that virtually any DuckDB instance in the wild can connect to MotherDuck by simply running '.open motherduck:' [1], and suddenly you get all these aforementioned benefits.

(head of produck at MotherDuck)

[0] https://motherduck.com/

[1] https://motherduck.com/docs/getting-started/connect-query-fr...

DuckDB 0.9 3 years ago

You should really get into a habit of disclosing your biases, as you seem to make an appearance in every DuckDB related thread with the same comments:)

(Head of Produck and Co-founder at MotherDuck)

DuckDB 0.9 3 years ago

For compute specifically MotherDuck is a scale-up, not scale-out system [0]

Storage is naturally distributed due to underlying IaaS.

Rather, that seems to be an implementation detail. What use cases are relevant is the key. Are we chasing big Apple-sized workloads? No. [1]

That said, scaling-up on EC2, especially with our one-instance-per-user architecture, fits vast majority of workloads we've seen in our past lives building Exabyte-scale systems.

[0] https://motherduck.com/blog/the-simple-joys-of-scaling-up/ [1] https://motherduck.com/blog/big-data-is-dead/

DuckDB 0.9 3 years ago

We at MotherDuck are doing this!

While we're running vanilla duckdb, our extension does sopen heart surgery on some of the components, to make it suitable for a multi tenant serverless cloud service.

(Co-founder and head of produck at MotherDuck)

I think we are much more than an analytics tool, but we don't yet have all the bells and whistles of the modern CDW. We think of ourselves as a new type of data warehouse - some things are different, but many things are the same.

We actually just published a new produck page that talks about how we see ourselves at [0].

[0] https://motherduck.com/product/

Head of produck and co-founder at MotherDuck here. Happy to answer any questions!

One minor correction on the title. While we're partners with DuckDB, and they're shareholders in MotherDuck, it is MotherDuck who raised the money, not DuckDB. On the plus side, we avoid the ethical dillemma of many OSS companies associated with being open-core - DuckDB is open, MotherDuck is closed (although contributing to open).