MapD: Massive Throughput Database Queries with LLVM on GPUs

http://devblogs.nvidia.com/parallelforall/mapd-massive-throughput-database-queries-llvm-gpus/

by bsprings • 11 years ago

79 29 11 years ago

devblogs.nvidia.com

vardump2• 11 years ago

I wonder how much effort the CPU implementation saw. I'd expect 1 socket, 4 memory channel Haswell CPU to be able to filter 40+-10 GB/s of rows per second, if row data can be in column major order. SIMD is pretty often memory bandwidth limited, whether we're talking about CPUs or GPUs.

etrain• 11 years ago

Yeah, I agree this is surprising. If all the data in their CPU implementation were stored in a well-packed columnar format, I'd expect something close to theoretical memory throughput on the query they describe (intermediate counts table fits easily into L1/L2 cache and results are commutative and associative). Cache-coherence issues in the aggregation code are a possible cause.

If I were building a speed-freak analytics database I'd be focusing on making my CPU implementations as fast as possible, since that's what 99% of potential customers are already running. Assuming you get to 40GB/s on this type of query, that's only a factor of 5 slower than the GPU implementation. I'd imagine that for most workloads, a factor of 5 speedup that requires new hardware and lots of energy is kind of a non-starter.

AceJohnny2• 11 years ago

If I were building a speed-freak analytics database I'd be focusing on making my CPU implementations as fast as possible, since that's what 99% of potential customers are already running.

Well, this is by Nvidia, so of course their incentive is otherwise...

tmostak• 11 years ago

Hi, one of the original authors here.

You're confusing rows per second with bytes per second. We're measuring here in rows per second, with random data that takes four bytes per record. So the group-by is showing around 24 GB/sec for CPU and roughly 1 TB/sec for the GPUs. Admittedly we are a small startup and there are some optimizations still to be made on CPU (we're working on being NUMA-aware, for example), but the CPU performance is not bad and still much higher for group by than you see in other databases.

You have to remember that we're building a full database and visualization system and not just optimizing for a single benchmark. In addition we're trying to make the point that you can hit this 1-2TB/sec (depending on the query) on a single server with GPUs, which means assuming the compressed data fits in 192GB of GPU RAM you can get much higher performance that you would see out of a whole rack of beefy CPU servers, particularly when you take into account the network overhead that distributed databases suffer. Furthermore, the bandwidth of Nvidia's Pascal architecture (to be released next year) should have at least 2X the bandwidth by using High Bandwidth Memory and likely significantly higher memory sizes, so our speedups will only increase.

jandrewrogers2• 11 years ago

Yeah, I came here to say the same thing. A properly designed and vectorized in-memory database implementation on a modern CPU should essentially limited by the aggregate memory bandwidth across the two sockets, and I would assume the in-memory representation has been compressed. Based on my experience, the performance numbers suggest that the CPU implementation here is suboptimal.

Bedon2922• 11 years ago

They are using GPUs with large amounts of GDDR5 to store the most important data, so the increased memory bandwidth is a major part of the performance upgrade over CPU using DDR3.

edit: The maximum bandwidth for DDR3 looks to be ~50GB/s and they are saying over 500GB/s in the GPU system

I say DDR3 because they have been doing this since before DDR4 became available. Even DDR4 is something like ~70GB/s

rektide• 11 years ago

Spark SQL, with their new Catalyst optimization, is real time compiling their queries into code (assembling Scala quasiquoted fragments into an AST, compiling that). No ridiculously awesome GPU backend here, but some of the same general kinds of tech- compiled queries. https://databricks.com/blog/2015/04/13/deep-dive-into-spark-... http://people.csail.mit.edu/matei/papers/2015/sigmod_spark_s...

jandrewrogers2• 11 years ago

Real-time SQL compilation has been common in analytical database platforms for many years. Open source has really been the exception to this for some reason, so it is nice to see that people are starting to close that gap a bit.

Compiling queries should really be viewed as table stakes for a modern data platform.

threeseed2• 11 years ago

I use SparkSQL every single day and it is not anywhere close to this.

You could combine SparkSQL with Tachyon but it is still pretty far from what MapD seems to be accomplishing.

Would love to see parts of Spark skip the JVM and call out to Rust or LLVM for some aspects.

paulasmuth4• 11 years ago

Are problems that fit in memory on a single machine (96GB in this article) considered "big data" now? This buzzword is becoming absolutely meaningless. But yeah, "Hyper-interactive visualytics at scale.", go nvidia PR.

Also, while it's absolutely amazing that they are able to scan 240B rows/sec, I wonder what one would use this capability for if they can only keep a few hundred million records around at a time? The difference between taking 10 or 100ms to scan the dataset should hardly matter to a user that is running "interactive analytics queries".

reitzensteinm• 11 years ago

You can rent a 128gb machine at Hetzner for $130 a month.

I think we can all agree to draw a line in the sand and say if a kid doing a paper run could afford your servers, you don't get to call it big data.

(not to take away from the linked project, which looks technically awesome)

Smerity• 11 years ago

I disagree with the "big data" line in the sand being whether "a kid doing a paper run could afford your servers". Using spot instances on AWS, you can have a 100 machine cluster with 1.5TB RAM for only $3 per hour.

Working at Common Crawl, some of the coolest projects I've seen have been side projects or weekend projects by interested volunteers. WikiReverse[1] cost only $64USD to parse the metadata of all 3.6 billion pages (even cheaper if you avoided EMR fees) whilst Yelp extracted 748 million US phone numbers from 2 billion pages for $10.60USD.

These days, big data (regardless of how you define the term) is within the reach of the kid with a paper route.

[1] https://wikireverse.org/

[2] http://engineeringblog.yelp.com/2015/03/analyzing-the-web-fo...

riquito• 11 years ago

I appreciate the analogy but it's not like if the server were free big data would disappear, isn't it?

threeseed2• 11 years ago

Big Data as the industry sees it is more than just the size of the data.

It is about the nasty work of data validation, consolidation, ETL, unification e.g. single customer view, analytics etc. Some of these can be technically challenging even on multi gigabyte files and require a range of multi disciplinary techniques. And Big Data increasingly means taking old EDW concepts and reimagining them for a more real time world.

comboy2• 11 years ago

If you know how to do even a simple WHERE on 750M rows in memory in 100ms without using GPU then please share some info, I'd love to learn about it.

rburhum• 11 years ago

eh, any where query that hits an index where the 750M rows are not going to be scanned. The query can easily return in 100ms... fetching the records is a different story.

paulasmuth4• 11 years ago

I actually recently left my job at google to start my own company and that's what we do. We didn't consider using GPUs yet as the queries we are running for the specific usecase we are currently working on (web/ecommerce analytics) are all IO bound [so the first step if we wanted to speed stuff up would be moving the data onto ssds or into memory as we are currently storing everything on disk. however this hasn't been necessary so far]. Our product allows you to run interactive queries (segment/aggregate/etc) against a billion record dataset (in the terrabytes; basically a giant logfile that stores an entry for every interaction we observe on the website/app with lots of metadata) with O(seconds) latency.

We wrote our own code that does "the heavy lifting" but it's pretty much the same that everbody seems to be doing right now and comes down to minimizing the # of bytes read at query time; we index all data into a columnar format (so we only need to load a subset of the data and can pack it tightly for most queries) and then slice the query up into lots of individual shards that we compute in parallel (again, being bound mainly by the 200megs/sec disk read we get out of each machine).

Without being completely serious now but regarding your question; "a simple WHERE" (scanning each row and applying a predicate fn with no shared state/shuffle/merge phase) is pretty straightforward to run on a huge dataset as you can split the execution up into as many pieces as you want and run them individually. Since this allows you to basically choose the problem size per shard you could scan those 750m rows on a ginormous cluster of tamagochis if you can fit a single row onto one ;)

If you or somebody else would like to talk more, please shoot me an email: paul@deepcortex.io

DannoHung• 11 years ago

Array based columnar storage and SIMD comparisons aren't fast enough?

pella2• 11 years ago

similar : PG-Strom (past=CUDA ; now=OpenCL)

"GPGPU Accelerates PostgreSQL"

slide: http://www.slideshare.net/kaigai/gpgpu-accelerates-postgresq... ( Dec 20, 2014 )

HN: https://news.ycombinator.com/item?id=8787414

and Open Source! https://github.com/pg-strom/devel

pella2• 11 years ago

more info ( from : 2013 )

http://data-informed.com/fast-database-emerges-from-mit-clas...

now:

Demo: http://tweetmap.mapd.com/desktop/ Blog http://www.mapd.com/blog/2015/06/04/mapd/

smegel• 11 years ago

that maps OpenGL primitives onto SQL result sets

Here is a set of vertices, now draw me a 2d image that represents the 3d scene those vertices represent from a certain viewpoint.

It boggles my mind that this logic can be twisted into running arbitrary SQL logic. Does it actually do something as magic as overlying SQL set logic onto a graphics scene...where the results of the SQL query are equivalent to the set of vertices that are calculated to not be visible when rendering the scene? Amazing stuff.

paulasmuth4• 11 years ago

I believe these guys are actually going straight from SQL to "GPU instruction code" (so AIUI the SQL query execution is not some hack that is layered on top of the programmable shader pipeline [which is what's responsible for the "here is a list of vertices now draw me a 3d scene part" you mentioned] or something like that but runs directly on the more or less general purpose processor in the graphics card)

I understand the part where they "map the result set to opengl primitives" as that they write the output/result set of the sql query into some datastructure that allows them [after executing the query] to easily use it from opengl to render pretty stuff. This part shouldn't be much different from other texture/vertex data that you'd bring in e.g. from files on disk or a MySQL database into opengl specific buffers so that you can subsequently use it for drawing in your program.

MapD: Massive Throughput Database Queries with LLVM on GPUs

Related Stories