Pragmatic bitmap filters in Microsoft SQL Server

https://www.vldb.org/cidrdb/2026/i-cant-believe-its-not-yannakakis-pragmatic-bitmap-filters-in-microsoft-sql-server.html

by tanelpoder • 6 months ago

21 7 6 months ago

vldb.org

paunchy• 6 months ago

Unpopular opinion: SQL Server is Microsoft's best product. Cost aside, it's a smart choice for a relational database and reliably meets most needs better than the open source alternatives.

bob1029• 6 months ago

I think .NET is Microsoft's best product, followed very closely by MSSQL.

SQL Server has some incredible 3rd party tools build up around it. 3~4 Red Gate tools can one-shot your entire devops lifecycle.

I still prefer SQLite for anything that can handle reasonable, non-zero recovery point objectives. But, if it's serious business and we cant drop anything on the floor I am reaching for MSSQL without even blinking.

adzm• 6 months ago

The query optimizer is truly amazing as well. I've seen it take complete madness as input and still perform reasonably. Now that you can keep targeting old behaviors until explicitly requested, upgrades have been much less concerning too.

Additionally columnstore support is awesome!

badlibrarian• 6 months ago

For genuinely useful workloads, it's available for $5/month (per database, not per server) on Azure. There's a free version if you hunt around.

I'm paying $10/month since 2012 and happy. Cannot find even a hint of these DTU packages on the Azure site to link to. (Good lord, Microsoft.)

jiggawatts• 6 months ago

I used to think so too, but they've been increasing the per-core licensing fees while simultaneously hardware core counts have been increasing exponentially. The upcoming AMD EPYC 6-th generation servers will have 512 cores in a dual-socket configuration! The MS SQL licensing for that will start to approach the GDP of some smaller nation states.

The argument always was that it's worth paying the MS licensing tax because their proprietary engine is so much faster and the hardware is so expensive that the result is a net win. In other words, the "total cost of ownership" argument made by the Microsoft sales people was actually valid. In the era of two to twelve core servers, this was true! Not any more. Open source database engines at the same dollar spend levels run circles around MS SQL.

More importantly, MS SQL is falling behind on features. It has a handful of inexperienced developers keeping it on life support. Its 2025 release added just a handful of major features (Regex, better JSON support, and an AI index). None of these are "core" changes, they're all things that a junior dev can "tack on" to an existing product relatively easily and safely. Meanwhile, fundamental limits remain unchanged for decades and are starting become big problems:

1. The MS SQL parallel executor can't utilise more than 64 threads per individual query. On the aforementioned 512-core boxes, this limits it to 12.5% of the capacity of one rack mount server!

2. It can't natively use modern SSD drives with greater than 4 KB atomicity. This is practically all server-grade SSDs, including Azure Premium SSD v2 and all v6 or later VM SKUs! This leads to absurdities, like the Windows 2025 + SQL 2025 Azure Marketplace images reporting "not supported" on database-optimised Azure VMs and refusing to install.

3. Terrible support for external data in blobs, such as parquet files in data lakes. Just compare the "effortless" approach in something like DuckDB to the mess that is CETAS and the like in SQL PolyBase. All development effort in this space is being sunk into the cloud-only Microsoft Fabric product. The on-prem database engine is being left entirely behind.

Etc...

I wouldn't start new projects on MS SQL in 2026. The Titanic is starting to tilt a bit now, the lights are still on and the music is playing, but the smart people know it is time to go find a life boat.

hdz284• 6 months ago

I Can't Believe It's Not Yannakakis

tester756• 6 months ago

.NET Core and Visual Studio (both standard and Code) are good also

Pragmatic bitmap filters in Microsoft SQL Server

Related Stories