Towards a Learning-Based Query Optimizer

https://engineering.databloom.ai/2022/05/towards-learning-based-query-optimizer.html

by 2pk03 • 4 years ago

27 3 4 years ago

engineering.databloom.ai

This is cool, I’ve always thought db optimization would be a great place for ML

karsinkk• 4 years ago

While transforming the plan into vectors is interesting, I wish they'd gone into more detail about how the ML model prunes and filters the best plan. It is also not clear what attributes of a plan the corresponding vector encodes. I do not know much about Databloom, but it looks like this "Learning-Based Query Optimizer" is built for specific use-cases in a Data engineering/analytics setting(like K-means as cited in the article). It might not be a replacement for optimizers in traditional Databases.

BenoitP• 4 years ago

not clear what attributes of a plan the corresponding vector encodes

Fig 5, page 4 from [1]:

Topology Features
Operator Features
Data Movement Features
Dataset Features

For a single logical plan, meaning it will vary in its length for another query. (which is a part I don't get: you learn a new model per query? Can you learn with a variable feature length?)

[1] https://conferences.computer.org/icde/2020/pdfs/ICDE2020-5ac...

Towards a Learning-Based Query Optimizer

Related Stories