HN user
Thank you for this report. I'll try to make it work on Ubuntu, expect the documentation to be updated in the next few days.
PS: if you're the 'leopard' who requested a redmine account, it should be activated now, feel free to report it there.
It depends on what your use case is. If you have well targeted queries (eg, with filters on "From" or "Subject" headers, and only query the message headers (not the payload) you can have a pretty good performance.
What is your use case, exactly ?
I totally agree with you regarding PL/Python. I may try to ask upstream what they think about it.
Sorry to beat you at it ;)
We currently have some very rudimentary python to postgres type conversion, but this area still need a lot of improvements.
You should release your code, I'm sure you have a wide range of ideas worth merging into Multicorn !
Thank you for your interest!
The postgresql plan is parsed, and passed as a list of "quals", objects representing simple filters. As an implementer, you don't HAVE to enforce those, since postgresql will recheck them for you anyway, but they can be quite handy.
For example, you can look at the imap foreign data wrapper (https://github.com/Kozea/Multicorn/blob/master/python/multic...) to see how the conditions from postgresql are converted to an IMAP filter, allowing for server side filtering.
The required columns are also provided, so if you don't need the email payload, the foreign data wrapper will not fetch it.
For joins, it will depend on the execution plan. There is still plenty of work on parsing the postgresql execution plan into something more useful, but the current set of "optimizations" is sufficient for our main use cases.
Multicorn developer here, I'll be glad to answer any question.
On the top of my head, another list of features worth looking at: - ARRAY support: makes your life easier, especially the array_agg aggregation function - HSTORE extension: use postgresql like a key/value store - SQL/MED implementation, foreign data wrappers: allows you to define external data sources as an extension.
Things that I miss in postgresql: - materialized views !! - a better graphical management tool.