HN user

mgradowski

104 karma

https://mgradow.ski

Posts0
Comments38
View on HN
No posts found.
AI is different 11 months ago

I jokingly alluded to antirez as HN crowd pars pro toto. I agree it doesn't pass as an intellectually honest argument.

AI is different 11 months ago

Sorry boss, I'm just tired of the debate itself. It assumes a certain level of optimism, while I'm skeptical that meaningfully productive applications of LLMs etc. will be found once hype settles, let alone ones that will reshape society like agriculture or the steam engine did.

AI is different 11 months ago

I wouldn't trust a taxi driver's predictions about the future of economics and society, why would I trust some database developer's? Actually, I take that back. I might trust the taxi driver.

What you're really saying is that the database presented in OP is not useful because it only handles DQL.

1. SQL can be thought of as being composed of several smaller lanuages: DDL, DQL, DML, DCL.

2. columnq-cli is only a CLI to a query engine, not a database. As such, it only supports DQL by design.

3. I have the impression that outside of data engineering/DBA, people are rarely taught the distinction between OLTP and OLAP workloads [1]. The latter often utilizes immutable data structures (e.g. columnar storage with column compression), or provides limited DML support, see e.g. the limitations of the DELETE statement in ClickHouse [2], or the list of supported DML statements in Amazon Athena [3]. My point -- as much as this tool is useless for transactional workloads, it is perfectly capable of some analytical workloads.

[1] Opinion, not a fact.

[2] https://clickhouse.com/docs/en/sql-reference/statements/dele...

[3] https://docs.aws.amazon.com/athena/latest/ug/functions-opera...

I'm curious why do you think of DAX as a virtue. My poor SQL-shaped peg brain has never really fit the DAX hole of MS software.

Also, it always struck me as something too complex for the non-technical folks, and not expressive enough for tech-literate analysts/data engineers &c.

That's what I do at $dayjob whenever I have to do windowing &c. Figuring out this stuff in Pandas is a waste of time. Before I discovered DuckDB, I would re-learn the API every damn time. I came up with a little utility function, which you can implement yourself :)

``` def sqldf(df: DataFrame, query: str) -> DataFrame: ... ```

I like interface-only packages in the Julia ecosystem e.g. Tables.jl enables the development of several packages for querying tabular data that work across many concrete implementations; Plots.jl separates the high-level plotting interface from the plotting backend.

Better SQL JOINs 5 years ago

Neither did I. I tend to avoid RBDMS-specific syntax extensions of questionable benefit.

The only thing I really miss in PG is ASOF JOIN. It's possible to implement it via CROSS JOIN LATERAL, but in my case it resulted in nested loops — coming from ClickHouse it bothers me that there is no way (that I know of) to hint PG to use a specific join strategy.

Better SQL JOINs 5 years ago

Related — Postgres (other RDBMS's too, probably) has JOIN b USING (column, ...) and NATURAL JOIN. IIRC, both fail when there's ambiguity in column names.

Also, you will sleep better at night knowing that your column dtypes are safe from harm, exactly as you stored them. Moving from CSV (or god forbid, .xlsx) has been such a quality of life improvement.

One thing I miss though is how easy it is to inspect .csv and .xlsx. I kinda solved it using [1], but it only works on Windows. More portable recommendations welcome!

[1] https://github.com/mukunku/ParquetViewer

I use WireGuard with a server in my pantry as a router. Dynamic IP of the server is handled by DuckDNS, and WireGuard gracefully handles client roaming e.g. I can switch from home wifi to mobile internet without interrupting my SSH sessions. Would recommend.

That's a nice and tidy codebase, real pleasure to read.

I believe many (most?) opencv & numpy operations release the GIL.

Any idea how I can determine this? I am prototyping a real time machine vision application targeting 2x720p@240fps and I want to avoid writing any C++ for as long as possible.

Regarding [1], many OpenCV functions in Python support an optional dst argument, similar to how they do in the C++ API. This makes the memory management situation not completely hopeless.

In my experience, the only drawback of the Python API is the fact that it cannot utilize the multithreading module (probably does not release the GIL for long calls).