archiewood

My gameboy color still works

If you want to deliver data to your customers you have two main options:

1. Build something custom in js

2. Use a drag and drop BI tool

(1) works if you are a front-end dev, but then it means the dev team need to own it forever, deal with customer requests etc

(2) means the data team can run it, but there's poor release management, and normally doesn't look good in your app

Evidence's version tries to hit the sweet spot: It's still written in code so you have version control and flexibility for customization, but it's SQL and Markdown so the data team can maintain it

DuckDB is really having a moment

The ecosystem is very active, and they have recently opened up "community extensions" to bring your own functions, data types and connections. A barrier at the moment is that extensions are written in C++, though this limitation should be removed soon.

I've been building a lot on top of DuckDB, two of the projects I'm working on are linked in the article:

- Evidence (https://evidence.dev): Build data apps with SQL + Markdown

- DuckDB GSheets (https://duckdb-gsheets.com): Read/Write Google Sheets via DuckDB

DuckDB GSheets is a DuckDB extension that allows you to read and write data from Google Sheets.

-- Install

install gsheets from community;

load gsheets;

-- Authenticate with the browser

create secret (type gsheet);

-- Read

select * from 'https://docs.google.com/spreadsheets/d/11QdEasMWbETbFVxry-Ss...'

-- Write

COPY <table_name> to 'https://docs.google.com/spreadsheets/d/11QdEasMWbETbFVxry-Ss...' (format gsheet);

Wow this is much more complete than mine, kudos.

Oh definitely, I love DuckDB.

This was a learning exercise for me as much as anything

503 Service Unavailable - PUT https://registry.npmjs.org/package-name - Service Unavailable

Wanted an incredibly simple interface for plotting data.

Pass data from CLI arguments, or a CSV file, get a nice chart with zero config:

`plot data.csv`

Excel for mac is (intentionally?) crippled.

Any serious Excel user has a PC.

One of the few places remaining where Windows > Mac

You might be interested in what we’re building over at Evidence.dev

It’s basically RMarkdown for SQL

This is by far the most impressive link in the comments.

i was kind of messing around with this blinking idea but couldn't get it to render in a way I was happy with

would you use another keyframe or did you have something else in mind

Thanks - I do honestly think the keyframes are a bit of a nightmare, but they also kind of make the site

I cheated for brevity :S

repo was private, now public!

oops well i just saw this took off and have made it public - thanks

I built this website to see what was possible with only one div, and a stylesheet.

No JS, No other HTML, just CSS.

Source: https://github.com/archiewood/pure-css-site

I’d love to try this with the data from our public slack community.

What’s the best way to go about this?

I see you support csv uploads but it’s going to take me a decent amount of wrangling to unpack the json blobs slack spits out.

Feel free to reach out: archie at evidence.dev

It’s interesting to me how far you have pushed the SQL language in this framework, such that it truly is “SQL only”.

The challenge as I see it with enabling analysts to build websites is that you need to build abstractions to get from familiar (SQL, yaml) - the language of analytics, to new (HTML, CSS, JS) - the language of the web browser

As one of the maintainers of Evidence (https://evidence.dev), one of the things I’ve often considered is how accessible our syntax is to analysts. Our syntax combines SQL and Markdown, with MDX style components e.g. <Component/>

The </> are inherently webdev-ey, and I do think they put off potential users.

On the flip-side, by adhering to web standards, you get extensibility out of the box, and working out what to do is just a Google search away.

Anyway, thanks for the thought provoking piece.

Aha I did not notice that prompt. By default my brain is trained to ignore things that are commented out in code

I think I would execute the SQL query that the cursor was within.

Ie anything until the next ;

We're really excited to see all the stuff built on DuckDB-WASM too!

I love how fast your workbench runs, and how effortlessly it renders 60k rows in the browser

Some UX feedback, if you're open to it:

- It was initially unintiutive for me that I needed to highlight a whole SQL statement to get Ctrl-Enter to run it. Also a prompt that this was the correct shortcut would help!

- I dragged in a CSV file, and the name was too long to show up in your table explorer, so I couldn't tell what the table name had been called (it turned out I needed to `select * from table_name.csv` - the csv postfix was unexpected

- The CSV file had headers, but they were not auto detected - would be good to be able to configure this

This is a relatively similar architecture to that we are building at Evidence.dev (open-source data viz framework)

Architecture: https://evidence.dev/blog/why-we-built-usql/

1. Query SQL databases, APIs, or local data (eg CSV)

2. Compile all the data sources into Parquet files

3. DuckDB-WASM in the browser that allows you to aggregate across sources

4. Users write code in DuckDB SQL and Markdown, enriched with viz components (built in Svelte)

Some things that we have learned in the process:

- It can be pretty performant up to about 20M rows of data in the parquet files,

- Above a certain level, for speed, it's helpful to sort your data in your parquet files to take advantage of DuckDB's predicate pushdown

- It's helpful to map DB types into a smaller set of Arrow types when you convert to Parquet - otherwise you have to consider a lot of different cases in the browser when you render in JS

- DuckDB-WASM still has some rough edges, though is improving fast

Was very bummed out when Heroku started charging for Hobby Tier as I had to sunset a few of my hobby apps

- Heroku was my first experience deploying a website: `git push heroku master`

- Dataclips was my first experience writing SQL

Writing code was markedly better than building dashboards in Tableau, which inspired Evidence (https://evidence.dev) - an open-source SSG for creating data websites using SQL and markdown, which I'm now working on

Previous HN discussions:

https://news.ycombinator.com/item?id=35645464 (97 comments)

https://news.ycombinator.com/item?id=28304781 (91 comments)

I love Obsidian.

The Markdown <-> Markup typing experience is just so good compared to e.g. Slack, Reddit and other markdown-esque tools

  Traceback (most recent call last):
  File "/home/runner/2320236597/main.py", line 3, in <module>
    from email_sender import SendEmail
  File "/home/runner/2320236597/email_sender.py", line 7, in <module>
    my_secret = os.environ['gmail_pass']
  File "/nix/store/xf54733x4chbawkh1qvy9i1i4mlscy1c-python3-3.10.11/lib/python3.10/os.py", line 680, in __getitem__
    raise KeyError(key) from None
    
    KeyError: 'gmail_pass'

This is my current workflow, though with ChatGPT.

I was just trying to save a few clicks

My most common use case here is to then want to hit the API from python, and adjust the params / url etc.

Would love a "copy to python requests" button that

grabs the headers

generates a boilerplate python snippet including the headers and the URL:

    import requests
    import json

    url = '<endpoint>'

    headers = {
        'User-Agent': 'Mozilla/5.0 ...',
        ...
    }

    data = {
        "page": 5,
        "size": 28
        ...
    }

    response = requests.post(url, headers=headers, data=json.dumps(data))

    if response.status_code == 200:
        print(response.json())
    else:
        print(f"Error {response.status_code}: {response.text}")

Looks cool, will check it out next time.

I do quite a lot of this kind of stuff for my job. Some thoughts that may be useful.

Often the full IDE is needed. I record a lot of gifs of VSCode, where part of the gif is typing code, part is interacting with the rest of the IDE / terminal - perhaps to run the code and view the output.

For me the killer app would be one which could pre-record keystrokes (and maybe mouse actions) so that I could do them error free. I often attempt a gif 10 times before I'm happy with the outcome.

I don't personally love the transition animation. I would want the option for something that seems like it's being typed.

The closest tools I've found are:

Typewriter VSCode extesion: Allows you to copy text and then "types" it out for you. https://marketplace.visualstudio.com/items?itemName=dansilve...

Ascii Cinema: https://asciinema.org/

The linked source data from the RIAA is presented much better: https://www.riaa.com/u-s-sales-database/

Positive money is up on the y axis