hamilton

We do have plans. It's a question of effort, not business / philosophy.

it can, but it doesn't format. You can even run the ast!

Obviously one advantage of SQL is everyone knows it. But conceptually, I agree. I think [1]Malloy is also doing some really fantastic work in this area.

This is one of the reasons I'm excited about DuckDB's upcoming [2]PEG parser. If they can pull it off, we could have alternative dialects that run on DuckDB.

[1] https://www.malloydata.dev/ [2] https://duckdb.org/2024/11/22/runtime-extensible-parsers.htm...

Definitely something we want too! (I'm the author / lead for the UI)

I agree; one thing that is neat about Instant SQL is for many reasons, you can't do this with in any other DBMS. You really need DuckDB's specific architecture and ergonomics.

Great feedback! Thanks. We agree w/ the red errors. It's not helpful when it feels like your editor is screaming at you.

You should read the post! This is what the feature does.

Can you say more? Where does it error out? Sounds like a bug; if you could post an example query, I bet we can fix that.

What about it is bizarre?

Me too (author of the post here). In fact, I was watching a seasoned data engineer at MotherDuck show me how they would attempt to debug a regex in a CTE. As a longtime SQL user, I felt the pain immediately; haven't we all been there before? Instant SQL followed from that.

Correct. We only enable fast previews for SELECT statements, which is the actual hard problem. This said, at some point we're likely to also add support for previewing a CTAS before you actually run it.

You can use any variation of DuckDB valid syntax that you want! I prefer to put from first just because I think it's better, but Instant SQL works with traditional select __ from __ queries.

Indeed, we are! We worked with DuckDB Labs to add the query_location information, which we're also enriching with the tokenizer to draw a path through the AST to the cursor location. I've been wanting to do this since forever, and now that we have it, there's actually a long tail of inspection / debugging / enrichment features we can add to our SQL editor.

ftr I'm still on X & posting! https://x.com/hamiltonulmer

True; at the moment the UI is not open source. We've talked about releasing the Column Explorer as a standalone component, but haven't been able to prioritize it yet. We'd like to!

(I designed and built the Column Explorer feature)

Observable's column summary feature is very nice! But I do think there's a very common lineage around these kinds of diagnostics which motivated both Observables and ours. See Jeff Heer's profiler paper[1] for more.

I'm very passionate about this area because I think "first mile problems" are underserved by most tools, but they take the longest to work out.

We had to do some gnarly things[2] to make this feature work well; and there's a lot of room to make it scale nicely and cover all DuckDB data types.

[1] http://vis.stanford.edu/papers/profiler [2] https://motherduck.com/blog/introducing-column-explorer/

love this. I similarly use immer for just about every svelte project I build & maintain.

I find that Svelte is the most "jQuery-friendly" of the frameworks. I can easily add a single Svelte component back into other wise static pages without requiring anything other than a simple build tool to output the JS + css. It's really the best of both worlds – sprinkle it in where needed, or go whole-hog.

So great to see folks utilizing Pyodide, the WASM scientific python stack my Mozilla colleagues put together. The creator, Mike Droettboom, wrote a Mozilla Hacks post about it a while back:

https://hacks.mozilla.org/2019/04/pyodide-bringing-the-scien...

Thanks for the link. Incidentally I have that same talk bookmarked. Though for the record, I have needed to render thousands of dom elements per second :)

Oh, great! Thanks for the link. I had it on my to-do list to try something similar to what you did. Glad to learn from someone else's exploration.

I've posted before about my experience with svelte (https://news.ycombinator.com/item?id=20338175). Will try to reiterate a few of them here.

I've been using it internally at my current job to great effect since March, after I convinced my manager to let me try it on a one-off project. That small project has led to us using it for other bigger projects, again pretty successfully.

tl;dr – I probably wouldn't choose to go back to other frameworks after my time in Svelte-land unless I'm in a situation where I have to use React. I don't MIND React, but it's clear that all things equal it simply takes longer to make things with it and the thing I make tends to be harder to work on over time. Svelte is just too good and has probably saved my company some money in dev time & dev happiness. For reference, I work across the data stack – data engineering to stats / ml modeling to frontends / products that use data. It shines in all areas, but it REALLY shines in data viz.

pros:

- the time to first meaningful render in svelte is substantially, substantially lower than it is in React, and you face none of the drawbacks of using the React starter stuff (ie ejecting isn't a thing). This may seem like a weird metric, but I think it's pretty vital. I can easily start a side project w/ svelte and not have to think deeply about getting all the dependencies, all the webpack stuff fiddled with, all that. The ramp to productivity is pretty flat.

- you don't need the Friends part of React & Friends nearly as much, but you can definitely use the ones that are framework-agnostic (like immer, or redux, or whatever). W/ the bigger React projects I work on, I have really struggled with the development speed of 3rd party libraries forcing me to update my components and wasting time. Having fewer dependencies has made my life substantially better. Most projects I just use immer + d3 modules.

- I was pleased to discover that in practice, the boilerplate doesn't ever sneak in, even in complex components. This means that I can either (1) deliver the thing faster, and / or (2) in the same amount of time, make it that much better. It ends up being that both of those things happen.

- svelte stores are incredible. We tend to use them with immer to make redux-like app state work nicely, but we use stores for all sorts of things – contexts for complex components like data graphics (ie we need to share reactive changes arbitrarily to children), tweened and spring. One way that I enjoy using them – I take params from my app store (the one I make w/ immer) and create a derived store that encapsulates a cached dataset request. There are SO MANY cool patterns you can implement with stores that makes your app easy to reason about. Example of a svelte immer app here: https://github.com/hamilton/svelte-immer-example

- having a great story for both style & animation really makes you think that most other frameworks are feature-incomplete. The component authoring experience really makes html and css feel like first-class citizens again. I currently have a designer helping me tweak the css of a bunch of components I've written, and it's wonderful – he doesn't have to learn all the css-in-js stuff I've encountered in other frameworks.

- Svelte truly, truly shines with data visualization, where the reactivity has reduced the cost of authoring data graphics for me. I say this as someone who used to use Protovis and then spent a lot of time in D3-land. Hoping svelte gl lands publicly sometime soon so I can get off of the regl train.

- Svelte creates joy the way jQuery did in the past. I noticed a talk by Rich where he talks about jQuery. This has been how I've described svelte to folks @ my work, and they get it – it DID feel magical then.

cons:

- the testing story isn't well-documented yet, and it probably just doesn't exist yet. I tend to just move code that definitely needs to be tested into separate modules and test them there.

- typescript support would be really nice.

- the community is small, so you have to find answers yourself or make them. This may not appeal to a lot of corporate frontend devs, but it works @ Mozilla where I work, since most folks are pretty web-friendly. I personally prefer this since things that are hard in other frameworks are trivial in Svelte. That is, you just don't need that big a community to be similarly productive.

- your dev culture has to be right for this, and the risk of moving has to be understood. It takes time to figure out how new frameworks pay off.

From experience building more complex apps in Svelte, I can say that this has not been my experience, & doesn't accurately depict React in practice. Three aspects that make it really killer – a really great story for handling app & component state cleanly regardless of complexity, a great style scoping story, and powerful transition / animation primitives that fit neatly into the idiom – has made it easier to build large data apps than I've found with React (which I also use).

Another aspect I have really appreciated is that I have substantially fewer dependencies, since there is more included in Svelte than in React. I tend to only use d3 utility functions (format, scales, etc.) & immer (which I pair with svelte stores to get something kind of Redux-y, which is great for other devs jumping into Svelte from React-land). Fewer dependencies means I'm less likely to encounter awful dependency hell.

Generally, I feel like every time I see a Svelte thread here, there's a lot of random noise, but it's good to listen to the folks who actually use it. There are situations where it makes sense, and situations where it probably doesn't. The main reason why I might not use it is ecosystem friction – a lot of devs I know would prefer to just use what they know, even if it is painful relative to Svelte & friends.

Unfortunately not – they're all private, internal views at the moment. I do hope to clean up some of my components and write something about some of the cool workflow aspects I've been able to unlock.

In the meantime, I would recommend walking through the tutorials & examples. They're very well written and I go back to them regularly to try out different parts. Try just making a simple line chart! Now make six on one page! Now try to link the rollovers for all of them! You'll probably learn a ton doing just that.

As someone who has built a number of data products w/ React, I decided to give Svelte a try a few months ago. At this point I am using it on some internal projects to great success & ease and probably wouldn't go back to React for newer projects unless there was a compelling reason to (eg collaboration w/ other React devs). As a disclaimer, I admire the React community + devs and think React has made a lot of hard things much, much easier.

A few unvarnished takeaways from a data science / viz point of view that I was thinking of writing up, but will use HN as a sketchpad:

1. I don't really miss the React 3rd party ecosystem as much as I thought I would. Since most of what I do is data viz / data presentations, I really just need a few d3 helper functions, and most other things I can build myself w/ svelte's affordances. The biggest downside, however, is that there aren't any good accessible UX component libraries for Svelte, so I have built my own. This all said, I'm productively code-homesteading and I love it. Animating component lifecycles, component style scoping, and state management are 1st-class citizens, so these pieces are pretty high-quality, usable, and thoughtfully built.

2. my "time to first meaningful render" metric was at an all-time low with Svelte. Getting started on a new idea or project is far, far easier than w/ any React project I've ever done. Substantially less boilerplate means I can get much, much more done in the same amount of time. I feel this every time I jump back into the React projects I work on & maintain.

3. as my apps have grown in complexity, I've found that Svelte really holds up beautifully. The style/layout/behavior colocation strategy keeps the cognitive overhead of working on all the parts of multiple components to a minimum. The performance gains of Svelte, even with complex components and interactions, feels magical. Bundle sizes are bizarrely small.

4. as someone who feels CSS will absolutely outlive all of these frameworks, I think the design of Svelte really feels like the best of both worlds. And that's how it should be – css + markup should be first-class citizens.

5. the testing story isn't fully there yet, so I would say to that end, proceed with caution. I typically try to compartmentalize JS logic from presentation as much as possible, but perhaps you don't.

6. the reactivity parts shine everywhere and are fairly easy to reason about in all cases, but they REALLY shine when you're building complex visualizations.

7. this is an important one to me – building something in Svelte really tickles the same part of the pleasure centers of the brain that JQuery did for a previous generation (and d3 did for early data scientists). You just SEE the thing work, effortlessly, at a low cognitive cost, and everything fits together so nicely. I often start svelte-powered experiments and end with a great, reusable component. I think there is a large group of engineers who remember the JQuery days and sat out the React era because it just seemed like way too much to do simple things. I think Svelte is especially for them.

In sum, really highly recommended to at least try it. I wanted to find evidence to discount Svelte, but haven't found any at this point, despite my best efforts. There is probably a certain class of dev that should be trying it – one without a ton of organizational constraints, probably. But if you do data science and want to level up your data presentation skills, you should be using Svelte imo.

Feel free to reach out to us on Gitter https://gitter.im/iodide-project/iodide if you're interested in doing this. If you know anything about compilation, it's pretty straightforward, depending on the language & what's happening under the hood. Here are our docs on language plugins:

https://iodide-project.github.io/docs/language_plugins/

Iodide does actually support that workflow, both in JS and our wasm python stack (Pyodide). We have some data scientists internally who have made Iodide reports using only pandas & matplotlib. It's still early days for this tech, but it's quite doable.

(disclaimer: I work on Iodide)

+1 - this is the right attitude to have! If you have a productive data science team, you should continue to use the tools you have - we'd be the first to tell you that. Re: the JS thing, we've found through user testing and our own experience as data scientists that folks don't really want to use JS for anything, and we want to meet some segment of data scientists where they are. As explained in the post, JS as a language and ecosystem isn't quite there in the way that it needs to be. In a few years, it may be, if you follow along w/ the various TC39 proposals out there.

To clarify, we do have support for Python / NumPy / Pandas / Matplotlib / others with Pyodide, which is natively supported in Iodide with the `%% py` delimiter. It doesn't address all the other workflow issues you have, of course, and there are some differences b/t a server kernel and what we have.

(iodide dev here) Yeah, that's a good point. We called it JSMD before we even thought of starting Pyodide, and in hindsight seems a bit constricting.

This New Yorker piece on Andy Byford provides a lot more context for why the MTA is the way that it is:

https://www.newyorker.com/magazine/2018/07/09/can-andy-byfor...