thelastbender12

I think that's a fair opinion, but I'd argue against it being poorly thought out - pandas HAS to stick with older api decisions (dating back to before data science was a mature enough field, and it has pandas to thank for much of it) for backwards compatibility.

Sorry, how is this new pricing anything but honest? They provide an editor you can use to - optimize the context you send to the LLM services - interact with the output that comes out of them

Why does not justify charging a fraction of your spend on the LLM platform? This is pretty much how every service business operates.

Thank you for writing this - there is a very clear split you feel when using fastapi for single script web servers, vs trying to organize it. And I probably share all the mentioned annoyances around writing bigger projects with fastapi.

The speed here is super impressive! I am curious - are there any qualitative ways in which modeling text using diffusion differs from that using autoregressive models? The kind of problems it works better on, creativity, and similar.

Thank you, my bad - I wasn't aware.

I still think what drives languages to continuously make changes is the focus on developer UX, or at least the intent to make it better. So, PLs with more developers will always keep evolving.

Sorry, I think this comparison is just unfair. Odin might have "shipped" but are there are any projects with significant usage built on it? I can count at least 3 with Zig - Ghostty, Tigerbeetle, and Bun.

Programming languages which do get used are always in flux, for good reason - python is still undergoing major changes (free-threading, immutability, and others), and I'm grateful for it.

Do you use Jupyter notebooks in VSCode? It uses the same pylance as regular python files, which actually gets annoying when I want to write throwaway code.

You need to request a specific python version compatible with this project. Give `uv venv --python 3.11` a try.

https://docs.astral.sh/uv/pip/environments/

I meant Pylance isn't legally available in Cursor (Vscode license restriction, which is justified). It broke very frequently, so I switched to Based pyright, which works but just not as well.

Yep, exactly that. IMO agent workflows, MCP and tool usage bits are all promising, but the more common usage of LLMs in coding is still chat. AI extensions in editors just make it simple to supply context, and apply diffs.

An addon makes it seem like an afterthought, which I'm certain you are not going for! But still making is as seamless as possible would be great. For ex, response time for Claude in Cursor is much better than even the Claude web app for me.

Congrats on the release! I've been using Cursor but somewhat annoyed with the regular IDE affordances not working quite right (absence of pylance), and would love to go back to VSCode.

I'd love it if you lean into pooled model usage, rather than it being an addon. IMO it is the biggest win for Cursor usage - a reasonable num of LLM calls per month, so I never have to do token math or fiddle with api keys. Of course, it is available as a feature already (I'm gonna try Continue) but the difference in response time b/w Cursor and Github copilot (who don't seem to care) is drastic.

This is pretty neat.

IMO a hindrance to this was lack of built-in fixed-size list array support in the Arrow format, until recently. Some implementations/clients supported it, while others didn't. Else, it could have been used as the default storage format for numpy arrays, torch tensors, too.

(You could always store arrays as variable length list arrays with fixed strides and handle the conversion).

I see it being a trade-off between how explicit the state persisted for a workflow execution is (rows in a database for Temporal and DBOS) vs how natural it is to write such a workflow (like in your PL/compiler). Given workflows are primarily used for business use-cases, with a lot of non-determinacy coming from interaction with third-party services or other deployments, the library implementation feels more appropriate.

Though I am assuming building durability at a language-level means the whole program state must be serializable, which sounds tricky. Curious if you could share more?

I kinda agree with you but I can also see why it isn't that far from "reasoning" in the sense humans do it.

To wit, if I am doing a high school geometry proof, I come up with a sequence of steps. If the proof is correct, each step follows logically from the one before it.

However, when I go from step 2 to step 3, there are multiple options for step-3 I could have chose. Is it so different from a "most-likely-prediction" an LLM makes? I suppose the difference is humans can filter out logically-incorrect steps, or prune chains-of-steps that won't lead to the actual theorem quicker. But an LLM predictor coupled with a verifier doesn't feel that different from it.

The succeeded operations would skip because workflow has run completion record for same idempotency key. Is that correct?

This sounds about right. But you need to make sure the service being called in that step is indeed idempotent, and will return the same response which it earlier couldn't in time.

PyArrow string arrays store the entries (string values) contiguously in memory, so access is quicker, while object arrays have pointers to scattered memory locations in the heap.

I agree, I couldn't really figure how the new numpy string data type makes it work though.

I think that's a little harsh. Imo the reason for difference in popularity/github-stars is just different user bases- an order of magnitude more people use LLM APIs (and can leverage Dspy) vs those who finetune an LLM.

Agree about the abstractions btw. I found Dspy very convoluted for what it does, couldn't make sense of Textgrad at all.

This looks great!

I'd also love to hear from people working on compilers - what are some real/fun/cool problems to work on for amateur compiler writers?

Suspect the obvious candidates are deep-learning, sql engines but those already get a lot of attention.

I think this is a fine opinion, we like tools that do exactly how much we want them to. But I'd suggest setting up python (and virtual envs) was actually a big headache for a lot of newer users, and some of the old ones (me that is).

I also don't see why leaning into python being a wrapper around rust/cpp/c is a bad thing. Each language has its own niche and packaging/bootstrapping is more of a systems level language problem.

For that use-case, you can set up a `virtual project` using Rye. And use it just to create python envs, and sync dependencies.

Honestly, the biggest time-saver for me has been Rye automatically fetching python binaries that work everywhere, and setting up clean venvs.

- https://rye.astral.sh/guide/virtual/

Simon Willison's github would be a great place to get started imo -

https://github.com/simonw/datasette https://github.com/simonw/sqlite-utils

So, his code might not be a good place to find best patterns (for ex, I don't think they are fully typed), but his repos are very pragmatic, and his development process is super insightful (well documented PRs for personal repos!). Best part, he blogs about every non-trivial update, so you get all the context!

I'm curious where universal forecasting models are most useful. It is technically fascinating but forecasting specifically seems like a domain where you'd want interpretable modeling - you use it for big-value problems and it significantly affects your action/policy. So, the tradeoff between performance and model simplicity should lean towards the latter?

Hi Omar - thanks for engaging here. I have a similar question to simonw, it _feels_ like there is something useful here but I haven't managed to grok it yet, after sitting through the tutorial notebooks.

Specifically, to your description above, I'd love seeing specific retrieval examples, where you need more-complex pipelines. Zero shot QA (1-step), few-shot QA (2-step), retrieval + few-shot QA (3-step) all make sense, but when the README starts talking about demonstrations, I can't really follow when is that actually needed. Also, it starts feeling too magical when you introduce "smaller LMs" since I don't know what those are.

Transferring state across sdks sound very cool, how do you achieve that?

With all the workflows as code frameworks (like say, Temporal), this is what I imagine to be a shortcoming. With regular services + state in database implementations, you can query the database separately. While with a temporal workflow, you can't cleanly figure out where a particular workflow run is atm since the state is an append only stream of events?

So anyways, PEP 703 is a a nice effort, but I doubt we (everyday mortals) can enjoy it.

I'd suggest we can be more optimistic. There is a lot of python, regular users write daily, where we delegate orchestration to established libraries - asyncio/web-frameworks/pytorch. The GIL limits how much they can parallelize your code, and its removal will help with that.

The co-author Nathan Kutz is a great instructor too. I recall going through his optimization course on Coursera back when MOOCs were new; instructors excited about the course material make learning fun.

Why is this comment necessary on every Julia-related post? I don't even use Julia outside tutorials but this adds no value beyond things that have already been said N number of times.

Every programming language doesn't need to become _the_ language to do something. They are experiments in how to best express what you want to compute. Even if Julia never takes off, they explore multiple directions other languages might want to implement - multiple dispatch for polymorphism, nested parallelism, macros so you can create DSLs from regular Julia code, and so much more. Asserting that Julia is only successful if everyone is using it is just super reductive.

Oh, to be specific, the platform was oriented towards test practice and our objective was to recommend questions in a sequence.

One good strategy that correlated with session length for us was asking questions that were neither too difficult or easy, based on what we knew of the user's level at that instant. The post hoc analysis was really meant to dig into multiple user sessions and see if the current method was working and evaluate counterfactual strategies.

I imagine the Uptrain product could help us segment user cohorts, find out which ones aren't performing super well etc. Would love to hear what you ended up building too?

Congrats on the launch, looks like a cool product! Just scanned the docs, so not super sure if my specific use case is supported.

I previously worked on a content recommendation system for academic users. We often wanted to go back and look through specific user sessions to see if the recommendations made sense in the context of their activity. So, ground truth data was kind of available, but only at a later time.

Is this kind of post-hoc analysis in your product scope? Looking at the code examples, it seems like you have to provide ground-truth data at inference time?

But seriously, for open source projects actually looking for contributors (vis-a-vis companies with a developer product they just make open source), there is no better resource.

To understand a framework, you need a mental model of it. Good documentation is helpful, but it seldom walks through why specific design choices were necessary.