alangpierce

What if you just... removed the AI part?

Maybe I'm not fully understanding the approach, but it seems like if you started relying on third-party MCP servers without the AI layer in the middle, you'd quickly run into backcompat issues. Since MCP servers assume they're being called by an AI, they have the right to make breaking changes to the tools, input schemas, and output formats without notice.

The state already puts a lot of resources into improving road safety, and that's one of the primary goals of the DMV, DOT, etc. There's reason to believe that driverless cars will greatly improve road safety, so it feels reasonable that the state would allow their development and have a framework for responsibly expanding their use.

Interestingly, the ChatGPT Plugin docs [1] say that POST operations like these are required to implement user confirmation, so you might blame the plugin implementation (or OpenAI's non-enforcement of the policy) in this case:

for POST requests, we require that developers build a user confirmation flow to avoid destruction actions

However, at least from what I can see, the docs don't provide much more detail about how to actually implement confirmation. I haven't played around with the plugins API myself, but I originally assumed it was a non-AI-driven technical constraint, maybe a confirmation modal that ChatGPT always shows to the user before any POST. From a forum post I saw [2], though, it looks like ChatGPT doesn't have any system like that, and you're just supposed to write your manifest and OpenAPI spec in a way that tells ChatGPT to confirm with the user. From the forum post, it sounds like this is pretty fragile, and of course is susceptible to prompt injection as well.

[1] https://platform.openai.com/docs/plugins/introduction

[2] https://community.openai.com/t/implementing-user-confirmatio...

This article argues that there's no reliable way to detect prompt injection: https://simonwillison.net/2022/Sep/17/prompt-injection-more-...

One solution to some indirect prompt injection attacks is proposed in this article, where you "sandbox" untrusted content into a second LLM that isn't given the ability to decide which actions to take: https://simonwillison.net/2023/Apr/25/dual-llm-pattern/

Agreed that if you focus on any specific task, there's a safe way to do it, but the challenge is to handle arbitrary natural language requests from the user. That's what the Privileged LLM in the article is for: given a user prompt and only the trusted snippets of conversation history, figure out what action should be taken and how the Quarantined LLM should be used to power the inputs to that action. I think you really need that kind of two-layer approach for the general use case of an AI assistant.

Giving different permissions levels to different email senders would be very challenging to implement reliably with LLMs. With an AI assistant like this, the typical implementation would be to feed it the current instruction, history of interactions, content of recent emails, etc, and ask it what command to run to best achieve the most recent instruction. You could try to ask the LLM to say which email the command originates from, but if there's a prompt injection, the LLM can be tricked in to lying about that. Any permissions details need to be implemented outside the LLM, but that pretty much means that each email would need to be handled in its own isolated LLM instance, which means that it's impossible to implement features like summarizing all recent emails.

It looks to me like the profile viewer is actually speedscope ( https://www.speedscope.app/ ). I find it nicer for exploring profiles compared with Chrome's built-in viewer.

To use with Node.js profiling, do the `node --inspect` and `chrome://inspect` steps, then save the profile as a .cpuprofile file and drag that file into speedscope.

Another thing I've found useful is programmatically starting/stopping the profiler using `console.profile()` and `console.profileEnd()`.

Like any transpiler, Sucrase can be run in parallel by having the build system send different files to different threads/processes. Sucrase itself it more of a primitive, just a plain function from input code to output code.

What I do not understand is "Sucrase does not check your code for errors." So it's not a type checker?

That's correct, Sucrase, swc, esbuild, and Babel are all just transpilers that transform TypeScript syntax into plain JavaScript (plus other transformations). The usual way you set things up is to use the transpiler to run and build your TS code, and you separately run type checking using the official TypeScript package.

Hi, Sucrase author here.

To be clear, the benchmark in the README does not allow JIT warm-up. The Sucrase numbers would be better if it did. From testing just now (add `warmUp: true` to `benchmarkJest`), Sucrase is a little over 3x faster than swc if you allow warm-up, but it seemed unfair to disregard warm-up for the comparison in the README.

It's certainly fair to debate whether 360k lines of code is a realistic codebase size for the benchmark; the higher-scale the test case, the better Sucrase looks.

worse it disables esbuild and swc's multi-threading

At some point I'm hoping to update the README benchmark to run all tools in parallel, which should be more convincing despite the added variability: https://github.com/alangpierce/sucrase/issues/730 . In an ideal environment, the results are pretty much the same as a per-core benchmark, but I do expect that Node's parallelism overhead and the JIT warm-up cost across many cores would make Sucrase less competitive than the current numbers.

itertools.groupby isn't really the groupBy operation that people would normally expect. It looks like it would do a SQL-style "group by" where it categorizes elements across the collection, but really it only groups adjacent elements, so you end up with the same group key multiple times, which can cause subtle bugs. From my experience, it's more common for it to be misused than used correctly, so at my work we have a lint rule disallowing it. IMO this surprising behavior is one of the unfriendliest parts of Python when it comes to collection manipulation.

https://docs.python.org/3/library/itertools.html#itertools.g...

The big difference, I think, is that dx87's post encourages the reader to make fewer assumptions and to keep an open mind ("While we don't know the details, it's entirely possible..."), while your bullet points are statements that encourage the reader to make more assumptions.

FWIW I find that TypeScript types can sometimes be overwhelming or hard to decipher, especially some third-party library type definitions. The connect function from react-redux is one example:

https://github.com/DefinitelyTyped/DefinitelyTyped/blob/d07d...

Since types can get really complex, I feel a need to be conscious of avoiding complexity when writing types myself, unless the value is enough to justify a hard-to-read type.

Great project, really expands what feels possible with the new improvements in TS 4.1.

I wonder if this strategy could also be used to have strongly-typed GraphQL queries without the need to have a type generation step that you run separately. Currently when using GraphQL with Apollo, it's great that you get types corresponding to all queries, but having to run the type generator and maintain types in __generated folders is definitely an extra hassle.

I use gitk pretty often and really like the fact that it acts like a modal, showing a richer UI visualization based on a shell command I run, and when I close it, I'm back to the command line where I was. I've been hoping to write my own CLI tools like that, ideally using React for the UI, but it seems like it's a big pain to set up right now. Is anyone aware of a nice way to do that? Maybe Zenity is flexible enough for arbitrary UIs, not just dialog boxes?

the downside of functional testing is that it's slower

This is certainly true in some situations, but not all, and I think it's important to make decisions based on the tech stack and tools available. For example, I've found that React + Enzyme + jsdom is a way to write frontend functional tests that are both fast and reliable, so in a codebase like that, I strongly prefer functional tests because they provide much more confidence. I've also found this to be true for software running against low-complexity data stores that are easy to fake, and for things like compilers that are running fully in-memory anyway.

when a functional test fails it not obvious what is broken

FWIW, I've found that with a good debugger, it's usually pretty quick to start with a functional test failure and trace the issue to the level of specificity that I'd have if it was a unit test failure. But like the other point, I'm sure it depends a lot on context.

We usually work on challenging problems

At least for me (as someone who's more often answering the questions at my company), this varies quite a bit. Certainly sometimes there's a deep technical problem or an issue requiring genuinely tricky troubleshooting, but often it's a new hire who is still getting up to speed and didn't realize that there's some command that they need to run that will solve all of their problems. In a case like that, I worry about an environment where they feel afraid to ask a question unless they've already fought with the problem for a long time, and where they feel obligated to post an extensive write-up of the steps they've already tried. I would call that an "opposite extreme" since they're significantly less productive than they would be if they had been more willing to reach out for help early on. That's my main concern with the article: it has a tone of scolding people for the way that they ask questions, and I worry that it would cause people to be afraid to ask questions even when it would be best for them to do so.

I think the ideal approach is a balance: some initial detail on the problem, but with an understanding that deeper details can come in the follow-up (which might be over chat or over a call or some other medium).

In case others are also confused by:

The first screen is a long scroll of 20 elegant looking blocks and titles none of which mean anything on their own.

The page behaves differently on mobile vs desktop, and that quote is a description of the mobile experience. On desktop, it also includes a one-sentence explanation of each law without needing to click through.

Certainly many of them do, but there are also plenty of people who want to focus on investments that also have a positive impact: https://en.wikipedia.org/wiki/Socially_responsible_investing

One specific example of a robo-advisor that does this: https://www.wealthsimple.com/en-us/feature/socially-responsi...

Hi, Sucrase author here. I'm also having a hard time understanding the rationale here, but it looks like it's maybe a nuance around self-hosting. In general, for any regular TS project, I'd certainly recommend decoupling typecheck and transpile as separate operations and running typecheck alongside lint. IMO one of the nicest things about TypeScript is that you can run your code before you've figured out how to get the types to check out.

They do consider swc (which has roughly the game goal as Sucrase) in the conversation, so it seems like there's some consideration of a transpile-only approach. TypeScript's built-in `transpileModule` is probably the most reliable and officially-supported way to get the equivalent, and is much faster than running TSC with typechecking.

I solved this using the ts-interface-builder and ts-interface-checker libraries and have been happy with the result. You write a plain TS type (rather than a custom syntax like io-ts has) and then run a codegen step to make a runtime representation of the type. Then you can create a checker from that and do runtime type checking.

https://github.com/gristlabs/ts-interface-builder

https://github.com/gristlabs/ts-interface-checker

Example: https://github.com/alangpierce/sucrase/pull/468/files

It was an attempt at standardizing a browser setting to opt out of tracking, but it was ultimately unsuccessful.

https://en.wikipedia.org/wiki/Do_Not_Track

Got it. So I understand, maybe someone saved a filename as the latin-1 encoding of some non-ASCII text, and Mercurial would need to support such files (but also would have no contextual information that it's latin-1)?

I'm tempted to say "nobody should have filenames like that", but I guess a project like Mercurial needs to be as compatible as possible. Are there modern use cases for filenames like that, or is it fair to say it's all legacy data?

It's interesting that they wanted to add b' prefixes to all strings, and I wonder if they would have had a better experience by embracing regular strings instead. At least in Python 3, if your string only contains ASCII, then the underlying representation will use one byte per character, so ASCII-only strings are stored just as efficiently as ASCII-only bytes instances.

I think there are two mental models for how to approach the str/bytes split:

1.) A `str` is for unicode use cases, and a `bytes` is better for cases that don't support unicode.

2.) A `bytes` is an array of numbers between 0 and 255. A `str` should almost always be used when your value is conceptually a sequence of characters. `str` doesn't imply that arbitrary unicode is allowed, and it's fine to have a convention that a particular `str` is ASCII-only, just like other conventions you might have on variable values.

My impression is that #1 is the Python 2 mental model and is tempting for Python 3, but that #2 often works better when writing Python 3 code. Under mental model #2, asking for "%s" formatting is really asking for a replacement strategy that detects the number 37 followed by the number 155 in an array of numbers and fills in a sub-array, which seems more strange and likely to get false positives if you're really working with binary data like the bytes of a .jpg file.

That said, I'm sure the devil is in the details, and maybe a project like mercurial has to stay backcompat with bytes data that is neither ASCII nor valid UTF-8, or some other compelling reason to stick with bytes everywhere.

Sorry, I wasn't meaning to dismiss the point wholesale, just saying to be cautious of an "appeal to authority"/"appeal to precedent" sort of argument, especially in this case. I do think the severity was at least part of pjmorris's point, and they called it out in their response to me.

Hammurabi's code probably isn't a great example when looking for precedent on appropriate punishment. Another example from your link is 210: "if a man kills a pregnant woman, the punishment is that the man's daughter shall be killed".

On Mac, Control+Command+Space brings up the emoji picker. Every OS has something like that these days, I'm pretty sure.

Unicode and Punycode were originally designed for letting computers handle the wide variety of human writing systems, common ones like Chinese and Arabic, as well as rarer ones. Much more recently, the emoji characters were added to that same Unicode system, so any encoding system these days that's international-friendly also ends up supporting emoji as well.

So Punycode wasn't specifically designed to work with emoji, it just happens to work with emoji because emoji are part of Unicode.

Cultured meat is supposedly getting pretty close, maybe the next few years, though maybe longer. https://www.memphismeats.com/ is one company doing it.

I'm vegetarian and I would definitely eat it, and most of my vegetarian friends would as well (assuming they figure out how to produce it without fetal bovine serum, which isn't vegetarian). But certainly everyone is different. I imagine some people will still have ethical qualms, and I know a number of vegetarians who don't like the impossible burger because they don't like the taste and texture of meat, so presumably they wouldn't like lab-grown meat either.

Makes sense! The image we're referring to is the one for the "Built for macOS" section, not the full screenshot at the top of the page. That one doesn't have a cursor in it.

https://www.chimehq.com/assets/images/built-for-macos.png

From the screenshot, it looks like the intention is that "Prin" has been typed by the user and "tf" is a suggestion, so it makes sense that "tf" would be gray and "Prin" would be green. (Of course, that could be made more clear.)