nsypteras

https://nicksypteras.com

Analyzing frontier LLM performance on my favorite daily puzzle game (https://www.nicksypteras.com/blog/cbs-benchmark.html) Next step is to assess how well the LLMs can create their own new, logically satisfiable puzzles in the same style. Then I'll have them battle it out, with one model creating a puzzle and the other attempting to solve it!

Congrats on launching! One immediate thought is that people will always be wary of running LLM-generated code on their machines even if it's sandboxed. Is one of the future business cases for this to host a remote execution environment that pctx can call out to rather than running the code locally?

Ya interesting thought - would be fascinating if generating games w/solutions is part of the training data pipeline. There's been previous work done on on testing LLMs on logic puzzles[1][2][3] so they could possibly be building off those ideas to improve performance.

[1] https://huggingface.co/papers/2504.00043 [2] https://huggingface.co/blog/yuchenlin/zebra-logic [3] https://arxiv.org/pdf/2403.12094

I'm impressed it recommended so many books i've already read and liked! I have a big reading backlog but once it's whittled down I will likely come back to this. One feature request would be to also show a "why this is recommended" for each recommendation so I can further narrow down the list for what I'm looking for

"Counter Chinese Influence in International Governance Bodies" and grouping them in with US "adversaries" and "rivals" is quite undiplomatic language to throw in under "Lead in International AI Diplomacy and Security" section. Diplomacy with China should be an important part of this initiative but will inevitably be bungled.

1984: U.S. withdraws. 2003: U.S. rejoins. 2011: U.S. stops paying dues after Palestine joins. 2017: U.S. announces withdrawal (effective end of 2018). 2023: U.S. rejoins, pledges to repay dues. 2025: U.S announces withdrawal

Seems to be a revolving door

Same here! Kiwix comes in clutch on flights. I've used it so many times to get background knowledge on topics mid-read. Plus free and open source. Such a great service.

I think that would be one of the success cases described in the article because HITL is an integral part of good customer support chatbots. Support chats can be escalated to a human whenever the agent is unable to provide a satisfactory answer to the user.

The transportation agency has spent years looking for an innovative way to allow passengers to move faster through the security checkpoints.

I think the writer had some fun with this one

That's pretty cool! Now I can imagine a tool that gives you a prediction before you even post and then offers suggestions for how to increase performance...

A while back I made a little script (for fun/curiosity) that would do this for HN profiles. It’d use their submission and comment history to infer a profile including similar stuff like location, political leaning, career, age, sex, etc. Main motivation was seeing some surprising takes in various comment threads and being curious about where it might have came from. Obviously no idea how accurate the profiles were, but it was similarly an interesting experiment in the ability of LLMs to do this sort of thing.

Cofounder splits are extremely common. Cofounder "couples" counselors are a thing you could look into to help resolve your differences. Your VC might have recommendations for one. If you ultimately decide to split, I'd recommend at least one of you (or "the company"?) getting a lawyer to draw up a formal separation agreement you both sign in order to split in the cleanest possible way.

So much nostalgia for my Pebble. Got it right as I was seriously getting into programming. I still remember how magical it felt writing the C code to build my first watch face and how proud I was to show people. Amazing news <3

Reminds me of a modern version of The Entertainment from Infinite Jest.

Is there an ML engineering practice that isn't focused on building models but more on managing/deploying/scaling models? i.e. can I avoid learning all the maths underneath?

I transitioned to this in 2018. It was called MLOps. I was a mobile developer before that. Transitioning was pretty easy at that time (it might be more competitive now). What I did: worked on an intensive ML project for myself and realized I enjoyed working across the ML stack. I wrote a blog post on the project here: https://www.nicksypteras.com/blog/aisu.html. Then I applied to an MLOps team and leveraged the project to demonstrate skills/experience. As for avoiding the math, you could probably get away with it but learning the basics will make everything way easier. I feel like without some basic ML math under my belt I would have been flying very blindly.

thanks, just confirmed can create Signals via the API now :) Look forward to trying this out

Cool feature! Any plans to offer limited API credits in order to trial things? I just created an account but I'm unable to test (settings shows 0/300 credits).

Ditto. I would pay big money for this if it came in an inconspicuous form factor like airpods. Hopefully it's just a matter of time before Airpods themselves can do this.

So glad you said this! For some reason that's always stuck out to me as having been my biggest personal "wow" moment while watching AI development progress. ChatGPT is awesome but for some reason I've never felt as awed by it.

Clearly an extremely impressive demo and congrats on the launch. I do wonder how often the bugs Devin encounters will be solvable from the simple fixes that were demonstrated. For instance, I notice in the first demo Devin hits a KeyError and decides to resolve it by wrapping the code in a try-catch. While this will get the code to run, I immediately imagined cases where it's not actually an ideal solution (maybe it's a KeyError because the blog post Devin read is incorrect or out of date and Devin should actually be referencing a different key altogether or a different API). Can Devin "back up" at this point and implement a fix further back in its "decision tree" (e.g. use a different API endpoint) or can it only come up with fixes for the specific problem it's encountering at this moment (catch the KeyError and return None)?

Back in high school, I spent a good chunk of time reading the guides on this site purely just to fantasize about doing the trips myself one day. What a cool blast from the past, particularly given that the site hasn't changed!

Not sure how comfortable the "climatization" is given that many people in the pictures are wearing winter coats inside. Hopefully it's not sweltering in the summer...

silly fail or genius viral marketing scheme?

I'm pretty sure the ads displayed on my Kindle are all for AI generated children's books lately. It's actually kind of fun looking at them to find the AI generated artifacts (e.g little girl building snowman has three fingers, etc.) Not sure who's actually buying these things...

Same here. The watch part of my wrist normally wrests on the bottom edge of my Macbook, so I end up lifting my wrist slightly to avoid the band from scratching, which then leads to pain. I've found some luck from wearing long sleeves that cover my watch so that I don't have to worry about the scratching issue.

Thanks for the questions!

- The javascript and python runtimes are hosted on AWS. With the sandbox editor, resources are shared and code is executed in sandboxed VMs. For signed in users, each user is given a fully isolated environment with their own resources. - typescript: We'd love to add it, though we haven't decided when to do so yet. If more people request it we will surely expedite it! - environment variables are stored in AWS Dynamo DB and are encrypted at rest - Intellisense for installed packages: would love to add this soon as well, though like Typescript, still on the backlog for now. Would love to hear any other ideas for enhancing the editor experience if you have them! Happy to chat more anytime - nick@napkin.io

Nice! I really love the simplicity of Spotless! Thanks for sharing and definitely reach out if you'd ever like to chat more about this space or any general feedback - nick@napkin.io

If you're familiar with webhooks, you can code and deploy them easily with Napkin by just writing code in the editor and clicking "Deploy". With other tools, you usually have to install dependencies on the command line or set up more complex cloud services to do this. You can see an example of the webhook use case with this Slack bot example: https://napkin.wiki/slack

For a static website, there's a lot of great options that don't require a "backend" per se. Vercel and Netlify are great if you use React and/or Next.js. For a no-code solution you can check out Webflow. If you want to just serve static HTML/CSS/Javascript pages, my preference is to do that with just AWS S3 (https://docs.aws.amazon.com/AmazonS3/latest/userguide/Hostin...).

Congrats on learning to code! If you ever want to chat about backend stuff feel free to reach out to nick@napkin.io :)

Thanks for sharing the stack trace. If you could share your username here or with nick@napkin.io I will dig deeper to see what's going on there. Pandas never fails to both assist and haunt me :)