nichochar

yep, that's what i qualify as "not an advantage".

It's totally fine if thats what your company is like, but the labs have an unfair advantage.

This was often true when writing code manually to be fair.

You could get to "something that works" rather fast but it took a long time to 1) evaluate other options (maybe before, maybe after), 2) refine it, 3) test it and build confidence around it.

I think your point stands but no one really knows where. The next year or so is going to be everyone trying to figure that out (this is also why we hear a lot of "we need to reinvent github")

We ran some tests at mocha (we have a coding agent with our own harness to build web apps, with a lot of tools and medium length tasks (3min to 10min).

Our notes:

Sonnet 4.6 feels like a fundamentally different model than Sonnet 4.5, it is much closer to the Opus series in terms of agentic behavior and autonomy.

Autonomy - In our zero-shot app building experiments, Sonnet 4.6 ran up to 3-4x longer than Sonnet 4.5 without intervention, producing functional apps on par in terms of quality to the Opus series. Note that subjectively we found Opus 4.5 and 4.6 are better "designers" than Sonnet 4.6; producing more visually appealing apps from the same prompts.

Planning / Task Decomposition - We found Sonnet 4.6 is very good at decomposing tasks and staying on track during long-running trajectories. It's quite good at ensuring all of the requirements of an input prompt are accounted for, whereas we were often forced to goad sonnet 4.5 into decomposing tasks, Sonnet 4.6 does this naturally.

Exploration - In some of our complex "exploration" tasks (e.g. cloning/remixing an existing website), Sonnet 4.6 often performs on par or better than Opus 4.5 and 4.6. It generally takes longer, and takes more tokens, though we believe this is likely a consequence of our tool-calling setup.

Tool-use - Sonnet 4.6 seems eager to use tools; however, we did find that it struggles with our XML-based custom tool use format (perhaps exclusive to the format we use). We did not have a chance to assess with native tool use

Self-verification - Similar to Opus 4.5/4.6, Sonnet 4.6 has a proclivity for verifying it's work.

Prompting - We found Sonnet 4.6 is very sensitive to prompting around thinking, planning, and task decomposition. Our prompt built for sonnet 4.5 has a tendency to push sonnet 4.6 into incredibly long thinking and planning loops. Though we also found it requires significantly less careful and specific instructions for how to approach problems.

How are we thinking about this:

We can't launch this model day 0, it requires more changes to our harness, and we're working on them right now.

But it reminds me a bit of 3.5 to 3.7 --> It's a pretty different model that behaves and responds to instructions in new ways. So it requires more tuning before we can extract its full potential.

I think Karpathy[1] summarized why he thinks this is the case quite well (as described he was himself hyping it up a bit much, but there are some foundational reasons why it's a very interesting experiment).

[1] https://x.com/karpathy/status/2017442712388309406

We're building an AI powered app builder. We use elixir, phoenix and of course OBAN.

It feels like such a super power. What you're describing is particularly important in the era of long running AI processes. Something as simple as running a deploy creates pressure on your agent orchestration. But if everything is powered by OBAN you have wonderful ways to build durability.

By the way, it's all "free" and built-in.

In other language ecosystems, people literally pay for durability that has this shape (like temporal)

Little to no evidence was presented.

This is vibe argumenting.

The effort described in the article is maybe a couple hours of work.

I understand the "enjoy doing anyway" part and it resonates, but not using AI is simply less productive.

i knew simon would be top comment. it's not an empirical law

I'm building an app builder (getmocha.com) and one of my favorite use cases I have seen is "small private social network":

some people are building custom, tailored social networks only available to their family, church, community, sports team, school, etc...

This was previously impossible but now AI changes that. I don't know if it will materialize, but a more federally distributed web with tons of small private social networks could be a future of healthy social

Congrats on the launch, spreadsheets are getting lots of AI upgrades these days, exciting!

If i were to try this out with some somewhat sensitive company data, what is the security profile of this? Would it potentially leak the data to MCP servers? Do I have control?

Elixir and Phoenix are very underrated.

It combines the "opinionated" aspects of ruby and rails and the power of erlang. The BEAM is like no other runtime and is incredibly fun to work with and powerful once you get how to use genservers and supervision trees.

We use Elixir for Mocha, and my one issue with it (I disagree with OP on this) is that live-view is not better than React for writing consumer grade frontends. I wish Phoenix took a much stronger integration with React approach, that would finalize it as the top choice for a web stack.

I literally cannot read this article because of a paywall.

The title is deeply ironic.

yes same category

disclaimer: building a competitor (https://getmocha.com)

Lovable and bolt took a massive shortcut: they outsourced the backend to a third party (supabase).

This makes their ceiling to build "useful" software incredibly low.

The right approach takes a lot more time: pick an opinionated framework (think ror) and build up a full stack app builder from the ground up.

Took us months and months of work to get it working, but now people _can_ build "useful" software (thats our bar)

Isn't creativity always impressive if done well?

Code is a medium, painting is a medium, piano is a medium, and prompting is a medium.

This sounds a little bitter.

This was super interesting.

Unsurprisingly, I see he didn't have much to say about faceit and esea.

I think CSGO anti-cheats are a league above the rest (I'm not sure why, maybe because the scene is more competitive?)

I promise you they're negative on unit economics.

In addition to what you're saying: - hosting costs - live sandboxes costs

They're betting on LLM costs going way down and VC funded until then

This analysis is very much on point. I'm building a product in this space (https://getmocha.com), and can share a few more insider insights:

- The churn from companies like lovable is indeed very high, and user frustration is high also.

- There are sub-niches available. Building internal tools is not the same as landing pages is not the same as saas. In the previous website builder market, different players (webflow, squarespace, wix) found and dominated a sub-niche

- The market is way bigger than anyone realizes. Today hundreds of millions for basically early adopters and highly tech-savvy users. This tech can and will go mainstream

- A huge issue with lovable, solved by others like mocha and replit, is the app backend. Lovable took a shortcut and partnered with supabase but that deal will not last. Supabase is losing big on their free tier (had to raise 200M to support it) and both want to capture the margin from the customer. There will be a reckoning.

Even if what you said was true, it will be false within months or years.

What then?

This is the whole premise of the article. Just extrapolate and imagine that it can think and write poetry better than you (it will, and likely soon), what then?

It's a very important question. A cultural one.

Reasoning:

.com is the best TLD by a long shot but it's really saturated, so as a startup you have no chance.

As you say, the hope is to make it and be able to buy the X in getX.com where hopefully you've checked that X belongs to a squatter and not an existing company (they're both bad the latter is worse).

We're building our startup infra on cloudflare over the other major hyperscalers and it turned out to be an amazing decision...

Generous free tiers, pricing scales very competitively after that, and their interface is not nearly as bad as GCP / AWS.

I highly recommend this stack.

Yeah I don't agree. I'm building a product in the space, and the number one problem is correctness, not latency.

People are very happy to sit there for minutes if the correctness is high and the quality is high. It's still 100x or 1000x faster than finding 3rd party developers to work for you.

I wish the models were getting better but recently they've felt very stuck and this is it, so agent architectures will be the answer in the short term. That's what's working for us at srcbook rn.

try the thing i'm building, it will build a website for you from a simple prompt: https://srcbook.com

I am building https://srcbook.com which is in this category but focused on webapps.

It's unreal what the AI can do tbh.

I have a masters degree in math/physics, and 10+ years of being a SWE in strong tech companies. I have come to rely on these models (Claude > oai tho) daily.

It is insane how helpful it is, it can answer some questions at phd level, most questions at a basic level. It can write code better than most devs I know when prompted correctly...

I'm not saying its AGI, but diminishing it to a simple "chat bot" seems foolish to me. It's at least worth studying, and we should be happy they care rather than just ship it?

As someone building a client which needs to sync with a local filesystem (repo) and database, I cannot emphasize how wonderful it is that there is a push to standardize. We're going to implement this for https://srcbook.com

Notebooks are hot these days! We also shipped our own version of a TypeScript notebook[1] but it takes quite different sides of the tradeoff: we want to run backend node code, so unlike this or observable we're not looking to run in the browser environment. Still, for many applications, this idea is a better take!

Kudos to the author.

https://github.com/srcbookdev/srcbook

Tailwind is amazing for LLMs. You can't beat it:

- concise

- inline with the rest of the code

I am willing to bet it's going to become a standard because of its existing popularity + the insane tailwinds that codegen give it.

We love feedback, let me know what's missing / what you wish was better. We have a discord if that's easier, or just email me at nicholas <at> srcbook <dot> com

Same category. Differences that I can spot (I'm not very familiar with Marblism):

Currently, Srcbook is:

- open source

- local

- focused on a different stack

- also offers a notebook product