Codemaps: Understand Code, Before You Vibe It

by janpio • 9 months ago

315 116 9 months ago

I really think more people should give Windsurf a go. It's really good. I'm a senior engineer and do a mix of agentic and regular coding and I really think people are looking past Windsurf.

As the conversation shifted towards Cursor vs Claude code vs Codex people seem to have stopped mentioning it which is a shame.

Source: user for 12 months - not a shill.

Codemaps was a very pleasant surprise when it showed up.

sama0042• 9 months ago

I just tried out windsurf yesterday, The only thing I hate for now is that when there are changes and I accept one of them, then trying to accept the others gives an error saying the file was changed

lord_sudo• 9 months ago

I’m sorry to hear about that. What version are you on? Looking to fix / repro this asap

all22• 9 months ago

This is enough for me to give this a go. I've tried a few different tools; abacus.ai (and their IDE), claude CLI, crush-cli. My workflows are still mostly on the command line, and a little in VS Code. I haven't found a flow that works "right", yet.

swyx17• 9 months ago

first mention i've heard of abacus.ai and IDE. what do you think stands out about them?

you might struggle with Windsurf since you're so command line heavy. but pro tip - ask for command line work to be done inside of Windsurf's Cascade agent. they were first to the terminal-inside-aichat pattern and i really like how it's much better at command line stuff than i am (or can do the legwork to specify command line commands based on a few english descriptions)

dingnuts2• 9 months ago

I've used it, and I thought it was absolute trash. Goes crazy doing shit I don't want. I spend more time deleting crap I didn't want and reviewing and changing its code than I do just writing it myself.

I know what you're going to say: I need to learn to use this groundbreaking technology that is so easy to use that my product manager will soon be doing my job but also is too hard for me a senior engineer, to find value in.

Kindly: no, I trust my judgement, and the data backs me up.

Have you taken measurements of how many features and bugs you've shipped over the last twelve months or are you just like the engineers in the METR study who self reported an improvement but when measured, had been impaired? What evidence do you have that your attitude is not simply informed by the sunk cost of your subscription?

Please share your data below

Madmallard2• 9 months ago

Wholeheartedly agree.

Nothing my friends that heavily use AI for is groundbreaking at all. It's stuff they already entirely know how to do, describe in full detail what they want implemented, then double-check all of the results. I'm not convinced at all that they're doing architectural and long-term design thinking in this process. They're just "making the thing". I don't think they really care enough to do any of that hard thinking either. Not that they should be, considering the state of the industry and the lack of loyalty companies have to developers.

gslepak• 9 months ago

I prefer IDEs like Zed that don't lock me in to their ecosystem and force me to "log in" to use them.

gnarlouse6• 9 months ago

`codeium` which is now `windsurf` started out as a vscode fork IIRC

gnarlouse6• 9 months ago

Agreed. I'm back and forth about whether I want to spend the time with an agentic coding editor yet, because it's sitting right on the cusp of distraction/enhancement.

I've also tried the 3 C's, and it still feels like Windsurf has the net best user experience.

mpalmer2• 9 months ago

I co-sign this as a similarly-credentialed person. I use windsurf at work and recently started enjoying Claude Code, but the UX of Windsurf is actually a legit value add. Codemaps especially - been using them for weeks and they're excellent. Ask me again in a year maybe; churn in code could make maintaining codemaps annoying, but even that seems solvable.

swyx17• 9 months ago

appreciate the feedback! just a reminder codemaps are based on snapshots of your code when you run them; technically there's nothing to maintain, because you just rerun them if you need to.

corefinder2• 9 months ago

I'm surprised that people still aren't discussing Github Copilot baked into VSCode. I pair agent mode + Sonnet 4 + Sequential Thinking + Tavily MCP servers and it works wonders. I recently prototyped the first version of our SaaS with this setup in a minimal amount of time. Also worth nothing, the pricing is extremely reasonable. Free credits + pay per use. I frequently max out the free tier and have never spent more than $40 per month.

nake89• 9 months ago

I second this. As a previous AI skeptic. VS Code with Agent Mode is amazing. Perfect amount of control over what the AI is doing. For me its been a game-changer. I would describe it as letting the AI do lots of the work and me being the guiding hand.

I will say, it is extremely important to have a good AGENTS.md file and other .md files that the agent can refer to when it needs to. Also having tests is more important than ever.

And when you notice common hiccups, document it in the AGENTS.md.

CSMastermind2• 9 months ago

I as a big Windsurf advocate, miles ahead of Cursor IMO but I've fully switched to Codex these days. The cloud environments are just such a nice feature.

Still like Windsurf though their pricing is what drove me to not roll it out across my company.

swyx17• 9 months ago

cloud envs are good :)

1) did you compare codex cloud with devin?

2) how about the new claude code teleport feature from web to cli?

just wanted to pry for more opinions on what matters to you

swyx17• 9 months ago

(coauthor) happy to take any questions! see 1 min demo video here https://x.com/cognition/status/1985755284527010167

this is brainchild of cognition cto steven who doesn't like the spotlight but he deserves it for this one https://x.com/stevenkplus1/status/1985767277376241827

if you leave qtns here he'll see it

bluelightning2k4• 9 months ago

Less a question and more a strong suggestion: codemaps should be viewable in the main pain. The sidebar is FAR too small. Either default or a button or something to open it like an editor tab.

swyx17• 9 months ago

acked. you can also open it in a browser window for now https://windsurf.com/codemaps/9e2791c4-0b14-4757-b4be-a71488...

potamic• 9 months ago

Do you have any examples/demos for very large codebases? Because it's easy to get decent quality output using LLMs on small codebases. But large, messy codebases is where you really need help with understanding.

swyx17• 9 months ago

we dogfooded this with some v large customers i cant name before launch so .. yeah reasonably confident, but ofc no guarantees overall.

ChrisbyMe• 9 months ago

this is the right way to try and tackle this problem imo. too much focus in AI dev tooling has been on building "products" that only half work.

making codebases understandable to humans, and LLMs etc, is a better approach

self documenting, interpretable systems would actually solve a lot of dev churn in big companies

plus it's not like artifacts have to be limited to code once that's figured out

esafak• 9 months ago

I don't think it's a choice; I use both. Code understanding is especially useful in new code bases, but once that's over you need to get work done.

dennisy• 9 months ago

Looks an interesting enough feature to give Windsurf a try!

yunyu• 9 months ago

Great idea. I always end up having to tag the relevant files/abstractions anyways to avoid having the LLM produce duplicated slop, and something like this makes collecting this info much easier.

rmonvfer• 9 months ago

This looks awesome. I’m a very heavy Claude Code user (and Codex) in both the CLI and VS Code (and now in the web too!) and it’s quite infuriating when the agent just gets lost after context compaction and I have to point it to read CLAUDE/AGENTS.md (and update it if a lot of changes have been made)

I tried Windsurf a while back but I’ll definitely come back ASAP just to play with this and see how it does in a somewhat complex project I’m working on.

Kudos to the team!

asdev3• 9 months ago

A feature like this isn't useful because knowing what connects to what, dependencies, etc. means nothing without business context. AI will never know the why behind the architecture, it will only take it at face value. I think technical design docs which have some context and reading the code is more than enough. This sits in the middle ground where it lacks the context of a doc and is less detailed than the code.

CharlesW3• 9 months ago

AI will never know the why behind the architecture…

That's true only if you don't provide that context. The answer is: Do provide that context. My experience is that LLM output will be influenced and improved by the why's you provide.

dingnuts2• 9 months ago

it takes longer to explain the context to the model than it does to just write the code based on the context I already understand, especially since code is more terse than natural language

asdev3• 9 months ago

if you know that context, you don't need a codemap

swyx17• 9 months ago

you might be surprised how much business context leaks into a codebase and that's plenty to work on :)

https://deepwiki.com/search/vimfnfnname-lets-you-call-neov_e...

but also how much you kinda dont need it when you're just debugging code

https://windsurf.com/codemaps/87532afd-092d-401d-aa3f-0121c7...

asdev3• 9 months ago

agree that AI can kinda infer business context sometimes. in my experience, it doesn't work that well.

a lot of the time, debugging isn't a logic issue, but more of a state exploration issue. hence you need to add logging to see the inputs of what's going on, just seeing a control flow isn't super useful. maybe codemaps could simulate some inputs in a flow which would be super cool, but probably quite hard to do.

philippta• 9 months ago

To add to that, a lot of business context is stuck in people‘s heads. To reach the level of a human engineer, the coding agent would have to autonomously reach out and ask them directed questions.

nsonha• 9 months ago

because knowing what connects to what, dependencies, etc. means nothing without business context. AI will never know the why behind the architecture

since when has coding become so trivial that things are only useful if it helps with the "why" and "business context"?

Closi• 9 months ago

AI will never know the why behind the architecture, it will only take it at face value.

There is no reason to believe that at some point in the future AI will know the business context for apps when they are vibecoded (because the prompts should contain the business context).

alansaber• 9 months ago

I really like this kind of applied statistical data infrastructure approach, feels much more natural than just raw text + immediate HIL

swyx17• 9 months ago

huh? whats statistical about it? i hope i didnt give the wrong impression in the post.

but yes, keeping the human in the loop, in charge, on top of the code, is the way to prevent ai slop code

gnarlouse6• 9 months ago

A few things to point out after reading and thinking about this:

- Another AI firm building products focused on Fortune 500 scale problems. If you're not at a F500, this tool isn't necessarily a good fit for you, so YMMV.

- static analysis tools that produce flowcharts and diagrams like this have existed since antiquity, and I'm not seeing any new real innovation other than "letting the LLM produce it".

They say it's ZDR, so maybe I don't fully understand what problem they're trying to solve, but in general I don't see the value add for a system like this. Also onboarding isn't necessarily just presenting flow charts and diagrams: one of the biggest things you can do to onboard somebody is level-set and provide them with problem context. You COULD go into a 30 minute diatribe about how "this is the X service, which talks to the Y service, and ..." and cover a whiteboard in a sprawling design diagram, or you could just explain to them "this is the problem we're working on", using simple, compact analogies where/when applicable. If the codebase is primarily boilerplate patterns, like CRUD, MVC, or Router/Controller/Service/DB, why talk about them? Focus on the deviant patterns your team uses. Focus on the constraints your team faces, and how you take the unbeaten path to navigate those constraints.

_jayhack_• 9 months ago

static analysis tools that produce flowcharts and diagrams like this have existed since antiquity, and I'm not seeing any new real innovation other than "letting the LLM produce it".

Inherent limitation of static analysis-only visualization tools is lack of flexibility/judgement on what should and should not be surfaced in the final visualization.

The produced visualizations look like machine code themselves. Advantage of having LLMs produce code visualizations is the judgement/common sense on the resolution things should be presented at, so they are intuitive and useful.

gnarlouse6• 9 months ago

Although I haven't personally experienced the feeling of "produced visualizations looking like machine code", I can appreciate the argument you're making wrt judgment-based resolution scaling.

bigiain• 9 months ago

static analysis tools that produce flowcharts and diagrams like this have existed since antiquity

Today I am apparently on of xkcd's "Lucky 10,000".

Does anyone have any recommendations for such tools? Ideally open source, but that's not a hard requirement. (Although "Enterprise - if you have to ask the price you can't afford it" options will not work for me.)

I'm particularity interested tools that work with Python, Java, and Javascript (Angular flavoured Javascript, it if matters)?

gnarlouse6• 9 months ago

https://www.ensoftcorp.com/products/atlas is the Java/c oriented flavor I'm most familiar with. I've used them for Javascript before previously, although I'd have to do some digging to find the particular package I used. I am confident that you could find one with an npm/pypi search.

recursivecaveat• 9 months ago

It's been a few years, but I remember jetbrains IDEs had these, though I'm not totally sure it was built-in rather than a plugin. Personally I find automated diagrams are not that useful. You generally need to have some understanding of what's happening to know what to hide and collapse to tame the spaghetti. So it needs someone in the know to be explanatory, and as a tool it requires a similar amount of fiddling to just drawing some boxes in slideware.

seafisher• 9 months ago

I'm currently building one based on tree-sitter: https://github.com/CRJFisher/code-charter. It's still in development but will be released soon (this year). Will support Python, JS/TS, Rust to begin with others (like Java) to follow.

szjanikowski2• 9 months ago

We are building https://noesis.vision/ a similar tool to extract the system architecture from the source code according to the patterns. We are now in beta for .NET.

After working with the topic for multiple months I can tell you that introductions for new-joiners are not the only use case for this kind of extracted knowledge. Many ppl in the organizations need insights into the software structure as they either impact decisions shaping this structure (e.g. analysts) or depend on the decisions about this structure (e.g. testers, or support agents)

It's all the matter of giving access to reliable architecture knowledge structured by a consistent ontology. Garbage in / garbage out - the higher the knowledge quality, the better the output - both for human and agentic knowledge consumers.

swyx17• 9 months ago

one of my earliest blogposts at Netlify https://www.netlify.com/blog/2018/08/23/how-to-easily-visual...

jrochkind12• 9 months ago

Figuring out new codebases is definitely one of the most challenging and time-intensive things I have had to do in my jobs.

swyx17• 9 months ago

we obviously agree but one of the problems i had with onboarding as a key proposition is that onboarding "seems" like a onetime problem. i lacked the datapoints or anecdotes to convincingly pitch "onboarding = context switching", a more recurring problem as the size of your team and size of your codebase and size of your tenure grows, even if its technically the same codebase, you're "always" onboarding to wahtever it is you're working on or maintaining or putting out fires.

jrochkind12• 9 months ago

Depends on how big the codebase is and how many people work on it and how often you need to switch to an unfamiliar context, but yes agreed, it's context switching, and is a regular thing. It is part of the job of software engineering, learning piles of code you were previously unfamiliar with.

stevenkplus• 9 months ago

for people too lazy to download windsurf to try it, codemaps is also in deepwiki

example: https://deepwiki.com/search/how-do-react-hooks-work-under_7a...

this does a pretty good job of going in the weeds of how the useState hook works in react

apstls• 9 months ago

Sounds very cool.

I wanted to try this out, so I opened Windsurf for the first time in ages and clicked the "Upgrade Available" button, which sent me to: https://windsurf.com/editor/update-linux

  Did you install using apt or apt-get? If so...
  
  1. Update package lists
  
  sudo apt-get update
  
  2. Upgrade Windsurf
  
  sudo apt-get upgrade windsurf

Whle `apt-get upgrade windsurf` will technically upgrade Windsurf, instructing users to run a command that will attempt to upgrade all packages on their system is nuts when the command is provided in a context that strongly implies it will only upgrade Windsurf and has no warnings or footnotes to the contrary. Good thing I didn't ask Windsurf's agent to ugprade itself for me, I guess.

EDIT - I don't want to detract from the topic at hand, however - after upgrading (with `sudo apt-get install --only-upgrade windsurf` :)) and playing around a bit, the Codemaps feature indeed seems very nifty and worth checking out. Good job!

kps• 9 months ago

So `apt-get upgrade $PACKAGE` has ridiculous semantics that no one would expect, and the actual syntax for upgrading a package is in neither the man page nor the command help.

normie3000• 9 months ago

So `apt-get upgrade $PACKAGE` has ridiculous semantics that no one would expect

Especially not an LLM!

pxc• 9 months ago

Sure it is¹ (kinda):

  --no-upgrade
      Do not upgrade packages; when used in conjunction with install, no-upgrade will prevent packages on the command line from being upgraded if they are already installed. Configuration Item: APT::Get::Upgrade.

The canonical way to do the thing you want via apt-get is `apt-get install`. And if you read the man page from start to finish, it'd be clear to you... but it is tucked away there in the most obtuse, indirect, ungreppable way. :'D

That would be a great addendum to an EXAMPLES section! In the meantime, this is documented well and clearly in the tldr page for apt-get².

Fwiw, apt-get not only sucks, but has been known to suck for many, many years (more than a decade at least). Its interface sticks around because it's basically plumbing at this point. But you, as a user, should never use it (or `apt-cache` or `apt-*`, if you can avoid it.

Aptitude is preferable for a whole host of reasons, not least of which being that its upgrade commands have the semantics you'd intuitively expect³. They take packages as an optional list of positional args, and upgrade everything only if you don't pass any. (Aptitude also has a ton of other nice features and I highly recommend it.)

There's also an official new porcelain in APT itself, aptly called "apt". It preserves⁴ the semantics of apt-get's `upgrade` command, but its usage message actually matches that syntactically— hopefully it'll barf if you tell it `apt upgrade windsurf` or whatever.

But automation needs to rely on the ugly, old, disparate APT commands that have been around forever and can't really change. That probably goes, too, for things guides want you to copy and paste, or instructions handed over to LLMs.

(This is one reason that if you only learn to use APT from guides/tutorials whose primary concern is something other than documenting or teaching how to use Debian-based systems, you'll probably never learn to use the correct tools (the nicer, newer ones).)

1: https://manpages.debian.org/trixie/apt/apt-get.8.en.html

2: https://tldr.inbrowser.app/pages/linux/apt-get

3: https://manpages.debian.org/trixie/aptitude/aptitude.8.en.ht...

4: https://manpages.debian.org/trixie/apt/apt.8.en.html

GuB-422• 9 months ago

I have been using Debian for literally decades and I didn't even know "apt-get upgrade $PACKAGE" existed. It is weird, it doesn't appear in the documentation, it doesn't work with the "apt" command, it means it is probably a relic of the past left there for compatibility reasons and you probably shouldn't use it.

My guess is that someone or some LLM hallucinated this command, "apt-get upgrade" is for upgrading your system, not for upgrading a single package, and it takes no extra argument.

For upgrading a single package, just do "apt install $PACKAGE". It is the same command as for installing. The semantics is rather clear to me, upgrading is like installing the new version on top of the old version. It also makes no sense to install a package you already have or to upgrade a package you don't have, but if you want to be sure, for example because you don't know if you already have the package installed or not, there are the --no-upgrade and --only-upgrade options.

swyx17• 9 months ago

hiya! team noticed your comment and agreed - and it is fixed.

    - const CodeSnippetTwo = `sudo apt-get upgrade windsurf`;
    + const CodeSnippetTwo = `sudo apt-get install windsurf`;

blks• 9 months ago

Did you also generate this with “AI”?

Lvl999Noob• 9 months ago

Why not use apt?

zamadatix• 9 months ago

Wow, what's the upside to that syntax? I never would have guessed.

ashirviskas2• 9 months ago

So it is the same thing when I ask Claude to build me mermaid charts of code flows? So no point in this tool?

bigwheels• 9 months ago

This was my conclusion, too! Over time as agentic coders get better at handling higher-complexity tasks, this kind of bracing will become less and less necessary.

ravila4• 9 months ago

Btw, claude code is a lot better at graphviz than mermaid! I have been using it a lot for architecture designs.

ashirviskas2• 8 months ago

I just experimented and it seems like mermaid has much better support everywhere, including gitlab and github, graphviz seems to be mostly forgotten.

Are you sure it is better at graphviz?

kingjimmy• 9 months ago

Out of nowhere Cognition with a banging product. Probably not 100% yet but the idea is so good I'll be surprised if within 6 months all the other IDEs aren't copying.

swyx17• 9 months ago

hey we been shipping!!

https://cognition.ai/blog/swe-1-5

https://cognition.ai/blog/swe-grep

https://cognition.ai/blog/devin-agent-preview-sonnet-4-5

bluelightning2k4• 9 months ago

Actually another piece of feedback, the "waves" release notes/videos were pretty cool. Might want to consider bringing them back.

fHr• 9 months ago

this is amazing!

casey2• 9 months ago

Aggressively ironic for a pitch deck that has no respect for the readers critical thinking skills

charcircuit• 9 months ago

whereas people get into trouble when the code they generate and maintain starts to outstrip their ability to understand it.

If you are trying to understand code, then you are not vibe coding by definition.

Madmallard2• 9 months ago

Is this like an entire astro-turfing for AI thread? Can mods like ban these topics? This is clearly just like monetarily incentivized amoral behavior.

As far as I understand still working with this stuff regularly. None of the actual problems have yet to be solved at all. AI still produces garbage for anything complex. And if it doesn't, it's because you specified in full detail how it should do everything and heavily hand-held it and reviewed the results, taking more time than just doing it yourself.

And it's either that or they are flying by the seat of their pants with the thing and free-balling their way to a broken system.

pyrale• 9 months ago

So we're back to UML?

zknill• 9 months ago

I created a side project ~3 years ago based on a similar idea. It was before LLMs were a big thing, and AI could render the code relationships for you.

I started with go and java (the two languages I was using in my job) and built AST parsers that would extract the relationships between the classes and functions and draw them with graphviz. Then I created a filter syntax based on regex to filter the graphs.

I never followed through on the idea to a production ready version, but it helped massively as a personal tool when working on unfamiliar codebases.

The site is still here, but lots of it is probably broken by now..

https://packagemap.co

divan• 9 months ago

I did something similar but for non-classes based language (Go) and in 3D [1]

But I saw it as next step towards shifting programming from sitting and scanning texts into something more tangible, where developer has broad overview of software, and can work on smaller parts while seeing context of how these parts are connected. Ended up concluding that this stuff should work in VR.

[1] https://divan.dev/posts/visual_programming_go/

BinaryIgor• 9 months ago

Well, interesting idea, but can you trust that it generates it properly? Because if it doesn't, then your understanding of the code will be incorrect, even worse than lack of knowledge; and if you do need to check all the things it has generated for you, as a description - doesn't it defeat the purpose of the tool?

soco• 9 months ago

Arguably, if you vibe code you don't really care about the code, thus even less about the diagram. So you'll vibe something, get a diagram of something to show your boss, and you can move on.

tleyden5iwx2• 9 months ago

Yeah but there are a lot of vibe engineers, and we care about the code, because we have to own it

raumgeist• 9 months ago

I really like the idea of visualizing code in any other way than text and have given it some thought from time to time. However, I think the problem here can quickly become that you tend to fall in love with a bad idea. No one, and I don't mean that in bad faith, wants to look at these diagrams. Usually they do not communicate the meaning they intend to do, and I find that I have to spend some time understanding what exactly is meant by any type of box or arrow. What people might want to look at is their mental visualization of the code or math they are working on (or their LLM made for them). At least to me, that is much more tied to what the data will look like at runtime and how different parts from different data-structures will interact with each other. If you were to visualize a flutter app, and nowhere in that visualization the tree-like structure of the widget-tree would appear as such, that would collide with my mental model of how such an app functions. This visualization will be induced by reading the code, much like reading a novel will produce pictures in your head. I'm not sure LLMs are the technology that will produce code-movies you would rather watch.

shklqm• 9 months ago

I feel like understanding the codebase should be the default thing the AIs would need to do first. This sounds promising though.

tleyden5iwx2• 9 months ago

Feature request: add a Github Action so I can generate a codemap for my repo and throw it on my README. Then update it when major PRs change the codemap.

abdellah123• 9 months ago

This is extremely useful and helpful ! How great is this use case for AI !!!

tinyhouse• 9 months ago

Looks cool! I've been doing this a lot recently - a workflow I set up to create diagrams for code using the Claude cli and mermaid. It works pretty well but it's just a diagram - no links to the actual code. The latter is a neat addition that will likely get me to try codemaps.

Codemaps: Understand Code, Before You Vibe It

Related Stories