Architecture.md

https://matklad.github.io//2021/02/06/ARCHITECTURE.md.html

by todsacerdoti • 5 years ago

1349 153 5 years ago

matklad.github.io

mholt2• 5 years ago

Ah, we actually have one of these at Caddy: https://caddyserver.com/docs/architecture

(The filename on disk is literally "architecture.md" -- it is a Markdown file rendered by Caddy's template handler: https://github.com/caddyserver/website/blob/master/src/docs/...)

It could use some improvement, but it's been really great for helping people learn how Caddy 2 works at a high level. Beyond our docs, I always encourage new contributors to thoroughly explore the godoc and code: it's very well-commented and organized, especially once you know how it all comes together. A single document will never be sufficient. But it can help you map between concepts and code.

Code search is also invaluable for this; I recommend Sourcegraph: https://sourcegraph.com/github.com/caddyserver/caddy

Edit: One other valuable piece is explaining why the architecture is the way it is. Our architecture.md doc links to a video that explains how I arrived at Caddy 2's architecture (and why it's not arbitrary): https://www.youtube.com/watch?v=EhJO8giOqQs

Spearchucker3• 5 years ago

Thanks for putting that out there. It's super interesting to see what we consider architecture to be. Your approach appears operational, focused on how the completed system functions, i.e. getting a new team member up to speed on the codebase.

Typically I try to start with tiers (1, 2, n-tiers...?) that show how the system might be deployed. I then list list layers (user/facade/business/data), interfaces between these layers, and components within each layer. I do that for each tier.

After that maybe something about quality objectives and how they might be met, eg: availability (MTTF / (MTTF + MTTR) * 100), efficiency, flexibility, integrity, interop, usability and so on.

This leads to a physical delpoyment model, which shows layers deployed to tiers. And yes, I'm rather fond of Visio.

Then a bit about approach (dev/deployment and operational management) risk and stakeholder management, technical reviews (change control board maybe?), and project reviews.

To be fair I come from a predominantly critical systems world (telecoms OSS and BSS, healthcare, transport and some fintech. And in that world architecture is very far removed from actual code until eventually.

kortilla2• 5 years ago

How do you distill down critical outcomes of the architecture for people considering using your project? Based on my experience so far, engineers will look at a giant document, see phrases like “stakeholder management” and nope the fuck out.

What I want to know is, what are the key performance considerations, failure modes, recovery procedures, etc.

tejohnso2• 5 years ago

Your approach seems like it would reduce the likelihood of implementation-time surprises. It also seems like different sections would satisfy different audiences. Do you have any book recommendations?

the_arun• 5 years ago

Off topic - what is the theme used in Caddy doc site? looks super cool.

mholt2• 5 years ago

Matt's Custom Theme? :) I dunno, I haven't named it. Glad you like it!

ibraheemdev• 5 years ago

The specific rust-analyzer architecture document the author refers to was discussed here a couple days ago: https://news.ycombinator.com/item?id=26026309

_hl_• 5 years ago

I think it's funny how the blog post is essentially an architecture documentation of the rust analyzer's architecture documentation.

wiremine• 5 years ago

One of the lessons I’ve learned is that the biggest difference between an occasional contributor and a core developer lies in the knowledge about the physical architecture of the project. Roughly, it takes 2x more time to write a patch if you are unfamiliar with the project, but it takes 10x more time to figure out where you should change the code.

This feels about right to me. Not sure a single doc will help solve that, but even if it cuts the time from 10x to 7x or 5x, it feels worth it.

DyslexicAtheist• 5 years ago

what they suggest is very similar to Architecture Decision Records (ADR's). https://adr.github.io/

TL;DR: ADR's are a design choice for a lightweight process to store and manage the history over what architecture decisions have been made in the past and why. They should be tracked within git so that the history of decisions and how these evolved is provided for free. Just track all this within an `adr/` subdirectory at the root of each project.

"Communicating and documenting architectural decisions" - David Ayers LeadDevNewYork(2019): https://www.youtube.com/watch?v=rwfXkSjFhzc

NicoJuicy• 5 years ago

Indeed, I was thinking about ADR's too

orthoxerox• 5 years ago

ADRs are more about the "why" (and are absolutely indispensable in any long-running implementation project). Architecture.md is mainly about the "how".

ablekh2• 5 years ago

I believe that an optimal architectural document should cover both "why" and "how". Having said that, back when I was writing and modifying such documents at a CMM Level 3 division of a large and well-known tech company (using the waterfall SDLC process!), relevant information was split between high-level and low-level design documents. I found it quite inconvenient and think that having relevant sections (with cross-referenced info) within the same document makes so much more sense.

villasv• 5 years ago

I thought of ADRs too, but it seems the objective here is to be a bit more high level. It doesn’t help that the term “architecture” is so wide in meaning.

I’ve adopted ADRs and it has been good for decision making, but it doesn’t help much new contributors to find exactly what module they need to focus on for a task.

crabmusket2• 5 years ago

It seems like ADRs are the deltas of architecture.md: they describe how it has changed over time. It's useful to have one document which just describes things as they are, but it's also great to preserve the history of how things came to be that way.

harikb2• 5 years ago

Any word on supporting diagrams inside GitHub flavored markdown?

monocasa• 5 years ago

I always just use ASCII art in code blocks.

_joel• 5 years ago

Same here, simple works. I use this tool to help http://asciiflow.com/

arminiusreturns• 5 years ago

I tend to prefer diagrams as code, so yoi can just embed an image that be updated, version controlled, etc, instead of ascii diagrams. Any reason in particular you want ascii instead of images?

harikb2• 5 years ago

Can you describe what 'diagrams as code' mean?

I don't mind images, just wondering if Github would make it easier to generate those. Something like the syntax of web-sequence/uml but not limited to data-flows and something more simpler, ideally.

ausjke• 5 years ago

pure text is easy to work with most of the time, in fact ascii diagram is used heaviyl in RFCs

jtr1• 5 years ago

Looks like there is an open feature request on GitHub for supporting Mermaid charts: https://github.community/t/feature-request-support-mermaid-m...

jhund• 5 years ago

I've started using draw.io [1] for diagrams I want to embed in markdown files. The cool feature about draw.io is that it can embed the data structure describing the diagram in a png file. So you get a PNG file that is both source code for your diagram (so you can edit it later), as well as the presentation (you can embed png files and they will render fine in Github hosted md files).

Then I use the markdown-images package [2] for Sublime Text, and I can see those png images in my markdown files in the editor as well.

Benefits of this approach:

* Powerful diagram editor, free to use.

* Editable diagram and embedable image in one file.

* Diagrams rendered in text editor.

[1] https://github.com/jgraph/drawio-desktop

[2] https://packagecontrol.io/packages/Markdown%20Images

corysama• 5 years ago

You could publish "html" that is actually http://casual-effects.com/markdeep/

pronoiac• 5 years ago

The method I've considered: add a separate .mermaid file, render it as a png or svg file, and put that in the markdown file.

amw-zero3• 5 years ago

How do you keep this up to date though? That's the biggest problem with documentation. Having some kind of append-only format, like ADR, can help, since the documentation specifically is tied to a decision at a single point in time.

Still, those can drift from the actual implementation to the point where they are both misleading and confusing. Such is the entropic nature of software.

matklad10• 5 years ago

Revise it twice a year. If the document gets stale faster than that, just delete the stale bits: they are probably too low level for this kind of documentation.

simonw• 5 years ago

I find filing issues whenever I find out-of-date docs helps. They tend to get fixed pretty quickly, because they offer a nice change of pace from working on bugs or features.

amw-zero3• 5 years ago

Oh, so the solution is to just add another thing to “remember” to do. That must be what I’ve been missing all these years.

Yes, that’s extra thick sarcasm.

Uehreka• 5 years ago

That’s a fair point, although given how “high level” this is supposed to be, I’d imagine that needing to update it a lot might actually be a canary that the project in question is in a lot of flux.

quantum_state• 5 years ago

Great advice!

vergessenmir• 5 years ago

The k8s project did this very well in the early days with their design docs. It made the codebase much more accessible than it ordinarily should have been. There are few projects that do this and I wish the JavaScript projects did this a lot more.

If anyone knows of other open source projects that do this, it would be particularly useful.

londons_explore• 5 years ago

I would encourage people to have one ARCHITECTURE file per directory of source code files. Don't duplicate documentation in these files - if there is a well documented header file for some module, just link to that.

In a big project, when I'm hunting for code that does something and I have no familiarity of the codebase, I want to be able to follow a chain of ARCHITECTURE documents from the root of the project to the implementation of the feature I'm looking for.

If your documentation is poor enough that I resort to finding a string used in the feature and grepping the whole source tree for it, then your codebase will be tricky for someone new to get started on.

kondu• 5 years ago

That sounds a bit excessive. If your project is going through a lot of changes and refactoring, the multiple architecture.md files will be hard to keep up-to-date

andrewflnr• 5 years ago

Disagree. If I have a question about the architecture, I don't want to go hunting through your directory hierarchy for the right doc file if I don't have to. Not saying you shouldn't have some kind of design doc in each directory, but I'd say it's more important to have a single roadmap than lots of little interlinked ones. If links were enough, I would just read the code.

tcbasche• 5 years ago

I can almost guarantee either the docs won't get updated, or new source code directories will get created without the ARCHITECTURE file.

At least having a single file means only one file potentially goes out of date, instead of dozens

ChrisMarshallNY• 5 years ago

I like the idea. I may do it for some (not all) of my projects.

exabrial3• 5 years ago

So one thing we've done is write all of our applications the exact same way with well defined terms (on a wiki) and a commitment to no more than 4 layers.

* Initiators (things that receive, decode, and validate input)

* Controllers (Business logic containers. One function refers to one business action)

* Services (Used by controllers to effectuate commands. Services absolutely cannot call other services)

* Cross Cutting concerns (Common model objects, logging, top level error handling, etc)

This allows even a new person to pick up a project and orient themselves immediately.

kenniskrag2• 5 years ago

Why shouldn't a service call other services? One Service may extend the features e.g. network < encryption < http.

Ma8ee• 5 years ago

I don’t think the comment implies that a service can’t be layered.

exabrial3• 5 years ago

Three reasons:

First: we found that it creates a spider web of dependencies if we din't have this rule. Instead, if you return control to the controller after performing a discrete action, it makes sure that business logic stays out of services and keeps service functions short and directed.

Second: it makes it easy to keep the entire design in your head.

Third: It promotes composition. This leads to easier testability with mocks rather than having to resort to full blown integration tests for even the smallest things. (We still do integration tests, but mock tests can be churned out in volume and are less fragile).

closed2• 5 years ago

I love architecture docs, but find they're often written using a funny process:

  1. Spend a long time writing the doc.
  2. Wait for a person to chance upon it.
  3. Hope you anticipated their questions.

It seems like the most important thing a person can do is reverse this:

  1. Say who the doc is for.
  2. Find that person. Ask them to try a lil contribution.
  3. Frantically write / revise the doc.

IMO it's a lot like creating a presentation. The earlier the feedback the better!

gjvr• 5 years ago

Yep, feedback is valuable. And the earlier the more valuable (it's like NPV...).

I love coding so much, and find it really hard to express ideas in natural language, so that in the end documentation... doesn't happen as much as it should.

What I find that really helps is the following:

1. Write down architecture specs (with interface specs etc), before coding. Not bloated, but really minimalistic. 2. Review these ideas with peers. 3. Happy coding and refine the docs.

steveklabnik• 5 years ago

I like this idea a lot, but you will cause a lot of people to bounce at step 2. Or at least, that has been my experience over the years. No matter how much you reassure them that it is okay if stuff is confusing and in fact you'd like to know about it so you can fix it, they'll say "great" and then go radio silent 99% of the time.

ralmeida• 5 years ago

Sounds right to me too. One quicker way to improve the 'first process' is to change only step one - do not spend a long time, but instead write a few paragraphs with what's most important and/or top-of-mind. Often, this opens the door to more contributions and questions.

Of course, update accordingly whenever you find yourself in a discussion about something with a contributor (no matter if the architecture doc is even part of the discussion or not).

dognotdog• 5 years ago

I feel like for any long-running project, that person is at least ME. If I haven't written down some architectural information for complex projects, when I revisit a project after it being dormant for half year, I need to poke around to figure things out again.

If I have written down architecture notes in the first place, they are very helpful at this point; and if I haven't, it's a good time to start because I'll be acutely aware of the non-obvious parts as I re-familiarize myself with the code.

closed2• 5 years ago

I usually reach for a friend, or someone I've met before, since using the first version of a doc is asking a lot! (And they're often part of the target audience).

j1elo• 5 years ago

A question can be seen as a bug reported against the documentation...

Apart from answering them, I end up converting 15% - 20% of questions into some revised content of my docs.

ryanSrich• 5 years ago

Most documentation follows that first path. With a step 4. that is basically "only update this when something is broken, or when we're hiring someone new".

Doing documentation well, especially if you're working on brand new tech, is very frustrating and difficult. You're often moving too fast to find the time to retroactively update documentation, and you're right back into the viscous cycle of it constantly being out of date. I don't know what the solution is.

rapnie• 5 years ago

This is also the idea behind Readme Driven Development. See https://news.ycombinator.com/item?id=25222601

kstenerud• 5 years ago

I've started doing this in my larger projects e.g. https://github.com/kstenerud/go-concise-encoding/blob/master...

An architecture document should be the code equivalent of a combined street map and tourist guide. Its purpose is to bring strangers up to a minimum level of familiarity with the code as quickly as possible. That includes where things are, why it was architected this way, things to look out for, and a few interesting points of weirdness perhaps.

syoc• 5 years ago

I really like this DESIGN.md[1]. Might not be as code specific as what tfa is talking. I find design decisions really helpful when evaluating if a tool or solution is useful for my needs. Especially if it elaborates on priorities made in the project.

[1] https://github.com/google/stenographer/blob/master/DESIGN.md

gorgoiler• 5 years ago

This is predictably unrelated but how can someone go to the effort of having such beautiful typeface selection and still have an unhyphenated ragged right?

Great advice though, otherwise.

matklad10• 5 years ago

Yeah, I indeed spend some effort to steal the fonts from asciidoctor and https://www.teamten.com/lawrence/programming/ (highly recommend both).

And yes, I myself am struggling with ragged right. I really wish to have proper text hyphenation and justification, to have a book-like feel. Sadly, justification without hyphenation looks ugly, and `hyphens: auto` doesn’t work well (and wasn’t supported in chrome last time I checked).

I am pretty baffled that such basic (in terms of impact, not in terms of complexity of implementation) feature isn’t widely available.

Although I am not a web designer, so I might be missing some simple way to solve this.

alderz• 5 years ago

I came to the comment section to compliment the choice of Garamond.

The web would look much more beautiful with proper justification. It's a pity technology is not there yet. TeX solved the problem for DVI/PDFs long ago, so maybe we should start blogging on PDFs!

matklad10• 5 years ago

Actually, chrome added support for `hypthen: auto` not so long ago!

bezout• 5 years ago

Absolutely, I was just thinking about adding a similar document to my new project. Apart from contributors, it also benefits casual visitors who just want to browse around.

utborin• 5 years ago

As someone who is in week two of spooling up on a multi-million-line codebase where most of the original authors have moved on to other projects, please, I beg you to heed this advice. I spend the vast majority of my time figuring out where a change needs to happen. The patches themselves are no more than 10% of the work.

(The other significant factor is running tests.)

Aeolun• 5 years ago

Well, at least it sounds like you have tests :)

danaliv2• 5 years ago

Good point. :)

fukmbas• 5 years ago

It's called job security bub. I'm not going to write out everything you need to know. Hire an expert and whatever time it takes him to figure out is how much it's going to cost you to churn through employees. Don't like it? Be a better employer so your employees don't leave.

yjftsjthsd-h• 5 years ago

Lovely - now when you get in a car accident, your teammates get to be upset by losing you and have to reverse engineer the mess you left. Hope you don't plan to ever take vacations either - you're too important to leave.

danaliv2• 5 years ago

I’m not “bub” to you, or to anyone else. I’m also not an employer, but if I were, I would absolutely not hire anyone with your attitude.

notapenny• 5 years ago

I suppose you're being sarcastic but I certainly have met people with this attitude. Sometimes its not explicit or intentional either, just the lack of time/importance from the business side of things that creates a situation where things are poorly documented/understood and it creates a dependency on certain people/companies. Ultimately I feel its up to us developers to stress the importance of investing time on documentation, but also be mature enough ourselves to realise that this is part of the job.

notafraudster• 5 years ago

Even on relatively small projects (1k-10k LOC), I incorporate this, broadly, into my README file for my repo: what are the various use cases, which components do what, in one sentence what are the main functions contained in each of the source files, sometimes a flow diagram showing which functions call which other functions.

alixanderwang• 5 years ago

I think diagrams are usually quite helpful in accompanying an Architecture.md doc, especially if it gets complex.

I work on a diagram maker that syncs with a Github repo. So whenever you make changes to this architecture diagram, it'll push changes to the repo, with screenshots directly in the README (turning the repo into a diagram presentation).

This also allows people to colocate the diagrams alongside the code or docs by including the synced repo as a submodule.

https://github.com/terrastruct-bot/Demo

Terretta• 5 years ago

An alternative is to use an embedded diagramming syntax like dot or mermaid such that the diagrams are described as version controlled text and optionally rendered inline by e.g. VSCode. Cleaner than a litter of side car images in which it may be unclear what changed.

See Markdown Preview Enhanced: https://marketplace.visualstudio.com/items?itemName=shd101wy...

Or add Mermaid to the built-in markdown preview: https://marketplace.visualstudio.com/items?itemName=bierner....

You can also edit and enjoy these markdown pages with diagrams on Mac, iPad, iPhone: https://www.mweb.im/

Lastly, if anyone is not sure what to diagram for readers of architecture.md, consider the C4 model:

https://c4model.com/

Gehinnn• 5 years ago

Text based diagrams definitively have their advantages over wysiwyg editors. However, sometimes the layout of a diagram also tells a story and I find it hard to express that with available text based diagram solutions.

That's what I use draw.io aka diagrams.net for. I made an extension that helps to create markdown-embeddable/versionable draw.io diagrams in vscode: https://github.com/hediet/vscode-drawio (there is also an IntelliJ extension)

I have to admit that I sometimes struggle with pixel perfect layouts though and rearranging nodes is time intensive.

However, most text based systems don't have a nice github integration and rely on manual export, which you can avoid when you diagram is not a plain text file but a rich diagram editor.

matheusmoreira• 5 years ago

This is so useful. Even a simple description of the source code tree helps a lot. Otherwise people will have to find entry point files and recursively search the entire repository until they find what they're looking for.

baby7• 5 years ago

I have a similar advice, but I will go one step further: add README.md to other folders as well. It is dope to have a map of your whole system in an Architecture.md (or a README if it's not too long), but it's even more dope to be able to click through it and have submaps of how other components are structured.

Displaying the folder/file structure and explaining what is what is a must. An example from Diem[1]:

  consensus
  ├── src
  │   ├── block_storage          # In-memory storage of blocks and related data structures
  │   ├── consensusdb            # Database interaction to persist consensus data for safety and liveness
  │   ├── liveness               # RoundState, proposer, and other liveness related code
  │   └── test_utils             # Mock implementations that are used for testing only
  └── consensus-types            # Consensus data types (i.e. quorum certificates)
  └── safety-rules               # Safety (voting) rules

I recently digged into dependabot's code, and I found it extremely well structured. For example you have an Architecture section in the first README[2] with a diagram (how awesome is that!) and with links to the README of all the sub components, which themselves live in the subfolders[3].

What I dread the most is going through a new codebase and not seeing any documentation in internal packages. Like how the fuck am I supposed to understand anything in there? By reading all the code?

[1] https://github.com/diem/diem/tree/master/consensus#how-is-th...

[2] https://github.com/dependabot/dependabot-core#architecture

[3] https://github.com/dependabot/dependabot-core/blob/main/comm...

xyzzy_plugh• 5 years ago

Like how the fuck am I supposed to understand anything in there? By reading all the code?

This is one of the superpowers of Go: for most Go projects, this is exactly what I'd do. Just read the code. It's easy to follow, it's all formatted the same, very little implicit behavior, and I don't need an IDE to do it.

Few languages were designed to be read by others. Thankfully Go is one of them.

systemvoltage2• 5 years ago

I am afraid, this doesn't quite cut the mustard. Code just can't replace a human readable Architecture diagram + explanations of whys of choices and hows of the system. Programming language doesn't replace this no matter how clear and modular it is. While Go is nice, you're vastly under-appreciating architectural documentation.

mssundaram2• 5 years ago

very little implicit behavior

Actually I find it quite implicit the way Go structures packages. You definitely need an IDE (I use neovim + coc.nvim) to jump to a definition unless you want to grep the folder for where a struct is defined...

baby7• 5 years ago

I agree that Golang has the easiest code to read, and jumping into a Golang project (even massive) is always a happy experience.

But still, you can't replace documentation.

michael_j_ward• 5 years ago

When I first started programming, I thought the description next to the folder / file name on github was actually describing the item - as you did above, and not just the message from the last commit that altered the file.

Many years later, I still believe that's how it should be.

baby7• 5 years ago

Agree! I gave that feedback many times, who cares about commit message? I want documentation here!

I think Github should really start taking a stance on convention, and come up with project structure conventions that would help project navigation. For example, youtube does that with chapters in videos, if you write a description with timestamps it will display these chapters in the timeline of the video[1].

IMO Github should do the same if you have a README in your folder, describing the folder with a certain syntax, then it should extract this info and show it next to folders.

If anyone at Github is reading this, pretty please :D?

[1] https://www.youtube.com/watch?v=XfP862hCrDM

jlg232• 5 years ago

This will help people browsing code in github, but folks browsing code in their IDE will most likely be lost, as the documentation you advice to put into "README.md"s should, IMHO, be in the module/package documentation in the source.

One can still link to those from the global README.md

snicker7• 5 years ago

READMEs and API docs aren't mutually exclusive. But they cover different scopes. READMEs are more suited for high-level documentation, focusing on the /why/ instead of the /how/.

baby7• 5 years ago

I'm not sure I understand why someone in their IDE would be lost, you can't read README.md and see the file structure in your IDE?

Documentation in the source is different from a directory organization documentation IMO. Rust has a lot of tools to write good documentation in source, and it is here to help produce good documentation for users of a library, not to explain how to contribute to the code or understand how everything is architected.

jzer0cool• 5 years ago

Any well known tools which print above like folder path trees command-line?

pbkhrv• 5 years ago

On Linux there is a nice little utility called “tree” https://packages.debian.org/stable/tree Not sure about Mac or Windows.

house9-2• 5 years ago

tree

brew tree (if you are on Mac)

avodonosov• 5 years ago

I also use such hierarchical style, with lower level REAODMEs refferring iddividual source files. So that user continues from READMEs to code comments.

Example: https://github.com/cl-test-grid/cl-test-grid#system-architec...

On every level I first explain the task solved by this module and then go into the implementation details. Knowing the purpose first helps reader to understand impl details.

azangru• 5 years ago

If you maintain an open-source project in the range of 10k-200k lines of code

What happens past the 200k mark?

qbasic_forever• 5 years ago

You can take an approach like the Linux kernel where _one_ person is the sole arbiter and approver of all changes. This person is tasked with having complete, 100% depth and breadth knowledge of the entire system. It is their responsibility to ensure changes adhere to the goals of the system. It is of course an enormous bottleneck, and a big risk for management (the old 'what if they get hit by a bus, what do we do then?' concern).

Realistically, past the 200k lines of code point you aren't dealing with a codebase anymore; you're an _organization_. You need knowledge management--where do architecture decisions live, how are they approved, how are they taught to new developers, how are they updated as maintainers come and go, etc. It takes strong engineering management and leadership to keep it together.

matklad10• 5 years ago

It makes sense to invest into more thorough multi page doc, and have people/process in place to maintain it. rustc guide would be a good example: https://rustc-dev-guide.rust-lang.org/about-this-guide.html

Xunjin• 5 years ago

I read the title at HN and I was like "okay seems interesting" then I saw "matklad" and went "Holy shit, must be great stuff".

I know I will sound another Rust evangelist, but people this person is the main maintainer of RA (Rust-analyzer), a LSP protocol implementation, anyone who tried RLS (Rust Language Server) then RA knows how great this tool helps you at learning and developing stuff with Rust. I use the nightly version (which updates everyday) and ohh boy... Never had the "opportunity" to caught a nasty bug or anything.

Architecture.md

Related Stories