Ask HN: How do people keep track of organizational knowledge?

https://news.ycombinator.com/item?id=48996526

by kadhirvelm • 1 day ago

26 22 1 day ago

news.ycombinator.com

With code coming in faster and faster, we've been losing track of context, what's out of date, the source of truth, etc. Also with individuals spending less and less time on any one problem, finding an expert to answer questions has become challenging. People don't have that single domain of expertise like we used to. There's too much surface area to cover.

Are we the only ones dealing with this? How's everyone else handling it?

toomuchtodo2• 1 day ago

Confluence, Notion, Wikis, etc.

kadhirvelm2• 1 day ago

Do you all rely on people writing things down as they go? or do you have some automated system?

toomuchtodo2• 1 day ago

People. Time is carved out for both documentation and its review, like a PR review. Otherwise, how would you trust what is being documented? A human must remain in the loop if accuracy is a component of success criteria for organization documentation. I say this familiar with Karpathy's LLM Wiki pattern.

1337h4xx• 1 day ago

Just do what AI does to manage context: write docs, regression tests, content index, tools/skills, and CLAUDE.md

midzer• 1 day ago

I organize myself in a some_project/ directory.

Tadace• 1 day ago

We still make sure to maintain a healthy directory and "backlinks". And as soon as new updates are in, we have someone on the team dedicated to clarifying changes and updating the Wiki - Confluence pages, Notion, which are adequately linked to tickets and shared drives where necessary.

This way, new and updated context is not lost on the next user.

andyjohnson03• 1 day ago

How do people keep track of organizational knowledge?

In my experience:

1. Well meaning people try to build wikis/confluences/whatever that turn into backwaters of slowly rotting stuff

2. Beyond a certain extent that they can hold in their heads, most other people give up.

3. Some people hoard knowledge and use it for personal advantage.

I dont know what the solution is. Maybe an ai that reads every chat and email and commit - and builds some kind of browsable, queryable, executable knowledge trove.

kadhirvelm2• 1 day ago

Do you think having an AI do that is 1) possible and 2) people will like that? Seems really invasive, no?

al_borland4• 24 hours ago

Going into all chats is a horrible idea. Some chats between small groups are meant to stay that way.

andyjohnson03• 14 hours ago

Its possible - teams chats where I work can have ai transcription enabled, for example.

As to whether people like it: I was thinking of a work context, where expectations of privacy are lower. I guess it could be an opt-in for group chats.

al_borland4• 24 hours ago

I’ve started hoarding knowledge, but not for personal advantage. There were two trends that led to it.

1. People would go in and “clean up” what they thought was a rotting Confluence. They were new-ish to the company and didn’t have the knowledge or experience to know what they were looking at. Rather than ask, they just deleted stuff. Some of those things they deleted were docs I wrote, several over 20 pages long that I referenced regularly… gone.

2. Constant re-orgs and tool changes meant it was becoming difficult to know where anything goes. Not long ago I had 4 bosses in 4 years. I got tired of starting over and shuffling things around.

If I keep what I need, no one else is going to delete it without consulting me, and I don’t have to try and migrate and re-organize it every time I get a new boss or management decides we’re using a new system.

I guess a third thing, which is less of a trend, is friction. The collaboration platforms are much higher friction than something like a local Obsidian vault.

I’ve probably written and shared more documentation than everyone else on my team combined, but at some point, I got tired of the uphill battle. I still do write and share stuff I think will benefit the team, but I no longer try and make everything public for the reasons listed. If someone asks for something I have, I will always send it over, and having my notes in markdown now makes that easy.

lyfeninja• 1 day ago

Isn't that the purpose of wikis? Not saying they work all that great but I think that's the goal.

I do like the idea of having a live editable wiki as the knowledge base a LLM can reference.

kjellsbells• 1 day ago

Two things.

1. If it's not written down, it does not exist. This needs to be a ruthlessly enforced rule. I know we all love those Mel the Programmer stories, but that's a story about fragility. Similarly if you are relying on SRE or dev heroics to solve field problems, you have a gun to your head and you need to solve it yesterday.

2. The TCO of knowledge is dominated by the maintenance cost, not the creation cost. I see this mistake made a lot. Anyone can create a confluence page, Slack comment, SharePoint site or a git comment. The org must make it someone's job to grow, weed, and maintain these gardens. It does not matter what system you use, but someone needs to look after it.

You get a bonus for building information "data structures" that make it easy for people to pull data out as your org changes, but this is probably something you should accept that you'll get wrong as your org grows. AI hides this problem because it makes search so easy, sometimes, but it will come and bite you hard one day, so be ready for it.

AnimalMuppet• 23 hours ago

If it's not written down in a place where other people can find it, it does not exist. So the place has to be either standardized or searchable, or preferably both.

ipiyer• 1 day ago

Obsidian.

tenzo• 1 day ago

we've faced this same problem but thinking about it a little deeper reminded me that this issue is no different to pre-llm world. someone creates a spec, someone or a team build/implement the spec and after actioning any feedback from testing or from customers, the product evolves and is no longer aligned with the initial spec.

i think one way to solve this is having an someone or an agent update the initial spec with any new changes as and when they are made.

al_borland4• 1 day ago

I’ve seen at least one person where I work doing this. They have an agent doing massive updates on Confluence constantly. It seems like a mess. It checks the box for having documentation, if that’s all anyone cares about. But there is no way people are actually reading it, looking for changes is impossible, since the LLM re-writes the entire page every time, and even not knowing their project well, it was full of hallucinations and nonsense.

If the LLM is going to write the doc anyway, why not just do it on demand from the code if/when it’s needed, instead of burning tokens on documents that no one will ever read?

I also think a lot of the value in those historical docs are the reasons why certain decisions were made, as it can be important context if there is every a discussion on doing things differently. The LLM likely won’t have that information, or may just make up a bunch of reasons the business doesn’t actually care about.

elenaviter• 24 hours ago

We capture organizational knowledge during each change in the product. Contributors can be agents and people and have different roles. Agents must follow our procedures, and one of the most important assignments is to journal their work in our knowledge base, in a feature session journal, so they are not losing the track somewhere in their harness.

Our "project knowledge" consists of artifacts of such types:

- product documentation (made by human). Intended product behavior and capabilities.

- technical documentation (maintained by human and agent who work together on feature). Implementation documentation.

- feature session journal (made by agent and human who work together on specific feature). Per feature session. Contains original problem, gaps, problems, misunderstandings and failed attempts.

- wiki and ontology articles for targeted knowledge retrieval. Built by Knowledge Keeper procedure under human supervision on demand. Consolidate the knowledge across subsystems.

- code and tests (mostly written by agents).

All these knowledge artifacts (except code and tests) are mostly markdown articles. They are stored in git - this allows versioned storage, authorship and diffs. All articles have front matter with summary, keywords, status (draft or reviewed and approved by human) and links to related articles, so we and agents know what is relevant and what the source of truth is. Wiki and ontology articles hold the semantic relationships for better retrieval.

Typical process of changes: As agent and human work on the feature, this is captured in the session journal. At any moment in this process, the engineer who works with the agent can ask the agent to hand off their work to other agents, integrate in the technical documentation or handoff to a Knowledge Keeper. Such handoff includes the message with documentation links and the link to the session journal, so one who continues starts with the context.

The Knowledge Keeper is a supervised procedure that keeps wiki and ontology aligned with the product documentation, technical documentation, known issues. Knowledge integration process is currently manually initiated by the handoff of the human who worked with agent on the feature. The Keeper prepares the integration plan which a human approves. Keeper builds the index on top of wiki and ontology after integrating the knowledge and tests the knowledge retrieval via MCP to check if integrated knowledge retrieves correctly and reports to the human maintainer. MCP allows the consolidated view of the product and its capabilities. Its answers point to the primary docs, therefore users do not need to find one person who remembers everything.

richardprince4• 22 hours ago

I use federated wiki for just about everything

4lx87• 19 hours ago

All I know is creating yet another place to store data is not going to solve your problem.

The problem of scattered data is to put data in one place.

Ask HN: How do people keep track of organizational knowledge?

Related Stories