Building a Knowledge System That Enhances Rather Than Replaces Thought

https://nsavage.substack.com/p/beyond-rag-building-a-knowledge-management

by nsavage • 2 years ago

166 52 2 years ago

nsavage.substack.com

uludag• 2 years ago

This article touches on a tension I've been feeling too with the rise of LLMs. I feel that analogies can be drawn to time management and systems of productivity, like GTD. Like in a GTD system, you almost need the various projects, tasks, next steps, deadlines, commitments, etc. to live in your head so you can make intuitive decisions about what you should be working on at the moment; something that AI almost by definition can't do, since your subjectivity is essential in this process, but in certain ways may be able to assist.

treetalker• 2 years ago

I've been sensing this too: people seem to forget about the principal and beneficial internal effect on the mind that results from mental exertion. I raised a similar point in another HN post today and someone downvoted me because they interpreted my statement as being too anti-LLM!

We're reliving Socrates's lament about writing.

nuancebydefault• 2 years ago

Gave you an upvote to compensate

Terr_2• 2 years ago

From Socrates worrying that writing would destroy memory [...] Think of it as a partnership: the computer handles the organizational heavy lifting, while you focus on the thinking.

I'm less worried about memory per se and more about failing to think, or getting brainwashed/ring-led by a system with its own biases and quirks. Any sufficiently complex organizing is thinking!

Perhaps the simplest example is when quantities (numbers, easy to record) get a mental weight that overshadows and hides their dimensions (the definitions, what they really mean.) For example, a tendency to automatically assume a rising GDP number is an unqualified good sign.

Stuff like LLMs bring that into newer and more-dangerous territory, because the model also contains uncountable subtle biases from its training data, and even if you know it isn't aligned (heh) with your own mental models you can't reliably change it. Much like false-memories implanted by interrogators, patterns in those systems can and will leak into the users. Whenever we can't "think about how our thinking is being changed", I'd say that's axiomatically bad.

Terr_2• 2 years ago

P.S.: Lest anyone think I'm a Luddite--not that I think that appellation is actually that bad--an example where I would use an LLM would be to help me generate synonyms or alternate inputs to a more-traditional search for a discrete external piece of information.

For example, I might remember a book with a jester playing a lute and singing about ogres, and I just can't find any clear search results, because it was actually a bard strumming a harp with a poem about giants.

This is much less dangerous than just throwing every dang thing into LLM inputs (since prompt injection isn't a fluke, it's a way of life) or filtering the results back through the same model in an opaque fashion.

vouaobrasil2• 2 years ago

What parts of note-taking should we digitize? What aspects should remain firmly in human hands? And most importantly, how do I create a tool that enhances rather than replaces human thought?

My personal philosophy is to use the most primitive methods possible and only use technology when there really is a strong need to go to the next level. It exposes what I really need, what are the weaknesses, etc. For example, I take all my notes with pen and paper. But if I find that I'm really referring back to something, I might write it up in a document. I don't see the point in digitizing everything right away if I'm never going to use what I write.

Moreover, writing things first by hand helps me remember them better and "feel" the knowledge through my hands.

Same thing with photography. I don't tend to use the burst mode on my camera unless I REALLY need it. When it comes to accomplishing things, I found (personally) that asceticism with tools is best.

RaftPeople• 2 years ago

Moreover, writing things first by hand helps me remember them better and "feel" the knowledge through my hands.

There was a science article recently that studied taking notes and it's results were that handwritten notes improved recall compared to typed notes.

mh-• 2 years ago

It cramps my hand pretty badly to handwrite the amount of notes I prefer to take in live meetings, so I really dislike that this is true. (And anecdotally, it is true for me.)

hunter-gatherer• 2 years ago

My personal philosophy is to use the most primitive methods possible and only use technology when there really is a strong need to go to the next level.

This is a good take on life in general. My parents were atmy house recently and my mother commented on how we never have dirty dishes. I told them the solution to me was always obviouse; don't own a dish washer. My wife and I have never owned a dishwasher since we were married, which has forced us to be more discriminatory about our dishware & forces us to wash dishes immediately after use so they're available again. People with dishwashers tend to have double what they need because they wait to start a load until the washer is full. Lots of the available technology we have has progressed past the level of optimization IMO.

Kalq• 2 years ago

With regards to zettelkasten, I've always worndered where serendipitous discovery of notes goes from being actually worthwhile to frequently getting distracted but justifying it as productive.

randomcatuser• 2 years ago

create something that thoughtfully augments human intelligence rather than replacing it

I htink this is a really cool design space, particularly with the framing of the purpose of Zettelkasten as reflecting your personal understanding of the world, your views, rather than a cold, dead categorization of facts and objects.

How can we design software systems that enhance our sensemaking?

- Encouraging us to ask more questions, or write down our thoughts? You can imagine the AI bringing snippets to your attention, asking: "What do you make of this?" or "How might we connect these?"

- Augmenting our notes? Finding nubs of thought, and presenting research/articles to help us develop those lines some more

- Collaboration suggestions? People who wrote about the same things?

gmuslera• 2 years ago

Try to build bridges, not roads. Sometimes your flow is interrupted or is inefficient because you are searching for the right word, or have a small understanding of a concept. or a better way to express a concept. Try to have a tool that does that small push. but without overwhelming you with data and disrupt your own process.

You usually are not writing for someone else, but for you, and the reward is the process you went through more than the final text. Maybe the user want to go with an AI all the way in, or not, but it must be user decision,and by default it should be something small and not intrusive.

le-mark• 2 years ago

Recently I’ve been imaging a world where social media algorithms were tuned to help people instead of “driving engagement” with ever more outrage bait. Oh you’re watching clips about machining and by your data profile you’re an uneducated adult? Here are some trade school, financial assistance, and self help links to nudge you toward a better life! What a world that would be.

BadHumans• 2 years ago

Isn't this what Tiktok is in China? It mostly promotes educational and science content.

drekipus• 2 years ago

yes, for natives.

but it also ties in with censorship/propaganda, ie: "no such thing as homeless people in china"

If you're a westerner, it will show how bad US is, lusty images, drug use, gambling. etc.

It's a super double-edged sword. We could have it in the west but we'd need a government that is looking out for the interests of people, not israeli/multinationals/lobbies/industry etc.

henning• 2 years ago

it's hard to understand what problem this person wants to solve.

is a version controlled directory of Markdown files + grep enough? or a super trivial wiki Rails app that will turn WikiText into links that you could probably prototype in an hour or less? why or why not?

UmYeahNo• 2 years ago

What you're describing sounds an awful lot like a barebones obsidian vault.(https://obsdian.md)

I'm not familiar with the OP's notion of a "zettlegarden" but I get the impression that they want something a bit more than just markdown files. Perhaps the idea is to increase the notion of connection between captured notes, whether intentional created, or "serendipitous".

visarga2• 2 years ago

When you're dealing with ten notes, it's trivial to dump them into an LLM and generate connections. With a hundred notes, it's still manageable. But what happens when you hit 10,000 notes? Or a million? We quickly run into the limitations of context windows and processing capacity. Vector search helps narrow things down, but it's prone to missing important connections that a human mind might naturally make.

I run into the same problem. I made a RAG system and imported 15 years of reddit and HN content (my own message logs) + 2 years of LLM chats, totaling about 80MB of text. I can use it to retrieve fragments but there is duplication, and almost all duplicate fragments have slightly different approaches. How do I merge all of them, and how do I get a deduplicated taxonomy? I got about 290K unique keywords extracted from the text, it doesn't fit into LLMs.

I am gravitating towards building graphs of ideas, and having a way to generate unique (non duplicate) new ideas while ingesting new text.

austinjp• 2 years ago

Intriguing. I've been thinking of something similar. Do you have any notion of what constitutes an 'idea' in this context?

visarga2• 2 years ago

I am using a LLM to extract ideas from text, as a numbered list, each one self contained, about 50-100 words long. It also generates relations between ideas (supporting, contradicting, expands on, etc...)

The idea extraction step is simple when starting from scratch, but when you got megabytes of text deduplication becomes a problem. I like to circle back a lot.

nsavage• 2 years ago

Author here. Glad I'm not the only one with this problem!

What I'm experimenting with is building a graph out of "entities", which I'm calling people, places and things found in each card. The point I guess is to allow the LLMs to make connections between cards, but this won't be replacing user made connections.

This lets me pick up all cards that reference something like, I don't know, "factories" as a concept, which isn't something the user will necessarily want to directly connect.

Downside is that deduplicated taxonomy: I have, from a scan through my list, "factories", "Factory" and "Factories". My attempt to head this off is by vector searching based on the proposed entity keywords and having an LLM decide if they are the same or not. I think I need to work on my prompt for this though.

lukev• 2 years ago

I have a thesis that RDF is good for this sort of work (I gave a talk on it at Clojure Conj last year, https://youtu.be/OxzUjpihIH4 if you're interested.)

It's good for bridging the gaps between LLMs, ontologies, traditional structured data, and logic programming.

voidhorse• 2 years ago

After trying out a bunch of digital zettelkasten tools, I just went back to paper.

This take may be a bit hot, but I actually think paper and pen is already the optimal maxima for thinking—not for retrieval, mind you, but for helping us produce new thoughts. Zettelkasten (at least the way Luhmann used it) is meant for this purpose—it is not a system for storing information and retrieving it but rather for supporting the creation of new connections.

The computer aided tools are suboptimal for this because they lack the good constrains of paper (severely limiting the search space) and the good features (seamless ability to incorporate a variety of representational modes, text, image, equation, with zero overhead, and the ability to organize things freely in space). As much as the digital knowledge base sounds good in theory, I don't think it will ever be as optimal for generation of thought. If all you want to do is summon existing information, digital tools are evidently superior. I personally think a hybrid system where one "thinks" on paper and "archives" digitally (after the thinking is done) might be best, but ultimately, we will be most productive with whatever system we actually enjoy using.

harlanji• 2 years ago

Tried Inkscape? I was all paper for a long time but it's taken over as I've gotten used to it an the hot keys. I carry a little Wacom tablet that works good with it too, but seldom even get it out of my bag.

It does have a (configurable) fixed page size, no ability to multi-page which was kind of annoying but I started thinking of it as a feature as well.

exe34• 2 years ago

have you come across scribus?

https://impagina.org/features

you don't have to use all the desktop publishing features, but it does have multi page, flows the text from page to page, some basic vector drawing etc.

dflock• 2 years ago

Inkscape does have multi page, now, if you still want that.

JellyBeanThief• 2 years ago

the ability to organize things freely in space

This is the key. It's not enough for digital tools to just put things in folders or tag them. It's the links themselves that need elevation. People need to add metadata to links, they need to apply rules specifying what links to crawl and how to arrange them in space, they need to specify how content should be displayed in islands of connected content. Then they need to be able to arrange islands on a 2d, 2.5d, or 3d canvas.

We have information input and retrieval solved. For some reason it's taking a real long time for people to get to spread out.

limandoc• 2 years ago

This is the core principle of LimanDoc.com I am building, a canvas that holds files and diagrams, but most importantly an inner board (a folder) that you can see through.

egormakarov• 2 years ago

I sometimes dream of video stickers, where you could put some short video on it, and then just put it on the paper notebook page

buggy6257• 2 years ago

Hypothetically with a sticker printer like a Cricut or something you could do little QR stickers…

Dennip2• 2 years ago

would something like an ipad + apple pencil not bridge this concept? The usage of pen & paper with the addition of the digital world; OCR, tagging etc.

sdwr2• 2 years ago

You would think so, but the micro-interruptions of switching modes, selecting documents, selecting tools, etc make a huge difference in terms of flow.

It sounds small, but "taking a second to select a new brushstroke from a list of options" is exactly what OP is talking about in terms of raw creation vs synthesis.

LeroyRaz• 2 years ago

Why do you prefer paper to a word processor or latex?

jrm4• 2 years ago

Wow, so glad to see the author's idea and this as well.

After playing with this stuff it hit me why these systems are simultaneously so attractive and also not amazing.

Human brains are still way better at the very cool linking thing. We make those connections better than the bots.

Which is why I dream of, and tried to start work on one that was basically "cards."

I'd love to see one with "the card" as the main atom/unit -- but that had a zillion different visual "arrangement" methods. Give me alphabetical, give me semantic, give me random etc etc.

two_handfuls• 2 years ago

The Livescribe pen goes well with this philosophy.

smj-edison• 2 years ago

http://dynamicland.org or https://folk.computer too.

BenFranklin100• 2 years ago

My experience echoes yours. There’s something about writing longhand that is very difficult to reproduce digitally, and I have tried them all, including Obsidian and LogSeq.

The only digital exception I’ve found is OneNote. It’s 20 years old but nothing else comes as close as to reproducing the experience of pen and paper. In fact, I found it superior and I regretfully gave up my fountain pens and paper long ago. Recent research has shown that tablets with a digital are equivalent to pen and paper for comprehension, both of which beat typing:

https://pmc.ncbi.nlm.nih.gov/articles/PMC8222525/

The advantages of OneNote over paper are practical: digital notebooks are more portable, can be automatically synced to other devices, converted to text if desired, searched, and has the superior feature of being able to easily insert images and screenshots.

My only regret with OneNote is that it is closed source and syncing requires a MS account. Using it on iOS also requires an account. It’s high time someone came up with an open source alternative that can be set up locally.

metalman• 2 years ago

I disagree.The human mind is adequate for coming up with new ideas, ;) and it has been reported by many good authorities that staring at a blank page, :) no matter the quality and fineness of one pen can be madeningly unproductive. For my day to day work, a pen and paper is indispensible for the making lists of, and crossing out things needing doing. The aquisition of perspective is vastly more important than meer knowledge.Removing blinders and having the scales taken from our eyes, a new light, etc. VS getting a bigger market share by whatever method, works, best.

kirso• 2 years ago

I agree with your disagreement. The retrieval of pen and paper are awful.

The problem is that we are overly complicating PKMs and and knowledge systems and focusing on storage instead of thinking or active retrieval which mostly comes from the fact that people tend not to know what is important and how to engage with it.

A simple journal timeline with bullet points and insights / interesting things that are relevant to me made a difference.

I have no idea how I would have stored these in a paper format dragging with me whilst travelling.

Rochus• 2 years ago

It's reminiscent of Luhmann's original zettelkasten system, where physical proximity created a kind of "neighbor effect" that digital systems struggle to replicate.

That's what outliners are good for. You automatically generate context and proximity when adding/indenting outline items. And we have the thing called hypertext for cross-referencing and navigation. I think Luhmann's zettelkasten is a quite poor concept in comparison, because the cards are small and don't adjust to the size of a topic, so we have to split topics, and we have to take care of identification and referencing the cards ourselves. When translating zettelkasten to digital, then at least we can get rid of some of these limitations, but we still don't have all the advantages of outliners.

In my experience, it is purely a matter of habit whether you write down ideas or thoughts by hand on paper or enter them directly into an outline on the computer. Before I had tools like Ecco or my own CrossLine (https://github.com/rochus-keller/crossline/), I also produced a lot of notes on paper. After a few years of working with these tools, I got rid of paper completely. With CrossLine, I also solved the problem of linking to other documents or emails, so that I can link any information or put it in the necessary context. And it scales well. I have outline files with thousands of notes, minutes, concepts, specifications and the like, everything cross-referenced and full-text searchable. I can still reproduce e.g. meetings or paths to decicions taken ten or more years ago with little effort.

pxeger1• 2 years ago

I write with a similar approach to zettelkasten. My technique is purely digital. Paper is too easy to lose, damage, or not have on you when you need it, compared to a digital file that can be copied, synced, and backed up.

I keep everything in just one long file, which I think gives a similar feeling to what the article describes about notecards - by using folds and reordering where necessary it’s quite easy to always have whatever you want visible; there’s nothing forcibly hidden from you by a file structure.

I use plain text (with a tiny bit of custom syntax highlighting) in vim, because I like the simplicity and it fits my needs. But I think a one-file approach could work fine in a Markdown or WYSIWYG format.

j45• 2 years ago

I never had the impression it was all or nothing between paper and not.. everything can end up in one place for beting connected.

kavasmlikon• 2 years ago

excellent article. IMHO it tackles the gist of what personal knowledge management should be about.

true learning/understanding (and intellectual depth for that matter) seems like something that (due to neurocognitive reasons) cannot possibly be achieved only through the process of reading, but is rather a function of the reader's quality of elaboration on what has been read. this inherently requires the reader's wilĺingness to invest work in the form of structured thought and cognitive effort as part of the reading process, which in turn - again due to neurocognitive reasons - for somewhat non-trivial thought and intellectual endeavors requires systematic note-taking and writing (https://www.youtube.com/watch?v=vtIzMaLkCaM). as a result, pure reading vs the reader's quality of elaboration on what they have read makes the difference between "feeling smarter" and actually "becoming smarter" as Sönke Ahrens (https://www.soenkeahrens.de/en/takesmartnotes) puts it.

I fully agree with the "Zettelgarden Philosophy" on note-taking as described in this article - it recognises the real purpose and value of personal knowledge management which is based on notes that embody personal comments, thoughts, insights and connections to other concepts/ideas rather than mere facts and summaries pulled out from articles. Though, my "tooling philosophy" differs in that I prefer to have my digital zettelkasten stored in simple text-files so that I can leverage the freedom and flexibility of text as a universal interface (grep for search/filter, git for version-control, unison for syncing across devices and operating systems, export/publishing through static site generators, etc) rather than being locked in and bound to the limitations of a monolithic tool (https://roamresearch.com/, https://obsidian.md/, https://www.notion.com/, etc) which https://zettelgarden.com/ seems to be a kind of.

"We need tools that respect the importance of human engagement while eliminating the friction that gets in the way of thinking and creating." my tool of choice has been org-roam (https://www.orgroam.com/) - it combines the power of emacs (for powerful, efficient text-editing) and org-mode (as a way to structure content in simple text-files) and adds the very missing layer for personal knowledge management on top: a powerful cross-referencing system incl. bidirectional linking between individual notes/nodes/zettles. to me, bidirectional linking seems like a crucial feature for personal knowledge management systems (something that the WWW at its inception as a "collaborative" knowledge management system probably should have adopted, but Tim-Berners Lee opted for the much easier, rapid implementation of one-way links (see Jaron Lanier https://www.youtube.com/watch?v=cCvf2DZzKX0&t=3009s, and Ted Nelson's Xanadu Project)

at the end of the day, a tool is just a tool and only as useful as the user is capable of using it. I found the usefulness of my zettelkasten depends largely on the quality of my note-taking process, which is a learning process in itself. I found Ahrens' book and this blog post very helpful in teaching / reminding me of the real purpose and value of personal notes.

Building a Knowledge System That Enhances Rather Than Replaces Thought

Related Stories