Programmable Notes

https://maggieappleton.com/programmatic-notes

by MaysonL • 4 years ago

132 29 4 years ago

maggieappleton.com

georgeoliver• 4 years ago

Lots of nice examples, but I didn't see anything for Org. What are Org users doing with programmable notes?

kkfx6• 4 years ago

For a small extent I'm one, while the "programming" is essentially limited to:

- inserting headings click on an org-mode elisp: link, like a note for a retail order, like

    * orders (new)

where new is the link

- inserting elisp: links for org-ql queries, like in book notes, a book a note (all org-roam managed), with queries per author, publisher etc using org-mode drawers as key-value meta-info storage

In the past I've done a bit more, but it prove to be too long to maintain: since until recently my ISP was sending regular monthly bill as a pdf attached to a notification email I've crafted a script (run by MailDrop) what extract (uudeview) the attachment putting it in a relevant dir (I have almost all files org-attached) and adding it to the relevant org-mode note. Unfortunately they choose to send just notification emails without attachments just saying "go to your profile page to download it" and at that point automation is simply too long to be maintained. I've tried a bit https://woob.tech but just for a monthly bill it's not worth the effort.

However my notes are not just notes, are the entry point to all my digital information:

- almost all my files are now org-attached, so to access anything instead of traversing a tree with a filemanager I just hit a key (bound to org-roam-node-find) few relevant chars, tab/enter and the relevant org heading link specific files, or are linked inside or I can run org-attach-reveal case-by-case depending on what I'm looking for;

- I access my mails, partially, via notmuch: and notmuch-search: link types, again in org-roam managed "notes", partially in the sense that a single key bring me to notmuch for tag:unread (unified inbox) and shift+the key bring a new mail composition buffer (notmuch-mua-new-mail), in notes I link relevant threads, individual messages etc;

- notes themselves are almost my documents, they act as metadata-rich directories with a search&narrow UI (both org-roam-node-find, org-ql-view, org-agenda views etc) but are also meaningful by themselves, for instance I have a note per contact, that note link relevant files, mails, logs anything else about a contact, inside the note I can compose a new mail to the contact on click (elisp:(message-mail again)

Emacs/EXWM (like the web + a browser) offer the environment, org-mode the textual language (like html+CSS), elisp and countless packages offer the user-programming language (like js) and the mix my desktop, a poor man's modern classic desktop.

jimenezrick• 4 years ago

You can do literate DevOps: http://howardism.org/Technical/Emacs/literate-devops.html

tveyben• 4 years ago

Thank you- I was wondering why org and it’s code block abilities was not mentioned…

I really like Howard’s tone (and emacs knowledge!) when he’s sharing knowledge (eg the video linked in the linked article).

It was really fascinating when I found his videos the first time a few years ago - very recommendable!

whacked_new• 4 years ago

Long time (mostly ex-) org-mode user here. Org-mode is incredibly capable, but it's often not the best tool. The last time I used active code blocks in org-mode, it was for reading headings from the org file's TODOs to create an at-a-glance summary table of completion state.

It's super interactive because you're executing code on the very same file where it's written, but tightly couples the functionality Emacs, and adds irrelevant content (code) into the file. It also requires you debug, which can become a huge distraction from what is a simple goal. You also have to remember to run the sync code block, unless you have a method of auto-running the code or use another mechanism to remind yourself. Auto-sync is a whole other exercise.

The workflow stuff in the OP article looks completely doable in org-mode, but if we can see the difficulty in engineering the workflow system directly in emacs-lisp, and good choices exist for developing the system separately, then all of a sudden you're left wondering if Emacs + org-mode is actually the best client.

So now I mostly write markdown. That said, adding markdown code-fence execution to Emacs is surprisingly simple... you can send it to org's evaluate!

kkfx6• 4 years ago

Emacs buffers are just buffers, not canvas, so dynamic information display is limited, but much can be done and it's already there (see Nicolas Rougier's Nano Emacs as a showcase).

You can substitute for instance

    (push '("#+BEGIN_SRC" . "↦" ) prettify-symbols-alist)
    (push '("#+results:" . "") prettify-symbols-alist)
    (push '(":PROPERTIES:" . "") prettify-symbols-alist)
    (push '(":END:" . "") prettify-symbols-alist)

you can avoid tags using in-property-drawer kev/value tagging, that's also easy to query with org-ql. A code block is run by a function as pretty anything and a function in Emacs can be called in countless way, like:

- hitting a key

- on file save with a hook

- when certain conditions happens with a hook

And that's in a global or file/buffer-local approach

riidom2• 4 years ago

Hmm I wish the author would have mentioned the name of the relevant Obsidian plugins, since their plugin site is pretty bad to navigate, no sorting, can't open links in background tabs and stuff.

Beside that, I find these features pretty intriguing, but I wonder if I would want that inside a tool, or rather externally. Like a set of individual scripts, that operate on defined folders of text files, and then make use of the OS notification system to communicate, or along these lines.

The blog itself is also very interesting imo, subscribed to the feed.

throwaway6849362• 4 years ago

I think shell scripts and plaintext (maybe markdown) files is the way to do it if you're on a unix-y OS. I already have a system similar to the article but not as advanced/developed.

kkfx6• 4 years ago

Just see https://youtu.be/B6jfrrwR10k and https://youtu.be/5ViUBaarsbw you'll understand why all modern software and unix CLI are actually limited and limiting crap :D

sofixa• 4 years ago

Shell scripts don't scale and are hard to maintain. Literally any programming language would be a better choice for the same task.

EleanorKonik• 4 years ago

I can speak for Maggie, but the relevant Obsidian plugins are probably Dataview, Templater, and the Spaced Repetition plugin(s).

You might find this community resource for navigating plugins by category helpful: https://publish.obsidian.md/hub/02+-+Community+Expansions/02...

riidom2• 4 years ago

Wow! Many thanks! :)

zmmmmm• 4 years ago

I think there's a lot of application of this concept to be had in regulated environments. Very often procedures are described as passive documents and then humans have to read them, periodically demonstrate their knowledge (training), execute the process, and then finally track that they did what it said in some kind of audit trail.

A format that lets us do all of those things with just one document (not duplicating things everywhere) would be great.

melony• 4 years ago

We should call such a format "hypertext". Add a universal cross platform renderer, a scripting language, and maybe a subscription mechanism too.

YousefED• 4 years ago

Love the examples in this blog (both use cases and prior art). Thanks for sharing.

I'm currently exploring what the ideal end-user-programming environment for this could look like https://www.loom.com/share/6fd5637254584cfea8de315f0f3b532c. I'd love to see end-user programmable applications that can be customized like described in the blog - in an environment that reacts as spreadsheet software (immediate results, no need to fire up an external editor or wait for a build-loop).

rmk2• 4 years ago

Is this feasible for people who have families and things to attend to outside of a life in technology? It seems to me that there is a lot of time and effort that is expended into note-taking and note-keeping. This is generally something only University Professors or professional research people do. Practicing software engineers have some overlap, but rarely need to keep such notes or have spaced repetition etc. for work.

knighthack• 4 years ago

I'm a lawyer. Notes are absolutely crucial to my workflow, as there are literally millions of legal cases, and precedents, and you don't want to repeat yourself or repeat unnecessary research. I'd see it no differently than programmers who keep gists or repositories of useful configs.

rmk2• 4 years ago

Sure, I can see notes being very important for lawyers. But I don’t see that for software engineers and programmers. For a programmer, the facts are the code. The end. The code is what gets executed. Anything outside of it is secondary. This is of course besides persistent data that usually lives in some sort of database, and configuration, which falls outside of the other two buckets. None of these items require note keeping. Programmers learn by doing: taking notes is very much a peripheral activity there. So I’m struggling to understand why or how this is a big deal and the brouhaha about org-mode, zettelkasten and the like on HN!

terafo• 4 years ago

Is you wish programmability in your notes system, you should consider Trilium notes[0]. It is FOSS, pretty fast(even though it is electron-based), has pretty good feature-set and is very programmable(no mobile app, though, but self-hosted web version isn't that bad).

[0] https://github.com/zadam/trilium

TheOtherHobbes• 4 years ago

Interesting ideas. I'm not sure my note-taking life is programmable enough to benefit from any calendar-based approaches. But something project-based would definitely be helpful.

The problem is that poor/random suggestions are worse than no suggestions, because instead of helping they'll take you out of flow.

The ideal solution would have contextual awareness - not just smart enough to understand what you're working on, but to know you well enough to accurately predict what you're thinking about and the kind of help of you need.

That seems like a tall order with current technology.

kkfx6• 4 years ago

Try to see some org-mode Emacs videos on YT and co, you'll discover the real meaning of "notes" and "computer usage" :-)

For instance https://youtu.be/B6jfrrwR10k but also, https://youtu.be/5ViUBaarsbw a bit exaggerated but nice. Notes are just personal documents, text means information, and automation on information means IT. Actually a notes-based computer system/environment is a desktop, is what we need and miss today.

Crappy compartmentalization of modern "apps" is the opposite, is the need for business against the user need and interest. Personally I have in "notes" essentially anything thank to the Emacs ecosystem:

- my files are org-attached, accessed via search&narrow thanks to org-roam

- links in notes point to pretty anything, mails threads, specific messages or mail search queries (all local, locally indexed maildir by notmuch, something better than GMail and under my total control), feeds (posts I decide to keep), bank transactions (any kind, from transfer to stocks passing through grocery expenses)

- notes form my agenda with SCHEDULED, DEADLINE, TODOs meta-information adding to a heading, org-agenda scan all notes to form a view, a single page or more than one, letting me selecting what I want to see (look for) or not

- queries on notes (org-ql) and transclusion means the ability to form/see my web of information "putting on the desk" (screen) just what I want in a semi-automated manner

- notes are also literate programming environments, spreadsheets, slides, all integrated with all meta-information I might want to add, I can place, link-to-execute, pass data from a code block to another, with code blocks of various languages, like pipes in the shell, but in a 2D environment instead of the CLI, I can create a table with all "formulae" I want like a spreadsheet but without the need to open separate tools and with the ability to use any language for them (included the built-in "calc" CAS)

Peoples these days use Google search and the modern web just because they are unable to make their own, collaboration included, notes with desktop computer environments are the tools need to substitute the centralized web of the giants to a web of personal information network shared as we want.

astrobe_• 4 years ago

That seems like a tall order with current technology.

The best way I know to simulate a very complex system is randomness :P. Turn your notes into a fortune file [1] and have those fortunes displayed where you look often (e.g. your browser's new tab page).

[1] https://en.wikipedia.org/wiki/Fortune_%28Unix%29

sinuhe69• 4 years ago

I love the idea. It’s certainly not easy to write a software to facilitate the vastly different ways people process their informations, especially while they are still in the tacit state. But it gives me ideas for a personal note taking system where I can reconfigure it to fit my needs and my workflow.

zimpenfish• 4 years ago

select a few random notes from my database

I've got a Shortcut which surfaces 10 random links from Pocket (which haven't already been surfaced - they get tagged) to the top because I am woeful at remembering to read things I've bookmarked.

(Unfortunately it also relies on having your Pocket links as a text file because their API doesn't have a way to find out how many URLs you've saved which means you need to download the entire set if you want to make a random selection...)

kixiQu• 4 years ago

I scripted up a button for my Tiddlywiki to open a random note matching certain criteria for processing. It's pretty cool to have an OS for your notes, I gotta say! It does feel like you can take a declarative approach to what you want. But... I wouldn't make the effort in many of these systems which are harder to extend.

Programmable Notes

Related Stories