Tree-sitter grammar for org-mode

https://github.com/milisims/tree-sitter-org

by happy-dude • 4 years ago

146 31 4 years ago

happy-dude• 4 years ago

From the readme:

Org grammar for tree-sitter. It is not meant to implement emacs' orgmode parser, but to implement a grammar that can usefully parse org files to be used in neovim and any library that uses tree-sitter parsers.

This grammar is in active development and is being used by nvim-orgmode/orgmode [1], a org-mode neovim plugin.

Some additional resources some might find useful:

* Org Syntax - https://orgmode.org/worg/dev/org-syntax.html

* EBNF grammar - https://github.com/200ok-ch/org-parser/blob/master/resources...

* Tree-sitter - https://tree-sitter.github.io/tree-sitter/

[1] https://github.com/nvim-orgmode/orgmode

tgbugs• 4 years ago

We're in the middle of updating the org syntax document. I've been preoccupied and haven't gotten a chance to take another pass, but there should be a new version out in the next month or so. See also [0].

0. https://github.com/tgbugs/laundry/tree/master/laundry/gramma...

milisims• 4 years ago

I'd had the 1.0 commit sitting on my machine for a while, this post gave me the activation energy to bump it. The grammar is fairly stable now.

nonrandomstring• 4 years ago

Great to have a generalised parser to help Org-up (org as a more widespread markup language) to florish. I'd really like to see Gemini adopt instead/(alongside) its markup.

spicybright• 4 years ago

I don't care for org mode but this could be huge for people that do.

I find being able to open + edit notes quickly helps my thought process more, so hopefully this can help with that.

lazyier• 4 years ago

I find being able to open + edit notes quickly helps my thought process more,

I use org with roam in Doom Emacs. The org roam notes directory is backed by git.

https://www.orgroam.com/

It allows for using a Zettelkasten style approach to note taking inside of Emacs. The basic idea is small quick notes that are cross referenced, indexed, and searchable.

Doom Emacs is like Spacemacs were access to commands are activated through hitting the space bar while in Evil's command mode.

This way if you stumble across a link or something notable or get an idea you want to hit later it takes about 15-20 seconds to generate a new note card, link it to other ones, and then get back to whatever you were doing before you got distracted.

And then later you can go back and refine and expand on the idea with more note cards and have something you can easily use for a future reference.

Nvim has it's own Zettelkasten-style note taking plugins, of course. But what sets Org apart, IMO, is just how flexible and FAST it is as a text-based UI for writing documents. Makes using MarkDown seem plodding in comparison.

My favorite is, though, reStructuredText. RST was a work of genius.

But the clincher for Org, when compared to RST, ends up being Org-babel. The ability to embed multiple formats into a single document, be able to quickly render those other documents to separate files, and being able to execute source code and maintain state between multiple src segments is crazy effective. Completely awesome if you want to get into literate programming.

This turns your personal text-based notebook into having capabilities akin to Wolfwam or Jupyter Notebooks. Can even turn your indexed notes into an interface for Jupyter or Wolfram Mathmatica if you really want to.

It's pretty nuts.

Having something like that for NeoVim would be extremely nice. With new versions of Emacs having Wayland and Native Compilation it has improved performance significantly, but I don't think it will ever approach to just how fast NeoVim can be.

bananamerica2• 4 years ago

Having something like that for NeoVim would be extremely nice. With new versions of Emacs having Wayland and Native Compilation it has improved performance significantly, but I don't think it will ever approach to just how fast NeoVim can be.

Always good to remind that emacsclient makes starting an Emacs frame/window instantaneous.

justinhj4• 4 years ago

I’m a long time emacs and org-mode user but I switched to neovim for 90% of my development work. I still use emacs for org-mode and I doubt I would ever use org-mode in neovim because it relies on being implemented in the entire emacs system and the emacs lisp language . Nothing short of embedding emacs on neovim would make it tenable. I have the same problem in the other direction, viper is not a substitute for neovim.

tconfrey4• 4 years ago

This is very exciting.

IMO the Tools For Thought (#TFT) ecosystem is crying out for a standard interchange format to break down the silos. Org-mode could be that format. It's not just an outliner format, it's the best markup[1] and supports pretty much everything you need for journaling/task-management/project-management etc.

What's been missing thus far is a single canonical definition that everyone can build on (beyond the org-mode source code). I'd love to see this be the reference implementation of Karl Voits Orgdown proposal [2]. (BTW there was a recent relevant discussion here [3])

[1] https://karl-voit.at/2017/09/23/orgmode-as-markup-only

[2] https://gitlab.com/publicvoit/orgdown

[3] https://news.ycombinator.com/item?id=30879327

preek• 4 years ago

All good points. As a daily user of Org mode, I couldn’t agree more.

Hence, we are working on building a parser that also acts as a canonical definition here: https://github.com/200ok-ch/org-parser

tconfrey4• 4 years ago

Thanks @preek. I mentioned you guys in [3] above. BTW I'm actually using this parser: https://github.com/orgapp/orgajs for my product (https://braintool.org), so there are other choices. I guess the key thing is a single well defined grammar.

Is GDrive syncing working in Organice these days? I've wanted to demonstrate interop with BrainTool (which syncs to GDrive files) but last I checked there was some bug.

scottrblock• 4 years ago

I agree! I'm the creator of phonetonote [1], which tries to integrate with different tools for thought. I just started a `tft-interop` repo [2] with similar goals. We're just getting started thinking through how to make the TFT structures (edn mostly) more interoperable. I'm not personally an org-mode user, but agree it could be a good format to parse these trees in and out of. I've mostly been thinking about OPML so far. Feel free to hop into the discussions where we're sorting through these ideas — https://github.com/phonetonote/tft-interop/discussions/2

[1] https://phonetonote.com [2] https://github.com/phonetonote/tft-interop/

altgans• 4 years ago

Imo the problem with Org-mode is that it "afterloads" all the complexity into the :PROPERTIES: drawers. Those quickly transform into unreadable hideous beasts of metadata that _require_ you to use Emacs. Every plugin developer likes to create their own set of properties, which directly leads to the problem of "multiple Markdown flavours". Extensibility is not always good.

tconfrey4• 4 years ago

Thats a fair point. But I'd argue that as long as there's a large enough core of agreed upon functionality having app-specific data packed away and carried for free (as plain text) is not a bad thing.

My app, for example, sticks a couple of useful pieces of data in properties but when it creates a TODO item or adds a [[link]] or *heading that semantic meaning is available to any other outliner or task management app using the same backing store.

I guess the argument is that org-mode encompasses a large enough set of core components that its the best place to standardize what can be standard in this PKM/Productivity/TFT space.

sfink3• 4 years ago

Offtopic, but I use org mode for my notes files (both general and technical), and I've never really figured out how to use it effectively.

My pre-org notes file was just a giant text file with entries delimited by manually-entered '--------------' lines. I stuck a topic in each entry for searchability and then the rest was freeform text. It worked pretty well.

org mode gives me more power, but it's getting a little unmanageable. I find myself expanding and collapsing sections a lot, especially when just trying to find the right place to add a new entry. (I'm using a hierarchy rather than the flat append-only list.) The file is pretty huge at this point, and I split it up once but that broke my most common usage of just isearching through the entire file, and it was annoying to have to load multiple files to do a complete search. I had links between files, but that didn't help for searching.

I imagine there are tips and tricks to resolve all of my issues and nuisances, but I've never taken the time to figure them out. Is there a good workflow description somewhere that I should be looking at? It's hard to beat a basic text file, but after a decade or so it has gotten pretty big and I'd like to eg be able to cluster related things together so I can purge obsolete stuff.

Maybe my problem is that org mode is intended for organizing things, and my use case is too free-form?

hnrj95• 4 years ago

try org roam

drunkpotato• 4 years ago

I have a daily.org file I use to capture meeting notes and quick notes into. org-mode automatically keeps it organized as a datetree. I set all my other org files to archive into a subheader of the day it's archived (or done, for todos). So far I like it, but I've only been doing it for a few months. We'll see when it gets to years.

One nice thing is it's easy to see at a glance roughly what I did in a given week or month. It also doesn't insert much process into my flow, which I appreciate.

sfink3• 4 years ago

I had a separate 'daily.txt' file for this sort of thing, and it's desperately in need of a replacement. Your setup sounds pretty good to me. It seems like org mode may be a better match for the daily stuff than it is for my notes file. I especially like the idea of archiving into a subheader of the right day.

tconfrey4• 4 years ago

Scaling your system is always going to be a problem, no matter what tools you use.

I hear you on isearching. To support multiple files you could look at ag.el for fast cross-file search (although not incremental like isearch).

Alternately you can use :tags: to add a different dimension of information by which things can be grouped. org-agenda can then be used to see items by tag (also since they are just text you can search them in a buffer). I use org-agenda in 'F'ollow mode whereby a second buffer shows the selected agenda item in its original context. It helps visualize where things are while navigating.

The new hotness, ala org-roam, is bidirectional linking with tiny files. That would probably be a pain to migrate to, and isn't really aligned with your free-form model. But it might be the way to scale over the long term.

sfink3• 4 years ago

That sounds promising. You understood perfectly where I am and what I'm looking for, thanks.

I'll check out ag.el. The incremental search is not that big a deal, I'm usually searching for a keyword anyway.

I looked briefly at org-roam, and I agree with your assessment. It looks like it adds friction to the part that needs to be as smooth as possible for me: adding notes. I could probably dummy something up for searching by making a command that shells out to grep or whatever, but that feels like fighting against the data model.

unhammer• 4 years ago

I typically just use capture and refile, almost never enter new things directly. I have a few templates for often-used things (work, personal, funny things my kids say), and then I either just file it directly or if I know it should go under some tree I C-c C-w to immediately refile it there. I use selectrum for completion with selectrum-prescient so my most used refiling goals are at the top of the list, and if I have to filter the list it's typically just two or three characters until I find the heading I'm after.

Tree-sitter grammar for org-mode

Related Stories