Hope is an alternative to markup languages

https://poef.github.io/hope/

by gjvc • 6 years ago

49 40 6 years ago

poef.github.io

gjvc2• 6 years ago

repository at https://github.com/poef/hope/

bovermyer• 6 years ago

Since my project doesn't use this, you could say... it has no Hope.

abnry• 6 years ago

Interesting idea. I've never thought of separating the markup and the text into two different files. To me, it suggests a more machine friendly way of keeping track of markup. Directly writing sounds very painful.

gjvc2• 6 years ago

Yes, this is looking in the direction of a content API for documents, and is not meant to be human-generated.

abathur• 6 years ago

My mind's been picking at related ideas from different angles for at least a decade now, I guess.

I'm not sure how I feel about this one, but one thing I like about separation approaches in general is that it's easy to imagine how N different annotations can all exist for the same underlying document, so you could use something along these lines to share/exchange/discover/crowd-source annotations.

That said, my general yearning isn't to kick all markup out of my documents--but to kick presentational markup out. Semantic markup can add a lot of affordances to your documents, and if I'm already committing to working with something that requires a toolchain--I suspect I'd rather just generate plaintext or presentational markups from there?

yarnover• 6 years ago

This reminds me of how Nisus (the word processor program) styled text. The document, a Classic-Mac-OS file, contained plain text in the data fork and styling information in the resource fork. If I recall correctly!

Mikhail_Edoshin6• 6 years ago

Yes, this was a rather good way to deal with it. Even if the content wasn't rich text, it still may have things like current position, folded sections and so on. It totally pays to keep this stuff separate from the main content as the latter stays readable by any other program that supports text.

In general the whole business of file type plus creator type felt much better than the single file type of now. E.g. in desktop publishing Illustrator EPS were very different from Photoshop EPS and it was very convenient to be able to tell them apart.

porpoise• 6 years ago

One of the my favorite apps, Scapple (from the same company that produces Scrivener), stores their data using just this approach (plaintext + character ranges), but in XML. It does make the data file much more readable, and it makes it possible to grep for text without having to worry about regex to skip through tags, but IMO the convenience is outweighed by the fact that there is no way to do mass search and replace (that is, using external tools) without totally messing up the formatting.

IMO the best approach is inline makeup but for the text editor to have the ability to hide tags. An example is PowerGrep's docx editor. You can open an .docx file and it will be editable entirely in plain text, but when you save it, all the formatting is preserved. In some way existing text editors (e.g. Sublime) are already capable of doing this via some kind of fold by regex functionality

hansvm• 6 years ago

I've toyed around off and on with FUSE drivers for various purposes, and hiding information like tags and then preserving those tags on save is something you can delegate to your filesystem. The advantage is that tag-hiding now works automagically with ripgrep, your favorite editors, and any other dev tool.

You _are_ limited in that you can only detect edits during interactions with the filesystem, which is strictly less information than you'd have in an editor tracking every keystroke, but a good diffing algorithm still mostly gets the job done.

O_H_E• 6 years ago

On the left you can edit the content and below it the markup. On the right is the rendered output

I am not sure if I can see anything on the right/left. this is what I see on chromium/firefox https://i.imgur.com/quySrXT.png. No rendered veiw!

Poefke7• 6 years ago

Original author here, hope is an old experiment and I hadn't updated it to avoid mixed http/https content. This solved now in the online demo.

lisper2• 6 years ago

The problem with this is that if you want to make any changes to the plain text you have to edit the markup in unison if you want the result to look coherent. So this gives up one of the principal benefits of markup languages: you can edit them with a simple text editor in a straightforward manner.

qsort2• 6 years ago

To be fair the author acknowledges this. To make an analogy, this seems to me more like the equivalent of ASM than the equivalent of C, the only reasonable way to make this work is to have a high-level language that "compiles down" to Hope.

lisper2• 6 years ago

Sure, but now you've come full-circle: what is that high-level language going to look like, if not a traditional markup language? And if Hope is intended to be object code rather than source code, what's the point of making it a text format at all?

neolog2• 6 years ago

one of the principal benefits of markup languages: you can edit them with a simple text editor in a straightforward manner.

That wouldn't be much of a benefit for me because I don't use Nano. I don't think many people do either.

byecomputer• 6 years ago

I can't put my finger on what alternative you're alluding to here. Writing HTML/Markdown/whatever feels mostly the same whether I'm doing it in Nano or Notepad or VSCode, and I probably use all three of those on a daily basis depending on what I'm editing/how long/what privileges I need.

githubalphapapa• 6 years ago

Buffers and strings with range-indexed metadata is also how Emacs's text properties work. For example, this shows the print syntax for strings with properties:

    (concat (propertize "foo"
                        'face '(:weight bold))
            " "
            (propertize "bar"
                        'link "http://bar"))
    ;;=> #("foo bar" 0 3 (face (:weight bold)) 4 7 (link "http://bar"))

gioele• 6 years ago

The technical term for this kind of markup is "stand-off markup" or "stand-off annotation".

There is wealth of research done on stand-off markup since the SGML times, with a renaissance in 2005-2010.

Historically, the main drawback (as highlighted by other comments) is that stand-off needs a way to refer to the content, i.e. a pointing mechanism. Pointers in a pointing mechanism can be either a) hard to write but easy to maintain or b) simple to write but prone to breaking after a change (and thus requiring support from authoring tools). Nobody has found a viable ergonomic balance yet.

Poefke7• 6 years ago

Agreed, you cannot edit one part without the other. But there are many situations where this is not a drawback. Nobody is writing Word documents in a text editor for example.

But I'm looking at a way in which you may be able to add annotations to a document that isn't under your control. If the document is either guaranteed not to change, you won't have a problem. Documents hosted on IPFS or a versioned URL to a Hyper document fit this bill.

If you want annotations to automatically change when a remote document changes, you need to be able to 'follow' the changes. Between when you originally 'linked' to the document and now, what has changed in the document. Which parts are deleted, which parts are added. If you know that, you can update the offsets in your annotations. So I'm looking into making a versioned document format to allow just that.

OJFord• 6 years ago

But why?

Why is a plaintext unformatted file with just words and no markup any improvement? They're both editable, readable in a raw form, and probably not how you want them to actually be consumed.

So why not just keep it in one file?

slightwinder• 6 years ago

Markup und plaintext are seperate entities by nature, and by seperating them physically, tools which don't understand the markup would work better with the pure plaintext.

catlifeonmars• 6 years ago

What problem does this solve?

Poefke7• 6 years ago

Hi, original author here. When I started out I wasn't trying to solve a problem. I was exploring a concept I was curious about. But in the end, we (me and a colleague) ended up building it into our WYSIWYG HTML editor as an alternative to the browsers execCommand editing API. When editing rich text, an API that is build around ranges for everything is a perfect fit.

The editor (https://simplyedit.io/) converts a contenteditable area's content to hope, we apply some markup (annotations) and convert the result back to HTML. I wrote the Hope-HTML conversion routine once, and all the range based manipulations are now one-liners instead of treewalking horror shows.

valenterry• 6 years ago

This is 6 years old with no progress...

Poefke7• 6 years ago

Yup, it was an experiment. There is a follow up, called cobalt: https://github.com/poef/cobalt/

But its not meant as production ready code, instead I'm exploring the concept and what it means for editing, rendering, etc.

Mikhail_Edoshin6• 6 years ago

XML is a serialization format. It's trivial to mechanically separate the markup from the content when reading and to merge them back when writing. (It should even be the preferred approach for many usage scenarios as it's rather efficient.) The description implies content and markup are physically separate; it can be done, but why? Although there is at least one upside: the same text can have multiple different markups attached (e.g. syntax and revision control or syntax and extra highlighting).

simonh3• 6 years ago

Why is you want revision control of both the text and the syntax markup? This seems incredibly brittle.

Mikhail_Edoshin6• 6 years ago

No, I meant that the same text may have two or three kinds of associated markup runs: syntax, revision control, and highlighting so that we can nicely publish a snippet.

bloak• 6 years ago

Where do you draw the line between "content" and "mark-up"?

* Word break nearly always gets treated as content, denoted by the space character.

* Line break can be either: '\n' or <br>.

* Paragraph break is usually treated as mark-up: <p>.

Is there a good reason for that, or is it just historical accident?

martyalain• 6 years ago

Reminds me of John McCarthy's lament at the W3C's choice of SGML as the basis for HTML : « An environment where the markup, styling and scripting is all s-expression based would be nice. »

Could this project be the beginning of an answer : http://lambdaway.free.fr/lambdaspeech/

Alain Marty

Mikhail_Edoshin6• 6 years ago

One thing that Hope doesn't seem to support are contentless elements. E.g. some drawing may look so:

    100:100:svg:svg
    100-100:svg:g /* group */
    100-100:svg:rect
    100-100:svg:rect

These elements have no text content, so their extent is 0 and offset is same. Yet they also have hierarchical relationships and you need to store extra data to express those. E.g. in this example you can guess that 'g' and 'rect's are descendants of 'svg' and the first 'rect' is a child of the 'g' but only because you have some preconceived ideas about the meaning of these elements. There's not enough data to tell if the second 'rect' is a part of 'g' or not. You need to store at least the level in the hierarchy. If you stored the data in a database (i.e. without implied order) you'd also need the relative position of the element, but here you have the order built-in.

Poefke7• 6 years ago

Original author here, this is exactly right. Hierarchy and contentless annotations don't mix well. This is the main disadvantage when compared to xml/html and the like. On the other hand, using annotations with offset pointers to the main text make things like overlapping annotations and multiple sets of annotations possible, which are difficult to pull of in xml/html.

Poefke7• 6 years ago

Hi, original author here, I just found out my old project featured here and love the thoughtful responses. If you are interested, I've been working on a new version of and on, called cobalt (https://github.com/poef/cobalt/). Hope was the proof of concept, cobalt is a more rigorous implementation. Specifically the range algebra is complete there.

It's still a hobby research project. There's a lot I've learned building and using it, about the things that HTML is good at, and what it isn't good at.

What I'm trying these days is to see if I can get a WYSIWYG editor with a user experience that comes close to the user experience of writing markdown. This isn't easy and its interesting to explore what makes that so.

Hope is an alternative to markup languages

Related Stories