Dataspace 0: Those Memex dreams again

http://natecull.org/wordpress/2017/06/26/dataspace-0/

by rauhl • 8 years ago

55 12 8 years ago

th0ma5• 8 years ago

I think this is a pretty good start to reconceptualizing our systems. I think about this a lot.

I also think about what are the more higher level systems that will be required in the far future... something that can run a whole probability distribution of the kind of program I want to create and as it quizzes me derives entire systems around the context of my desires... something like that would require a language around systems that we can't do now because we feel a generalized higher level thing will have to be so dumbed down as to not be useful. I think we forget that a higher level system, if it worked way harder, could actually do the right thing at all the lower levels.

TuringTest2• 8 years ago

I've also thought a lot about this problem (I recognize a lot of the intermediate steps); I've got even a good idea on user applications that could benefit from that unified data and computation model.

In fact, I'm more interested in the practical applications than I am in the underlying model. However, I'm glad that such model is something I'm familiar with as is logic programming, and not some obscure branch of category theory full of point-free abstract operators.

pdfernhout• 8 years ago

Yeah, similar dreams for about 35+ years -- and they are still not fully realized as a spare-time hobby. It's a harder problem than it looks at first. I use the names Pointrel and Twirlip for my experiments on GitHub, SourceForge, and elsewhere related to triple stores, shared documents, and more. My undergrad work in that area indirectly helped inspire WordNet -- started by my undergrad advisor at Princeton, George Miller, just as I graduated.

One related post on that from me from 2005: "Python implementation of Memex" https://tech.slashdot.org/comments.pl?sid=156379&cid=1311190...

A more recent experiment: https://rawgit.com/pdfernhout/Twirlip7/master/src/ui/twirlip...

Would love to help realize with that, say: https://decentralizedweb.net/

infradig• 8 years ago

I would have included Gelernter's "Mirror Worlds" as a reference.

thallukrish• 8 years ago

Very similar thoughts. I have tried a universal data model for a while for solving personal data as well as business data (small and medium businesses)...I had built an App and then put that in cold storage as I am still not clear in my head on how I can go about presenting it to the world.

nailuj• 8 years ago

Interesting musings. I think the author dismissed graphs a bit too early. Making lists another first-class object (e.g. by always having nodes be lists) might be a bit less elegant but would work for his constraints.

The data structure he describes in the last article is actually well known in Prolog as difference lists (except that you can't do higher order logic programming with them). Converting between terms and lists is also well supported by Prolog. Granted, the syntax is not as nice as using S-expressions with special sugar for terms.

TuringTest2• 8 years ago

A free-form graph is hard to serialize. If the goal is to have the syntax close to the meaning, tree terms are simpler, and then graphs can be modeled on top of them.

As for lists of lists, the most recent article uses them for an in-memory optimized representation:

http://natecull.org/wordpress/2018/08/16/dataspace-10-an-arr...

elvinyung• 8 years ago

Here's an orthogonal thought. Kubernetes is really popular, but it really should've been Erlang or some other kind of distributed Smalltalk that vaguely smells like serverless. Distributed objects failed originally because RPC was a leaky abstraction.

The idea of a "server" at this point, then, is basically an extremely coarse-grained unit of work and locality. You deploy some binary onto a box on a rack, you ensure that the stuff on the binary stays together in one place.

But what we really care about is objects and functions/methods, and having related objects be closer together. In the ideal world, we should just be able to throw more and more machines at a cluster, and have it transparently rebalance objects underneath, taking into account load and affinity.

scroot• 8 years ago

One of the more important ideas here is that you have to simulate time within your Actors/Objects [1]. There is so much possibility here, but since the status quo more or less "works" in market terms, it's hard to push for anything better.

[1] https://oneofus.la/have-emacs-will-hack/files/2005_Teatime_F...

lcall• 8 years ago

I have attempted to address this in the vision & internal structure of a tool I wrote (AGPL): a work in progress which is planned to meet most of the requirements mentioned in the article. I wrote it (after trying org-mode, collapsible outlines in other tools, etc). It uses postgres, and I hope that "sharing" data exchange features between multiple instances are coming (though it already does export to org-mode-like text, to readable outlines in the latest dev code, and to html). The best code is that in github, though an older and useful downloadable .jar is available. Currently, it is keyboard and desktop-only (text-oriented). It also is the most efficient/effective thing I have found for organizing notes/lists/details of all kinds, and should be easy to learn to use, as all the essentials are on the screen. There is a tutorial.

It makes an internal knowledge graph as one uses the product (stored in postgres, runs fast). It builds an object model on the fly as a side-effect of using the product, using relationships, numbers, etc as knowledge at an atomic level where words are secondary. The best info organizer (for my style at least) that I know of, though (so far) less feature-rich than many products. I hope the About page at that link explains the present and future well.

(Part of the eventual purpose of the tool is to help individuals and mentors via maturity models that apply more generally to life before getting into any particular topic. That could easily be misunderstood, and there is more about that at the web site (below). Part of the idea is to help in the development/growth of self and others -- skills, knowledge, positive character traits, anything good -- in any area of life that one chooses, so we can have good goals instead of trying to make life seem fulfilling with empty searching for more pleasure/power/attention.

And by marking things done or "archived", it also provides a journal feature or personal log, of entries created or archived in any date range (defaulting to "yesterday and so far, today", to help with daily standup reporting).)

http://onemodel.org (And yes I plan to move to https fairly soon :) ...as well as adding anki-like and other features someday.)

Candid feedback welcome, especially via the site (2 mailing lists; direct email in footer). Thanks.

Myrmornis• 8 years ago

And files are just too big for the small pieces of data we want to point at.

Files can be any size you want. Why does the author say this?

yunyu• 8 years ago

Files are associated with inodes, which have nonzero size. They are also segmented into multiples of the block size of a filesystem, so even a 1 bit file effectively takes up 4KB if your block size is 4KB.

Dataspace 0: Those Memex dreams again

Related Stories