Mu: making programs easier to understand in the large

by peterkelly • 10 years ago

122 64 10 years ago

sja• 10 years ago

I'm a little hesitant to learn how to 'understand large programs', or how to structure them for that matter, from a repository that has 100 code files in the root directory.

I'm not sure what I'm looking at?

mintplant• 10 years ago

Try the README: https://github.com/akkartik/mu/blob/master/Readme.md

It's a new programming language.

edit: The source seems to be structured intentionally in a recommended reading order. See the comment at the top of

https://github.com/akkartik/mu/blob/master/000organization.c...

Layers of code are filled in as you read down the list of files. It's an interesting concept, similar to literate programming. (And FWIW, these source files are C++ code for the compiler, not an example of Mu code.)

The author also has a blog post up here: http://akkartik.name/post/mu

akkartik24• 10 years ago

Thanks for figuring all that out and laying it out so clearly!

More details on my flavor of literate programming: http://akkartik.name/post/wart-layers

ktRolster8• 10 years ago

His idea is that the way to understand a large computer program is by running it, testing it. So he's made (what I understand) a very nice UI for introspection of a running computer program. You can see what is happening while it's running.

Designing a programming language is kind of like a 'rite of passage.' I think most of us have at least thought about how to make languages better.

akkartik24• 10 years ago

Heh, I've already had my programming language rite of passage[1]. Mu was my attempt to not build yet another language. Hence the lack of syntax.

Thanks for your kind words! I'm gratified that somebody understood what I'm trying to do in spite of my crappy writing skills.

[1] http://akkartik.name/post/wart

akkartik24• 10 years ago

Author here. For my part, I have never understood why people consider it to be a good thing to squirrel code into a bunch of different sub-directories.

a) It makes the build scripts more complicated, which means they'll be more likely to break on some poor noob, and that when they break it'll be less likely the noob will be able to tell how to fix it.

b) Invariably the codebase accumulates dependencies between directories that are uneconomic to reorganize. At least a flat directory has a shot at under-promising and over-delivering.

c) Maybe people do it to make the place look neat, they way I used to 'clean my room' by stuffing all my dirty clothes into drawers. But codebases that are messes at a deep level are less likely to be cleaned up if they look clean at a superficial level. And all codebases eventually turn into messes, the way we've done things so far.

btrask• 10 years ago

Sure, directories can be used badly (and the Java and Ruby projects I've seen are often very difficult to navigate because so much is the structure of the language, rather than the structure of the project).

But they can also be used well, and the promise of a meaningful directory structure is really appealing.

Here's a question, given all of what I understand about your layered way of structuring code: how easy is it to spin off logical portions of a project? If my project accretes its own HTTP server and I want to turn that into a stand alone library, could I? Would it be possible to "rewrite history" so that the HTTP-related changes never appeared in the layers to start with?

Thanks!

i573323• 10 years ago

That same line of thinking could be used to just put everything into a single huge file.

dman4• 10 years ago

I think the reason why people (I put myself into this camp as well) prefer directories for organization is because there is an entire ecosystem built around them. Backing up, recursive search, exploring via ui, programmable access, programmatic access using language of ones choice are all choices available to the user. Code organization isnt the only place this shows up, organizing music and photos has similar semantics - apps which try to do the organization via higher level metadata rarely put in the engineering effort to come up with robust solutions for all the above use cases I cited abode.

antiquark• 10 years ago

How is "subtract a, 3" better than "a-3"?

I can already feel my carpal tunnel acting up!

FenugreekAcerb• 10 years ago

Some say (for example Moore) that arithmetic is only a tiny part of programming and don't deserve special syntax (especially with a language that lacks C-style for loops).

Terr_• 10 years ago

...Well, a tiny part of whose programming?

Programming languages (at least the not-so-academic kind) need to mesh with the problem-spaces being solved, and there are definitely some contexts where most of the work is encoding and applying arithmetic rules.

ktRolster8• 10 years ago

Really all you need for programming are > < + - . , [ ]

Anything more than that is for wimps. Brain is the only language I ever program in.

akkartik24• 10 years ago

It translates to assembly/machine code with a lot less code. I don't need a compiler, I don't need to write optimizations.[1]

I actually don't consider Mu to be a language. (I'm the author.) It's a low-level starting point to explore ways to make the standard OS primitives more testable. I don't need it to look nice. Higher level languages can come later, once the foundations are in place.

I actually am growing increasingly suspicious of all our navel-gazing about syntax. We spend all our time in places like HN thinking about how to make small screenfuls of code look nice, but all that effort hasn't really translated into helping me understand the large-scale structure of random open-source codebases more easily.

Perhaps one way to think about it is that syntax helps insiders keep a codebase in their head. I'm more concerned with ways to help more outsiders import a codebase in the first place. Because people leave and move on, and it's very hard for software projects today to improve once their original authors move on. More on this: http://akkartik.name/post/readable-bad

(I'm susceptible to carpal tunnel, but it hasn't become a bigger issue. I take wrist breaks.)

[1] I can even imagine doing the translation in machine code, so that Mu then becomes self-hosting without needing any of the current stack. I might go down that road..

skewart• 10 years ago

"I'm more concerned with ways to help more outsiders import a codebase in the first place."

This. The pain point that mu addresses is huge. I haven't really looked at the project in much depth yet, or played around with it, but I love that you are tackling this issue. I have spent way too much time wandering around a big code base trying to build mental models of what's going on, and thinking to myself the entire time "there's got to be a better way."

chipsy• 10 years ago

This kind of consideration has led me down the road towards XML, because it's the syntax that is powerful enough to model documents that bundle a mix of schemas(including source formats with their own idiosyncratic syntax). It is not a syntax anyone wants to type in or read, but that one advantage counts for a lot, given the right problem domain, because it pushes the semantic content to the center.

dorfsmay• 10 years ago

Why the comma? Why not

    substract a 3

or even:

    (- a 3)

userbinator• 10 years ago

I don't need a compiler, I don't need to write optimizations.

You've effectively created a sort of portable assembly language, but it's more verbose than usual assembly language ("sub eax, 3").

but all that effort hasn't really translated into helping me understand the large-scale structure of random open-source codebases more easily.

I think the root of the problem is that software is being written to be more complex than it could/should be, so the solution is to encourage reducing complexity.

desireco42• 10 years ago

Definitely could use better presentation. It is interesting, reminds me of forth a little, where you compose a lot of small commands.

kazinator• 10 years ago

Imagine a world where you can:
[...] think of a tiny improvement to a program you use, clone its sources, orient yourself on its organization and make your tiny improvement, all in a single afternoon.

If we alter that to "three programs you use [...] all in a single afternoon", we can plonk that into my resume.

vinodkd2• 10 years ago

Things I really liked:

+ recording runs as tests

+ one library for all layers of abstraction

+ building in assembler to avoid the big runtime.

+ using literate programming techniques to manage coding in assembly.

+ scenario, "screen-should-contain" and "assume-keyboard". awesome.

+ spaces: nice primitive of the closure and similar concepts

+ attributes for meta programs. again, awesome.

+ labels and using [,] as labels. genius.

Suggestions:

* If there's a valid reason to call them recipes, go ahead. If they're actully just functions, please stick with what everyone knows and gets. Same thing with "reagents","reply", "ingredients" and "products". Update: I did read your blog post about this. still think the regular names are better in the long run.

* I dont know where you're going to with multiple types of the "number:list" kind, but if so, the map must look the same, not lispy.

In all, I really like where you're going with this. Kudos! I find a lot of resonance of ideas I've had for a long time now.

akkartik24• 10 years ago

Wow, that's a very detailed look. Thanks! Feel free to email me; my address is in my profile.

The terminology is definitely a work in progress. It's mostly for my attempts to teach programming using Mu. I noticed that mathematical words intimidated some students. I tried to write up my rationale here: http://akkartik.name/post/mu. But I've certainly started mixing up the terms with my student, so it might not last.

vinodkd2• 10 years ago

Will do. And yes, I did read your rationale. The intent is not bad, the names probably are.

slang8004• 10 years ago

Functions, arguments, classes, methods, objects, threads, locks, all these are reassuring everyday words, and yet their meaning in programming (and math) bears no relation to their everyday meaning. (from http://akkartik.name/post/mu)

I can see how the use of the term "arguments" could be confusing ("parameters" or "inputs" would make more sense), and "threads" is a rather tenuous metaphor for how scheduling works within a kernel, but all of the others mirror their real-world meaning pretty well. I'd be more worried about students needing to unlearn "containers", "ingredients", and "reagents" when they start reading material from outside of Mu, talking to other developers, or learning calculus (which uses functions, sets, and arrays).

Someone• 10 years ago

Here's an argument against using the term "ingredients" for arguments: in the real world, if you bake a cake, its ingredients are gone. In your language, they still exist (if you write a recipe for eat in mu, you can have your cake and eat it)

From that observation, I think one should conclude that using analogies from cooking, as you do with recipe in the kitchen sense is not the best idea.

vmorgulis2• 10 years ago

The preprocessor reminds me a bit MyDef:

https://github.com/hzhou/MyDef

akkartik24• 10 years ago

Indeed! I had a stimulating conversation with Hui Zhou last year.

dman4• 10 years ago

I think the project would benefit from a Terminology / Definitions section which lists in one table the names and crisp definitions of the concepts being introduced. It appears that you are using non standard terms so having a single list of the terms would allow readers to have that list open when they are reading through the layers.

akkartik24• 10 years ago

That's a good idea. Might http://akkartik.github.io/mu/html/010vm.cc.html [1] be such a big-picture map? Let me think about how to improve it; I haven't touched it in a while. Further suggestions most welcome.

[1] Colorized version of https://github.com/akkartik/mu/blob/379244666c/010vm.cc

dman4• 10 years ago

I was thinking something much simpler. If you were writing a book about Mu what would be the definitions you would print on the back inner cover? A table with two columns - first column being a term and the second column containing its description in one or two sentences.

Could you elaborate on the target audience for the doc you linked to? I am unable to make out whether that doc is intended for serious programmers in other languages coming across Mu for the first time or whether its intended for new comers to programming.

enqk3• 10 years ago

So I wrote an implementation of linear search in Mu using its array type.

https://gist.github.com/uucidl/df81b6ab0fe65713f5ba

My notes:

- creating an array w/ create-array is possible, however I could not find how to take its address and pass it around

- one cannot get the address of an invalid position of the array, so one cannot construct bounded ranges (STL style) using a begin/end pair of addresses

- the interpreter strongly suggests to use refcounted pointers (address:shared) instead of addresses even when one does not borrow the memory

- could not figure out how to write a test scenario that uses checks named memory addresses rather than ordinal memory addresses (memory-should-contain instruction)

akkartik24• 10 years ago

That's great!! Yes, create-array was just a version I created so I could teach arrays separate from new and addresses. But it creates an array on the 'stack' (default space) so there's no way to take an address of it. I quickly take my students past it to new.

memory-should-contain doesn't currently support named locations, sorry. Part of the problem is that with spaces a name can have many different addresses in different functions. So my tests write stuff I want to check in raw numbered locations, and the first 1000 addresses are reserved for tests so that names can never be clobbered.

I looked at Rust's borrow checker for a bit but wasn't smart enough to understand how it works or transplant it easily. I also noticed that it wasn't smart enough to deal with things like doubly-linked lists without reaching for ref-counting. So I figured I'd keep things simple and just use ref-counting for everything. That way I punt on all the complicated static checks in favor of a simple runtime one.

The rule is: new returns shared:address, and get-address and index-address (and maybe-convert) return address. Use shared:address to pass things around between functions, and reserve non-shared addresses only for short-term operations, usually mutations. Since non-shared addresses are not dynamically allocated, there's no possibility of use-after-free so they don't need to be refcounted, and you can copy them around as much as you like.

Use-after-free and related memory corruption is really the only thing I'm concerned about protecting my users from. Memory leaks I plan to have tools for, so that programmers can identify and break them down when memory becomes a concern, but not worry about until then. I call this "zero-developer-cost abstractions" :) It feels less restrictive and more dynamic than Rust.

Edit: I just took a look at your code, and it feels perfectly idiomatic. Nice job. Only issue I found was that you forgot to specify the outputs of find in its header. So its calls end up doing some runtime type-checking. I should probably raise a warning in this situation. Thanks again.

enqk3• 10 years ago

Indeed, I missed the output for find! A warning would definitely be helpful.

I like that you have users and thus will be able to test things out and get some genuine feedback.

This might however introduce a bias guiding your language and standard library in a certain direction (I think it's inevitable and what's happening in all languages anyway) .. Therefore I would add some of your peers into the mix.

Otherwise I really like the directions you are exploring, as I've come to very similar conclusions at this point in my career: - being able to modify a system through safe additions - not focusing too much on local details of a system

Mu: making programs easier to understand in the large

Related Stories