HN user

catenate

170 karma
Posts1
Comments86
View on HN

I've been using a little literate-programming shell script for years, for various personal and professional projects. I find it makes a huge difference in my ability to hand off a code file to another developer, because I tend to document justifications and explanations for why the code is the way it is, where it came from, what else was tried, alternatives and variants, and how to understand aspects of it that are not obvious from the code.

My script doesn't try to shuffle code around, which has the advantage that if you are familiar with the general structure of the code, the documentation follows the same structure. (This may not be the best order to explain the theory of the program, as is noted in several introductions to literate programming).

Instead of code blocks, my little shell script has a per-line approach, where each code line is preceded by the name of the file to which it is extracted. This approach allows me to name a variant immediately after the filename, so that I can code alternative lines, and decide at the time of extraction which sets of lines to use. This is also useful for extracting multiple very similar files from a single markdown source. This use of variants has been very effective in supporting alternative implementations, since I can quickly switch between them by the list of variants I give the tool to extract.

https://github.com/catenate/sharedo/blob/main/lit.md

Since 1 June 2012, I've been taking notes in unicode text files, which contain (occasional or adjacent) lines starting with 'nb ' and then a list of tags. I wrote a simple tool ("nb") in Inferno's shell (thanks to Robert J. Ennis for the port to Plan 9's rc), to (1) search for given keywords in per-directory index files pointed to by the global index, (2) index all of the nb lines in files in the current directory, and (3) if necessary, append, to a global index file, a reference to the index file in the current directory.

https://github.com/catenate/notabene

I've found that I'm comfortable with the eventual consistency this offers, in exchange for fast lookups when I want something (as opposed to indexing first, and/or indexing globally, and so waiting for indexing to get a result). This distributed-file approach also allows me to add tags to a variety of files: local files, or networked file-system files, or sshfs-mounted files, or Dropboxed files, or files under version control, or files with varying text formats; and find tags across all of them and across all the time I've been indexing.

It runs in linear time with respect to the number of tags I've entered, plus the time to read and process the global index, so obviously there are many ways I could improve the time performance (as an easy example, I could permute the index to list all the tags in alphabetical order, and next to each tag list the files that contain that tag).

I also wrote other tools, since the layout is so simple: for example, "nbdoc", to catenate the actual contents of the references returned by the primary tool (nb); and "so" (second-order), to return all the tags which appear in any nb line with the given tag(s).

I've also found that it's not easy for me to remember what tags I might have used in the past, or how I was thinking about something, so I try to use the conjuction of several tags to narrow down search results, rather than try to remember one specific tag (this seems to correspond to the observation that it can be difficult to remember exactly where in a hierarchy you put something).

The modular approach, of per-directory indexes referenced in a global file, also makes it easy for me to combine work-specific notes, with public notes, with private notes, all in the same global index file, at work; but only have the same public and private notes at home.

Mafia (party game) 10 years ago

I made my own Secret Hitler deck, and have been playing it for a few months with both adult and teen groups for whom I usually run Werewolf games. SH has been really well received, keeps everyone more involved with the game (both because of fewer deaths, and the basic mechanics), and is well on its way already to supplanting Werewolf as the social game of choice.

I used Emacs from 1993-2004, and switched to Acme for the 11 years to present. I don't miss trying to memorize all the key combinations from Emacs. I like that Acme presents a clean, simple, and direct Unicode interface to what I work with: mostly editing shell scripts, and running shell commands, as a build engineer. It takes a while to get used to mouse-button chording, but I don't even think about it now. I constantly use guide files, in many directories, to store and modify commonly used commands to highlight and run, so I make many fewer typos now, and don't forget which commands to run or how I run them. I can also switch contexts a lot faster, both because commands are laid out in the directories where I use them, and because the Dump and Load commands store and retrieve sets of files in the tiled editor subwindows. When I had to work on Windows I enjoyed having a pared-down unixy userland that I could write scripts in, to use also in my Linux Inferno instance (mostly communicated from one instance to the other through a github repo for backup and version control). The biggest drawback to me with Inferno is that so few other people run it, that I have to compile it myself on any new platform on which I run it (there are not really rpms/debs/etc available to just install it). But your experience with Plan 9 Acme might be better, I just prefer also working with the Inferno OS improvements, such as bind, /env, sh, etc.

I told Facebook to delete my account as soon as they implemented this feature, years ago, and I found that (at least at the time) I could not block it on my Android phone. So, at least, I've not since then been party to giving Facebook information (they most likely have anyway) about other people, since the Facebook app is not installed on my phone.

Composing a larger program, by combining functions in different languages in a new framework, seems to take things in the direction of large and complicated programs. Would it not be simpler to write smaller, independent, individually named and reusable programs in several languages, according to each language's strengths, and pipe the output from one of these programs to the other? As an added bonus not all the programs need to understand the entire problem domain.

I use Inferno to develop software, as a virtual OS over top of Linux or Windows. Its community doesn't see any value in writing software just to make it easy to use for newbies. It is still actively maintained, with new changes from Plan 9 development. There are even some new software tools developed in it (eg, I wrote a build tool). Maybe this qualifies?

The redo-inspired build tool I wrote abstracts the tasks of composing a build system, by replacing the idea of writing a build description file with command-line primitives which customize production rules from a library. So cleanly compiling a C file into an executable looks something like this:

Find and delete standard list of files which credo generates, and derived objects which are targets of *.do scripts:

cre/rm std

Customize a library template shell script to become the file hello.do, which defines what to do to make hello from hello.c:

cre/libdo (c cc c '') hello

Run the current build graph to create hello:

cre/do hello

Obviously this particular translation is already baked into make, so isn't anything new, but the approach of pulling templated transitions from a library by name scales well to very custom transitions created by one person or team and consumed at build-construction-time by another.

Also see other test cases at https://github.com/catenate/credo/tree/master/test/1/credo and a library of transitions at https://github.com/catenate/credo/tree/master/lib/do/sh-infe...

I think this approach reduces the complexity of the build system by separating the definition of the file translations from the construction of a custom build system. These primitives abstract constructing the dependency graph and production rules, so I think it's also simpler to use. Driving the build system construction from the shell also enables all the variability in that build system that you want without generating build-description files, which I think is new, and also simpler to use than current build-tool approaches. Whether all-DSL (eg make), document-driven (eg ant), or embedded DSL (eg scons), build tools usually force you to write or generate complicated build description files which do not scale well.

Credo is also inspired by redo, but runs in Inferno, which is even more infrequently used than Go (and developed by some of the same people). I used Inferno because I work in it daily, and wanted to take advantage of some of the features of the OS, that Linux and bash don't have. Just today I ran into a potential user that was turned off by the Inferno requirement, so I'll probably have to port it to Linux/bash, and lose some of those features (eg, /env), to validate its usability in a context other than my own.

EDIT: Replaced old way, to call script to find and delete standard derived objects, with newer command.

Inspired somewhat by redo, I wrote "credo" as a set of small command-line build tools, so build description files are in the shell language rather than a standalone DSL.

https://github.com/catenate/credo

(I don't like how makefiles have so many features that reimplement what you can do in the shell. I also don't care for big languages with build-tool DSLs--though you could say credo is a build-tool DSL for the shell, like git is a version-control DSL for the shell. With only language directives, no constructs.)

I wrote it in the Inferno shell to take advantage of some nice OS features and its cleaner shell language. One of these days I should port it to bash, so other people might use it.

In high school I was blacklisted from an admin position for demonstrating that you could write in Digital Command Language a program that simulated the login environment, stored login attempts, and then after three tries exited to the real login environment to let the user in. In college I was nearly expelled for just mentioning to the IT guys that they didn't have a password on some database, and I could get in with just telnet. These attitudes haven't changed much since 1990 at least.

I think the most important design criteria in the evolution of Unix to Plan 9 to Inferno was that each step consciously made things _simpler_. Simplify the programming model, use fewer different concepts, get rid of workarounds and hacks by bringing useful features into the OS. Remove redundant utilities and even options to utilities, focus on improving the one may to get something done so it hardly takes any effort to do it, and integrate all the various bits well with each other. It's not an approach that asks for newcomers to hack on all sorts of things to "improve" the OS by adding everything possible to it, which is part of why it doesn't foster a big community.

I think the biggest difference between Plan 9 and Unix is that Plan 9 rids systems programming of many special cases by reusing the filesystem metaphor as far as possible. This is a deep and pervasive change, and makes doing most things on the system just a matter of reading and writing to files. This leads to neat things like the /env filesystem, which is a per-process view of the contents of shell variables as files. (This makes it easy to write make (mk) rules that depend on shell variables, just by listing the files/variables as prerequisites.)

Inferno's biggest difference is that it runs the entire OS on a virtual machine called emu. Kind of like Java, but not just a language. (Inferno has a language too, called Limbo, which is closer to Go than C.)

Laughter and joking is a coping mechanism for dealing with uncomfortable and hostile situations. It's a way to blow off steam and accept things without reacting as aggressively as the humiliation you feel. Take out that release valve, and people are going to start reacting more badly than a simple joke, which will escalate security's perceived need to be there in the first place. I'm very glad I don't fly any more for my job twice a week, because I never bothered to restrain my comments in the face of such misdirected, useless, and intrusive delay and obviously unconstitutional search. I will continue to take 500-mile road trips rather than fly with my family, because there's no way I'll teach children that being either irradiated or caressed by a uniform agent is acceptable.

Inspired by redo, I'm working on a similar project that takes advantage of some of the nicer features of the Inferno OS: sh features like apply, <{}, and quoting rules; /env for shell-variable dependencies; and bind to overlay directories. Its user interface replaces build description files (eg makefiles) with a set of command-line tools to manage build rules and dependencies, an approach I haven't yet seen in other build tools.

https://github.com/catenate/credo

I was able to teach myself Basic programming on a TI-99/4A, Atari 2600, and Commodore 64 from 4th grade (in the early 1980s) because the languages were simple, the machine was simple, there were magazines (Compute!) to crib code from, all the freeware and shareware programs were in source, and my dad said "There's the manual." when I asked him to teach me. :)

I spent time programming because I didn't have a modem. It turns out this was the best thing for me, since I didn't waste my time on BBSes. A friend of mine who had, dropped out of the CS program at Penn State after a digital logic course I loved.

In high school I took a year and a half of Basic and Pascal, but more importantly spent three periods a day in the computer lab exploring every nook and cranny of programs I had to write, programs I wanted to write, and the shell language of the school district's VAX.

Seemed clear to me that I wanted to do computer science, but I switched to computer engineering after that digital logic course. Took college classes in Pascal, C, C++, assembly for microprocessors, common lisp for ai, vhdl for processor design, operating systems, and wire-wrapping a microcontroller system.

Grad school courses in Smalltalk [edit: and computer architecture and petri nets and, most importantly, self-taught makefiles for a class project build system].

After I got out of school I took company courses in Java and Perl, but more importantly taught myself the Unix environment from books, and had opportunities to use it every day at work.

On my own, I read books and did programming exercises in Haskell, Ruby, and Go.

[edit: I dug seriously into make and bash for ten years, writing the build system for a telecom network.]

I have a real passion for functional parallel programming in the Inferno shell, so I'm working on a blue-sky build-tool project. [edit: https://github.com/catenate/credo]

On my job I'm now picking up Python because I have to maintain a build system code base in it.

Overall, I'd say I've probably taught myself more than been taught, but only because I had about 8 years of formal instruction in a programming span of 30 years. The education was essential though, because there were so many concepts I would not have gotten to on my own.

Unix fmt double-spaces after periods, except periods after single letters. Of course, that's only useful to you if you compose text in a text editor, and pipe it through fmt (or par, if you are that picky), before publishing it.

Thanks to HTML, however, you wouldn't know this text, for example, had lines about 72 characters,¹ was carefully hand-justified after fmt(1) to leave no dangling words alone on a line before or after punctuation,² and had double spaces after all periods inside paragraphs, except T and S in T. S. Eliot.

¹ Except the first paragraph, better broken at “letters”, “editor”, and “it”, and this footnote, also broken after “editor”.

² Though I would rather do that than visit violence upon decent sentences, like this one, broke at “sentences”. Obviously it's preferable to expand and contract margins a bit to suit the text. Nobody does this any more though, because auto-sizing to columns destroys manual formatting.³

³ Incidentally, these footnotes had to be separate paragraphs, to keep them from merging with the previous one, and with the body paragraphs. What a piece of work is HTML… it delights not me.

Here is this comment before the autoformatter got to it. https://gist.github.com/catenate/6567903

There is an invention without which no other invention can spread beyond its inventor. An invention which makes possible inventions beyond the ability of one person to create. An invention which permits a person to do something with their life other than kill for food, skin for clothes, and hide in natural shelters. An invention so important and fundamental, our physical brains and the thoughts of our consciousness are continually shaped by it. An invention so powerful, it defines the scope of what we can accomplish as a species. An invention impossible to leave behind or survive on most of the planet without, as defenseless physically as we are. An invention so complex and manifold, we continually reinvent it, and almost cannot teach more than the barest rudiments to any other species. Have you got it yet?

Like the buckle on a seat belt, they are designed to be strong in the directions of forces that might be applied in normal use, and require pressure in an entirely different direction in order to be released.

Let's pretend you build a bridge out of this material. Could the direction of force you'd need to apply to dissemble it capable of being generated by an earthquake, or by harmonic oscillation of winds (eg Tacoma Narrows)? It would be tragic to see a bridge with cars on it dissolve into cubes like the spaceship in Super 8.

People like to put down scripting, especially shell scripting, as "not a real language." I prefer shell scripting, since it lets me tie together programs written in many languages, each doing something that language does very well. The opposite of this would be to try to do everything in one language, which seems inefficient to me. Difficult applications of one language are often handled efficiently in some other--why not take advantage of that? Of course, I've made my peace with living in a polyglot computing world, and having to at least superficially know many programming languages and little-language tools.

The best difference I've seen between "programming" and "scripting" is Knuth v McIlroy, summarized here. http://www.leancrew.com/all-this/2011/12/more-shell-less-egg...

I noticed right away that the title had no capital letters. But I didn't notice that the body had none until I read this comment and looked back over the text. Neither time did the lack of capitals bother me.

Impressions of Go 13 years ago

Aka polymorphic function parameters. If I manage to abstract a bit of functionality into a function, I don't care what the type is as long as the operations within the function work on it. Haskell uses typeclasses to provide this guarantee for specific types. Go seems to try to do something like this with interfaces, but it doesn't seem to apply to the map/fold/filter style of programming

Impressions of Go 13 years ago

I'd like to use map/fold/filter idioms instead of for loops, because this encourages me to write reusable functions instead of embedded blocks of code. To do this in Go, I need to write type-specific functions for every combination of passed function, source collection, and return-collection type. The alternatives I see in Go are to lose type checking or to fumble around with introspection and casts.

I'd like a minimal smartphone, which comes unlocked or has some other way of assuring I won't ever get into this situation again. I think this is fair, since I have had to live with the consequence. Locking down the OS version of a phone is very much enough for me to never do business with that kind of company again. I would never buy a computer with an OS I couldn't upgrade, and Android phones are computers.

Further, I'd like a phone whose processing power is devoted to Android apps I choose, rather than carrier's undeleteable bloatware. I was hoping for something like a Nexus 5 this year, since the Nexus 4 seemed to have these traits.

I worry that the Moto X would replace a Nexus 5 in their roadmap, and that their desire to compete with the S4 will lead them to mimic it instead of having the kind of product I'd like.

So I see Google focusing so much effort on a Moto phone as a problem for me and what I want, not something to look forward to. Google once had the product I should have bought (Nexus 4), especially at that much lower price point.

I don't believe that Moto is any different now, because I haven't seen them implement better policies. From my POV, the burden of proof is on them to demonstrate they changed, since they decided to screw over me, and everyone else with a Photon. I don't need to give them any more money, unless they are obviously the very best phone available for me, and my wants aren't what people or the carriers apparently want.

Don't really see the point in getting another Motorola phone. I paid a premium for my Photon, and it really annoys me every single day with how long it takes to wake up and actually do something, and how often it crashes. That might be due to the fact that I have no choice but to still run Gingerbread, because Motorola decided to lock the Photon to that version.

This response explains how I use Acme to store (and later modify) commands in files in each directory.

https://news.ycombinator.com/item?id=5566567

In addition, I wrote a generic tag-indexing tool (in Plan 9's rc and Inferno's sh) to find random bits of information (including commands) without caring where they live in my file hierarchy.

https://github.com/catenate/notabene

http://swtools.wordpress.com/2010/03/05/nb%E2%80%94search-an...