DSpinellis

Given that AI-generated code is not 100% perfect, we need to take into account the time and cost of reviewing when we discuss the productivity boost AI developers receive from code.

I convinced a software company to use a version control system (RCS on shared disk) back in 1993. To make it work we had to setup a network — Ethernet over (thin) coaxial cable at the time. This was so new to us that we didn't know we needed to use terminators on the two cable ends.

Indeed! The repo includes some v4 elements: https://github.com/dspinellis/unix-history-repo/tree/Researc...

The provided kernel predates the actual edition by a few months. It is based on https://www.tuhs.org/Archive/Distributions/Research/Dennis_v..., which matches V4 more than V3.

I don't think the concept of a rebase existed before Bitbucker and Git.

30 years ago (1995) open source offerings: mostly CVS for large projects and RCS for smaller ones. On the proprietary side, the aged SCCS was available and used, while Perforce and Microsoft Visual Source Safe were being launched.

I published an updated extension of this post's linked article in Empirical Software Engineering. You can read it without a paywall at https://rdcu.be/b7FzE. You may also be interested to see the actual GitHub repository at https://github.com/dspinellis/unix-history-repo.

That's the point: non-dependant tasks can run concurrently in Airflow. In sh/BAsh/dgsh dependant tasks can also run concurrently, as in tar cf - . | xz.

A nicer syntax and a lower probability of deadlocks.

And, more importantly, assign a name to a process, so that it can appear multiple times in the graph.

Admiring your multi-tasking!

I'm curious: what do you mean by "dgsh will use iteration under the hood too"? Dgsh does several things under the hood, but I wouldn't characterize any of them as iteration.

Manually playing around with fds is definitely unmaintainable. My hope is that a clean syntax can help create maintainable complex pipelines.

Thank you for the suggestion. This would mean that you'd also then create some mapping from each name (like git_log) to its implementation, right?

Nice benchmark! This is a (not at all efficient) awk one-liner.

awk -F\; ' $2 > max[$1] { max[$1] = $2 } !($1 in min) || $2 < min[$1] { min[$1] = $2 } { sum[$1] += $2; count[$1]++} END { for (n in sum) printf("%s=%.1f/%.1f/%.1f, ", n, min[n], sum[n] / count[n], max[n])}'

Can't see how dgsh could be applied to it.

I've found creating pipelines with Python to be messy and intuitive. Other than creating a DSL to express them I can't see how DAGs can be expressed naturally with Python's syntax.

Even creating tools in Python that can be connected together in a Unix shell pipeline isn't trivial. By default if a downstream program stops processing Python's output you get an unsightly broken pipe exception, so you need to execute signal.signal(signal.SIGPIPE, signal.SIG_DFL) to avoid this.

I went through two iterations before adopting the current syntax. Truth is neither me nor Doug McIlroy, the inventor of Unix pipes, who kindly and generously provided feedback during dgsh's development, had something better to propose.

What syntax would you propose?

Author of dgsh here. This is definitely not what I had in mind.

Apache Airflow solves a very different problem. Its DAGs are static dependencies between sequentially executed processing steps, whereas the DAGs of dgsh express live direct data flows.

Fifty years ago Marc Rochkind wrote SCCS, the first version control system, which later inspired RCS, CVS, and Git. In the March issue of the IEEE Transactions on Software Engineering he provides a retrospective of SCCS: its innovations, its problems, and how subsequent systems improved on it.

I advocate the following rules for when to write and when not to write a shell script.

# Write a shell script:

* Heavy lifting done by powerful tool (sort, grep, curl, git, sed, find, …)

* Script will glue diverse tools

* Workflow resembles a pipeline

* Steps can be interactively developed as shell commands

* Portability

* Avoid dependency hell

* One-off job

# Avoid shell scripting:

* Difficult to see the preceding patterns

* Hot loops

* Complex arithmetic / data structures / parameters / error handling

* Mostly binary data

* Large code body (> 500 LoC)

* Need a user interface

A need for associative arrays (implemented in Bash as via hashmaps) moves the task to the second category (avoid shell scripting).

Pipe-related concepts in various restricted forms were floating around for years. Doug McIlroy indeed proposed them in 1964 and was heading the Bell Labs team when they were implemented in the Third Research Edition of Unix (1973).

See https://youtu.be/FbDebSinSQo?si=xbWIqES80hI2S802&t=901

ChatGPT can generate wrong code but it can also be a great helper. I've summarized the good, and bad, and the ugly in a recently published IEEE Software column: https://doi.org/10.1109/MS.2024.3363848

The ai-cli library attaches to programs that offer interactive command-line editing and modifies their interface so you can obtain generative AI suggestions with a single keystroke.

A repository with GitHub Markdown and troff-inspired extended typography hotstrings for the AutoHotkey program. Input thousands of math symbols and emojis in your editor, chat, CLI, IDE, email, browser. ∄ a,b,c,n∈ℕ∧n>2 ∋ aⁿ+bⁿ=cⁿ 2H₂O + 2e⁻ → H₂ + 2OH⁻ I +

There is a series of conferences MSR — Mining Software Repositories — with research papers looking at such questions. http://www.msrconf.org/ In fact, I presented this work in the 2015 MSR conference.

Very interesting comment. Having initially taught a more theoretical software engineering course and now teaching this one, I've realized that grasping the theoretical aspects of software engineering without practicing them in a realistic context is difficult and results in pretty shallow learning.

Innovation had peaked long before the Unix wars. I think that the diminished innovation rate comes from rising inertia due to the large code volume, diminishing returns, and compatibility concerns.

Tue, we trace the evolution of the Unix code along the line that currently is FreeBSD over 50 years. The Linux kernel started its life decades later than Unix and it didn't (initially) share any code with the original Unix code base. Furthermore the GNU and other programs and tools required to form a GNU/Linux distribution have their own separate history, further complicating the picture. Linux's evolution, though shorter, is also interesting, but it is a very different story.

The post claims (without any reference) that in New York City only 8,500 hospital beds were occupied. This Bloomberg article reports at around the same date that 23,000 existing beds are occupied. https://www.bloomberg.com/graphics/2020-new-york-coronavirus... I'd put more trust in Bloomberg's reporting than on the post's claim.

Indeed: I try to give priority to people who haven't yet been heard. But giving also some priority to those who were first incentivizes students to participate more actively and energetically. Being first to raise your hand is more difficult than tagging along the others.