Dt: Duck tape for your Unix pipes

https://dt.plumbing/

by signa11 • 3 years ago

226 146 3 years ago

dt.plumbing

theobreuerweil2• 3 years ago

It wasn’t mentioned on the landing page (I think) but is this a concatenative programming language?

tgv• 3 years ago

It's stack based, like Forth and RPN. So 1 2 3 * + means something like

    push(1); push(2); push(3); multiply-top-of-the-stack(); add-top-of-the-stack();

So `status pls` means: call the function status (which leaves its results on the stack) and print the top of the stack with new-lines (multiple lines if it's an array).

Then `status upcase pls` would do something like the above, but it calls a "to uppercase" function before printing.

theobreuerweil2• 3 years ago

Got it cheers

mst3• 3 years ago

While I think 'pl' and 'pls' for print line and print lines is probably an excellent choice overall I can't help but see 'pls' and try and parse it as an abbreviated INTERCAL.

buildbot2• 3 years ago

This is really interesting and fills a nice niche for when you don’t want to reach for python or whatever.

As an aside, I wish I was cool enough for lobste.rs!

buildbot2• 3 years ago

Thank you to the two people who sent me invites, that was very kind :)

Jolter• 3 years ago

Title edit: original title is “duct tape for your unix pipes”

dimator• 3 years ago

I read the page a few times but I'm still scratching my head. How is this intended to be used?

stingraycharles2• 3 years ago

Yeah, the examples are focused on dt’s internal state and features. A few real world practical examples of what you can do with it would be much better.

quench• 3 years ago

For example I am trying to figure out how I would split each line on a delimiter. I cant figure out what the syntax is

dt [ "," split ] pls prints split

dt [ "," split ] map pls prints stack underflow

I think I could definitely use this if it had better docs and I could figure out how to.

bvanderveen• 3 years ago

Presumably it's meant to be used when you're writing a shell script and you have some problem in front of you that would be trivial to solve in a real programming language and you find yourself saying a sentence starting with "I just wanna…" and rage-googling or asking ChatGPT or whatever.

stingraycharles2• 3 years ago

Would be great if it could show some things it can do that awk can’t. Because awk is typically my Swiss Army knife if I run into these situations.

usrbinbash• 3 years ago

you have some problem in front of you that would be trivial to solve in a real programming language

Which is the exact point in time when I solve the problem by using a real programming language.

NullInvictus• 3 years ago

My read as well.

I write shell-scripts when the current tools solve the problem easily. I distribute shell scripts to colleagues (never customers) only when I absolutely do not want to install extra software on their system.

I avoid awk and perl because if I'm going to introduce a second language to a tool, I'm not going to pick the niche ones everyone only learns opportunistically if at all. At that point I'd rather pick something my colleagues are deeply familiar with.

And on a small level, writing these little binaries that truly do one thing and do it well and that I understand intimately is a private joy.

gnfargbl2• 3 years ago

This article links to https://en.wikipedia.org/wiki/Vesta_Stoudt, which links to https://en.m.wikipedia.org/wiki/Cotton_duck, which explains that the word "duck" here comes from the Dutch doek, or "linen canvas".

reportgunner4• 3 years ago

Yeah which explains that duck tape is not for ducts (or pipes) since it's made of fabric.

She is often misattributed as the inventor of duct tape. However, numerous variations of adhesive cotton duck tape had existed for decades, nor did she invent the specific formulation of the popularized duct tape.

dynisor4• 3 years ago

That is not what the thing you quoted says, or even correct. All duct tape is made of fabric and it is… well duct tape, so ducts were the intended use case when the name was created.

Duct tape has kinda become a general term for all cloth-based tapes though, so you can indeed find duct tape not intended for ducts, but the fact the name was originally “duck tape” has nothing to do with that.

p-e-w11• 3 years ago

TIL that "duck tape" is actually the correct/original version, and "duct tape" is a modern derivative.

I had always assumed it was the other way round, and some people simply don't know what a duct is so they substitute a similar-sounding, more common word.

gnfargbl2• 3 years ago

Me too! The situation isn't helped by the registration of "Duck Tape" as a trademark for a specific brand of tape -- but that appears not to have happened until 1981, well after both names were in common use: https://trademarks.justia.com/733/00/duck-73300816.html

taneq• 3 years ago

Sounds like “duck tape” is adhesive fabric tape (maybe water resistant?) and “duct tape” is shiny metal tape for connecting air ducting? And both names are used informally for either or both types of tape by most of the population?

And then so many people post “facts” about a clearly fuzzy issue and proceed to argue about who’s “right”. I’m getting real blegg/rube vibes from the whole argument here.

JohnFen2• 3 years ago

Me too. This explains the thing about "duct tape" that never made any sense to me -- it is not the sort of tape you'd want to use on ducts.

But I grew up calling it "gaffer's tape", because it was popularly used by gaffers on film and TV sets.

qbasic_forever• 3 years ago

Ironically the kind of tape you actually use on metal ducting isn't even duck tape, instead it's a thin aluminum foil tape.

Duck tape kind of sucks for most temporary applications as it leaves so much residue behind. Gaffer tape is an awesome replacement (although quite a bit pricier).

NoZebra120vClip3• 3 years ago

It would be amusing if this were merely an elaborate citogenesis hoax and the original name was indeed "duct tape", but we're all being gaslighted now.

ttctciyf• 3 years ago

ISTR from my scout days that "ducking" is a fabric surround at the bottom of tent walls that you tuck under the groundsheet - probably from the same root, I guess - though few online dictionaries seem to have heard of it.

hoyd• 3 years ago

In Norwegian too, 'duk'. That's also called 'lerrets tape'

Lio• 3 years ago

The same cotton duck material is still used for old school Caradice cycle touring bags.

It's a nice waterproof alternative to oil based materials as the cotton swells when wet to seal its own seams.

https://carradice.co.uk/

bshacklett• 3 years ago

I can’t find a source at the moment, but I’ve read that it was also branded as duck tape because of its waterproof nature, making for some clever word play.

alxmng• 3 years ago

Nushell has filled this void for me. It elevates pipes by structuring them into lists or tables. It has all sorts of goodies for manipulating those data structures like inserting/updating rows and cells, better string interpolation, useful utility functions like case transformation, and more.

p-e-w11• 3 years ago

Agreed. Nushell fulfills the Unix philosophy promise that the standard Unix tools have been breaking since forever. Traditional Unix commands feel like a hodgepodge of hacks by comparison, that only work together occasionally, and even then mostly by accident.

imiric6• 3 years ago

The Unix philosophy doesn't dictate any specific data format to interoperate between commands, besides this being data streams, and typically text. Commands are free to interpret these streams however they want, which is why this seems like "hodgepodge", but in no way were tools breaking any sort of promise.

FWIW I've been using *nix machines for decades, and very rarely have I had the need to use structured data between commands. Yes, it would make things cleaner or more robust in certain cases, but in practice there are common ways of handling this. E.g. most tools support outputting or inputting null-terminated lines.

klysm• 3 years ago

The “Unix philosophy” is whatever the writers philosophy is - it has no real meaning

shortrounddev23• 3 years ago

That's why I use powershell

bafe• 3 years ago

Isn't this what powershell tries to do,with the difference that it's structured objects being passed instead of tables?

lionkor2• 3 years ago

Title should be

Dt: Duct tape for your Unix pipes

      ^^^^

ly3xqhl8g92• 3 years ago

Curiously enough, not really, from the post:

    Excuse me, is it "duct" or "duck" tape?
    I mean sure, "duct tape" and "duck tape" are both fine. "Duct tape" is the more common name today. The older name is "duck tape" after the duck cloth under the adhesive.

    As an aside: If you care about such things, make sure to read up on how today's common version became popular. A 51-year-old woman named Vesta Stoudt, mother of 8, mailed her idea for a better ammunition box seal to the president (who approved production) after her bosses didn't do anything with the suggestion.

wizofaus• 3 years ago

Barely a day goes by I don't learn something surprising on HN that's entirely unrelated to IT/software dev.

lionkor2• 3 years ago

The title of the article spells it "duct", and titles should not be editorialized.

mgaunard2• 3 years ago

I either use Bash (which is actually a decent programming language if you spend the time to learn it properly) or Python.

john-radio• 3 years ago

I use Python's `fileinput.FileInput()` a lot for ad hoc scripts, because its behavior is to read from STDIN if available, or to attempt to open a file from `sys.argv[1]` (possibly `[1:]`?) if not, which covers a ton of this kind of use cases.

Aerbil3132• 3 years ago

Bash (which is actually a decent programming language)

How so? I tried to seriously learn Bash a few days ago, and once I learned about word splitting, [[ ]] vs. [ ], "${MY_VAR}" just to properly reference a variable it seemed even more full of technical debt that Wat-full [1] Javascript. I figured it would be obsoleted by a modern shell language alternative by the time I will be 30 (currenty 18), it's not worth it, and gave up. I think the bar should not be Turing completeness for a programming language to be decent.

1: https://www.destroyallsoftware.com/talks/wat

mgaunard2• 3 years ago

Learning a programming language properly usually takes 5 years of daily practice using it to solve real problems.

There is no 'wat' situation for a tool you really are proficient with.

RadiozRadioz2• 3 years ago

There is a zero percent chance that Bash will be gone by the time you're 30. I recommend you use some of that youthly neuroplasticity of yours to get used to it; it is too valuable a skill to give up on simply because it's "wat-full". It is in part a valuable skill to have precisely because it is "wat-full".

tpoacher2• 3 years ago

For me, somewhat surprisingly, the best bash resource I've come across is the manpage. If you haven't read this and have instead relied on any other resource, I would recommend spending some time to read the bash manpage before deciding to give up on it. To me this has transformed bash from what I once considered a cryptic ugly mess to one of my favourite languages to write even relatively general-purpose code on.

The help command is the only other resource you need, imo.

dejj• 3 years ago

up: Ultimate Plumber for shell

https://github.com/akavel/up

jeromenerf• 3 years ago

Nice one.

There have been so many nice CLI tools these last 10 years. Quality of life kind of nice. ag/rg, fzf, entr, up, lf...

Go and rust have been appreciated enablers, even though ruby/python/js demonstrated the concepts, with somewhat poor performances and UX.

booniepepper5• 3 years ago

Oh my!

artemonster• 3 years ago

Forth/Joy/Factor vibes

frameset2• 3 years ago

In my profressional life I have cause to work with Powershell a great deal, and I love the object oriented pipelines that it has.

Being able to filter, sort, or transform properties of the objects passed over makes it much more straightforward to work with than bash et al.

brazzledazzle3• 3 years ago

I’m with you here. Once you get the hang of it you even miss the most awkward pipelines once they’re gone and you’re just passing around single dimensional strings.

The memory overhead with sets of objects and their properties vs vanilla strings is significant though and easy to bump up against when working with large data sets. Best to try to keep all of the processing for that dataset in a single pipeline if you can. But that’s tradeoffs.

frameset2• 3 years ago

Yeah the memory usage of the pipeline is real. Often using foreach instead to save memory is needed. Or good old fashioned array chunking.

shortrounddev23• 3 years ago

Personally I only use shell scripting for simple tasks, not anything that would run into a memory limit. If it requires more heavy lifting, I'll write it in a programming language like C#. Nothing we do in production relies on shell scripts, they exist only as shortcuts for our workflow

pmarreck3• 3 years ago

There's a lot of people trying to copy this to the Linux space. NuShell comes to mind: https://www.nushell.sh/

I had an idea where I'd make wrappers for all the popular commands that would accept JSON structured data instead of raw binary data and "do the right thing". And that way you could take advantage of it without having to change your shell tooling. And they'd all accept an extra argument called either --structured-out or --unstructured-out which would either emit JSON or render the output back to a "flat" string, as appropriate.

einpoklum• 3 years ago

I don't really see why existing shell tools and scripting languages are missing, that this project provides. Especially if you consider the expressive range from, say, grep/bc/tr, through sed, through awk, to perl or python.

mst3• 3 years ago

I write a lot of perl -e pipeline members for this sort of thing and I can absolutely see dt producing something that's easier to skim read for somebody who only really knows shell.

(I got sufficiently competent at perl before really noticing sed and awk existed that I'm pretty terrible at the latter two, so YMMV etc.)

yunohn• 3 years ago

Maybe I’m not the target audience, but this README is very light on real usecases and examples to help me understand when I would need this.

quectophoton• 3 years ago

Trivia: "DT" is Japanese slang for "dōtei" ("童貞", means "virgin").

Names are hard.

mst3• 3 years ago

It's generally considered a feature for your pipeline to -not- be screwed, after all.

alexvoda• 3 years ago

Is it nailed?

p-e-w11• 3 years ago

There's a fairly successful version control system whose name is an insult in British English.

And wait until you hear what the most popular open source alternative to Photoshop is called...

booniepepper5• 3 years ago

There are 5 pages of entries in Urban Dictionary for "DT" -- there is no winning with short names.

hiccuphippo• 3 years ago

What do the -- in the documentation mean? Also why does one command use double backslash (\\word) while others only a single one (\word)?

reportgunner4• 3 years ago

Isn't it the bash double dash[0] that signifies the end of cli options?

[0] https://unix.stackexchange.com/a/11382

WaitWaitWha• 3 years ago

"Duck Tape" (versus duct tape) is a trademark of Manco, now Henkel Consumer Adhesives.[0]

Did some fancy DB work for them on OS/2 and early AS/400s in the '90s.

[0] https://www.duckbrand.com/about

pmarreck3• 3 years ago

So this is basically just Forth.

I'd suggest that a functional shell like es-shell is more interesting than this experiment =)

https://wryun.github.io/es-shell/

Dt: Duck tape for your Unix pipes

Related Stories