In praise of --dry-run

https://www.gresearch.co.uk/article/in-praise-of-dry-run/

by Smaug123 • 5 years ago

395 158 5 years ago

gresearch.co.uk

Smaug12310• 5 years ago

(Author here.)

A whistlestop tour of the technique "insert a lightweight API boundary down the middle of a tool, for fun and profit". It leads you towards a structure that has lots of benefits, such as meaningful `--dry-run` output and more ready librarification.

tasogare• 5 years ago

Nice to see F# in posts from time to time. I might try the technic in the future.

b3morales• 5 years ago

This was a great read!

It's not material to the point, but I think there's a small typo on line 12 of the second "Finishing the example" snippet: it looks like

    let instructions = gather args

should be

    let instructions = gather inputGlobs

Smaug12310• 5 years ago

Thanks!

You're quite right; thanks for letting me know. I'll get that fixed.

nicoburns3• 5 years ago

For particularly dangerous operations, even better is to make dry run the default, and force users to explicitly opt-in to actually running it.

llimllib3• 5 years ago

I've written a couple tools that default to dry run and require `--yes-actually-delete` and similar, but I can't think of any publicly available tools that follow this pattern - anybody have suggestions?

TFortunato• 5 years ago

Not quite the same, but apt commands like apt-get require user interaction to confirm what you are about to do, unless you explicitly add a "-y" / "--yes" to the command.

btdmaster• 5 years ago

sed does not operate on files directly unless turned (-i)nteractive.

Smaug12310• 5 years ago

Also not quite the same, but `git clean` bails out extremely early unless you supply one of -i, -f, or -n (the latter of which is `--dry-run`).

OskarS• 5 years ago

”git clean” sort-of works like this. It errors out unless you explicitly specify if it’s a real run or dry run.

jerf• 5 years ago

rm has "--no-preserve-root", which isn't quite "--no-seriously-delete-my-root-directory" but is at least heading in that direction.

Izkata• 5 years ago

Not exactly the same, but along the same lines: mysql has an --i-am-a-dummy flag so that DELETE queries won't run without a WHERE.

Phlogistique• 5 years ago

https://github.com/dmerejkowsky/ruplacer

Uses `--go` to confirm.

wgarvin• 5 years ago

Perforce’s command line app uses this pattern for one of its most dangerous commands: p4 obliterate //depot/path/... which prints out exactly which files will be obliterated from the server (which not only removes their entire history, but can corrupt other files too if you don’t p4 snap them first).

To actually carry out the obliterate action, you have to add the -y option.

matja• 5 years ago

`hdparm --yes-i-know-what-i-am-doing --please-destroy-my-drive --fwdownload`

https://github.com/Distrotech/hdparm/blob/4517550db29a91420f...

tshaddox• 5 years ago

React has a deliberately inconvenient API for injecting unsanitized HTML markup straight into the DOM:

    <div dangerouslySetInnerHTML={{ __html: "Some raw HTML" }}></div>

https://reactjs.org/docs/dom-elements.html#dangerouslysetinn...

ben5092• 5 years ago

I've done that and called the counter option --for-real, but I'd like both args so runbooks can spell out:

    1. do-step-one --dry-run
    2. (do validation)
    3. do-step-one --real-run
    4. (do validation)

Many commandline parsing libraries assume you're doing booleans, and I think --dry-run and --no-dry-run is confusing. And, internally, you have a boolean flag so there's always the possibility of some code getting it backwards.

Internally, I'd like an enum flag that's clearly dry_run or real_run, so the guards are using positive logic:

    switch(run) {
      case dry_run:
        print("Would do this...");
      case real_run:
        do_real_thing();
    }

Smaug12310• 5 years ago

coughs gently I use `DryRunMode.Dry` and `DryRunMode.Wet` in the OP.

kirkules• 5 years ago

Definitely true. In this case, it's important to vet guides/examples to make sure you minimize copypastable examples that have the --no_dry_run flag

steego• 5 years ago

It’s a good opportunity to use long options like:

—i-acknowledge-this-is-dangerous-and-i-gave-this-more-than-cursory-thought

jakebasile• 5 years ago

Yep, this is what I do now. The dry run is the default, you have to flag `--really` to do anything permanent.

globular-toast• 5 years ago

I don't like this because it trains users to assume that commands can never do dangerous stuff by default.

overshard• 5 years ago

At my work I've gone through and setup a variety of "replacement" commands for commands that may have dire impact on production systems. All of these replacement commands dry run by default (or fake dry run/try to create a dry run the best they can) and require a "--do-it" (think emperor palpatine voice) flag to do the intended operation. I've had multiple coworkers thank me for this setup as it's saved people from silly typos and mistakes, one such being the different handling of programs in the use of and ending "/" in folders. ex. "/etc" is different than "/etc/" sometimes for certain operations.

I'd like to one day see a global flag on operating systems to dry run nearly all commands that changed anything.

danmur• 5 years ago

Ditto :). I use --really though.

MAGZine• 5 years ago

--void-warranty

maherbeg• 5 years ago

+1. I prefer having a --real flag that you have to invocate.

jdfellow• 5 years ago

I think I've done --really when doing something similar.

Hjfrf• 5 years ago

It's official best practice for powershell to enable - whatif in any potentially dangerous scripts.

Seems to be followed most of the time.

Twirrim• 5 years ago

I've largely switched to writing all my tools dry-run by default. We've had a few undesirable events from tools that had --dry-run modes and accidentally got run without. Making --run or similar necessary almost guarantees it's never run without intention.

nicoburns3• 5 years ago

All of these replacement commands dry run by default

Same here. In addition if there's never a use case for running in production (e.g. at one point we had a command that reset a dev environment to a clean slate) then the command will actually specifically check for the production environment and refuse to run as an extra failsafe.

IgorPartola2• 5 years ago

--make-it-so is even better.

andrewshadura2• 5 years ago

I sometimes create a makefile with "it so: build" so that I can make it so :)

greggyb• 5 years ago

It should be a parameter that takes as an argument the current user and logs the invocation, "--if-it-breaks-i-take-personal-responsibility $USER" to really make sure folks understand what they're doing.

jonnycomputer• 5 years ago

Good idea. I'm writing some db scripts, and I think I'll do this.

hnick2• 5 years ago

Exactly how I do it. It guards against people who don't read the docs or bump enter halfway through accidentally (damn laptop keyboards).

lbhdc• 5 years ago

I have been incorporating dry run flags into the tools I have been working on (typically etl like tools). Typically this flag only stops the final call to external services that would change the internal state of those services.

Its been really helpful when hacking on things, adjusting logic, or validating that input data does what I think it will.

im3w1l• 5 years ago

First I found myself nodding along, but then a troublesome question popped into my mind. What about Toc/tou race conditions?

Smaug12310• 5 years ago

Indeed, that's the thing I rather swept under the rug. I'll definitely take precautions if it's at all easy to avoid the assumption that just because my `--dry-run` stage said something was safe, it will definitely be safe by the time execution passes to the "do-it" stage. However, in general you are indeed doomed the instant you decided to split the tool up this way.

I have never actually encountered such a race condition while using any tool I've written this way. But I do remember one place where I specifically didn't add a feature to the tool because it was so vulnerable to a TOC/TOU race condition. Instead, I turned that feature into a "did I just manage to do the thing successfully, and if not, why not" check at the end of the "do-it" stage.

So yes, that concern is extremely valid, and if you're doing an operation whose validity is liable to change after you've done the check, then you do just need to follow a different approach for that operation. (But then there's probably no possibility even in principle of a `--dry-run` for such an operation anyway!)

mst• 5 years ago

Something I've found can work is "check, output plan, prompt for confirmation, if 'yes' execute plan in a way that maximises the odds of bailing out if anything changed while you were displaying the prompt" - for bonus points, recalculate after each action that had to be simulated and do something policy-defined if anything differs from your prior expectations.

Obviously the effort of doing this becomes a trade-off but it's worth considering.

iudqnolq2• 5 years ago

I think this can be mitagated with more abstract descriptions, like:

"Would have gotten all servers tagged foo (currently 1,341) and deleted them"

Smaug12310• 5 years ago

That's all very well for just `--dry-run`, but it's still a problem if you use the method in OP to architect the entire tool (and in particular the non-dry-run execution) around the existence of a `--dry-run` phase.

amirkdv• 5 years ago

I'm adamantly pro dry-run and like OP I've found that you really need to design for it. From my experience, OP's two main ideas are spot on: librarification and pushing dry-run logic deeper into library code.

Here are two other things I've found:

1. Regardless of where you push your run/dry-run dispatch to, the underlying "do it" logic really needs to be factored properly for individual side effects (think functional). Otherwise, you inevitably end up with loads of "if dry-run / else" soup or worse, bugs _caused_ by your dry-run support.

2. You still need safety nets for when someone is doing dangerous operations without dry-run. Pointing and calling [0] is a great trick for this. For example, say your tool is about to delete X, Y, and Z. Instead of a simple "Yes" confirmation which the user would quickly end up doing on auto-pilot, you could have a more involved confirmation like "Enter the number of resources listed above to proceed" and then only proceed to delete if the user enters "3".

Very curious to hear about design idioms folks have come up with!

[0] https://en.wikipedia.org/wiki/Pointing_and_calling

kbenson4• 5 years ago

Usually I just use dry run directives to make sure stuff is working right when I first develop it, so all that is worked out from the very beginning. I also like to develop with a lot of debug output gated behind debug as well, and just leave that there for the inevitable but at some point that would make me add it if not already present.

What I don't like is passing those as params as args. Currying this stuff around is very error prone and cumbersome. I prefer to set environment variables for DRY_RUN and DEBUG (with debug accepting higher integer levels for more debugging).

This works wonderfully for me since I never have to worry I didn't pass something along correctly, I'm always asking the global set at runtime.

Buttons8402• 5 years ago

So, for a top level bit of code to pass arguments to lower level bits of code you set environment variables? Seems a bit odd to me

Using environment variables for the user to configure their environment or pass information into a program seems normal, but code passing arguments to other bits of code in the same language with environment variables seems odd.

d4mi3n• 5 years ago

There are other approaches to this as well:

1. Passing around a configuration or context (similar to your gripe around params)

2. Referencing some kind of global configuration (a la env vars)

3. Referencing a local or scoped configuration (for example, a method of an object can check instance variables that dictate behavior)

I personally prefer either a passed context or a local configuration; I find both easier to test in isolation. Global contexts have their uses, but tend to become problematic when they clash with other libraries or tools that may also be present in the execution environment.

hinkley2• 5 years ago

I have some code that lacks permissions to run to completion except on Bamboo or other servers, so I practically have to have a dry-run mode anyway.

IgorPartola2• 5 years ago

What I like is a system where you first create a plan for what you will do, then there is an executor that will execute the plan. So --dry-run just doesn't run the executor. Of course, depending on what you are working with that might not be possible, but if you can design it like so, do it. It also makes everything nicely decoupled.

cortesoft• 5 years ago

Yeah, not sure how this would work with any workflow that caLLs out to another service and uses the result of that to determine the next step, if that intermediate call changes state somewhere.

klodolph4• 5 years ago

Yes, I like this a lot. You often end up with a main code path that looks like this:

    do a bunch of stuff

    make a plan

    if dryRun {
        plan.Print()
        exit()
    }
    plan.Execute()

renewiltord3• 5 years ago

I'm sure there have been older tools with this, but Terraform is the first I encountered with it and I really like this model.

whateveracct• 5 years ago

Regardless of where you push your run/dry-run dispatch to, the underlying "do it" logic really needs to be factored properly for individual side effects (think functional). Otherwise, you inevitably end up with loads of "if dry-run / else" soup or worse, bugs _caused_ by your dry-run support.

--dry-run is one of the best motivations for using granular, extensible effects

when you constrain literally every bit of IO your program is allowed to do, you can now 100% know you've stubbed them all out when doing a dry run

lytefm• 5 years ago

2. You still need safety nets for when someone is doing dangerous operations without dry-run.

Another important property is idempotency. Especially if the script involves network requests or moving files around, you'll want to reach the goal by re-running in case something breaks half-way.

catlifeonmars• 5 years ago

Is idempotence the right word for this? Honest question. This is more like eventual consistency after a small number of operations, where each individual operations is idempotent.

GuB-42• 5 years ago

Dry run is among the things you really need to design for, others are:

- cancel

- undo/redo

- progress bars

Progress bars with accurate timing are notoriously difficult, or even impossible to get right. But even regardless of timing, having a progress bars that really shows progress and doesn't freeze is hard. Every slow operation has to have some sort of callback mechanism to update progress, and you have to know the number of steps in advance.

Undo requires, for each operation, to know how to roll back. You also need to have every operation go through the undo stack. Another option is to have an efficient snapshot system, which may be just as hard or even harder.

Cancel is actually the hardest to do right because it combines the difficulties of both the progress bar and undo. You have to have a way to interrupt a long operation at any time and get back to before it started. And because "cancel" is most often used when things go wrong (ex: disk full, bad connection, ...) you have to be very careful with error handling.

monsieurbanana• 5 years ago

The thing I've gather from this whole thread is that I made the right decision by choosing a functional, immutable language.

hinkley2• 5 years ago

I've found that you really need to design for it

It's an architectural step a lot of people skip, and by doing so you often end up in a situation where the only way to make things faster is via caching, and chasing caching bugs for the rest of your tenure.

One of the less appreciated aspects of model-view-controller is that usually it demands that you plan out your action before you do it - especially when that action is read-only. Like a cooking recipe, you gather up all of the ingredients at the beginning and then use them afterward.

By fetching all of the data early, you reduce the length of the read transactions to the database, allowing MVCC to work more efficiently. You also paint a picture of the dependency tree in your application, up high where people can spot performance regressions as they show up in the architecture - where there's a better opportunity to intercede, and to create teachable moments.

To make dry-run work you are best served by book-ending your reads and your writes, because if the writes are spread across your codebase how do you disable all of the writes? And if any reads are after writes, how do you simulate that in your dry-run code? It won't be easy, that's for sure.

The problem is that we tend to write code stream-of-consciousness, grabbing things just as we need them instead of planning things out. This results in a web of data dependencies that is scattered throughout the code and difficult or impossible to reason about. This is where your caching insanity really kicks into high gear.

To my eye, Dependency Injection was sort of a compromise. You still get a good deal of the ability to statically analyze the code but you can work more stream-of-consciousness on a new feature. But it does rob you of some more powerful caching mechanisms (memoization, hand tuning of concurrency control, etc)

Smaug12310• 5 years ago

This is the first comment I've used HN's "favorite" feature on. How wonderfully insightful.

suls• 5 years ago

I think this is a case where the Free Monad could help a lot. I've only ever built toy-examples but I really liked the way it allows you to structure the program's plan separately from interpreting it. A dry-run mode could be implemented as a side-effect free interpreter .. without ever having to touch the plan or the program's grammar.

HereBeBeasties2• 5 years ago

These observations speaks volumes as to how testing is often done badly. It's a bit of a generalisation, but if you move your reads/writes to the periphery, the core tends to be pure functions that are easy to unit test without side-effects. If you don't move your reads/writes to the periphery, everything becomes an integration test, with the accompanying brittleness, lack of performance, and resistance to change. TL; DR: Stop designing/implementing things such that you have to mock everything; DI is just a tool, and as such it is used for both good and evil.

ryanbrunner• 5 years ago

One thing that I've used in the past for safety that has been extremely helpful is to always log results to a detail level that allows reversal of whatever you did. For example, for a big DB update, create a CSV (or just write to STDOUT) all affected records along with their old data. If you get into this pattern enough, and your operations are similar enough, you can even standardize on the format so you can have a common tool that can undo an operation for you given the log.

Certainly it's not something that can be applied for every use case, but when it can, it has so many benefits - an audit trail of what actually happened, ability to undo, and a common structure and approach for what dry runs should do (just output the log)

throwaway_2047• 5 years ago

I have yet to see anywhere implemented like this, but IMO mutation/deletion should be on dry run mode by default.

mr-wendel2• 5 years ago

For all you Bash heads out there:

  DRYRUN=1
  VERBOSE=1
  
  cmd() {
      [ $DRYRUN -eq 1 -o $VERBOSE -eq 1 ] \
          && echo -e "\033[0;33;40m# $(printf "'%s' " "$@")\033[0;0m" >&2
      if [ $DRYRUN -eq 0 ]; then
          "$@"
      fi
  }

And now you can put this all over your scripts to very easily implement dry-run behavior without any quoting worries. Note that you can't invoke aliases this way.

  cmd echo "don't worry about quoting"
  cmd do_something_dangerous "$scary_arg" "$scary_arg2"

tremon• 5 years ago

Why not simply

  DRYRUN=echo
  
  cmd() {
    $DRYRUN "$@"
  }

?

mr-wendel2• 5 years ago

The printf version will give you automatic quoting making copy'n'paste really easy to test. It' also mixes well w/ a verbose option for "--verbose" or such.

croehrig• 5 years ago

I use this all the time (I call it $EXEC instead), and have gotten into the habit of starting all my scripts with EXEC=echo and put it on every command that does any file changes. It has saved my bacon many times.

It doesn't work all the time as the previous poster noted, but it is very low friction which is especially important when writing quick throw-away scripts.

SavantIdiot• 5 years ago

Duplicity has this option. I love it. Especially when something bad can happen, like blowing up my disk because an ISO image is in the backup area.

zbentley2• 5 years ago

Even better than a dry-run flag is to have the noop mode be the default behavior and have e.g. "--execute" enable non-read-only behavior (or, for really scary commands like programmatic data destroyers, "--nuke-my-data-i-solemnly-swear-i-know-what-i-am-doing"). Prevents plenty of accidental mistakes by sleepy operators.

That's a quibble though, the article is well-put and quite correct about the value of a dry-run mode.

zbentley2• 5 years ago

Oops, I read the article but not the comments. I see I'm far from the first person to suggest this, apologies for the duplication.

yuchi• 5 years ago

Fantastic article, and I promote a stricter approach: `--dry-run` is the default, and you need to disable with `--no-dry`.

For irreversible (such as deletions) actions I even go one step further with a `--no-dry=<some-non-trivial-value>` in order to force some thinking.

Edit: apparently this is way more common than I thought, by reading comments here!

agumonkey• 5 years ago

I wonder how it would feel to have every program dry-runnable but composable.. basically FP/virtual systems. You can (dry-run cmd-0) | (dry-run ...) | (dry-run cmd-n) and it will yield a potential system state.

notafraudster• 5 years ago

How do you all write dry-run logic for utilities that perform file operations (edits, deletions, creations) and have some kind of sequential dependency in operations such that later steps depend on the previous steps having been executed?

I have found it a frustrating nut to crack because it seems like it needs to involve either writing a simulation layer that tracks the state of all the file calls virtually, or else copy all the files to a temp folder (so the dry run isn't dry, it's just on a separate version of the data). Both of these seem like bad solutions.

programmarchy• 5 years ago

Copying to a temp folder sounds pretty reasonable to me.

lamontcg2• 5 years ago

Yes, if you call a black box outside of your code base which does something then subsequent operations can't easily be dry-run'd.

So if you need to install a package which sets up a systemd service, the subsequent dry-run of managing that systemd service would fail because the service doesn't exist yet. Or you need to make assumptions in the service dry run that it should have been set up before.

You can just assume that the service would have been setup and report that you were, say, enabling and starting the service. Or you could require some kind of hint to be setup from the package installation to the specific service configuration (for well known things this could be done by default, but for arbitrary user-supplied packages this cannot be done). Or you could list the entire package contents and try to determine if the manifest looks like it is configuring a service.

And it still may fail because you don't know the contents of that file without extracting it and the service may not parse at all.

And that's just a simple package-service interaction example. You could spend a week noodling on how to do that fairly precisely, then there's a hundred or a thousand other interactions to get correct.

You're being told not to do the thing, so there's fundamentally a black-box there, and you need to figure out how much you're going to cheat, how much you're going to try to crack open the black box into some kind of sandbox so you can figure out its internals, and how much you're going to offload onto the user. Not actually an easy problem at all.

swiley• 5 years ago

Yeah, the ability to indicate a shell command is pure (or that it must be pure) is something that's really missing in POSIX-like APIs. It's something I've certainly missed. Something like fork that disables write outside a handful of file descriptors (like one the parent starts with popen) would be pretty awesome. Maybe BSD jails do that.

Eridrus• 5 years ago

Besides having dry-run support, I think this gets to the question of what the level of abstraction should be, and I think that is clearly application-specific.

Maybe in your case the level you are trying to report dry-run information at is too granular.

Or maybe you have too tight coupling between your code that determines what needs to be changed and that which actually does the change and you might want to refactor the code that determines changes to the code that makes the changes.

gameman144• 5 years ago

I don't see anything wrong with the "copy to a temporary directory" approach if your function actually does operate on files within a directory. In that scenario, copying to a different directory is actually probably exactly what you want for a dry-run, since then the code that you're executing is the same exact code that would run were you to execute in non-dry-run mode (as opposed to virtual operations which are prone to bugs if that virtualization layer and your real operations ever fall out of sync).

ItsMonkk3• 5 years ago

I don't know of any system that works like this currently, but it seems like you are asking if you can switch branches, do the work, then your "dry-run" is a pull request complete with all files changes which you can approve manually.

mpyne• 5 years ago

My kdesrc-build has a dry-run style of logic, not because of destructiveness but just because it was annoying to only realize after 4 hours of compiling software that a module you needed was somehow not in the build, or that your configuration change caused you to re-download all the repositories instead of pulling small updates.

This is problematic in multiple ways:

1) The script uses a single repository of project metadata to derive the rest of the build. This repo needs to be present and updated to figure out what to do.

2) There are sequential dependencies on a great deal of steps that can fail without much predictability (including network traffic, git commands, and the module build itself). But in a mode where you're compiling every day, even a transient module build failure does not necessarily mean you shouldn't press on to also try and build dependent modules.

I've ended up using an amalgamation of logical steps:

* If a pass/fail check can be performed non-destructively, do it (e.g. a dry run build will fail if a needed host system install is missing, or the script will decide on git-pull vs. git-clone correctly based on whether the source dir is present) * If pass/fail can't be done, just assume success during dry run (e.g. for git update or a build step) * For the metadata in particular, download it to a temp dir if we're in dry run mode and there's no older metadata version available. * Have the user pass in whether they want a build failure to stop the whole build or not.

For separate reasons, I've also needed to maintain logic to help modules with build systems that didn't support a separate build directory to act as if the build was being executed in the source directory (this was done by copying symlinks to the source directory contents into the proper build directory, file by file).

Having this separate, disposable directory made it safer to not have to be perfect on tracking files changes or alterations perfectly. And of course it helped greatly being able to offload the version control heavy lifting to CVS (back in the day, then Subversion and now Git).

wheybags• 5 years ago

I've made a further extension of this principle for a video game release script, by adding a --unsafe flag. The script was responsible for building the game on a bunch of platforms, uploading it to various storefronts and announcing the release on twitter, reddit etc. There was a --dry-run flag too, that just prints what it would do, but sometimes you want to test part of the script without doing anything public. There were various flags like --no-upload and --no-twitter etc, but it was easy to forget them, so I added in a top level --unsafe flag. At every point where we do something public, at the deepest level of the call stack just before doing it, I check for unsafe, and if its not present, throw an exception. It worked pretty well, and definitely saved my ass a few times, especially during refactoring.

aranchelk• 5 years ago

Dry-run can be implemented succinctly in Python with a function decorator wrapping those functions that should run conditionally.

Nice aspects of this technique:

Inside the decorator you have access to both the function’s name and its arguments so you can print descriptive messages to the user e.g. “In dry run, would have run function ‘delete’ on ‘/foo/bar’”

You can use the name of the decorator as de facto documentation, e.g. “@destructive”

kevincox• 5 years ago

Their example of pushing it into the type system is nice but requires a language with a fairly powerful type system. Once simple way to do this in basically any statically typed language is just requiring a "Run Token" for any bit of code that does something "for real". This method works well if you want the output to be similar or identical for dry run and actual run.

    struct RunToken;

    fn do_the_thing(_: RunToken, ...);

The general approach works for just about any statically typed languages (although null is the enemy).

You can make it a bit more tedious to generate that RunToken so that a `do_the_think(RunToken, ...)` looks more natural. But the idea is simply that if the only place you create a RunToken is when parsing the --no-dry-run flag you can't accidentally do_the_thing when you move the code out of the `if is_dry_run` block without noticing.

Of course this is a simple way to get fairly reliable dry run. I do agree that having seperate plan and apply steps is approach is the best when you can do it.

AgentME• 5 years ago

This is almost like one step toward an object-capability system. In an object-capability language, the main function would receive capability objects, like a reference to a module for accessing the filesystem, and functions could only use those capabilities if they were passed a reference to them. You could write your main function so it takes the filesystem module, creates a wrapper around it that only exposes read-only functionality, passes that read-only wrapper around to most functions that build up some kind of plan, and then the main function uses the raw unrestricted filesystem module to execute the plan at the end depending on the dry-run flag. You can then be easily sure that code doesn't do any writes to the filesystem if the unrestricted filesystem module isn't passed to it.

swyx• 5 years ago

all CLIs that do substantial work should have dry runs.

more examples I've collected:

- Angular CLI https://malcoded.com/posts/angular-fundamentals-cli/#--dry-r...

- AWS CLI https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-h...

- Rspec https://relishapp.com/rspec/rspec-core/v/3-8/docs/command-li...

- Serverless framework https://forum.serverless.com/t/dry-run-with-serverless-frame...

at netlify we didnt do a dry run but Netlify Dev is close: https://news.ycombinator.com/item?id=19615546

I'm requesting one at Gatsby: https://github.com/gatsbyjs/gatsby/discussions/16384

slver3• 5 years ago

Dry run is OK, but if we want to look at it closer, it occurs in tools with mixed query + command responsibilities.

For example

    rm PATTERN

Is really a shortcut for something like (pseudo code):

    find -name PATTERN | rm

If things were always factored for command/query seggregation you'd not need dry-run, you'd simply just run the query.

rwl2• 5 years ago

This is a good point, but notice that what makes the segregation easy in your example is that the query and the command have a shared way to refer to what the command will operate on: the filename.

Consider a conceptually very similar case: instead of finding and deleting files in a filesystem, think about finding and deleting lines matching a pattern within a file. Then the query is something like grep...but what do you put for the command on the other side of the pipe?

Of course, you can tell grep to output line numbers and use a command that operates on line numbers, or similar. The point is, in order to achieve this kind of segregation, you need some common way of naming operands on both sides of the divide. And naming things is hard, so segregating things this way is hard, and thus there's a lot of tools with mixed query/command responsibilities.

slver3• 5 years ago

PowerShell solved this problem by outputting easy to map objects instead of just plain text. Unfortunately they made it a bit more verbose than it could've been, and the industry didn’t recognize the potential, so it remained a niche product.

I still love PowerShell.

ItsMonkk3• 5 years ago

Finding is a query. Deleting is destructive. Make sure that you are not nesting your destructions within a query.

You would first build a pure function that read the files and returned matching lines. You could then display the count of the matched lines, and if the user wants they can see those lines and what line number it is. This looks a lot like a dry-run.

You can then pass these results into an executor. You then build a system that combines all deletes of a file and passes that to your DeleteLinesInFile fuction that does so in a single transaction. Cycle through each file and voila.

account42• 5 years ago

rm doesn't do pattern matching though - that part is in your shell so the responsibilities are already separated in this case.

da39a3ee• 5 years ago

This is obvious but a good way to implement dry run in some settings is to obtain a read-only connection to the relevant datastore / roll back a transaction without committing, etc.

xupybd2• 5 years ago

Gresearch puts out some interesting articles, and they work in F#. I wish I were in the UK, I'd love to try and get a job there.

dkarp• 5 years ago

They’re an interesting organization.

See this past submission: https://news.ycombinator.com/item?id=18499712

Smaug12310• 5 years ago

I believe (though can't swear to it) that we do provide immigration/relocation assistance and money, at least for quant researchers and possibly for all employees. If that would be enough to get you interested, feel free to drop me a line at patrick.stevens@gresearch.co.uk and I can find out actual details rather than half-remembered possibly-false snippets from years ago.

xupybd2• 5 years ago

Sadly money is not the issue proximity to family is. I'll just hope one day they open an NZ or south east Asia branch.

JustSomeNobody• 5 years ago

I use --dry-run with rsync almost every time I use rsync. I don't really use rsync enough to remember if I should use a trailing slash or not...

blt• 5 years ago

Abstracting a bit, this is an application of the principle "represent important intermediate results as explicit data structures". It shows up in many places: testing, extensibility, logging, and parallelization are all easier to implement when the result is represented explicitly instead of implicitly in the program's call stack.

Has anyone written about this idea? It seems to have some interesting interaction with laziness.

deckard1• 5 years ago

If you're doing a lot of modifications to a filesystem or a database, one pattern I've used and liked in the past is to have your code simply dump out the commands to run. Usually to stdout and you save it as a script.

That way you don't need to worry about issues with bugs in the dry run logic and you have a record (a script) of what you actually ran.

Even better if you can extend this output script so that it's idempotent and fails on the first failed command.

nerpderp82• 5 years ago

Even better is to invert the flag, that default is to dump the plan and have to pass an `--actually-do-it` to have it make the changes.

ayoisaiah• 5 years ago

This resonates with me. I recently published a cross-platform tool [1] for bulk renaming files with dry run as the default. In this mode, it shows the changes about to be made and any possible problems that may occur (such as overwriting existing files). To actually carry out the changes a flag must be used.

I felt it was a good design decision since the effects of bulk renaming can be substantial. Most other similar tools have it backwards with their dry-run mode relegated to a secondary action.

[1] https://github.com/ayoisaiah/f2

voakbasda• 5 years ago

Simulation is a great thing. I use this wrapper, even in my most trivial shell scripts:

  pretend=${scriptname_pretend:-false}
  run() { echo "$@"; $pretend || "$@"; }

Sure, this doesn't give copy/paste re-usable output, but it's really just a quick and dirty sanity check when things start getting complicated.

gm• 5 years ago

What do you guys think of "dry run by default" where you have to turn dry run OFF via an argument? I've done this on some of my utilities just because screwing up the arguments would result in much grief (lost data).

gorgoiler• 5 years ago

I don’t even trust dry run for some things.

Yesterday I wrote a cleanup tool. In fact it’s two tools: one that outputs a parseable list of things to clean up...

    # snapshotX # keep
      snapshotY # delete

...and another that consumes the list and does stuff. It’s the only way to be sure.

FounderBurr• 5 years ago

One lesson from making tools over the years having dry-run options is to always very explicitly state what you mean by dry-run.

The most common use is as a write/modify gate, but many tools have uses where it may need to avoid certain classes of reads as well (network, db, etc).

rlonn• 5 years ago

I've found that while having a dry-run code path is always super useful, having a --dry-run flag is an anti-pattern. I prefer to make dry run the default, with all state-changing actions being in the form:

if (clear_db_flag) {

  if (verbosity_level > 0) print("DROP DATABASE customers;")

  if (execute_flag) exec_sql("DROP DATABASE customers;")

}

I.e. instead of a --dry-run flag to make the script harmless, I have an --execute flag that weaponizes it. This means that while developing the tool I'm more likely to run in dry-run mode (i.e. without the --execute flag) and so test the dry-run path more than I perhaps would have.

It also means I'm not very likely to delete the production database by accident, through running my script in what I intended to be dry-run mode but actually ending up deleting something I really didn't want to delete.

I think it's a nice methodology, to always have a print statement first, then perform an action that the print statement informed the user about. Easy to remember to add the print statements and more likely that you keep the dry-run code path up to date.

jiggawatts• 5 years ago

In the PowerShell world: "-WhatIf" is your friend.

To implement this in your own scripts is not difficult at all. The template snippet below automatically wires up "-Confirm" and "-WhatIf" support:

    [CmdletBinding(ConfirmImpact='High',SupportsShouldProcess=$true)]
    PARAM(
        [parameter(ValueFromPipeline=$true,Mandatory=$true)]
        $TheThingToProcess
    )
    PROCESS {
        if ( $PSCmdlet.ShouldProcess( $TheThingToProcess, 'Do scary thing' )) {
            # Scary step
        }
    }

That even wires up the full "[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend" logic, so you don't have to track boolean variables all over the place.

It also passes through that preference to commands invoked witin the script, so things like "Format-Volume" will do nothing if you invoke the outer script with "-WhatIf".

In praise of --dry-run

Related Stories