Bash_unit – bash unit testing framework

https://github.com/pgrange/bash_unit

by arunc • 5 years ago

102 80 5 years ago

github.com

chovybizzass• 5 years ago

haha. i love it!

atsaloli2• 5 years ago

ice3• 5 years ago

I'd recommend checking out this fork instead: https://github.com/bats-core/bats-core

atsaloli2• 5 years ago

Nice! Glad to know development is continuing. Thanks!

williamdclt• 5 years ago

I used bats a while ago to test a homemade CLI to open github pull-requests, it worked well enough for me (https://github.com/williamdclt/git-pretty-pull-request if you're looking for an example, although I did not maintain the tests out of laziness so they are very red).

Cannot compare it to bash_unit, but I'm happy there's alternatives!

dnautics• 5 years ago

Yeah i think most people looking for this don't necessarily care about the distinction between unit/integration (i myself don't really know anymore because so many of my elixir "exunit" tests hit the db -- it's fine) they just want something TAP compliant and maybe also offers green dots.

lelanthran• 5 years ago

Funny, you can read the title as an instruction and it still sorta makes sense :-)

ahmetkun• 5 years ago

I guess another good example of that would be lobash. Assuming you're familiar with lodash, of course.

louwrentius7• 5 years ago

I appreciate the effort but I think that if you are unit testing shell scripts, I think it’s probably better to use a language like Python or something similar.

I am serious.

I have decided for my own efforts to use python for any kind of command line script and I’m always glad I did.

fnord123• 5 years ago

bats is used for running tests against command line interfaces of your program. This is useful for making sure weird strings and argument patterns are handled correctly.

loloquwowndueo• 5 years ago

I still find it a bit awkward to use python just to run some commands in sequence with some massaging of input/output data and parameters based on simple logic. bash scripts are great for that. And in that case it’s still a good idea to have automated testing instead of relying on running the script a few times to ensure it behaves as it should. I’ll definitely give this a try :)

I do agree though that if you need extensive massaging of output or arguments, python can help and make the whole thing easier.

disgruntledphd22• 5 years ago

The trouble is that bash is necessary to setup python. For instance, the lingua franca of Docker is bash (well sh, but you should obviously change that).

I've been resisting learning (any more) bash for a while now, but I think I'm going to have to. This looks like it could reduce some of my terror at the insanities of shell scripts (why does one even need two quote characters that do different things?)

ducktective• 5 years ago

Is there an elegant way to do unix-like piping in python?

esamatti• 5 years ago

"sh" for python is actually pretty cool

https://amoffat.github.io/sh/

Available as `python3-sh` in Ubuntu.

louwrentius7• 5 years ago

If it’s just a few lines or some one-off thing, I understand the use of bash.

I start to get the feeling that if you feel the need for shell scripts, it might be wise to pause and wonder of this is really the right approach long-term. Especially if you feel the need to put it in git or something.

My experience is that there are often I need to put in some checks for safety and before I know it, you create a mess of grep awk cut sed and you wish you started out with python.

Are you really that much in a hurry or do you have the time to calmly spend a little bit more time to ‘do it right?’

sethammons4• 5 years ago

I agree that you should go to a more robust language when shell scripts get bigger, esp if you are needing or wanting unit tests. I would avoid Python if there are lots of dropping into the subprocess module; it suffers the same problem as Go - lots of ceremony and boiler plate to run shell commands. I recommend Ruby or Perl because they can drop into a shell-like mode and you can execute shell commands more “naturally.”

louwrentius7• 5 years ago

Ar this point anything else seems better that shell scripting. Sure not brainf@ck or COBOL, but I think the point is clear.

torvaney• 5 years ago

In a similar vein, there's also babashka for these "bash+" use-cases: https://github.com/babashka/babashka

Huffers2• 5 years ago

Agreed entirely. I inherited a system with 50k lines of unreliable shell scripts. I figured out ways to unit test them, do mocking, and pushed developers to unit test their scripts. I even wrote long articles of best practices for writing reliable scripts, including massive lists of gotchas.

Eventually I realised it was a lost cause and really you just shouldn't use shell scripts for anything that you want to be reliable above a trivial level of complexity.

That was with PowerShell, but I wouldn't be surprised if the same applies to bash as well.

majkinetor3• 5 years ago

That is what you get when you let anybody write scripts without code review.

Bash people are the worst when they switch, since they keep the same mentality and continue parsing strings or use other bashizms.

See my answer above also.

louwrentius7• 5 years ago

With Powershell it is possible to write decent code imho.

Powershell is on another plane, more like Python and less like bash shell scripting.

I used Powershell and Pester a lot and it worked great.

Bash becomes a mess of cat grep sed awk cut

dcolkitt• 5 years ago

One of the biggest downside is that the startup time for python is significantly slower than the shell. With just one script this isn't really noticeable, but if you're composing a lot of small scripts together, python becomes noticeably less responsive for interactive commands than bash.

Along similar lines bash's syntax is incredibly streamlined for composing standalone scripts and programs through pipes. A simple bash one-liner like the below would be much more awkward to write in python:

diff <(netcat $server | grep town | sed 's/street/St' | cut -f 3 | head -n 5) <(cat ./$(psql $query).dat)

louwrentius7• 5 years ago

I wonder if that ‘lots of small scripts together’ is a desireble situation. Can’t it be just one app that performs all the steps?

It is all about context to me. A shell oneliner is not something I would replace with python but as soon as you start up an editor, think again I would say

majkinetor3• 5 years ago

We have tones of PowerShell and it works like a charm, although we have some experienced posh devs

We also test REST backend in PowerShell using Pester and home made Posh rest client.

PowerShell is preferred in this house because

a) you can run it on any Windows OS on the spot and modify it in ad hoc manner, you can even debug it with breakpoints etc easily. Also our Linux machines have it so it is unifying admin interface.

b) its powerful, you can do anything in it with few lines of code (one case: we did 10 million SOAP requests using certificate per day for entire country)

c) many Windows tools use it like SqlServer, IIS etc. which makes management way easier - for example we use [1] to install sql server on all dev/prod machines or use [2] to monitor all our servers or use [3] to send CI metrics to influx (all those are just minor samples, we have bunch of stuff like that)

d) we find it way easier to keep CI/CD vars in PowerShell hashtables then in yaml, so our yaml fiels are one liners and everything works locally.

e) Python, ruby and friends are NOT designed for shell work. Its akward, unfriendly and most of all not there on Windows OTB.

---

[1] https://github.com/majkinetor/Install-SqlServer

[2] https://github.com/majkinetor/flea

[3] https://github.com/majkinetor/psinflux

a1o2• 5 years ago

Hey, since you appears to use Chocolatey and there's few people using it, how do you feel it? Do you think it will keep existing? I am not that into the way MS is doing the winget thing, but I also noticed it appears to impact how people adopt or not chocolatey. I see a lot of manual scripts for installing things on Windows CI systems (some using now defunct fciv and soon to be defunct bitsadmin)... I really like chocolatey but I am worried it will disappear soon.

louwrentius7• 5 years ago

Powershell is an entirely different matter to me.

Powershell is so much powerfull, I’ve written a ton of code in that and also used Pester for unit testing.

Entirely different world compared to shell scripting.

CraigJPerry2• 5 years ago

There’s clearly some good thought gone into this - i especially like the ergonomics of the “fake” command and arguments.

However, having been down this path myself, i don’t think it’s a great idea.

Beyond the hello world examples you start to run into the need to defang commands being tested or getting better visibility of what they’re doing just so you can make a useful assert, you end up LD_PRELOAD’ing shims to intercept calls and it gets a bit horrifying, very quickly.

acdha• 5 years ago

I agree: this is great as an intellectual exercise but I’m reminded about how many times I’ve been glad that I set a policy in the 2000s that any bash script too large for a normal size screen should be rewritten in Python.

fooflarney• 5 years ago

or just make

JackuB• 5 years ago

If you fancy more BDD-style testing with a Gherkin-like syntax, there is also Shellspec https://shellspec.info / https://github.com/shellspec/shellspec

ryanianian• 5 years ago

Shellspec is delightful to use. Every time I go to test with it I end up finding bugs. But also every time I go to test with it I end up convincing myself to not use bash. I imagine the same would be true of bash_unit.

justinholmes• 5 years ago

Shunit2 as well

https://github.com/kward/shunit2

actionowl• 5 years ago

If you fancy posix shell (and C/C++ support) https://github.com/jmmv/atf/

oweiler• 5 years ago

I've built a Bash testing framework myself, and my realization was that Bash is a bad language for this.

https://github.com/helpermethod/bash-specs

a1o2• 5 years ago

Hey, any ideas how to leverage your current testing framework to add coverage or if there's any pretty way to print coverage results that are somehow as standard as TAP?

bewuethr3• 5 years ago

I have successfully used https://github.com/infertux/bashcov in the past.

jasonpeacock3• 5 years ago

For everyone who's saying "if you need unit tests then you should rewrite it in a different language", you're not wrong but you're not right either.

Bash is a lowest-common-denominator language, available on almost all platforms, with a very stable (old?) API, which makes it ideal for broad distribution & bootstrapping of systems.

When you use another other languages, like Python, suddenly you're not only worrying about whether Python is installed, but also which version is available, and you still can't use anything that's not part of the standard library (because then you enter package management/dependency hell).

I agree that even Python with stdlib-only is still better than Bash, but I guarantee that you'll find Bash >=3.0 on every host.

titzer• 5 years ago

I used bash to write a testing framework for a project I have that needs minimal dependencies. E.g. I don't want to depend on having Python (and the transitive closure of its batteries-included experience) installed. For that purpose, bash works pretty well. I also want to run on some older machines, like 32-bit macs.

Turns out that bash does evolve...slowly. E.g. I found out that the default bash on MacOS 10.4 doesn't support the "=~" operator...oi...

arendtio• 5 years ago

In addition, the default MacOS Bash is buggy :-/

hiq2• 5 years ago

I agree that even Python with stdlib-only is still better than Bash, but I guarantee that you'll find Bash >=3.0 on every host.

Shouldn't you go for POSIX at that point? I'm not an expert, but I still have some idea how to test that my script is (somewhat) POSIX compliant (e.g. use shellcheck, run the script with dash or a shell in POSIX-compatible mode). I have Bash 5 on my Debian, how do I test that a script I tested with this version works on Bash 3 as well? Or have the changes between Bash 3 and Bash 5 been so minimal that I would have to do it on purpose to find an incompatibility?

I'm also curious about the number of machines where Bash scripts are run but don't have, say, Python 3.4 installed, and projects where it's a better trade-off to spend time developing and testing in Bash rather than have a Python 3 dependency. I'd say it's relatively rare.

chubot2• 5 years ago

Bash also has a -o posix option, which changes a number of things which may be significant, but I rarely see it used.

But none of those 3 solutions really check for POSIX compliance (including -o posix, since of course you can use many non-POSIX features when it's on). I am not aware that ShellCheck actually checks POSIX compliance; I am pretty sure it just checks for common errors in your bash/sh scripts (which is what it says on the home page).

I think what you may mean is "portable shell", i.e. portable between shell X and Y and system X and Y. That is a decent goal but many people use bash just because it alone is portable enough! The system tools are often unportable though. Limiting yourself to POSIX in that case is pretty painful and also virtually untestable. It's better to say "portable between X and Y" because I think that's what you mean.

In other words, there is plenty of stuff that's portable but not POSIX that you probably want to use. local variables are probably the biggest example. Every shell I know of supports those, including dash, but it's not POSIX (though maybe they're thinking about adding it; the spec is pretty behind)

I addressed this somewhat here: http://www.oilshell.org/blog/2021/01/why-a-new-shell.html#li...

ian-g• 5 years ago

I had something like this in my last job: I had to create an installer for a VCS aimed at chip designer.

The installer would be ignored as much as my coworkers could manage, and as many customers as possible had to be able to just use it unmodified. Since our supported OSes were RHEL6, RHEL7, SLES12, and SLES15, the best option was bash. Also had to build it on the off chance one of our more restrictive customers was still on RHEL5.11

Pain in the butt, but it worked.

ko1nksm• 5 years ago

There are platforms where bash is not available. I wonder why people who want to run it on various platforms use bash.

ShellSpec (https://shellspec.info/) is a POSIX compliant testing framework supports all POSIX shells (Bash >= 2.0, dash, ksh88, zsh >=3.1, etc). I'm the author of ShellSpec.

cle2• 5 years ago

What code are you writing that is pure Bash? Every Bash script I've written or find in the wild is always used to control subprocesses, and then you have to worry about compatibility of those programs, which are all over the place.

Python stdlib does way more than Bash, it's generally easier to just install Python and run a script that sticks with stdlib, than to install Bash and then figure out what subprocesses it invokes, what packages those map to in the OS package manager, what versions they are, if they're compatible with what you wrote, etc. And if you need more libs, Python also has a cross-platform package manager with a consistent interface. As an example, consider coreutils between macOS and Linux--macOS ships with dramatically different coreutils programs (such as "ls"). The only feasible way to get them consistent across OSes is to stuff everything in a Docker image, but that has its own problems and limitations. Or ask the user to figure it out for themselves, e.g. "use Homebrew".

jasonpeacock3• 5 years ago

My point is that Bash is already installed almost everywhere, including across WSL, Linux, and OSX.

Yes, when working with sub-processes you need to account for platform differences...which is also true when Python.

A Bash script "just works", while I have to direct customers to install the appropriate version of Python (or whatever else).

simias• 5 years ago

Bash is a lowest-common-denominator language, available on almost all platforms, with a very stable (old?) API, which makes it ideal for broad distribution & bootstrapping of systems.

/bin/sh is, bash isn't. Portability is a good excuse for using /bin/sh, bash is just a crappy middle ground, it's not as portable as the POSIX shell and it's still not a decent substitution for a proper scripting language.

meddlepal• 5 years ago

Portability is not important in most cases. Most shell scripts are written for and run on a single platform usually as setup or teardown for something else.

tyingq• 5 years ago

Bash isn't installed default on OpenBSD or FreeBSD, and the bash on MacOS is a very old version. Also not installed by default on Alpine Linux, which would be fairly popular in the container world.

prussian• 5 years ago

When I was writing my IRC Bot In Bash, I was looking for something akin to this. for the most part I got away with simple mocking for the IRC bits, but all the functionality (plugins & commands) is still untested. Around the time I created it bats was a thing, but it didn't seem that unobtrusive. this seems more in line with how I'd probably do testing for bash, though at this point I'm not so sure what I'd use it for. Maybe It's worth going over some personal or work projects and trying this out.

jascii• 5 years ago

On one hand, like a lot of posters are saying, maybe use another language if it gets complicated enough to require unit testing.

On the other hand, a lot of the stuff that gets written in bash is quite mission critical, so increasing test coverage is a good thing!

pdkl952• 5 years ago

On the 3rd hand, a major aspect of the 'engineering' part of being a 'software engineer' is learning how to choose which side of various tradeoffs is appropriate for a project. Should we perform the expensive computation on demand at runtime, or is trading time for space with a lookup table more appropriate? Is the task simple enough for a basic 'glue language' like bash, or does it require a full-featured language like ruby or python?

Regardless, as you pointed out, regardless of the language some sort of testing is still a good idea. Even small, supposedly 'trivial' scripts deserve some testing!

synergy202• 5 years ago

posix sh, be it dash or ash, can do 90% of whatever bash does and is way more portable.

chimpansteve• 5 years ago

I'm not sure I understand this. Those shells only have 90% of bash functionality, but are somehow more portable?

I get the POSIX argument, but I'm not sure I agree. In my experience at least, POSIX is less supported than bash.

I'm pretty confident that any random terminal I have to ssh into will have a bash shell. I ssh into a lot of terminals.

I know where you're coming from, it just doesn't chime with my personal experience.

synergy202• 5 years ago

on embedded boards which are deployed 100x more than desktops and are typically resource restricted you have to use posix shell instead of the mega-bytes bash there.

Bash_unit – bash unit testing framework

Related Stories