A simple web server written in Awk

For tcp. There's another one which uses GAwk's internal tcp stack, but I wasn't sure if that would work in normal awk, and I'm a purist like that so you got this one.

pmarreck5• 3 years ago

I’d like to see both, please

nine_k• 3 years ago

Socat is a trivial stream converter, like a pipe fitting with different thread sizes on two ends.

The interesting part is the logic and the actual.string processing.

ISL• 3 years ago

My first interesting question was, "How do I open a port in awk?!"

eichin2• 3 years ago

True, but you can go the other way and do it with only socat: socat -v tcp-listen:8888,reuseaddr,fork exec:'cat RESPONSE' is a (Very Static) entire web server, if RESPONSE has 2 header lines, a blank, and an html body. (A little too late in the morning for me to do the golfing needed to eliminate the "cat" and do the cr-nl from whatever shell you're already running, though.)

metadat2• 3 years ago

It's funny how closely this resembles nginx.conf.

dosk• 3 years ago

23 years earlier https://marek.terminus.sk/prog/hawkh.tgz

homepage https://marek.terminus.sk/prog/hawkh.shtml

metadat2• 3 years ago

Marek's achieves it without socat, using only cat and sh built-ins. Badass OG.

calvinmorrison• 3 years ago

then on top of that you can run werc a web framework written entirely in rc, save for a few of the filters which i think are awk files (like the markdown2html converter)

mdale• 3 years ago

Awwwwk such a cute web server :)

iamcreasy• 3 years ago

Interesting, but what is a TCP wrapper?

kstrauser2• 3 years ago

Something that handles the TCP connections and passes them to the wrapped program’s stdin and stdout. Back in the day, lots of network daemons were written as command line tools and executed via inetd.

kqr• 3 years ago

Is there a modern solution for this that isn't some variant of FastCGI? I know it's inadvisable but sometimes I would like to write a proof of concept that doesn't require me to code a reliable, long-running process.

dmux• 3 years ago

If you enjoy stretching what you can get done with limited environments, I recently discovered BusyBox includes not only an AWK implementation but an HTTP server[0] that supports CGI as well. I spent a weekend setting up a really simple web-app using recutils [1] as the database and BusyBox AWK as the programming language.

If you enjoy bricolage and you feel like wasting a weekend, I would recommend giving it a try!

[0] https://openwrt.org/docs/guide-user/services/webserver/http....

[1] https://www.gnu.org/software/recutils/

Aachen4• 3 years ago

That looks so featureful (thinking of attack surface here), you might as well install a regular web server at that point. What's the benefit of using busybox for this?

Edit: ah is this file size? Or does this refer to RAM? Either way, sounds like that's the benefit

BB httpd can be compiled with only basic features like CGI and ETag and will have only 8Kb

Crestwave2• 3 years ago

BusyBox provides a complete set of POSIX utilities and more in a single minimal binary; there are lots of embedded systems with pretty much just BusyBox installed, although I'm not sure how common it is to compile it with httpd. GP linked an OpenWRT article, so I'm guessing (the site seems to be down) it's included in that OS.

manonthewall• 3 years ago

Busybox is certainly a gem.

notorandit• 3 years ago

You can achieve the same in bash. With no so at.

cf100clunk6• 3 years ago

Could you post a bash string or script that does this?

LispSporks22• 3 years ago

I thought you could listen() accept() in modern awks.

tyingq6• 3 years ago

Sort of. Gawk supports magic file names like "/inet/tcp/8080/0/0" to be a tcp client or server, but you don't get control of accept()...it's all mooshed together. So you can make a slow, single threaded webserver with it.

Which is too bad, because the usually shipped gawk extensions also give you fork. If they had separated listen() and accept(), you could actually make a reasonable webserver. See my other comment for an example of what you're limited to.

Yannik_Sc• 3 years ago

If I got this term correctly, these days it would rather be a "Serverless webserver in awk". Also sounds more catchy.

hackideiomat4• 3 years ago

gsub(/\/\.\./, "/", request_filename) # avoid directory trasversal

Hmmmm

http://localhost:8888/..../..../..../..../..../..../.../.......

tmsbrg• 3 years ago

damn, you beat me to it.

Was gonna write:

http://localhost:8888/..../..../..../..../..../..../etc/host...

mypc

These regex substitutions are so easy to bypass :)

hackideiomat4• 3 years ago

Always fun :D

x-complexity• 3 years ago

All this tells me is that preventing directory traversals can only be done by checking absolute file paths are within a bounded range, and nothing else.

nonameiguess• 3 years ago

Running the server as a service account that can only read its own directories, running it in a chroot, running it in a mount and pid namespace, using SELinux to further restrict what files it can read even in principle.

Of course, if you're trying to go superminimal anyway, it's not that big a deal to create a server that doesn't even have sensitive data on it. You can make init simply mount a root filesystem that only has busybox and whatever files you want to serve and starts up the httpd process and nothing else. Turn Linux into a unikernel basically. If you compile busybox yourself, you're also able to remove all the subcommands you don't actually need.

pmarreck5• 3 years ago

change that to:

gsub(/\/\.\.+\/?/, "/", request_filename) # avoid directory traversal

source: am "regex expert" >..< (and know how to spell)

hackideiomat4• 3 years ago

U sure? echo -e "GET /../.. HTTP/1.0\r\n\r\n" | nc localhost 8888

tyingq6• 3 years ago

Ah, I had posted a golfed silly gawk webserver here some time ago. It uses gawk's built in tcp.

https://news.ycombinator.com/item?id=22085459

Pretty printed: https://gist.github.com/tyingq/4e568425e2e68e6390f3105e58878...

tripflag• 3 years ago

neat! i was aware of bash's built-in tcp client at /dev/tcp, but gawk being able to function as a tcp server is way cooler. Thanks for that bit of knowledge :>

dzove8552• 3 years ago

Actually, bash has now a loadable builtin to listen on tcp socket. I wrote a script and the patch. https://github.com/dzove855/Bash-web-server

The patch is now included in bash 5.2

Aachen4• 3 years ago

What does the loop and sleep 1 do? Is that to respawn upon crashes (why'd it crash?) or does it exit socat after handling a request?

I recently made a netcat webserver returning only one static response just to have a tiny info page for my new email server, that needs a while true loop but no sleep. I then benchmarked the performance and was very surprised to find that a slow VPS manages [spoiler answer] https://lgms.nl/p/cau/?b64&bY%2FBTsQwDER%2FZbivViDxAxw58Q1pO... Performance graph: https://snipboard.io/Vtn0MO.jpg

tyingq6• 3 years ago

It's a one-shot awk script. Handle one request and exit.

Aachen4• 3 years ago

Huh okay, then why the sleep?! That kills performance but why'd you do that

drzaiusx11• 3 years ago

For those who like simple shell tools taken to extremes, this sed script implements "Super Mario Level 1 for the NES"

https://github.com/chebykinn/sedmario

wigster• 3 years ago

crazy. lol

thricegr8• 3 years ago

I know this seems the antithesis, but I've always wondered if you could fold and mangle cURL to actually act as a web server?

tyingq6• 3 years ago

I don't think so. Poking around the source code, the only thing exposed in the cli client around listen() and accept() is related to FTP, because of the way FTP works. It does mean, though, that libcurl has functions like Curl_conn_tcp_listen_set() and Curl_conn_tcp_accepted_set() that could be used for what you're describing. It's just that they are only used for FTP now.

hi41• 3 years ago

I love when people do projects with awk! I received my awk 2nd edition book just yesterday! I am so excited about reading it!

abecedarius• 3 years ago

Cool news -- I liked the first edition a lot back in the 90s.

pmarreck5• 3 years ago

I've heard of Awk for years and (probably like many) only used it for single-line snippets for the vast majority of that time.

Imagine my surprise when I just decided to look into it one day (after finding slightly more complex Awk scripts that did a lot with very little code which piqued my curiosity) and finding this very nice line-oriented DSL that has aged SHOCKINGLY well given how old it is.

I just wish it had interrupt handling of some sort without running a custom fork/patched version

debarshri• 3 years ago

It must be a very awk-ward service to deploy.

A simple web server written in Awk

Related Stories