Show HN: A little web server in C

https://github.com/robdelacruz/lkwebserver

by robdelacruz • 3 years ago

178 111 3 years ago

github.com

A little web server written in C for Linux.

Supports: CGI, Reverse Proxy.

Single threaded using I/O multiplexing (select).

rahmeero• 3 years ago

Interesting. Would be good to get more info, beyond having to read the code.

Curious to know how it compares to micro_httpd [1] which is about 200 lines of C. Or others like thttpd and tiny_httpd.

[1] https://acme.com/software/thttpd/benchmarks.html

forgotmypw17• 3 years ago

Thank you so much for sharing this!

This is exactly the type of web server I'm looking for my project.

lionkor• 3 years ago

Shameless plug for when I attempted something similar, though multithreaded with pthreads: https://github.com/lionkor/http

sgloutnikov• 3 years ago

Here's also althttpd [0]. Heard Richard Hipp mention it and fossil in an interview.

[0] https://sqlite.org/althttpd/doc/trunk/althttpd.md

qwertywert_• 3 years ago

The amount of written functions/libs for opening c sockets has to be in the 100,000s at this point. And they are all slightly different.

skulk• 3 years ago

Cool project, but this project demonstrates the reason I've stopped writing things in C. The standard library has garbage string functions and it seems every project has its own version of this file:

https://github.com/robdelacruz/lkwebserver/blob/main/lkstrin...

It's fun to write this (and read others' versions) the first 3 or 4 times, but it gets old quickly.

xmonkee• 3 years ago

I have the same issue, but I blame the absence of good package management. If it had that, one of the thousands of these libraries would have won out and become quasi-standard.

le-mark• 3 years ago

I always thought the glib part of gtk was really nice for this, but adoption is spotty at best.

alex_lav6• 3 years ago

This is how I felt writing Go. Writing the same nongeneric functions with slightly different type signatures and/or endless interfaces and/or interface{} signatures. It's 2023.

2h3• 3 years ago

never mind than MANY people dont need generics, and that generics have a significant compilation and runtime cost, in terms of time and memory. who cares right?

and never mind that Go has had generics for over a year now right? sometimes having a small, stripped down language is better than having a huge bloated monster. I would point to examples, but you know what they are.

earthboundkid2• 3 years ago

You know Go has had generics for the last 15 months, right?

dajtxx• 3 years ago

I know it's not your code, but related to this comment, it looks like it missing a check for lks == null?

https://github.com/robdelacruz/lkwebserver/blob/main/lkstrin...

denotational• 3 years ago

That assert is checking a library invariant; it should never fail unless there’s a bug in the string library itself (although I’m not entirely sure this string library would tolerate a malloc failure from a quick glance through).

This is distinct from checking the parameters; if lks is null then the user of the API has made an error. Some libraries may sanitise user parameters, others don’t. At any rate, an assert would be the wrong choice to check user parameters since this would result in a (recoverable) user error leading to an abort unless the assert is disabled at compile time (-DNDEBUG), returning an error would be a better choice.

robdelacruz4• 3 years ago

Thanks, much of the asserts were scaffolding while I was programming the helper functions. They function similar to comments, reminding me of the internal conditions that should always be valid.

userbinator• 3 years ago

That's the "lazy, dumb" way of doing it --- write another string library. A much better way is to design your algorithms so they need a minimum of string manipulation, which is unfortunately on the more difficult side for text-based protocols like HTTP.

Personally, I wish HTTP messages were closer to something like ASN.1 DER; there's little in the way of string manipulation necessary for those, and all the lengths are prefixes instead of "try to find the terminator" (and don't forget to not run past the end of the buffer...)

jeremyjh• 3 years ago

Peak HN right here folks. Pack it in we are done.

jerrysievert• 3 years ago

or just parse in place: https://github.com/celcius-labs/tiny-http/blob/master/src/ht...

pjc503• 3 years ago

ASN.1 DER

This has also had serious security bugs because it's so hard to understand.

krapp• 3 years ago

It's fun to write this (and read others' versions) the first 3 or 4 times, but it gets old quickly.

I mean, you can always just use an existing string library or reuse your own. There's no reason to rewrite string operations for every project.

samtho4• 3 years ago

This may sound sort of “old man waves at cloud” of me but one thing I’ve found sad is the gross over-complication of later versions of standards such that the sort of project linked here may not be as practical for something like HTTP/3 for example. Similarly, the large, muddled tool chain that is “required” to make modern JavaScript applications makes it hard for newer learners to really understand what is going on because the minimal code version still needs its own transpiler, build system, linter, process managers, etc. Maybe we need all this complexity, but I suspect that some of the overzealous, solve-everything systems design we have come accustomed to is mainly serving to create a larger problem set instead of creating elegant abstractions that are agreed upon.

skaushik922• 3 years ago

I see what you’re saying and agree that HTTP3 is complicated but I would that since it’s a backwards compatible standard, the added complexities are completely optional. For most use cases the basic protocol is perfectly suitable and only as the scale evolves does it require the additional complexity.

samtho4• 3 years ago

I understand what you’re saying, but if someone decides to post a link to their project that is an HTTP/3 server in under X lines of code but only implements HTTP/2 features, is it really an HTTP/3 web server?

GuB-42• 3 years ago

The big thing that makes HTTP/2, HTTP/3 and even Gemini hard to implement is TLS. I theory, HTTP/2 don't require it, but in practice, it does.

You need to implement a variety of ciphers, manage certificates with expiration dates, etc... And if you wanted to implement all that yourself, people will yell at you for doing your own crypto. So yeah, you need a library. But not just that. You need a way to update your certificates, so you can't have a package (or even a single executable) that you can just run and have a server that serves static pages. You could make a self-signed certificate that lasts a thousand years, but good luck getting it accepted.

wahern2• 3 years ago

Generally speaking, you also need some sort of an operating system to make use of HTTP, and yet that doesn't figure into the complexity of HTTP.

In classic HTTP TLS was layered beneath it, providing important degrees of freedom, including the freedom to not use TLS, which can be especially important for experimentation and development.

Prediction: If HTTP/3 manages to substantially replace classic HTTP+TLS, QUIC is destined to become a kernel-provided service like TCP, shunting all that complexity behind an OS abstraction and freeing user space. The fact QUIC uses UDP is an important aspect here because a performant userspace QUIC stack conflicts with classic, high-value abstractions like file descriptors and processes; abstractions which make it viable (i.e. cheap) to have a rich, diverse ecosystem of languages and execution environments in userspace. More importantly, HTTP will have come full circle.

1vuio0pswjnm7• 3 years ago

Seems like there are some unspecified assumptions here. For one, an assumption that TLS is the only game in town. Another is that "you", as in "you must do this" or "you must not do that", applies to every person equally,

When one uses TLS today, chances are very good that it's using something from djb, someone who "made his own crypto". Maybe the assumptions, stated as "rules", do not apply equally to everyone. For example,

"And if you wanted to implement all that yourself, people will yell at you for doing your own crypto."

In fact, before HTTP/2 existed, djb did exactly that, as a demonstration that it could be done.^1 It succeeded IMHO because it worked. People can "yell" all they want, but as above, these same people if they use TLS are probably using cryptography developed by the person at which they are "yelling". Someone who broke the "rules". Perhaps there is evidence that HTTP/2 would exist even were it not for the prior CurveCP experiment. But I have yet to find it.

The word used in the parent comment was "implement" and the suggestion is that attempts to "implement" would not succeed. Perhaps the reason they might "fail" is not a technical one. Perhaps "success" in this instance really refers to acceptance by certain companies that are making "rules" (standards) for the internet to benefit their own commercial interests. It may be possible to implement a system that works even if these companies do not "accept" it. If so, then the problem here is the companies, their fanboys/fangirls (watch for them in the comment replies), and the undue influence they can exert, not the difficulty of implementing something that works.

IMHO, getting something "accepted" by some third party or group of third parties is a different type of "success" that getting something to work (i.e., "implementing"). It's the later I find more interesting.

1. https://curvecp.org

sharikone• 3 years ago

Some elegant abstractions are created too. JS modules, for example. Or, in another field, the Language Server Protocol.

But yes, humans are humans and everything that is "simple" will get built upon and then become the backbone of something complex that will engulf and smother it as it evolves.

kragen2• 3 years ago

i'm concerned about this too

google isn't really concerned about creating elegant abstractions so much as they are about improving the performance of their browser talking to their server, though sometimes these do coincide

paulddraper2• 3 years ago

It can be simple, or it can be fast.

Or it can be neither. But it can't be both.

If you want fast web at all cost (and you need encryption), you get HTTP3/QUIC.

samtho4• 3 years ago

At the end of the day, the question is never one debating simplicity. If you consider the amount of complexity implemented just to even boot into an operating system to run these systems, the lack of simplicity is already a forgone conclusion. The crux of issue is going back to abstractions and my worry is specific to how our abstractions are getting larger and larger and doing more work at even level with very tight coupling rather than creating systems that use more levels as needed.

jacquesm3• 3 years ago

Similarly, the large, muddled tool chain that is “required” to make modern JavaScript applications makes it hard for newer learners to really understand what is going on because the minimal code version still needs its own transpiler, build system, linter, process managers, etc.

We built pianojacq.com without all that, just plain vanilla JS and a minimum of dependencies, see: https://gitlab.com/jmattheij/pianojacq

cpach2• 3 years ago

Hey, I really like the choice of default song you chose :)

treve• 3 years ago

I relate to the general concern about complexity creep a lot, but in the HTTP/1/2/3 case I'm just not terribly worried. So much effort has been put into backwards compatibility. The only potential true new concept that comes to mind is PUSH, which clients can ignore and mostly failed. HTTP/2 has been around for almost 10 years and I haven't seen a single implementation require it.

ttymck• 3 years ago

This is a result of trying to make HTTP "do everything well", right? Instead of leaving it to each application to select a layering of successive protocols tailored to it's needs, everything has been jammed into the HTTP spec, which has the nice property of giving you all its features in web browser clients (which is the delivery mechanism for this wide array of apps).

Certainly some applications need not have access to the entire HTTP suite. If the goal is not to offer a "full-featured" client then an HTTP client may not be terribly difficult, it can just fail gracefully if the other party tries to do something it doesn't support?

raxxorraxor2• 3 years ago

I will yell at the cloud too. I get why TypeScript exists. For large projects it really helps.

But loosing one of the main benefits of scripting languages for some convenience? You could argue that you use webpack anyway. Same question applies here...

If you have to use JS anyway, be glad that you can have an extremely shallow toolchain for deployment for smaller projects. I am not a JS dev, I just use it for stabbing at bits occasionally.

adamrezich3• 3 years ago

this is a great opportunity to ask something I've been wondering about for awhile:

what are the best options out there for hosting websites built as HTTP-serving executables (either Windows or Linux)? is it possible to do this relatively cheaply?

I ask because I've been working on a framework[0] for building websites in a compiled language recently, and while it's been a ton of fun to build and test locally as a hobby project, I have absolutely no idea if it's even remotely financially viable to host a (small- to medium-sized) website made this way, compared to all of the managed hosting solutions out there for PHP/Node/etc.

I don't want/need to pay for a whole dedicated server—I just want to serve HTTP (eventually HTTPS) from a single executable, using one or more SQLite database files. ideally, it would cost as close to your typical shared PHP host as possible.

I have almost zero experience with "cloud" hosting—I made a small game with Node on Azure years ago, and accidentally racked up charges just playing around with it in development—so I don't know if this, or AWS, or whatever else is a viable solution for this. I've seen that it is indeed possible to host a single executable on Azure, but I haven't actually tried it myself, or determined what the pricing for this would end up being.

[0] https://github.com/rezich/Newf

torstenvl• 3 years ago

Not exactly what you're looking for, but I'd use lighttpd and a CGI program.

zokier• 3 years ago

For AWS simplest thing is to throw it into docker container (`FROM scratch` can work) and run in Fargate (AWS' container runtime). But its not super-cheap solution; baseline cost is something like $20/mo for Fargate and additional $20/mo for load-balancer. Lambda can work too, but it is in some ways bit more involved.

Cheapest AWS solution would be just EC2 instances (=basic virtual machines). t3a.nano instances cost just $3.5/mo and do not require additional load-balancers.

The modern cloudy approach would be to look into stuff like CloudFlare Workers, iirc they can run WASM, so if you manage to compile your code to that then it might work.

schemescape• 3 years ago

Maybe look at container hosting with your (statically linked) binary being the only thing in the container?

Otherwise, why not just get a cheap VPS and host the binary there? I’ve used Vultr and it’s $3.50/month all-in, at the low end. There are even cheaper providers, although I don’t know about their quality. I bet this option would be the cheapest.

This is the beauty of a single binary—it’s trivial to deploy!

adamrezich3• 3 years ago

that was my thought exactly! I wonder if we're ever going to see a shift toward single-compiled-binary websites, away from increasingly complex deployments that exist solely to facilitate interpreted language stacks.

this is the question driving the framework I'm building—it even has support for simple HTML templates, but they're interpreted, type-checked against the structs that get passed into them, and baked into the executable, all at compile time. this is all coming off of building a website for a client using PHP for the first time in over a decade—on one hand, I appreciate the relative simplicity and ease of deployment compared to modern backend stacks, but on the other hand, it's still an interpreted language, with all the baggage associated with that. I believe it is possible to take the ease of use and speed of iteration of interpreted languages, and the benefits of strongly-typed compiled languages, and get the best of both worlds—at least, for the scale and complexity of website that I want to build and maintain.

lelanthran• 3 years ago

Cheap VPS. I run multiple things on a single cheap digital ocean droplet.

Multiple websites served by golang written server, some static sites, also gitea and Jenkins.

robdelacruz4• 3 years ago

This site has a good tutorial for this. It recommends vultr (which is what I use), for $2.50 - $5 a month.

https://landchad.net/

adamrezich3• 3 years ago

what a fantastic resource—thank you!

easytiger3• 3 years ago

You need a VPS. Hundreds of providers available.

gwking• 3 years ago

I am currently working on a low traffic (back office) web app for a client. It runs off of a single AWS EC2 instance (2 vcpu arm). When I looked at the hourly pricing I thought it would be $30/month, but the actual bill has been more like $4/month, I suppose due to an idling discount or perhaps some free tier credit or something else.

The web app is fronted cloudflare (free tier). On the box itself I have caddy set up as a reverse proxy with the cloudflare cert, then uvicorn serving my python asgi (starlette) app. The app uses a local SQLite db.

I’m still working out some of the operational stuff like backups and monitoring but so far I am very pleased with the setup. I’m learning a lot and for the first time I do not feel like there is some monstrous pile of complexity behind a curtain.

Setup takes some time but I have detailed notes and it gets easier every time I run through. Feel free to get in touch if you’d like to hear more details.

leetrout• 3 years ago

Reminded me of a small C webserver: G-WAN

http://gwan.com/

davidw• 3 years ago

For fun, does anyone have something like this in Rust? Curious about that language and this sort of thing would be fun to look at.

imran-iq• 3 years ago

Isnt it in the book? https://doc.rust-lang.org/stable/book/ch20-01-single-threade...

password4321• 3 years ago

I use dufs when I need a webserver for minor things, and reverse proxy to it through Caddy if things get serious enough to need an SSL certificate.

https://github.com/sigoden/dufs

kragen2• 3 years ago

if you like this, you might like httpdito

http://canonical.org/~kragen/sw/dev3/httpdito-readme

http://canonical.org/~kragen/sw/dev3/server.s

it's a 2-kilobyte executable written in i386 assembly that can handle 20000 requests per second on my laptop, but only serves up files from the filesystem; no cgi or reverse proxy

instead of being single-threaded or preforking it just forks a child per request

jaza• 3 years ago

If you don’t do that part, it will be over 4kB instead of under 2kB. If you really care about that, you’re probably insane.

Love it!

amadvance• 3 years ago

Take care that select() is not good for a webserver. From manpage:

       WARNING: select() can monitor only file descriptors numbers that
       are less than FD_SETSIZE (1024)—an unreasonably low limit for
       many modern applications—and this limitation will not change.
       All modern applications should instead use poll(2) or epoll(7),
       which do not suffer this limitation.

pjc503• 3 years ago

Lol.

22 years ago, I worked for Zeus Web Server, which was built entirely around one-process-per-core webserving off select(), and it was so much faster than Apache for serving static content that the developers had built a business out of it. At the time it could saturate a gigabit ethernet link off the largest HP-UX server we could find.

Sure, you should use the modern interfaces, but 1024 connections per process can get you surprisingly far.

robdelacruz4• 3 years ago

Thanks for this. Saw the bug you logged on github too. Agree that poll() should be used instead to avoid the FD_SETSIZE limitation. Maybe in the future when littlekitten webserver grows to be a big cat...

stevefan19992• 3 years ago

I hate to say it, old man, we have Rust today and we can do things as efficient and as performant as you while being safer overall.

We have tokio to handle all the IO stuff, we have hyper to handle HTTP parsing, and we even have tungstenite to handle websocket out of the box. While I appreciate your work but it will not be practical to write C anymore in the modern age. Well, unless you need to target something LLVM isn't there yet and maybe you need some weird GCC toolchain (cough cough AVR)

easytiger3• 3 years ago

While I appreciate your work but it will not be practical to write C anymore in the modern age.

been hearing that for 20 years

capableweb• 3 years ago

If you goal is to just reuse as much stuff as possible, which you seem allude to, then you should just use Apache/nginx/caddy directly. But the goal here doesn't seem to be that, so it makes sense to implement it without using as many libraries as possible.

Both approaches are valid and serve different purposes, you seem to have misunderstood the purpose here.

raxxorraxor2• 3 years ago

People will use C just to get away from Rustafaris (I know...) out of spite some day.

easytiger3• 3 years ago

Like the people who took up python to get away from the unbearable Ruby zealots.

Religious wars will persist

robdelacruz4• 3 years ago

Confession: I write in C because it's fun and feels like you can do anything in it. Agree it's probably not the most practical or commercial-friendly solution. I like C!

stevefan19992• 3 years ago

C is like a footgun with naive bullets while Rust is also a footgun but with smart AI assistant. One will faithfully cripple you, but the other will cripple your mind before you shoot. I prefer the latter as it makes you consider

icar• 3 years ago

https://github.com/robdelacruz/lkwebserver/issues/2#issue-17...

Show HN: A little web server in C

Related Stories