Web server for Linux written in amd64 assembly

by nkurz • 12 years ago

122 57 12 years ago

Luyt• 12 years ago

    xor rax,rax

I used this construct often to zero a register, in the time that memory and CPU cycles were scarce. But nowadays, my time is a more valuable resource, and I tend to write:

    mov rax, 0

It takes a somewhat longer instruction code, and a few CPU cycles more, but it conveys meaning better.

sneak• 12 years ago

It takes a somewhat longer instruction code, and a few CPU cycles more, but it conveys meaning better.

So do webservers written in Python. I don't think that was the point of this project.

XorNot• 12 years ago

Couldn't you just find-replace this in a makefile?

Then again almost everything about assembly gets a bit "isn't that what we made other languages for?"

xyzzy123• 12 years ago

Personally I find the xor form easier to follow, it's SUCH a long-standing convention that if I saw a mov rax, 0 I would wonder what was going on.

rjzzleep• 12 years ago

same here. _but_ if you weren't proficient in writing and reading asm the statement would no longer be true.

now, you could argue if that were that the case, wth is that person doing there anyway.

also if you compile something like `return 0;` it used to be compiled down to `xor eax,eax; ret;`, but meh barely anyone i know coding these days even knew this to begin with.

lttlrck• 12 years ago

A few more instruction cycles? Which architecture and which decade?

ksk2• 12 years ago

This decade :)

For modern out-of-order CPUs - xor reg1,reg2 is problematic because the result depends on the previous contents of the register. Hence it cannot be executed out of order.

However as a special case xor reg1,reg1 (along with sub reg1,reg1) will be detected by the cpu (intel anyway) as a 'zero idiom' instruction and because it has a smaller opcode than mov reg,0 its preferred.

There are also other complex reasons - for detailed info see 3.5.1.8 in the Intel optimization manual.

csmithuk2• 12 years ago

xoring something with itself is a common idiom in ASM. Been doing it since the Z80 (XOR A). Not sure that meaning is enhanced at all.

makmanalp• 12 years ago

Pretty neat! It's awesome how assembly these days is reasonably high level:

https://github.com/nemasu/asmttpd/blob/master/http.asm

It dawns on me why we couldn't have shortcuts for several patterns that show up everywhere:

- mov, mov, mov then call/syscall could just be written as call(arg, arg, arg) since it's not that difficult to figure out which argument needs to go to which register if there was a defined order of arguments.

- push push push push <function body> pop pop pop pop <ret> could just be a define really. I realize that there are cases where you wouldn't necessarily do that, but that seems to be the minority. The compiler could just figure out what registers you use in the routine and push / pop those. If you want to keep a register, there could be added syntax for that.

It seems in both of these cases the language optimizes for simplicity and flexibility, ignoring the common case. Neither of these strike me as situations where introducing the abstraction would require a lot of compiler "magic" to guess and optimize. It's almost just string replacement.

Then again, you could just write all this in C and let the compiler figure it out. Interesting stuff!

vidarh2• 12 years ago

Most decent assemblers supports macros, some of them fairly complicated.

And there is Randall Hyde's High Level Assembly: http://www.plantation-productions.com/Webster/

pjmlp4• 12 years ago

It's awesome how assembly these days is reasonably high level:

There were macro assemblers already available in the 80's!

csmithuk2• 12 years ago

This. Microsoft MASM is the canonical example of one.

Born in 1981, still being updated today:

http://en.wikipedia.org/wiki/Microsoft_Macro_Assembler

mjn2• 12 years ago

In my OS/architecture class we used a textbook whose author had piled on so much macro assembling on top of SPARC asm (macros all in m4, naturally) that he in effect was writing the book using a personal high-level language constructed out of gobs of m4. Like the bizarro-world version of personalized language construction in Lisp-land...

Was this book (the first review complains about the same thing): http://www.amazon.com/gp/product/0130255963/ref=as_li_ss_tl?...

gillianseed2• 12 years ago

Indeed, I pretty much learned programming on the C64 using Turbo Assembler which had macro functionality.

wglb• 12 years ago

Try the 60's.

sehugg• 12 years ago

Also CISC instruction sets (VAX, 68k) were already pretty high-level.

J_Darnley• 12 years ago

If you or anyone is looking for something which takes care of a lot of the entry and exits to functions I would suggest they look x264's x86inc.asm [1]. It was designed for using SIMD in DSP functions not for writing whole programs but I don't think it would get in your way of doing that.

Plus it is BSD licensed for those that hate the GPL.

[1] http://git.videolan.org/?p=x264.git;a=blob;f=common/x86/x86i...

oso2k• 12 years ago

TASM from Borland had bunch of macros/shortcut keywords for the patterns you describe. There's dozens of library files to do similar things in MASM as well. mammon_ had written a bunch of macros [1] that do same thing for NASM. See how he exploits NASM's preprocessor for this in his "Extending NASM" article in the Assembly Language Journal, Feb/Mar 1998, Issue 3. He also demonstrates how do all the control structures in NASM's preprocessor (for, if, switch/case, while, do/while, etc.).

[1] http://mammon.github.io/Text/nasm_apj.txt

NigelTufnel• 12 years ago

I wonder how fast is this server. Is handwritten assembly faster than GCC/clang-written assembly?

0x0• 12 years ago

That depends on how skilled the developer is, and what cpu subtype they are targeting :)

vidarh2• 12 years ago

Maybe, maybe not, but for a web server, if asm vs. C is making a noticeable impact on the overall performance, one of them is doing something very wrong - they should spend most of their time in sys-calls to shuffle data to/from the network, not executing web server userland code.

userbinator• 12 years ago

Speed is probably not going to differ that much but size is one area where even a novice asm programmer can easily beat a compiler.

nknighthb• 12 years ago

Not in the general case. It's been a long time since x86 assembly developers could commonly beat a decent optimizing compiler.

The thing about compilers is that they're leveraging, even if imperfectly, the collective wisdom of their authors and of the companies who actually built the chips and have offered insight, advice, and sometimes even code. It's very probable they know more performance tricks than you do.

One problem is landmines in the ISA, such as instructions that look like they exist to be used, but are really traps implemented in suboptimal microcode for the unwary programmer who didn't look closely at their performance characteristics. Or certain sequences of instructions that might combine to do something ridiculously slow[1].

These landmines vary by microarchitecture. An instruction that's incredibly slow on one line of x86 chips might be a wonder-drug on another. This both increases the probability that your code will hit a landmine on at least some CPUs, and gives you a possible "in": Compilers aren't going to optimize perfectly for every microarchitecture. If you know exactly what you're doing (or spend a hell of a lot of time on trial and error), you might be able to come up with optimal codepaths for specific chips that the compiler didn't.

By and large it's not worth it, though. Hand-tuned assembly still ends up in places, but increasingly rarely, and it's confined to small hot-spots. A particular algorithm or part of an algorithm gets re-implemented in assembly because the compiler just can't get it right.

[1] I could have sworn there was a story about this just recently, but I can't seem to find it. Something like a piece of code running way slower than anyone thought it should, until an AMD engineer piped up and said "Oh yeah, don't do that, it causes a pipeline flush." for reasons that were utterly non-obvious to anyone who didn't know the internals of the chip.

mjn2• 12 years ago

Is handwritten assembly faster than GCC/clang-written assembly?

Sometimes, but the biggest case is if you can carefully arrange a tight inner loop, especially one that case make use of SIMD, like some DSP and scientific-computing code. Auto-vectorizers are getting better, but still miss lots of cases, so a skilled asm programmer can beat the compiler. The more "spread out" the performance-critical code is, in general (i.e. performance not dominated by one or two tight loops), the harder it is for hand-coding asm to beat a compiler; humans are not that good at doing whole-program optimization on large codebases. The more cross-platform the code has to be, the worse for the asm programmer as well: beating gcc's code-gen on one architecture is easier than beating it everywhere.

drv• 12 years ago

I don't want to be too harsh - this is a fun idea, and I'm prone to silly fantasies about rewriting slow code in assembly myself - but this particular assembly doesn't take advantage of many of the "dirty tricks" that are available in low-level code.

As one example, check out the content-type detection, which is essentially a long chain of repeated strlen + strcmp; assembly language doesn't magically make bad algorithms fast.

yohanatan• 12 years ago

Not to mention that long chain is ugly to read. I would rather see a macro defined and called multiple times than to see the same block of code copy/pasted over and over.

diestl• 12 years ago

Why would you do this, write it in C and let the compiler do the hard work.

xedarius• 12 years ago

I agree. There's little justification for this other than 'I can'.

If you need to hit the metal you can inline assembly in both c and c++.

mhurron• 12 years ago

There's little justification for this other than 'I can'.

And why isn't that enough?

exDM69• 12 years ago

Why not? There are several http servers written in C already.

I am pretty sure that this was an exercise in practical user space assembly programing rather than an attempt to write yet another http server.

Tepix• 12 years ago

I'm surprised that asmhttpd doesn't use sendfile(2) anywhere. Isn't that the fastest way to send a file to a socket?

nemasu3• 12 years ago

Yeah, I'll be switching over to sendfile and getting rid of the large read/write buffers. So much cleaner and simpler ( and faster? ).

colanderman• 12 years ago

and faster?

You'll get way more bang for your buck by minimizing userspace→kernel round trips and memory copies than by hand-optimizing assembler code. sendfile is one great way to do this.

You'll get even more bang for your buck by eliminating the kernel from the packet processing path by using netmap, PF_RING/DNA, or DPDK, and a user-space TCP/IP stack.

Assembly only really helps alleviate GCC's moronic decisions resulting in excessive stack spills and alignment-unaware loads & stores.

Something about carts and horses.

nemasu3• 12 years ago

Done ... so pretty now. :)

edsiper2• 12 years ago

asmttpd - 0.04

Using Document Root: htdocs/ An error has occured, exiting

nemasu3• 12 years ago

It's a crappy error message, but usually means the port is taken.

callesgg2• 12 years ago

Runs with about 3% less CPU load than apache on my server. However apache also does ALLOT more stuff.

driverdan• 12 years ago

Under heavy load?

callesgg2• 12 years ago

Yeah downloading a large file. Apache was about 14% CPU Load asmttpd about 11% CPU Load

gtirloni• 12 years ago

No libraries required? That is awesome!

notastartup• 12 years ago

so what is the implication of this? Does this mean that this webserver will be much faster and using less resource because it's written for linux directly?

Web server for Linux written in amd64 assembly

Related Stories