Learning to Read X86 Assembly Language

http://patshaughnessy.net/2016/11/26/learning-to-read-x86-assembly-language

by adamnemecek • 10 years ago

591 234 10 years ago

adamnemecek5• 10 years ago

Also Matt Godbolt's gcc explorer is the the bee's knees for understanding assembly

I think that playing around with it for 2 hours will teach you more than most classes on the topic. It really drives home why interactivity is such a bit deal in education.

You should also try writing a script for counting instructions in binaries. It's pretty illuminating. Here are some sample statistics https://webcache.googleusercontent.com/search?q=cache:j0gebK...

makmanalp3• 10 years ago

This is amazing. After a few minutes I learned so much already - try with an empty function that returns 0, then return an argument, then return an argument +1, argument^2, etc.

As an aside, I was looking at the header code generated by gcc to handle the initial function call. What's the convention on how we assign parameters to registers? I'm trying stuff out, and the first paramter always seems to be edi, then esi, eax. Except when I change the types, and it turns into xmm0 and the like. And the result returned is eax too, except when it's a float, then it's xmm0. How do things know where to look?

adamnemecek5• 10 years ago

How do things know where to look?

The compiler generates code that uses the correct register. So the compiler picks a register into which it will put the result and then generates code after the calling location that gets the result from the correct register.

And yeah, there's quite a bit of surprises. E.g. I found out that gcc is smart enough to perform tail call optimizations https://godbolt.org/g/MZDmwP

Merad• 10 years ago

This is determined by the calling convention [0]. If you're interacting with code that you didn't write, you need to know the calling convention that they use in order to set up arguments correctly.

0: https://en.wikipedia.org/wiki/X86_calling_conventions

richchurcher• 10 years ago

This is such gold: thanks!

JoshTriplett2• 10 years ago

One great example to try: add -O3 to the compiler options, then write a function returning its argument multiplied by 2, 3, 4, ...

You'll see the various ways the compiler avoids emitting an actual multiplication instruction. The smallest multiplier I found that actually produced an imul: 46.

boterock• 10 years ago

Interesting example, I came up with clang 3.9 producing an imul with 11 though.

MichaelMoser123• 10 years ago

You still need an overview of the instructions and registers. Knowing where the operator and operand of the instruction is also helpful. (i always mix up intel and at&t notations)

jwr• 10 years ago

Wow, thanks for posting this — it is a fantastic resource!

I love it even more for using Intel x86 assembly syntax, instead of the horrible AT&T syntax.

Graziano_M• 10 years ago

AT&T is still an option. Which is fantastic because it's a quick way to see how the two differ.

pmoriarty2• 10 years ago

One of the most useful tools that helped me learn assembly was the Ketman Assembly Language Tutorial.[1]

As I step through an assembly language program, it gives me instant visual feedback of the contents of each register, flag, and memory.

I'm really surprised that there's nothing remotely like this on the web yet, and that I have to resort to running dosbox or freedos to have access to this super useful tool.

[1] http://www.oocities.org/siliconvalley/office/6208/

umbs• 10 years ago

Unfortunately, the tool linked from that page (it's archived) can't be downloaded.

Jugurtha• 10 years ago

You might like this:

https://news.ycombinator.com/item?id=9322723

qwertyuiop92431• 10 years ago

x86 is the worst ISA. If you want to play with assembler without feeling a desire to stab yourself and end it all, I recommend ARM.

Or go learn Z80, x86's weird, 8-bit cousin (it had a 16-bit version, but it sold poorly), which had a greater emphasis on backwards compatability (you can run code from the original 8080 on a Z80, unchanged), and is nicer to work with (because it wasn't extended in unticipated directions far beyond its original capabilities, while keeping fetishistic backwards compatability by stacking hack on top of hack on top hack. It also didn't have memory segmentation, otherwise known as The Worst Thing.)

There are only two common reasons to learn Z80 assembler, though: to program the Gameboy (which runs on a modified Z80 with all the cool instructions removed), and to program a TI calculator, thus making all highschoolers in your area immensely happy.

TI calculators are a comically overpriced scam, that have only survived because of the College Board, but that's another story.

umanwizard2• 10 years ago

This is about learning to read, not learning to write.

If you're ever in a situation where you need to read assembly, it's typically not up to you what ISA it'll be in. There are two situations where I've had to read assembly: either reverse-engineering a compiled binary or trying to understand the compiler output for a small piece of a program I'm working on in a higher-level language.

qwertyuiop92431• 10 years ago

Oh. Well then, yeah.

Besides, you don't have to worry about the worst of x86 in those cases.

I'm pretty sure everyone has to learn at least that much x86 at some point, like it or not.

rzzzt• 10 years ago

My use case is inspecting a backtrace from a core dump.

muterad_murilax• 10 years ago

There are only two common reasons to learn Z80 assembler, though: to program the Gameboy [...] and to program a TI calculator

What about the myriad of other (mostly vintage) computer systems and video game consoles out there? ;)

Sega Master System and Game Gear, for example.

qwertyuiop92431• 10 years ago

They're not as common reasons.

But yes, if you want to program your TRS-80 (but only the original: later ones were 6502), or your ZX* (How many of you lot know the ZX line? Spectrum? No?), or your Game Gear, or your Master System, or any of the various CP/M machines, you have to learn Z80.

david-given• 10 years ago

The Game Boy doesn't quite use a Z80 --- like the Z80 it's based on the 8080, but in a different way. So you don't get things like the IX and IY or the alternate register banks, but you do get things like (a very crude set of) stack-relative addressing modes, which makes it a better fit for modern programming languages than the Z80.

http://gbdev.gg8.se/wiki/articles/CPU_Comparision_with_Z80

As an aside: most of the old, classic 8-bit micros are complete pains to write modern code for, because modern programming languages all assume fast stacks. The Z80 has no stack-relative addressing, which means you need to reserve a precious index register as a frame pointer at the top of every function, and then indirect off that --- but the Z80 designers didn't realise that people would want to do it so often and as a result it's verbose, deal slow, and doesn't handle 16-bit values. So you need to do:

    ld h, [iy+8]
    ld l, [iy+9]

...for a total of 8 bytes of code and lots of cycles.

The Game Boy processor (which doesn't have a snappy name) allows this:

    ld hl, sp+8
    ldi a, [hl] // load and increment
    ld l, [hl]
    mov a, h

...which is (IIRC) five bytes. Still not great, but shorter, and also loads faster.

If you look at the instruction encodings, the Z80's actually a pile of nasty hacks. The original 8080 is way more elegant; and there's lots of software and tooling for it, too. (But it still can't run C efficiently.)

blueatlas• 10 years ago

Another good one for learning is the 6502. Don't forget to buy a copy of Lance A. Leventhal's "6502 Assembly Language Programming". Great book for learning not only assembly, but also the fundamentals leading up to assembly.

qwertyuiop92431• 10 years ago

Of course. Who hasn't wanted to write their own chiptunes? Grab your (emulated) Ricoh 2A03 (and your emulated Konami VRC VI, for some extra fun), and get hacking.

And thanks for the book reccomendation.

_JamesA_• 10 years ago

Amen.

I learned to program on the 6502 based VIC-20 using Lance Leventhal's "6502 Assembly Language Subroutines" as a guide by manually assembling and poking into memory. What a fun time.

honkhonkpants3• 10 years ago

I never felt more like stabbing myself than when trying to cipher out exactly which immediate values are possible on ARM, and which are not. X86 I happen to enjoy. It is not "the worst ISA" by any means. It has wonderful code density, which turns out the be very important. There's a reason that x86 won and continues to win.

JoeAltmaier3• 10 years ago

When originally invented the x86 instruction set was efficient - the most-used instructions had shorter byte code sequences. But eventually some instructions got 'left behind' by the compilers. There are a whole host of single-byte instructions that are never, ever used by a compiler - the register exchange instructions for instance (xchg eax, ebx). Compilers just schedule destination registers carefully, never need to swap them around.

Also the whole set of exchange-register-with-itself were defined but never used. E.g. xchg ax,ax which does nothing in one byte. In fact that one was considered useful, its used as the 'no-op' instruction (0x90) right? But what about xchg bx,bx, xchg cx,cx and so on? Just wasted single-byte opcodes. Leaving actual common instructions to use longer bytecode sequences.

So maybe an executable should begin with an opcode-decode-table that is loaded with the code, that tells the hardware what byte sequences mean what instructions. So each executable code can be essentially compressed, using optimum coding for exactly the instructions that code uses most often. Just thinking out loud.

qwertyuiop92431• 10 years ago

Well, yes. That part sucks. But on x86, everything is like that. I'd rather have one weird, inconsistant thing than an amorphous, ever-shifting mass of them. Which is x86 in a nutshell.

Here's a handy heuristic: if somebody claims to know every x86-64 instruction (or even every x86-32 instruction), you can be at least 90% sure they're lying.

lgeek3• 10 years ago

I never felt more like stabbing myself than when trying to cipher out exactly which immediate values are possible on ARM, and which are not.

It depends if you're compiling for ARM or Thumb. The main rules are:

ARM and 32-bit Thumb instructions: 12 bits for arithmetic data processing instructions and load/store, 8bits with even rotate right for bitwise data processing instructions + MOV and MVN

16-bit Thumb: 3 bits for arithmetic instructions with Rd != Rn, 5 bits for shifts (so the whole range is covered) and load / store (but shifted left by the size of the data, i.e. the maximum offset for this version of LDR is (0x1F << 2) = 124), 8 bits for arithmetic instructions with Rd == Rn, SP-relative loads / stores and literal loads.

I doubt x86's popularity has much to do with its instruction set at this point. In particular, the variable length instructions are a pain for decoders (both hardware and software).

ris• 10 years ago

There's a reason that x86 won and continues to win.

It is definitely, definitely not the ISA.

wolfgke13• 10 years ago

x86 is the worst ISA.

x86 is clearly not a beautiful ISA, but it is not as black as it is painted. The first thing that one should understand is large parts of the encoding of the instructions make a lot more sense once one writes them down in octal instead of hexadecimal (something even the Intel people who wrote the reference manual seem to have missed):

http://www.dabo.de/ccc99/www.camp.ccc.de/radio/help.txt

qwertyuiop92431• 10 years ago

...that really doesn't help with the issues I have: the instruction set is an absolutely ugly jumble of almost 40 years of religious backwards compatability, odd hacks, and various extensions.

SeanDav• 10 years ago

In my day I wrote a fair amount of x86 assembly language. I found it fairly easy and straight forward. There are probably even less reasons to learn it these days but nothing gives you a better idea of how a computer works (arguably, besides microcode, but that is a different kettle of fish).

qwertyuiop92431• 10 years ago

The simple stuff's alright, but if you want to do anything performant, it gets hairy pretty fast, as you wade through almost 40 years of expansions and ugly hacks.

woofyman• 10 years ago

It also didn't have memory segmentation

The Z80 only has a 16-bit address space.

My favorite CISC architecture is the 68000 series.

qwertyuiop92431• 10 years ago

Yes. But you could install an MMU.

The 68k (and it's 8-bit semi-cousin, the 6809) were very nice. However, unlike the Z80, the 6502, and x86, they're no longer being made, and are increasingly rare.

busterarm• 10 years ago

I'd recommend learning 68k instead. Or MIPS.

But definitely Z80 over x86. x86 is pain.

qwertyuiop92431• 10 years ago

I wish 68k, but it's out of production (as is the 6809, its equally awsome 8-bit cousin).

MIPS and ARM are fun, though, AFAICT. As is 6502.

minimax• 10 years ago

It also didn't have memory segmentation, otherwise known as The Worst Thing.

The examples in the post are clearly x86-64 running in 64 bit mode. I.e. it's running in flat model... there is no segmentation to worry about.

wolfgke13• 10 years ago

The segment registers fs and gs still exist in x86-64 and are used.

Under Windows gs (under x86-32 fs) points to the Thread Information Block (TIB):

https://en.wikipedia.org/wiki/Win32_Thread_Information_Block

Under Linux gs is used for thread-local storage (TLS).

But "as a typical programmer" indeed you only have to worry about the internal details of this if you are an OS developer, otherwise you can simply use the appropriate segment override prefix and otherwise don't care about it.

EvanAnderson2• 10 years ago

I agree that the x86 ISA is pretty warty (though I have a strange fondness for it), but I'd recommend 6502 rather than Z80. There are a lot of fun retro-computer platforms that are 6502-based. Thinking of the zero-page functioning as a register-bank is really fun, too.

spc4765• 10 years ago

For 8-bits I'd recommend the 6809. Two 8-bit accumulators that can be used as a 16-bit accumulator, 4 16-bit index registers (that can largely be interchanged, except for S which is also the stack register) and you can generate pure relocatable code. And the zero-page isn't restricted to address $0000.

DonaldFisk3• 10 years ago

x86 is the worst ISA.

I can only assume you've never had to program a Burroughs B90 in assembly language.

qwertyuiop92431• 10 years ago

I haven't. It's probably not fun.

Annatar18• 10 years ago

x86 is the worst ISA. If you want to play with assembler without feeling a desire to stab yourself and end it all, I recommend ARM.

Yes, intel is really bad, especially for learning, and while ARM is certainly better, it's pretty esoteric, and also backwards (right to left) like intel.

If you want a nice, orthogonal ISA to learn assembler on, MC68000 family is a song. The instructions are human readable, the processor is big endian, and the moves are src, dst. It's almost like a high level programming language.

qwertyuiop92431• 10 years ago

I would argue for the 6502 over the 68000.

The 6502's a bit less simple to learn, but I'd say it's worth it. It worked its way into many important computers, and is arguably one of the most emulated and most used processors in existence.

pselbert• 10 years ago

Excellent article that has context around the how and why, which I appreciated. Pat has a great "explorer" writing style.

I was going to post this last night but figured it must have been posted already. I was wrong! Nice to see it ok the front page.

pat_shaughnessy5• 10 years ago

Thank you! Knowing that people enjoy reading this sort of thing makes it worthwhile.

iamahacker2• 10 years ago

Excellent article.

JoeAltmaier3• 10 years ago

The examples seem odd to me: the argument order is reversed from every Intel disassembly (or assembler) I've ever used. Addi edi, 43 is the normal way to say "Add 43 to edi". The destination register is normally first; the source register 2nd in the disassembly, right?

ycmbntrthrwaway• 10 years ago

gdb uses AT&T syntax by default. [0]

[0] https://en.wikipedia.org/wiki/X86_assembly_language#Syntax

sigjuice• 10 years ago

This article uses lldb, and not gdb. Both default to AT&T syntax, I guess.

wolfgke13• 10 years ago

In the examples given in the article it is written

  addl $42, %edi

which is correct in AT&T syntax. In Intel syntax it would be

  add edi, 42

raimue• 10 years ago

The examples in the article are in AT&T syntax, you seem to be used to Intel syntax. Just keep on reading, the article will discuss the differences between AT&T and Intel syntax.

Gruselbauer• 10 years ago

It turns out x86 assembly is much simpler than Hungarian

Well, what isn't? I have a knack for languages but that beautiful beast seems like an almost impenetrable fortress of strangeness.

On the other hand, that encourages me to learn to read basic assembly.

_l4jh• 10 years ago

A video I like to send friends asking about how to understanding x86 assembly is https://www.youtube.com/watch?v=yOyaJXpAYZQ

I think the video maker does a good job of mapping a simple C program to its disassembly.

gnuvince• 10 years ago

Nice article! It take a subject that is scary for many and does a great explaining a little bit very clearly using good visual aids. I look forward to the next articles!

pat_shaughnessy5• 10 years ago

Thanks a lot. I'm looking forward to writing it, I think it will be interesting.

RandomInteger4• 10 years ago

This is the first time I actually read all the way through an article on assembly. It was nice and concise. Granted, I'll probably forget this until the next article (due to focusing on other studies), but thank you none the less.

pat_shaughnessy5• 10 years ago

Thanks! Glad to hear you made it to the bottom :) I always worry that I tend to write on and on for too long.

ivanhoe• 10 years ago

To understand assembly it really helps to know at least something about how computers work on a low level. When I first tried learning it (long time ago, in a high school) I had no idea how computers really work on such a low level, how CPU's addressing registers and that kind of stuff, and while I managed to learn the syntax, even write some asm code, it was all really confusing to me. And only few years later on University, after I've learned in details about the cpu architecture, registers, buses, DMA, etc, it suddenly all started to make perfect sense and became 100x more clear and easier. So if you're interested in this, it will save you a lot of effort to invest some time first to learn the computer architecture basics, and then from there go to learn the assembly lang. Just my $0.02

theparanoid2• 10 years ago

It's helpful to realize x86 assembly is not what's executed by the machine; machine code is. One assembly instruction, e.g. ADDL, is translated to several different machine code instructions depending on the destination, source, and addressing mode.

amw-zero3• 10 years ago

Can you point to a source for this? All x86 assemblers that I know of map one assembly instruction to one machine instruction.

khuey• 10 years ago

There would be much less confusion about this in the descendant posts if this said "One assembly mnemonic is translated to ..."

adamnemecek5• 10 years ago

Yeah stuff like dma isn't getting enough attention in a lot of treatments of the subjects. Also in/out instructions (did you ever consider how a CPU talks to a HDD)?

spc4765• 10 years ago

Two ways. The first is IO mapped IO, which is what the x86 supports with the IN and OUT instructions. All this really is is a MOV to a different "address" space (16 bits of addressing). The second is memory mapped IO, in which hardware is mapped to the addressing space of the CPU (Motorola 68K uses this format). A CPU that allows IO mapped IO can also do memory mapped IO (it's not precluded).

userbinator11• 10 years ago

I always recommend this book for those who would like to know how computers really work:

https://www.amazon.com/Code-Language-Computer-Hardware-Softw...

RandomOpinion2• 10 years ago

While I like Petzold's "Code", it's really aimed at nontechnical audiences. Jon Stokes' "Inside the Machine" or Nisan and Schocken's textbook "The Elements of Computing Systems" are far better if you have a technical background.

Grom_PE4• 10 years ago

As AT&T syntax is still being used, at this point I'm willing to believe it's to purposely make x86 assembly hard and unpleasant to read and write. Perhaps so people will want to stay away from it, and in a way, to reduce the amount of code that is tied to the x86 platform. It spreads the thinking x86 assembly is terrible and ugly.

Intel syntax is much cleaner, in particular, Intel Ideal (as opposed to MASM), and specifically, FASM (flat assembler). FASM makes it as clean as possible and turns writing assembly into a joy.

Compare:

    movl %fs:-10(%ebp), %eax       ; AT&T
    mov eax, dword ptr fs:[ebp-10] ; MASM
    mov eax, [fs:ebp-10]           ; FASM

spc4765• 10 years ago

As I recall, the "dword ptr" stuff was only necessary if the instruction was otherwise ambiguous. Using EAX means you are using a 32-bit destination. But something like:

    move fs:[ebp-10],5

is ambiguous. Is that an 8-bit constant? 16 bits? 32-bits?

Grom_PE4• 10 years ago

I believe MASM-style assemblers require "dword ptr" at all times and many disassemblers keep outputting that in unambiguous situations.

Your example won't assemble because size can't be guessed, but this will:

    mov dword [fs:ebp-10], 5

burfog3• 10 years ago

All of those are misleading because the FS segment override isn't specific to an operand. It applies to the whole instruction, which commonly has one place (a memory reference) for the override to take effect. You can have more than one override, but only the last one remains active. Normally you can have an override even if it isn't used. There are a few instructions with more than one memory access; the override only applies to one access and you don't get to choose which one.

Grom_PE4• 10 years ago

No. This is misleading:

    fs movs byte [edi], [esi]

This is a syntax error:

    movs byte [fs:edi], [esi]

And this is a valid override with sensible syntax:

    movs byte [edi], [fs:esi]

userbinator11• 10 years ago

What a train wreck! It’s hard to imagine a more confusing state of affairs.

I'd say that's more attributed to someone many many years ago deciding they would not follow the official Intel syntax (for what reason I do not know), and somehow convincing the rest of the community to follow them. That's actually one of the things that could make for a very interesting article: how one processor family got two different and incompatible Asm syntaxes. The fact that the mnemonics and syntax don't correspond to those found in the manufacturer's datasheets and manuals just increases the barrier to understanding. As far as I know, the same didn't happen to ARM, MIPS, SPARC, and the others. Especially when the sense of the comparisons/conditional jumps is reversed, and some of the more advanced addressing modes look less-than-obvious, it's hard to imagine why anyone would adopt such a syntax:

http://x86asm.net/articles/what-i-dislike-about-gas/

Note that the GNU tools have option to use Intel syntax too, so you can avoid some of the confusion (in the DOS/Windows and embedded world at least until recently, Intel syntax is overwhelmingly the norm.)

pcwalton3• 10 years ago

And Golang uses a separate syntax from Intel syntax and AT&T syntax, so now there are three incompatible syntaxes in common use. What a mess. :(

In Go's case it's because they wanted to have the syntax reflect the MachineInstr-like abstraction they have in place. I wouldn't be surprised if something similar was responsible for the AT&T syntax as well.

(In case it isn't clear, my preference is to do what LLVM does and to treat MachineInstr and machine code as distinct objects, and to use the vendor syntax for the human readable representation of the latter.)

flamedoge2• 10 years ago

They should all have same binary encoding, which means they can all be decoded to any human readable format regardless...

theparanoid2• 10 years ago

For extra points the Go syntax has /no/ spec.

To quote "The details vary with architecture, and we apologize for the imprecision; the situation is not well-defined." [0]

[0] https://golang.org/doc/asm

adamnemecek5• 10 years ago

how one processor family got two different and incompatible Asm syntaxes

Do you know the reason for this?

userbinator11• 10 years ago

Unfortunately not, but I'd love to hear from those who do.

aap_2• 10 years ago

Familiarity with the VAX I'd say. Although that doesn't explain the % in front of registers. Maybe that came from SPARC? Looks like they tried to unify various asm syntaxes.

haberman3• 10 years ago

This. The AT&T syntax for x86 thing is a huge mistake. All the official docs are Intel syntax. Intel syntax is easier to read and write. Half the gotchas in this article are problems that don't exist in Intel syntax, like the instruction suffixes. The instruction suffixes get even weirder when you get to the sign extending instructions. I wrote an article about this here: http://blog.reverberate.org/2009/07/giving-up-on-at-style-as...

pjmlp5• 10 years ago

Additionally Intel syntax Macro Assemblers offer higher level constructs providing almost C like experience while using machine opcodes.

Annatar18• 10 years ago

The AT&T syntax for x86 thing is a huge mistake.

For someone who grew up on normal processors (MC68000 and UltraSPARC) AT&T syntax is the best thing since sliced bread: it's perfectly logical to move something to somewhere, instead of "move to somewhere something".

wolfgke13• 10 years ago

I prefer Intel syntax and admit that you have a good argument in your article why the AT&T syntax might be problematic, but for newer extensions (SSE etc.) the instruction naming in Intel syntax actually converged towards AT&T syntax: Just to give an example from SSE2 (from http://softpixel.com/~cwright/programming/simd/sse2.php):

  paddb - Adds 16 8bit integers.
  paddw - Adds 8 16bit integers.
  paddd - Adds 4 32bit integers.
  paddq - Adds 2 64bit integers.

pjmlp5• 10 years ago

A few years ago I decided to convert the Asm syntax of my toy compiler from Intel to AT&T, because I wanted to rely only on as, not forcing people to install another Assembler.

Since as had some issues with the Intel syntax I was using, I decided to not use its support for the syntax and convert it to AT&T.

Never again, it is just plain ugly.

Compared with DOS/Windows Macro Assemblers (TASM, MASM, NASM, ...), AT&T ones are just plain pre-historic.

matchagaucho• 10 years ago

When I worked at Intel in the Server BIOS group, the development process involved several iterations of developing macro abstractions until the x86 ASM code became more readable and maintainable.

No one particularly enjoyed working in raw ASM 100% of the time.

tptacek• 10 years ago

So, this is fantastic, but I want to make an appeal for the most important thing to understand about any assembly language, even before you work out the individual instructions:

Break your programs into basic blocks! Reverse engineers never read assembly in a straight line. Instead, they read the control flow graphs of subroutines, which is the graph where nodes are runs of instructions ending in jumps and branches, and the edges are the jump targets. I hope this doesn't sound complicated, because it isn't: it's literally just what I wrote in this paragraph. It takes about 15 minutes for most platforms to learn enough to recover CFGs from subroutines by hand.

To get a decent understanding of what a chunk of assembly code is doing, all you really need is:

* The code broken into subroutines (this is usually your starting point) and then CFGs (good disassemblers do this for you, but it's easy to do by hand as a first pass)

* The CALLs (CALLs don't end basic blocks!)

* The platform's calling convention (how are arguments passed and return values returned from subroutines)

There are two tricks to reading large amounts of assembly:

1. Most of the code does not matter, and you won't be much better off for painstakingly grokking it.

2. Virtually all the assembly you'll see is produced by compilers, and compilers spit out assembly in patterns. Like the dude in The Matrix, after an hour or so of reading the CFGs of programs from a particular compiler, you'll stop having to read all the instructions and start seeing the "ifs" and "whiles" and variable assignments.

kitsuac• 10 years ago

If you want to learn how to read X86, don't choose this AT&T style syntax as it's a monstrosity.

bogomipz2• 10 years ago

I thought this sentence was curious:

"To write code that runs directly on your microprocessor you need to know how memory segmentation works"

Although you can't completely ignore segments, in practice at least on Linux the only segments in use are user code/data and kernel code/data segments.

Does anyone know why the author might suggest that understanding segmentation is necessary to write Assembly code?

ktRolster• 10 years ago

Probably because you need to deal with the MMU. You can't just write raw assembly and expect it to work (ignoring the MMU), but the kernel takes care of that for you.

bogomipz2• 10 years ago

I am not following - what is raw assembly? I can write a complete userland program using only Assembly and it will run just fine. When do I need to deal with the MMU exactly?

ndesaulniers• 10 years ago

Nice illustrations. It was a good idea to get the prologue/epilogue out of the way. Note that for optimized code, it's usually unnecessary. For more info: https://nickdesaulniers.github.io/blog/2014/04/18/lets-write...

Vera527• 10 years ago

What a great read. Thanks for sharing op.

Annatar18• 10 years ago

And, of course, modern compilers will usually produce faster, more optimized code than you ever could, without making any mistakes.

This assertion comes up over and over again in the last 30 years. Every time I've had it asserted to me, it always came from non-assembler programmers, who always wrote in a high level language. I have yet to see evidence of optimizing compilers generating code even remotely close in efficiency to what we would code directly in assembler.

A coder would never write all those extra frame pointer setup code, nor would they waste encoding and tact cycles shuffling values from one register to another. For example, a human might write the code from the article thus:

  Add42: addb $42, %al
         ret

and that's it. No frame pointer or stack setup, that's all unnecessary overhead because compiler algorithms can't reliably make such contextual decisions.

aap_2• 10 years ago

Why couldn't they though? Doesn't sound very hard to only generate the wordy prologues and epilogues when necessary (i.e. when you have to save any registers). Why they apparently don't do this is another question then.

Annatar18• 10 years ago

I'm not aware of a general algorithm which is capable of deciding whether and how many processor's registers to use instead of setting up frame and stack pointers, and pushing a variable number of arguments on the stack, are you?

A human will know while coding in assembler, at any given time, how many variables are in the game; and will almost always manage to fit them all within processor's registers; There were only two times in my life where I actually had more than eight variables within a subroutine and had to use the stack, and even then, I didn't push everything, but only as many registers as I was actually coming up short, and the rest I still stuffed in the available registers. The other time, I figured out a more efficient algorithm where I could fit everything within the seven general purpose address registers (a0 - a6, since a7 is the stack pointer). A human will also know whether the expected result is within a byte, word, longword, or quadword range, and will only use those instruction and register sizes; a compiler has no chance to figure that out. It's trivial for humans, but as far as I'm aware, impossible for a general compiler algorithm.

In fact, even the best optimizing compilers are so dumb, that one is not allowed to mix and match 8-, 16-, 32- or 64-bit code; one must either compile everything 32- or 64-bit (the linker won't let one link 32- and 64-bit object code together). A human could easily write correct assembler code using all of those instruction / register sizes at once, and we often do.

I have yet to see a compiler capable of inferring that. If you know of one, please show me the generated code. I'd love to use such a compiler.

RX145• 10 years ago

Bear in mind that this isn't a release build, so the generated assembly will not be optimized at all.

Annatar18• 10 years ago

There isn't really much more to optimise: at higher optimization levels, the compiler will figure out whether that was a one time operation or not, and if he determines that it was, it will simply hardcode:

  moveq $52, %eax

but all the extra cruft with stack and frame pointer setup will remain unchanged, and will still be there, if only to comply with the ABI calling conventions. I guesstimate that there are up to 50 clock cycles used for each setup and teardown of the stack and frame pointers; now multiply that with the number of times a function is called, and you can easily waste hundreds of thousands, or even millions of clock cycles pretty much doing no useful work, just housekeeping.

xxs• 10 years ago

>because compiler algorithms can't reliably make such contextual decisions

They can actually. Compiler optimizations have come long way, even Java's JIT should be able to optimize that. (ok, not using the AL register)

My personal story - I used to use exclusively assembler for 6502 and 8086 as it actually ran fast enough. In the mid 90s I saw Delphi's code (and Delphi was not known for its optimizations) but it was able to use the Pentium instruction pairing which takes quite an effort to accomplish by hand.

While beating an old compiler was easy it was the time the compilers began making strides rivaling humans.

Still, hand written inner loops in Assembly might yield some performance (iirc, grep still relies on) but overall there are very limited amount of settings where there would be significant difference... to warrant the effort (incl. correctness and [micro]benchmarks)

Annatar18• 10 years ago

But a human would almost never use some of those more complex instructions, for a very simple reason: they eat too many clock cycles. When one is coding in assembler, one usually targets two constraints:

1. the least amount of clock cycles needed to pull off an operation;

2. the least amount of bytes to encode the operation.

Where those two meet is where the best coders get unbelievable performance out of the hardware. At least that's the case in the demo scene, although many nowadays cheat by banging the GPU's in CUDA or OpenGL.

RX145• 10 years ago

If you're experimenting with asm in crystal, you might want to use --prelude=empty to remove the standard library to make the asm output cleaner. You can then then require lib_c and use that directly.

pat_shaughnessy5• 10 years ago

I tried that while researching the article, but found the call to "puts" doesn't link without the standard library code. And without a "puts" or similar call to produce output LLVM optimized the entire program away :)

I suppose this could work if, as you suggest, I manually called out to a lib_c function like printf instead.

RX145• 10 years ago

Yeah, puts is part of the part of the standard library, and uses crystal's evented io framework, fiber scheduler and libevent. This is what most of the extra code in the asm output will be doing.

kruhft• 10 years ago

Reading assembly language is about having the computer in your head, just like regular programming. You read and execute the instructions just like any other language, just the operations are that much smaller and less abstract. Each instruction is a 'function call'; just like any low level language you leverage abstraction to build up these operations into greater pieces of execution knowledge using macros and functions in logical ways to get the outcome.

It's not magic. The best way to learn assembly is to program in it. I learned on the Gameboy by getting a job and programming 2 games in it. Fun as hell, especially when the machine is small enough to really fit in your head and clock cycles count at 4Mhz.

dman• 10 years ago

Btw if someone has a local copy of intels intrinsics tool before they converted it into a web version - could you please share it?

Learning to Read X86 Assembly Language

Related Stories