C Is Not a Low-level Language (2018)

https://queue.acm.org/detail.cfm?id=3212479

by goranmoomin • 5 years ago

204 192 5 years ago

queue.acm.org

johnklos• 5 years ago

This is a rewording of the original title, which is:

C Is Not a Low-level Language Your computer is not a fast PDP-11.

"Trying to expose a PDP-11" is misleading here, because how it's written in this title suggests that C is not a low level language because it fails at exposing a PDP-11.

Rather, the article suggests that modern processors are made in a way to expose abstractions which are similar to a fast PDP-11, which lacks real ways to represent their true complexity.

It's an interesting article, but this misleading title might give the wrong idea.

nanodano• 5 years ago

C Is Not a Low-level Language Anymore

Pretty sure C was always considered a high-level language.

rcoveson• 5 years ago

"High" and "low" are relative adjectives. The meaning of "high-level language" has changed. C was considered a high-level language, and it is now considered a low-level language. Not by everyone, of course. There are still a pair of definitions for "high-level language" and "low-level language" that draw the line right above assembly. I won't say "nobody" uses those definitions anymore; lots of people learned them and many still use them. I will say that it is pointless to act as if those are the only definitions of the terms anymore.

BoorishBears4• 5 years ago

People don't call C high-level, but I also don't see people call it low-level even in a casual setting with newer programmers. Instead I see it called a system-level language.

eesmith6• 5 years ago

I think I'm confident in saying that K&R saw C as lower-level language. As you say, it is relative [1]. I just don't think enough people considered C to be a high-level language (given that Smalltalk, APL, and Lisp were about) to make your broader characterization that "C was considered a high-level language."

Here's my reasoning:

From "The C Programming Language" book (1st. ed., 1978) at https://archive.org/details/TheCProgrammingLanguageFirstEdit... we can read 'C is not a "very high level language"' (p. ix) and 'C is a relatively "low level" language' (p. 1).[1]

They describe what "low level" means to them: "This characterization is not pejorative; it simply means that C deals with the same sort of objects that most computers do. namely characters, numbers, and addresses."

And on page 2 we see how they don't regard C as the lowest level: "Of 13000 lines of system code, only about 800 lines at the very lowest level are in assembler."

I also found "The C Programming Language" paper in The Bell System Technical Journal (1978) saying "All three languages [BCPL, B, and C] are rather low-level", at https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6770408... .

Now to the [1], I found the paper "Implementing LISP in a high‐level language" from 1977, where that high-level language is BCPL, which is a precursor to C, so clearly a good number of people at the time would have considered C a high-level language, at the very least in the context of developing a Lisp.

ncmncm• 5 years ago

C is the lowest-level language in common use, short of assembly language.

C lacks any high level abstractions. All the abstractions it does offer are hiding register and stack slot assignments (as local variables), code entry point addresses (as function names), ALU instructions (as operators), branching (as control-flow statements), and address arithmetic (as pointer operations). All of these are low, machine-level abstractions.

C was never a high-level language, even from its first day. It was specifically intended as a portable assembly language, by someone used to coding assembly language, to use porting an OS coded in assembly language to a new target host.

charcircuit• 5 years ago

C lacks any high level abstractions

macros

pjmlp4• 5 years ago

And structures, unions, pseudo-meta programming via the macro processors, no exposure to IO unless on a CPU with MMIO.

JOVIAL and Algol dialects were also designed for creating OSes and no one calls them low level.

adwn• 5 years ago

C lacks any high level abstractions. All the abstractions it does offer are [...]

By that standard, practically all languages lack high level abstractions. Garbage collection? Hides pointer chasing and marking of memory locations. Dynamic dispatch? Hides a pointer to a function table. Functional programming? Hides pointers to closures. Closures? Hide pointers to data and function pointers.

Jensson2• 5 years ago

"Low level" is relative. The article says that there doesn't exist any "low level" languages that programmers can use, this isn't a very helpful definition. If we go by the languages that programmers actually have access to, then machine code is as close as you get to the metal, and C maps very well to machine code. So by any reasonable definition C is a low level language for programmers. For a hardware engineer who works on CPU architecture it is of course different, but we are talking from a programmers perspective here.

whatatsentyways• 5 years ago

Good take - defining a "low-level language" feels like trying to define a "cold temperature". It's all very relative.

We could say that a temperature is "freezing" if it is under 0C, and "boiling" if it is over 100C. But it's hard to nail down "cold" or "warm", as any group sharing a thermostat will tell you.

You can easily place arbitrary assembly commands or memory values in C code. Plenty of embedded codebases are littered with asm blocks in performance-critical areas. It's hard to see it as a high-level language in the age of javascript/python, but maybe the overton window has shifted.

to11mtm• 5 years ago

Yeah IDK.

I know in college, a lot of my profs referred to C as a 'mid level' language (or by others as a 'systems' language), C# 1.2 and Java were 'high level'. They really only considered ASM as a low level, but that was almost 20 years ago.

drran11• 5 years ago

C is a 3-rd level of abstraction language. Currently, there is no 4-th level of abstraction, so C is a high-level language by definition of «level of abstraction».

Language levels of abstraction:

1. Machine code

2. Assembler

3. A high-level language

C does allow low level things, e.g. access of machine registers or embedding of assembler or machine code, but it is also capable of quite high level of abstraction.

Some of 3-rd level languages allows higher level of abstraction, or hide low level abstractions, e.g. memory management, stack, byte order, bits per number, etc., hover none of them improves programmer productivity by an order of magnitude.

rndm_access• 5 years ago

Which layer is Forth? xD

saberience• 5 years ago

I would define fourth level languages by what they allow you to do, or what foot-guns they provide you with.

For example, surely languages like Typescript are "level 4" as you cannot manipulate memory directly but only use the languages higher level type structures.

bee_rider• 5 years ago

That, in the conventional classification, C is considered a high level language, is definitely what I learned in some languages class. This seems like an odd result given that it is probably the least abstract language that the vast majority of programmers might ever encounter, outside the exercise or two on assembly that they might see in an architectures class. Sort of how, as far as I know, most (all?) PIC and AVR microcontrollers would be considered "very large scale integration" chips.

This the best possible outcome, though -- since they used up these kind of "biggest classification that doesn't sound totally silly" labels in the 70's and 80's or whatever, we don't have to worry about classifying languages anymore, which was not really all that productive to do anyway.

Although I do wonder what level Verilog should be considered.

Ginden• 5 years ago

Such definition is just useless and therefore should be abandoned.

hover none of them improves programmer productivity by an order of magnitude.

I disagree. I can write efficient SQL statement in 2 minutes, but implementing efficient multiple "table" join with spill to disk took me much longer than 20 minutes.

saagarjha3• 5 years ago

Respectfully: are these levels just something you created for this comment, or are they actually something you can back up with literature that agrees that they are the right levels or that they even exist at all?

jug• 5 years ago

I'd argue that the class of languages running within a virtual machine like Java (JVM) or .NET (CLR) ought to be placed in a higher level of abstraction. These are so detached from the physical reality that they become cross-platform, which in itself is also the goal.

junon2• 5 years ago

Err... what? Did you make these terms up yourself, or are these established in some literature already? Because they make no sense.

C isn't converted into assembler language unless you ask the compiler to. For example, LLVM lowers IR AST structures into target specific machine code. Assembler is never involved.

ASM is just a human readable representation of machine code for a particular ISA. So in your mental model, C and ASM exist at the same level.

Further, Java bytecode would exist above the the machine code level (again, alongside C then, since the VM is compiled to machine code) and Java on top of that.

It just doesn't make any sense to define languages like this. I don't see any benefit, and I don't even see it as being correct.

saagarjha3• 5 years ago

C maps very well to machine code

Again, that depends. If you're a compiler engineer, it's likely you might disagree with that statement, because your job is to create passes that change that C into machine code that barely resembles the original source but performs the same operation, faster–except that "same operation" isn't even defined in terms of the machine code, it's defined in terms of an abstract machine that doesn't exist outside of the C standard. Does C really map well to machine code when it reorders your statements and enregisters your variables and unrolls your loops and inlines your functions? I think that really depends on where you're sitting on the ladder of relativity.

noobermin2• 5 years ago

Sigh...

The point the article is making is this is a "low level" only in the programmer's mind. The point is that for example the mental model you have when programming in python is one mental model and the model you have for C is another. C's might be faster in most implementations, but it is still a mental model and it is a mental model that is somewhat removed from what actually happens when your cpu executes instructions.

People have trained themselves to think in the Python (or js or java or etc etc) mental model while remembering that it doesn't map to what actually happens 100% People have NOT made that leap for C because people still labor under the delusion that their CPU is a fast PDP-11.

vnorilo6• 5 years ago

Most of the time I've written performance sensitive C or C++, I spend most of the time looking at disassembly or pipeline models, trying to spin up an instruction mix that will utilize the CPU backend in a good way.

In the best case, the compiler finds the instructions I want, and often it does a very good job with details like register allocation.

So the abstraction leaks and inverts like hell. But still, in many cases the C program can still be portable, even if none of the performance tuning is - it targets the wrong pipeline and memory subsystem traits.

When you need to rely on intrinsics, it is different - but I'd say intrinsics are closer to inline asm than C. Saying that as someone who might have ported a DSP library from SSE4 to AltiVec with mostly #define. I know. Also I'm happy it wasn't the other way around!

Jensson2• 5 years ago

People have trained themselves to think in the Python (or js or java or etc etc) mental model while remembering that it doesn't map to what actually happens 100% People have NOT made that leap for C because people still labor under the delusion that their CPU is a fast PDP-11.

I don't see this, I worked on high performance software and everyone is aware that the CPU isn't just a fast PDP-11. This was before 2018, so teaching C programmers about their language isn't the point of the article, everyone who needs to care about performance already knows these things.

lapinot4• 5 years ago

The article says that there doesn't exist any "low level" languages that programmers can use, this isn't a very helpful definition.

This is not the definition used in the article. There is literally a paragraph called "what is a low level language" at the beginning:

Think of programming languages as belonging on a continuum, with assembly at one end and the interface to the Starship Enterprise's computer at the other. Low-level languages are "close to the metal," whereas high-level languages are closer to how humans think. For a language to be "close to the metal," it must provide an abstract machine that maps easily to the abstractions exposed by the target platform. It's easy to argue that C was a low-level language for the PDP-11.

C maps very well to machine code.

The article's core point is literally a contradiction to this. I'm not sure what to say. Perhaps you ought to read again or provide more arguments as to why you think this it is true.

A modern Intel processor has up to 180 instructions in flight at a time (in stark contrast to a sequential C abstract machine [..])
[..] the C abstract machine's memory model: flat memory. This hasn't been true for more than two decades.
Unfortunately, simple translation providing fast code is not true for C.

"Low level" is relative.
For a hardware engineer who works on CPU architecture it is of course different, but we are talking from a programmers perspective here.

Point is being made in the articles that programmers do care about cache and pipeline behavior (to get decent performance in intensive parts) or about threading, both of which are transparent to C. And also that languages otherwise seen as "high-level" (usually because of memory management) sometimes have aspects that map better to these features (hence are lower level than C in some other aspects).

vlovich1233• 5 years ago

You say “ This is not the definition used in the article. There is literally a paragraph called "what is a low level language" at the beginning:”

Then say C fails this test of being low-level:

A modern Intel processor has up to 180 instructions in flight at a time (in stark contrast to a sequential C abstract machine [..]) [..] the C abstract machine's memory model: flat memory. This hasn't been true for more than two decades. > Unfortunately, simple translation providing fast code is not true for C.

None of this is true for assembly either. To my knowledge neither x86 nor ARM expose primitives to control runtime instruction parallelism nor cache. You just have to know what that looks like and how to adjust that implicitly as best you can (still with no guarantees). Maybe a CPU architecture like Vulkan would work where there’s low-level primitives and vendor-specific plugins but I don’t know. GPU programming has gotten significantly harder and error-prone with Vulkan. Additionally, game-programming doesn’t need pixel-perfect behavior while you do typically want that out of your CPU (yes it’s slightly inaccurate with GPU compute advancements, but that’s still significantly more expensive as a development target and reserved for problems where the value is worth it)

dahfizz3• 5 years ago

For a language to be "close to the metal," it must provide an abstract machine that maps easily to the abstractions exposed by the target platform.

C does map perfectly well to the abstractions exposed by x86 CPU. By this definition, C is a low level language. The author himself states that modern processors are fast pdp-11 emulators. This is exactly the "abstractions exposed by the target platform".

If C is not a low level language by that definition, then nothing is. Assembly uses the same PDP-11 abstract machine model that C does. Hence the parents point that this definition is useless.

lowwave• 5 years ago

yeah low level is relative. Assembly would be the place to manipulate hardware, but then developer need to have knowledge of the chip design, etc.

Depending on what you want to do. One can also go a level lower with machine code, or physics as how some quantum computers are coming on the scene now days.

ori_b• 5 years ago

The article's core point is literally a contradiction to this.

No. The article's core point is that machine code maps poorly to what the CPU does.

Modern CPUs expose no low level access to the way they execute code.

mghfreud• 5 years ago

I am lost here, the mentioned bugs are a result of optimizations like speculative execution, branch prediction, prefetching etc.

These are language independent optimizations. For example, any language (that allow for loop like constructs) compiled to intel machine code and executed on intel processor will be exposed to these bugs, it is not C specific. Am I missing anything?

spijdar• 5 years ago

any language (that allow for loop like constructs) compiled to intel machine code and executed on intel processor will be exposed to these bugs, it is not C specific.

Well, that's kind of the point, that Intel/x86 and most/all modern processors implement an abstract machine that's basically made for C, papering over the underlying instruction parallelism and absurdly complex memory model with all kinds of crazy front-end decoder business to allow the CPU to ingest that machine code and pretend like instructions are executed in order, with C-style control flow.

You could (in principle) prevent these sorts of bugs by creating a new kind of machine, but that machine would be incapable of running C software, at least efficiently. There are several ideas being alluded to here, another is the idea of directly exposing the underlying instruction level parallelism -- that's been attempted before in VLIW processors like Intel's Itanium chips. You could make the argument a big part of their problem was at the compiler level, trying to map C-style semantics to the CPU, while maybe a different language/compiler would have extracted more performance.

Trying to summarize the author's idea, modern CPUs have a lot of hidden potential behind a restrictive "virtual machine". If that layer were stripped clean, we could (the idea goes) get more performance and parallelism, and potentially more security, at the cost of compatibility/interoperability with legacy software.

dahfizz3• 5 years ago

Intel/x86 and most/all modern processors implement an abstract machine that's basically made for C, papering over the underlying instruction parallelism and absurdly complex memory model with all kinds of crazy front-end decoder business to allow the CPU to ingest that machine code and pretend like instructions are executed in order, with C-style control flow.

The issue is that none of this has to do with C, really at all. C relies on the semantics exposed by the machine code. The ISA does not expose speculative execution or pipelining. C, and asm, and literally all software conforms to the ISA because that's all there is.

Calling out C specifically is just clickbait, IMO. The author makes a great point about how the x86 ISA may not be a great abstraction for modern CPUs.

dundarious3• 5 years ago

If hardware is significantly more advanced, but C is largely the same as it was in PDP-11 days, then C now is relatively speaking less able to control the low level compared to back then.

C has decent escape hatches to direct machine control via a simple ABI and even inline assembly, but the behavior of C code itself is arguably under-specified on modern hardware.

Too3• 5 years ago

Being under-specified towards the hardware is a feature, to allow optimization.

Now I’m not arguing all the UB in C is great, in fact the opposite. But that should be prohibited on the language layer and not on the machine layer. See rust for a better solution.

lelanthran• 5 years ago

I am lost here, the mentioned bugs are a result of optimizations like speculative execution, branch prediction, prefetching etc.
These are language independent optimizations. For example, any language (that allow for loop like constructs) compiled to intel machine code and executed on intel processor will be exposed to these bugs, it is not C specific. Am I missing anything?

1. There's machine code that is exposed to the ISA (public machine code that compilers generate) and there's machine code that exists and is used but is not exposed.

2. The author is making the argument that the machine code that is exposed is designed around the memory model of the C programming language, which itself was designed around the memory model of the PDP.

Put the above two together and (if you squint really hard, and ignore things like logic and reason) the conclusion is that the modern x86/X64 ISA is suboptimal because of the PDP.

The actual reality is that all the popular programming languages are imperative, have the concepts of stack, heap and in-order execution of instructions.

Because all languages appear to converge on the same basic concepts in order to be commonly accepted, I think that it is doubtful that any alternative machine and memory model would have arose in the absence of a language like C or a machine like the PDP.

I think this because of the existence of other languages that offer alternative machine and/or memory models, and those languages have existed for decades without being popular.

kaba05• 5 years ago

The actual reality is that all the popular programming languages are imperative, have the concepts of stack, heap and in-order execution of instructions. Because all languages appear to converge on the same basic concepts in order to be commonly accepted, I think that it is doubtful that any alternative machine and memory model would have arose in the absence of a language like C or a machine like the PDP.

But as we can see, this model could not keep up with performance improvements so much more complexity got implemented beneath the surface of the old model. The author’s point is to be aware of the mismatch here, and that perhaps we should stop believing the “lies” what C tells us.

I personally believe that we would be much better off with lower level instructions exposed to us, and putting the complexity in software. That way CPU vulnerabilities could be patched, and I believe we could create much better optimizations, and faster CPU design iterations.

Linosaurus• 5 years ago

Sounds like he's saying that a low level language without invisible abstractions no longer exists for modern CPUs. Thus the calls to rethink both the hardware and language.

Too3• 5 years ago

The bugs are not caused by speculative execution.

The bugs are caused by programmers believing they have control over memory addresses and registers in the CPU only because they are hardcore programmers writing in “low level C”. So when they peek behind the abstraction, the ship now has two captains, one being the programmer and one being the compiler. When both of them hit the gas at the same time all the undefined behavior bugs appear.

A bit extreme analogy but you could compare it to writing code without mutex locks because code will only be executed in single thread, then go multithreading anyway. On an old cpu and old compiler it will work, but once you rev up it will crash and burn. In the case of C, the language spec always catered for this possibility, but many programmers thought they were smarter than the compiler leading to todays bugs.

Other languages neither expose the control nor the temptation to insert such sticks into the machinery. Both sides of the abstraction have a much more mutual understanding of where the border of the language ends. With taller guardrails in place to prevent stepping over it.

petergeoghegan2• 5 years ago

The bugs are not caused by speculative execution.

Actually, they are.

It seems like you're arguing against the existence of any kind of low-level hardware/software interface. C is strongly associated with that interface, but it isn't the same thing.

pmontra• 5 years ago

As hinted by the article LLVM IR [1] is a lower level language and yet it's only intermediate as per the i in IR.

And it's true that the actor model makes writing parallel programs easier. I tend to use queues and message passing when I write multi threaded programs in sequential languages like Python or Ruby. That's easier to do in a language like Elixir. Unfortunately when I work with Elixir I'm a little discouraged by all the boilerplate needed to make supervisors and GenServers work. I think there is a lot of room for improvement for a higher level language that makes most of that disappear.

[1] https://llvm.org/docs/LangRef.html#instruction-reference

styluss• 5 years ago

I'm not sure if it's the appropriate solution but have you looked at higher level abstractions in elixir, like Flow, Broadway, GenStage?

riskneutral• 5 years ago

The Cell processor was a very different architecture from x86. It sacrificed cache coherency and required the programmer to manually manage each core's cache, in exchange for state-of-the-art performance. This was all done in C (although FORTRAN compiler was also available, of course). The Cell processor simply introduced new intrinsic functions to the C compiler, to allow the programmer to access the new hardware functionality. It all works perfectly fine with the rest of C, although people felt it was too difficult to program and the architecture quickly went extinct.

NVIDIA GPUs are also innovative hardware, mentioned in the article, and CUDA is also just an extension of C. CUDA is wildly popular, and lots of higher level abstractions had been built on top of it. The only thing lower level than CUDA is the NVVM IR code which is generated by the C compiler (eg. LLVM NVVM backend) and is only compiled into final machine code by the GPU driver at run-time. So C is the lowest level.

The problem doesn't lie with the language, it lies with the x86 processors and different trade-offs that companies like Intel must make, such as trying to sell processors to developers who have been instructed by their employers to be productive and use a "safe" and high level language (e.g. Java CRUD application developers, or JavaScript web developers, etc).

edit: typos

tubby12345• 5 years ago

NVVM IR compiles down to PTX which then is compiled by the driver to a shader language (or something like that).

qayxc5• 5 years ago

NVIDIA GPUs are also innovative hardware, mentioned in the article, and CUDA is also just an extension of C

CUDA uses a completely different programming model from C. There is no such thing as a C virtual machine behind CUDA and just because the syntax is identical (to make adoption easier), doesn't mean semantics are.

The problem doesn't lie with the language, it lies with the x86 processors and different trade-offs that companies like Intel must make

You are ware of the fact that the mentioned UltraSPARC Tx CPUs are also highly susceptible to Spectre? These CPUs feature up to 8x SMT ("hyperthreading" in Intel jargon) and thus are facing the same issues as x86 when it comes to parallelism and speculative execution.

The problems are evident across hardware architectures.

phdelightful2• 5 years ago

CUDA's programming model is not /completely/ different than C. It's not even /very/ different. Most of the C abstract machine (what I think you meant when you wrote virtual machine) carries over directly.

What is quite different is the performance characteristics of certain constructs due to the underlying GPU architecture (esp. memory access and branches). Obviously, there are extensions related to GPU specific things, but those are quite few and far between (though important for performance). Most everything related to GPU control look and act like library functions.

p_l2• 5 years ago

Niagara doesn't do speculative execution, and their SMT model is wildly different from intel's - specifically, they are a variation of barrel cpu where you duplicate minimal amount of resources to hold state then execute X instructions per thread in round-Robin fashion. Similar setup is used on POWER8 and newer (which allows you to dynamically change the amount of threads available).

AceJohnny2• 5 years ago

It's really fun (?) to work with hardware designers, for whom even the registers that the code accesses is considered "high-level".

Abstractions, all the way down... even our digital bits are a joke to the analog designers.

protoman3000• 5 years ago

Thanks for this interesting read.

Say a platform superseded that C platform the article is describing, allowing for an unleash of power and parallelism in computing, what are the implications for Linux and other operating systems?

qayxc5• 5 years ago

I doubt that outside of special cases this would actually provide significant benefits.

Suitable languages already exist (see Erlang), but the fact of the matter is that a vast array of problems don't benefit from parallelism or aren't parallelisable at all. Not to mention that a good chunk of the benefits would be negated by the increase in coordination and synchronisation between parallel tasks.

usrbinbash7• 5 years ago

The Problem is not with C alone however.

Speculative Execution, branch prediction and look ahead to the next 25 instructions, wasting huge amounts of power,...I mean, what?

The article even mentions that there is another way:

In contrast, GPUs achieve very high performance without any of this logic, at the expense of requiring explicitly parallel programs.

Yes, C doesn't support that very well. But it could, and other languages, namely Rust, Go & Julia already can. So maybe it's time to do in CPU design what Go did in language design, and hit the brakes on complexity?

We don't need smarter processors, we need processors that can do high throughput on many execution units, and languages that support that well.

jcelerier4• 5 years ago

Please no :( unparallelizable tasks are already slow enough. My guitar effects rack (where effects all have to be processed one after each other, by design) uses barely less CPU% in a 2020 computer than in a 2011 one and it is extremely frustrating.

usrbinbash7• 5 years ago

uses barely less CPU% in a 2020 computer than in a 2011 one and it is extremely frustrating.

But if that's the case, what's the point in all that extra-complexity in the CPU, if in the end the benefits seem to be miniscule?

pjc502• 5 years ago

we need processors that can do high throughput on many execution units, and languages that support that well.

Language support is very difficult for this, for a whole bunch of reasons - and it often requires redesigning the entire program and its data structures. It is still the case that most code the end-user is waiting for is JITted Javascript, which is why Apple focused so much effort into making that fast. Which is forced to be single-threaded. Hence all the big/little CPU designs; you get one or two high-speed high-power cores, and some low-speed low-power cores.

usrbinbash7• 5 years ago

    go doSomethingWith(x)

    Threads.@threads for x = 1:42

    thread::spawn

How is that difficult? The problem is that people are taught that concurrency and the capacity for parallell execution is somehow difficult. It really isn't.

It is still the case that most code the end-user is waiting for is JITted Javascript

That's a problem with JavaScript, not with language design. JS is simply not a very good language, and its lack of support of parallell processing is just one of its many problems.

eloff• 5 years ago

There are already processors that are highly parallel, high throughput execution. GPUs as you point out. The reason they haven't replaced CPUs is not laziness, it's that many problems are not trivially parallel. Today's mix of CPUs that are fast at serial execution plus multiple cores, SIMD, and GPUs seems to give a good mix of flexibility to program for.

IshKebab• 5 years ago

Rust and Go don't support GPU-style parallelism natively (I guess Julia probably does). You wouldn't even want that for 99% of programs. It's only useful for big mathsy tensor operations with very little flow control.

Most programs are not like that at all.

pyjarrett• 5 years ago

languages that support that well

Ada has had built-in tasking for a long time, and is now also getting a built-in parallel blocks and parallel loops (iterators and for) structures in Ada 2022.

smlckz• 5 years ago

In C, a read from an uninitialized variable is an unspecified value and is allowed to be any value each time it is read. This is important, because it allows behavior such as lazy recycling of pages: for example, on FreeBSD the malloc implementation informs the operating system that pages are currently unused, and the operating system uses the first write to a page as the hint that this is no longer true. A read to newly malloced memory may initially read the old value; then the operating system may reuse theunderlying physical page; and then on the next write to a different location in the page replace it with a newly zeroed page. The second read from the same location will then give a zero value.

What? What is the benifit of such behaviour?

What does other OSes do in this regard?

bodhiandphysics7• 5 years ago

Lots of programs malloc a lot of memory, and do nothing with it for a while. This allows the os to wait for a low load time to handle memory allocation.

pm2153• 5 years ago

Is there much benefit from delaying the zeroing to first-write rather than first-read (which I think is effectively where Linux does it)?

phendrenad2• 5 years ago

"Assembly isn't a Low-Level Language: Your heterogeneous multicore Apple M1 with integrated GPU and multi-tier SRAM cache isn't a 1970s PDP-11 minicomputer, you can't just MOV RAX, EAX and expect it to work, you complete and utter jackwagon!"

veltas2• 5 years ago

C's pre/post-increment is from B which was designed for an older arch, not PDP, and B still had the pre/post increment and decrement. It's now a well known misconception that C was based on the PDP's instructions, it doesn't hold water with the facts. C did redesign for one thing on the PDP and that's byte addressing... and we're still using byte addressing most places, and frankly if you don't then C isn't incapable with that.

shakow3• 5 years ago

and we're still using byte addressing most places

Which is a masquerade in itself, as in the silicon, addressing works at the memory line level.

bodhiandphysics7• 5 years ago

The cpu bus is what’s important. Bus addressing us at the byte level

flohofwoe3• 5 years ago

Yet pointers are byte-granular. A more interesting question is rather, why are pointers no bit-granular by now, it would just "waste" 3 bits, and 2.3 exabytes ought to be enough for anybody ;)

kovacs_x• 5 years ago

One should always think of a C as high level Assembler. Not less, not more. Everything else (parallel programming, threading, ...), are "higher" level paradigms, where C "robustness" is more of an obstruction, not a help.

Imo, from hw designers perspective C is a language as "high" as it goes, when for a software engineers it's often "as low as it goes".

lproven3• 5 years ago

Have you RTFA?

Because that is exactly the point that the article refutes.

Const-me• 5 years ago

which attempt to hide latency

It does exceptionally good job at these attempts. If you think manually managed caches are fun, read [1] for an illustration what amount of efforts is required to sum an array for an architecture where on-chip RAM is manually managed. Another interesting case was Cell CPUs in PS3, I don't have hands on experience but I've read that it was equally hard to develop for.

A low-level language for such processors would have native vector types of arbitrary lengths.

A low-level language would have native vector types of exactly the same lengths as underlying hardware. "arbitrary" is overkill unless the CPU supports arbitrary-length vectors.

Despite not specified as a part of language standard, all modern C and C++ implementations support these things. Specifically, when compiling into AMD64 instructions, the compilers implement native vector types, and vector intrinsics, defined by Intel. Same with NEON, all modern compilers implementing what's written by ARM.

you must be able to compare two structs using a type-oblivious comparison (e.g., memcmp)

Using memcmp on structures is not necessarily a great idea, these padding bytes can be random garbage, it's not specified.

with enough high-level parallelism, you can suspend the threads.. The problem with such designs is that C programs tend to have few busy threads.

Not just C programs. User input is serial, it can only interact with 1 application at a time. Display output is serial, it delivers a sequence of frames at 60Hz. Web browsers tend to have few busy threads because JavaScript is single threaded, also streaming parsers/decompressors/decryptors are not parallelizable.

ARM's SVE (Scalar Vector Extensions)—and similar work from Berkeley—provides another glimpse at a better interface between program and hardware.

Just because it's different does not automatically make it better. The main problem with scalable vectors, it seems to be designed for problems CPUs no longer solving. For massively parallelizable vertical-only FP32 and FP64 math, GPGPU is the way to go, an order of magnitude faster while also being much more power efficient. CPU SIMD is used for more than vertical-only math. One thing is non-vertical operations i.e. shuffles, trivial use case: transpose a 4x4 matrix in 4 registers. Another one is operations on very small vectors, CPUs even have a DPPS instruction for FP32 dot product. For both use cases, scalable vectors make little sense.

a garbage collector becomes a very simple state machine that is trivial to implement in hardware

People tried that a few times, first with Lisp, then with Java chips. General-purpose CPUs were better.

Running C code on such a system would be problematic, so, given the large amount of legacy C code in the world, it would not likely be a commercial success.

nVidia did precisely that, made a processor designed purely for compute speed. I wouldn't call them a commercial failure.

[1] https://www.nvidia.com/content/GTC-2010/pdfs/2260_GTC2010.pd...

dash2• 5 years ago

Previous HN discussions with many comments:

https://news.ycombinator.com/item?id=16967675

https://news.ycombinator.com/item?id=21888096

flohofwoe3• 5 years ago

This one caught my eye:

for example, you must be able to compare two structs using a type-oblivious comparison (e.g., memcmp), so a copy of a struct must retain its padding.

This definitely doesn't work in the real world because the padding bytes will contain random junk which isn't copied along in some situations, depending on the compiler and optimization level.

Also, IMHO what the article calls "low level" might be important for compiler writers, but isn't really all that relevant for most programmers when the comparison is to "high level" languages like Java, C#, Javascript or Python.

In my mind, the most important property of a low level langauge is to provide explicit control over how data is layed out in memory, this is usually an afterthought in high-level languages, if possible at all.

Or more generally: how much explicit control does the language allow before the programmer hits the "manual optimization wall". In that sense C is fairly high level, especially without non-standard extensions, but still much lower level than most other programming languages. I think there is definitely room for more experimental languages between C and assembly.

rightbyte• 5 years ago

In that sense C is fairly high level, especially without non-standard extensions, but still much lower level than most other programming languages.

In practice you are using gnu c etc. The Stackoverflow consensus that you are supposed to stick to the standard no matter the complexities of the workarounds is rather annoying and seem to be quite prevalent.

sharpneli• 5 years ago

The reason for sticking to standard is when inevitably someone will try to compile it with another compiler. It’s far easier just to stick to the spec instead of fixing non standard code.

For some reason it has been me several times doing the fixing as original dev thought it’s completely fine. I hope to save others the pain.

noobermin2• 5 years ago

This article is my go to url when arguing on the internet with people who are under the illusion that "C is portable assembler" when it really isn't and their mental model is actually not what a computer actually does today (or has performed in the past few decades).

pjmlp4• 5 years ago

Another way is to ask them to describe how to perform certain systems programming tasks with the restriction that it was to compile under pure ISO C mode.

The answer will always be external Assembly.

dunemaster• 5 years ago

This article could have been 1/8th of its length and the message would still be clear. Modern writing is vague, overly detailed and inelegant.

lproven3• 5 years ago

Not really; as evidence, look at the number of comments here that failed to get the point.

When you present a viewpoint to people whose profession and living depend on believing something contrary to it, it's necessary to present a lot of evidence and solid reasoning to get even a small number of the smartest of such people to consider your point.

marcodiego• 5 years ago

Looks like C being Low-level or not needs a precise or formal definition of low-level language. Without that, it falls down to opinion.

lproven3• 5 years ago

I am wondering if this excellent essay has surfaced again because I just tweeted it in reply to a popular Twitter account.

I dispute your conclusion, and this is the reasoning, as expressed in the article:

"High level" is tricky to define, because high is a relative assessment. But "low level" has a clear, agreed meaning: relatively similar to the machine's instruction set and architecture; close to the metal; comparable to assembly language.

And the point of the article is that C is close to the architecture of a PDP-11. Modern CPUs are nothing like the PDP-11 and haven't been for a third of a century or more. C models the architecture of a 1970s minicomputer, and 21st century computers are nothing like 1970s minis – they just run similar OSes.

If C is not close to the real architecture, then it's not low level.

The fact that there's nothing mainstream which is closer is irrelevant. The C programming model is nothing like modern multicore SIMD superscalar 64-bit CPUs with out-of-order execution, branch prediction etc.

If it isn't close to the metal, then it isn't low-level. C is neither any more. QED.

zaptheimpaler• 5 years ago

Is there a language that exposes the stuff below assembly? Like assembly language presents a nice sequential ordered view of processors as if they execute instructions 1 by 1, but we know the reality is quite different - each instruction is many micro instructions, they compute data dependencies and parallelize accordingly, pipeline & predict heavily etc. We do so much to fit async data dependent computation into sequential models, only for that to be translated back into async/parallel by the CPU. I wonder what that kind of PL would look like?

fault1• 5 years ago

perhaps HDLs like (system)verilog and vhdl? then you truly are interacting with the architecture and you care much more about things like timing, clock cycles, etc.

kazinator• 5 years ago

This article presents a false premise.

Intel (and other) processors present a serial programming model in spite of parallelism under the hood because that is required for stable semantics in assembly language programming, and for the instruction set architecture to be a stable target for any higher level language whatsoever. It's not because of the expectations of the C programming model.

The unoptimized, instruction by instruction fetch-decode-execute model of the instruction set pins down what the code means, which is super important, otherwise there is chaos.

Moreover, machine language executables must continue to work across evolution of the architecture family. The way an new Intel processor is today has as much to do with C compilers as it does with the need for someone out there to run MS-DOS or Windows 95.

Compilers could better deal with chaos at the architecture level, because breakages at the architecture level can be treated as a new back-end target, and code can be recompiled. It's the code that doesn't get recompiled that you have to worry about.

kaba05• 5 years ago

But converting (even just-in-time) between two architectures is not that hard — if every new code could get some better performance/efficiency at minimal overhead for “legacy” applications, it may well worth it.

dev_tty01• 5 years ago

Moreover, machine language executables must continue to work across evolution of the architecture family. The way a new Intel processor is today has as much to do with C compilers as it does with the need for someone out there to run MS-DOS or Windows 95.

Apologies, but I think this belief that compatibility must be maintained is false. Apple has made three incompatible binary architecture transitions and dealt with the issues via emulation. Each time, the performance gains from the new architecture have masked the inefficiencies of the emulation layer until applications could be rebuilt. It just takes a willingness to say no to legacy compatibility. Microsoft is finally trying to take a similar step with WinARM. I suspect the x86 architecture will get relegated to the dust bin within a few more years. The instruction set is just too polluted and chaotic at this point. As an example, the Apple M1 instruction decoder is much wider than x86 and part of why that was practical to implement in a power efficient way is that the M1 instruction set is much less variable in length.

JoeAltmaier• 5 years ago

Maybe this doesn't matter so much. Remember, the cpu may be re-writing your code and re-ordering many operations. The latest Intel processors can examine your code and write new microcode to do it more efficiently. Heck, even identifying a fault to an instruction in 'your' code has become problematic!

avidphantasm• 5 years ago

Effing great article. I am starting down the path of learning rust and wondering how its mutable-first design and ownership model alleviate some of the problems identified in making C fast on contemporary machines.

wly_cdgr• 5 years ago

Pretty sure K&R calls C a high level language. Just a matter of perspective, innit: a high level language to assembly programmers, a low level language to .NET/JVM/web programmers, and prob something like a lower-mid-level language to someone looking at the whole tower from a distance

jleyank• 5 years ago

I have an old button that says “C combines the power of assembly language with the flexibility of assembly language.” Might have changed now, but it was a humorous yet valid observation then.

C Is Not a Low-level Language (2018)

Related Stories