Programs to Read

http://wiki.c2.com/?ProgramsToRead

by mpweiher • 8 years ago

248 56 8 years ago

wiki.c2.com

sneak• 8 years ago

+ nginx

codezero• 8 years ago

+ redis

Agree on nginx - I wanted to make a crazy change and found it extremely easy to understand how I could have done it and why doing it would be stupid :)

beefhash• 8 years ago

On the topic of C, I'd also strongly recommend mandoc[1]. It solves a number of hard problems (indexing and searching of man pages, rendering and parsing markup and translating said markup to HTML, PDF and tty output. The code remains fairly accessible. Definitely one of the codebases I regularly refer to for style and practices.

[1] https://mandoc.bsd.lv/cgi-bin/cvsweb/

kristapsdz• 8 years ago

Thanks! :)

jolmg• 8 years ago

So, is wiki.c2.com something that people keep writing on or is it really readonly nowadays? Is there a way to become a member? I just mistakenly signed up to hypothes.is from the right side-bar thinking it was c2.com.

becca_smth812• 8 years ago

Seems like it hasn't been updated in a while. Each page has an "last updated" timestamp. The homepage says: "Last edit December 19, 2014". There's also a Github project but hasn't been updated in 2 years: https://github.com/WardCunningham/remodeling

fao_• 8 years ago

Hi, for some reason you're shadowbanned? As a new account I find this really unusual. It's likely you might need to verify your email or something. I've vouched for this comment, which is why it appears.

On Topic: there was a discussion a year ago about what the progress block is: https://github.com/WardCunningham/remodeling/issues/24

It seems that he's stuck with some files that are mixed-encoding and can't repair them, it's likely the wiki will be read-only until that problem is solved, unfortunately.

tabtab3• 8 years ago

Ward Cunningham decided to close c2 to new material in part because he got tired of battling a determined vandal who was all bent out of shape over grammer and spailing, and second to try out his new "Federated Wiki" concept. So far, Federated Wiki is a flop in my opinion.

C2 was interesting in that software developers and IT professionals tried to categorize debates and disagreements over software engineering issues. Almost nothing was ever settled, but you got different perspectives. For example, is the "best" software about maximizing economics & psychology (human grokking), or symbolic parsimony? Is it better to hire 5 geniuses or 10 ordinary engineers?

amyjess• 8 years ago

Ward Cunningham decided to close c2 to new material in part because he got tired of battling a determined vandal who was all bent out of shape over grammer and spailing

It's honestly pretty upsetting that one vandal is enough to ruin one of the oldest and most significant institutions on the web for everybody.

oraraonaro• 8 years ago

Is there a resource with the best snippets of the code sources listed here? Can't be bothered to trawl through for the good stuff.

hangonhn• 8 years ago

I went through the list and the comment about Apache Lucene really piqued my interest. I found the Lucene code in GitHub but some of the sources have more boilerplate Java comments than code! Anyways, I remembered Java Docs are often excellent so here's the Lucene Java Docs:

https://lucene.apache.org/core/7_5_0/core/index.html

From my experience, studying the API Docs for Java and using it as a guide to the code makes understanding the code much easier.

The code is here:

https://github.com/apache/lucene-solr

riffraff• 8 years ago

Many years ago I enjoyed Code Reading[0] which is a whole book which basically just discusses snippets from open source codebases.

It might be a bit old (I honestly don't remember the specifics) but most of the code was already "mature" at the time so I think it would still be valid.

[0] https://en.wikipedia.org/wiki/Code_Reading

kornish• 8 years ago

Another list of reads worth mentioning is the Architecture of Open Source Applications book [1]. Each application is small (< 500LOC) and contains design rationale in an approachable literate programming style.

[1] http://aosabook.org/en/index.html

lucaslee2• 8 years ago

I saw the Linux kernel was recommended many times here, but how many people actually read it? Where do you even start? The Linux kernel has around 60,000 files and 25 million lines of code...

I think smaller projects are better for learning purposes. If you are interested in reading some smaller projects, check out my project here https://github.com/CodeReaderMe/awesome-code-reading.

ken2• 8 years ago

Nobody ever wrote a 25MLOC program from start to finish, so I don't think it makes much sense to read it that way.

I'd read it the way it was written: from the beginning. What's Linux 0.01 look like? What's the next changeset after that release look like? What was necessary to add the driver for your favorite device? What changes were made for your particular CPU?

Programs are not static works (except maybe TeX and Metafont). They exist in the form they do in order to be amenable to changes. So look at the changes that drove it.

lucaslee2• 8 years ago

Read the early versions, that's exactly what I am doing too!

I read Bitcoin 0.1.5 [1], which only has 15,000LOC and is the first tagged version on Github. Compared with the current Bitcoin codebase with 320,000LOC, it's much less daunting!

[1] https://github.com/CodeReaderMe/awesome-code-reading/issues/...

afarrell• 8 years ago

Have you actually done this? What was your experience?

stdplaceholder• 8 years ago

The linux kernel has got, conservatively, thousands upon thousands of defects. The only reason to read it is in case you really need to know how it actually works, because the documentation for some syscall is wrong or your systems aren't working right in practice, or you need to know how some undocumented hardware works. Otherwise I'd say it's best left unread.

stiff• 8 years ago

Most of the kernel code is in the drivers, the general purpose subsystems (VFS, I/O scheduler, task schedulers, memory management etc.) are a small fraction of those 25 million LOC and largely independent of each other so it is not that hard to build some understanding of them.

Some ways you can start:

- Here is start_kernel(), the kernel entry point after booting up and handling the lowest level stuff in asm: https://github.com/torvalds/linux/blob/v4.19/init/main.c#L53...

- grep for SYSCALL_DEFINE to find definitions of syscalls, e.g. this is open(): https://github.com/torvalds/linux/blob/v4.19/fs/open.c#L1076

(understanding how the I/O and networking system calls work is quite helpful for application developers, even if you work in node.js, python or another high level language)

- this is the struct that represents each process in the system, you can pick some interesting field and search for where it is used and where updated: https://github.com/torvalds/linux/blob/v4.19/include/linux/s...

Finally, the linux-insides book is pretty helpful: https://0xax.gitbooks.io/linux-insides/

blattimwind• 8 years ago

(understanding how the I/O and networking system calls work is quite helpful for application developers, even if you work in node.js, python or another high level language)

Being able to semi-quickly figure out how the kernel actually performs some I/O operations and what the exact semantics are is tremendously useful when working with low-level I/O applications (e.g. database-like applications).

CodeArtisan• 8 years ago

I have found NTOS to be much better documented than Linux. For example,

https://github.com/markjandrews/wrk-v1.2/blob/master/base/nt...

Almost every line of code has comments.

megous• 8 years ago

Where do you start depends what you're after. You should start by defining questions, then by searching for existing documentation, then reading the header files and data structures, and then perhaps some code. Also read userspace counterparts of the interface you're studying if applicable.

Worked fine for me when getting into media subsystem to write camera and camera soc interface drivers.

Nzen• 8 years ago

James Hague's recommendations [0] of classical programs are on my bucket list.

[0] https://prog21.dadgum.com/210.html

cyphar• 8 years ago

In my view, "reading the kernel" is not really a useful exercise. Source code isn't a book, and most code isn't really a pleasure to read by itself.

Instead I would suggest that you try to debug a problem, or try to understand what a particular syscall does rather that starting from init/main.c. If you have a goal in mind (trying to solve a problem or understand a specific aspect of the kernel) you are far more likely to get useful information out of the exercise.

Another useful hint is that you should just ignore the majority of code that looks alien. Linux uses a lot of macros and synchronisation primitives that you probably aren't familiar with (RCU for instance). It's much more useful to take note of the things you don't understand and just move on to the code code that you do understand. Most of the macros and synchronisation primitives are used all over the kernel so you're very likely to get to grips with them over time.

I really hope that bpftrace will make it easier for people to get into kernel debugging and thus have a nicer "in" to kernel development.

potta_coffee• 8 years ago

I often recommend the Flask source code, if you're into Python. Very clean and well structured, great example code.

tomxor• 8 years ago

Some are GreatProgramsToRead and some are not

Indeed! It took me a while a appreciate (because it can be painful). Reading bad programs can be very educational, especially when you try to infer the decisions or processes that caused it to be bad in a particular way.

ryanmccullagh• 8 years ago

Question, where does one start, when planning to read the Linux kernel? There is so much code. I have read it, but randomly. I have read contents of net/ kernel.

I have read the main method where it attempts to launch pid1 of /bin/bash, etc.

Is there a really good place to start reading? For example how does Linux talk to the hard drive?

What's the first thing that happens in the kernel?

seotut2• 8 years ago

Well, some famliarity with the hardware doesn't hurt. Years ago when I studied the Linux kernel myself, I started with qemu, and the gdb stub, and walked line-by-line through the boot process.

But as others suggest, taking an older version, or even simpler unices(xv6,netbsd) might be easier.

varjag• 8 years ago

Start skimming through the included documentation files. You don't have to do a thorough read, but the Documentation folder is structured in a way mirroring the organization of the kernel. They will help you to understand the structure, and you can get deeper understanding of the essential parts that way.

Once you have mental map of major areas of the kernel it will be easier to relate to the sourcetree.

There used to be a number of great books on fairly esoteric issues of the kernel (like, comprehensive explanations of network stack). However most of them seem to be stuck at 2.6 and are no longer very up to date.

wwweston2• 8 years ago

Knuth vs McIlroy's solutions for a Word Frequency programming challenge:

http://www.leancrew.com/all-this/2011/12/more-shell-less-egg...

Knuth's solution has its own elegance and interest including various efficiencies and from-scratch facilities.

McIlroy's will make you wonder how much you're wasting in time and complexity.

YZF• 8 years ago

Anywhere I can see Knuth's solution online? Can't seem to find the link in there.

wwweston2• 8 years ago

Here's the paper:

https://www.cs.tufts.edu/~nr/cs257/archive/don-knuth/pearls-...

(ACM ref, too: https://dl.acm.org/citation.cfm?id=315654 )

Also, I found a blog post which links to a C version of the exercise in the spirit of Knuth's approach, and a Haskell version in the spirit of McIlroy's:

https://franklinchen.com/blog/2011/12/08/revisiting-knuth-an...

akkartik4• 8 years ago

One claim that caught my eye: Tex being unbuggy because it's written in the Literate Programming style. I like LP but that's a big claim. Occam's razor suggests a more likely answer: it was written by Knuth! A great programmer, and also one who grew up programming non-interactively: http://ed-thelen.org/comp-hist/B5000-AlgolRWaychoff.html#7

deathanatos• 8 years ago

On the one hand, I see your point. But on the other hand, if a master of some skill says "This is how I achieve the results I achieve." — perhaps imitating that might just be valuable. Does a dancer not get better by learning to imitate (at least at first) a master? (Though perhaps eventually, a deeper understanding of the artform might take hold, at which point experimentation and improvisation allow the dancer to explore undiscovered variations of the artform.)

And I don't think it is just that Knuth does it that catches my attention either. Ideally, I think (if I understand the premise of literate programming), literate programming would force the programmer to take the time to explain the code being written, and ideally, the rationale behind why that code solves the problem at hand. And to do that, the programmer must first understand the code being written, and I think upon attempting to do that, they'll find they have not sufficiently thought the problem through.

I fear too many of the coders I know would not honestly attempt it; they are in too much of a hurry to get the code written.

Now, I don't think literate programming is the only way to accomplish that; simply writing good docstrings and comments might suffice to a good degree (and good ones: I too often review code where the docstring is a basic repetition of the function name and arguments, or where the comments explain what is being done, but not why). But far too often I see no comments, no docstrings, no tests, just miles and miles of code, and so it is no wonder that Knuth is what he is.

akkartik4• 8 years ago

No disagreement there! Though you may violently agree or disagree with this thing I wrote a couple of years ago: http://akkartik.name/post/literate-programming

I also just looked at your profile, and it resonates a lot. Tell me what you think of mine: http://akkartik.name/post/about

emmanueloga_2• 8 years ago

Reading TeX original source is easier said than done... I think I managed to get PDF some time ago but took me like 30 minutes of browsing. Not talk about running it...

svat4• 8 years ago

It's available as a book (TeX: The Program, volume B of Computers and Typesetting), and Knuth considers the book as the canonical thing to read. You almost surely don't have a PDF of the entire book, but it's easily available in good libraries: http://www.worldcat.org/title/tex-the-program/oclc/826569131 (or you can buy it: as I was typing this comment I just finally bought a used copy from Amazon for $9 which is pretty good for a hardcover book).

If you have a typical TeX distribution installed, you can see an approximation by running `texdoc tex`, which brings up the PDF that is also available online e.g. here: http://texdoc.net/texmf-dist/doc/generic/knuth/tex/tex.pdf -- but it is a bit hard to make sense of if you're not familiar with the conventions of WEB (the book starts with a "How to read a WEB") -- as a workaround maybe you can read webman (http://texdoc.net/texmf-dist/doc/generic/knuth/web/webman.pd...) though that's meant for the programmer (working with the raw .web files) rather than the reader (working with the typeset output).

And it helps to be familiar with Pascal, and the typical conventions of programming style in the 1970s (e.g. that of the P4 (Pascal) Compiler and Interpreter, also published as a book and also available online: https://homepages.cwi.nl/~steven/pascal/ )

I've been reading the TeX program in bits and pieces for over a year now, and only now do I have a rough sense of it all :-)

e12e• 8 years ago

Seems like this thread needs a link to: http://aosabook.org/en/index.html

qwerty456127• 8 years ago

Such a small list, such a humble selection of programming languages :-(

dfrunza• 8 years ago

Programs to read ... first entry 'JavaUnit'.. SKIP!

kureikain• 8 years ago

If you like this kind of reading code list, I run a newsletter at https://betterdev.link/ where we have a small section that include one or two interesting Github repo per languages per issue.

injb• 8 years ago

The Subversion api should be on this list too. I had occasion to do some development with it before and it's beatifully written.

Programs to Read

Related Stories