Brian Kernighan on the Typesetting of “The Go Programming Language” Book

https://rkrishnan.org/posts/2016-03-07-how-is-gopl-typeset.html

by melling • 10 years ago

224 75 10 years ago

blue1• 10 years ago

It's sad that a complete structure writing/publishing workflow is still so hard. XSL-FO is awkward and limited, and will probably remain so. TeX is not design-friendly. On the other side, InDesign is mostly visual only. Framemaker seems to be only serious tool but it's very expensive.

jernfrost• 10 years ago

What about Apple Pages? I find that very good and easy to deal with typesetting. But I am no expert in the field so it might have sever weaknesses I am not aware of. Anyway I like that it seems to have been designed with layout in mind from the start unlike MS Word e.g. where doing layout has never felt natural.

coolsunglasses3• 10 years ago

Typesetting isn't the same as word processing, you need a lot more power than something like Pages gives you.

For example, our book is 308,000 words and 32 chapters. By the time it's done it'll probably be ~310-325k plus index. LaTeX means I can change one line in the header and have the build completely re-typeset the book according to those parameters programmatically, and it does a _very_ good job.

Re: index - how do you generate an index with Apple Pages? And I don't mean concordance, I mean a proper index. With LaTeX it's...programmatic. We provide the metadata in the form of `\index{blah}`, it generates the index for us.

It's extremely hard to find anything that's competitive on all these fronts and which is also open source. LaTeX is pretty much it, everything else is focused only on basic formatting & presentation and hasn't even gotten to a proper typesetting algorithm.

baldfat6• 10 years ago

A word processor is a totally different beast from type setting.

https://programmingforresearch.wordpress.com/2012/03/23/word...

Though people have been trying to solve this in Word and templates but those still fail at technical books still IMHO.

Finnucane2• 10 years ago

I was a devoted FrameMaker user until Adobe killed the Mac product. I used to have to do fairly complicated legal documents, and it was worth every penny. Plus, it was a very good word processor--you never had to leave the program to edit text. So I used it for everything I had to write.

A book like this could have been handled fairly easily in FrameMaker, but sadly Adobe has decided to position in it to be only for large enterprise shops running Windows. They're probably (rightfully) afraid it would cannabalize InDesign sales if they made it more accessible.

reacweb• 10 years ago

I have used it in 1992 on Sun stations. It was a delight. I am pretty sure it would gain a lot of attraction if they open sourced it.

shepard• 10 years ago

Hm. I never heard of FrameMaker before, but their 2015 release is available at $29.99/month, which is hardly "positioned only for large enterprises".

girzel3• 10 years ago

I keep an eye on the Scribus project from time to time. Who knows, maybe someday it can compete.

https://www.scribus.net/

baldfat6• 10 years ago

Sadly the tool used is not maintained currently

GNU troff is looking for a maintainer. If you’re interested, please take a look at this general information about GNU packages and being a GNU maintainer, and then email maintainers@gnu.org with a bit about your background and particular interest in this package. Thanks.

Please note that maintaining this package requires assigning copyright to the FSF, and ensuring that any future contributors also execute papers, since that is what past authors and contributors have done.

girzel3• 10 years ago

Written in XML?! That's the first I've heard. Though also a fine proof that what you're writing is much more important than how you're writing it.

(Still, though -- XML?)

insertnickname2• 10 years ago

Why not XML? XML was made for documents, they were writing a document.

girzel3• 10 years ago

For an actual typing-on-the-keyboard format, it seems like almost anything would have been better, particularly given the number of markup languages that can translate trivially into XML.

I may have misunderstood -- he was actually typing in XML tags as he wrote the book, right?

Sharlin• 10 years ago

XML may have been made for documents but it is certainly one of the worst language families ever for actually typing documents by hand.

jdeisenberg• 10 years ago

Also, XML + XSLT gives you the capability to easily transform your original to other formats.

Shorel• 10 years ago

XML is awful in a time before ZenCoding/Emmet.

Now, not so much.

dog>cat*3

tombert• 10 years ago

I agree that ZenCoding has greatly simplified XML development, but I find that even with it you're still beholden to ugliness when trying to edit it.

oxplot2• 10 years ago

Recently I've been playing with Prince [1] and I've found it to be very powerful with excellent support for paged media. Given that HTML, CSS, Javascript, SVG is used to build the document, it's also familiar to most people already. There doesn't seem to be any open source competitor to Prince though.

[1] http://www.princexml.com/

coolsunglasses3• 10 years ago

There doesn't seem to be any open source competitor to Prince though.

This would be my issue with using it. This is partly why I stay with LaTeX for our book as I can convert it to other formats if I really need to. I find LaTeX gives me more control over presentation & type-setting than web-based stuff as well.

katabasis• 10 years ago

I want to echo the support for Prince here, even though it's not open-source it is a very useful piece of software. I create web-based art books at a museum[1], and Prince has allowed us to generate high-quality PDFs[2] and even print-on-demand editions of the same book for very little additional effort. Full-bleed images, roman numeral page numbers, footnotes, multi-column text pages, all are possible through CSS.

In a past life I was a print designer relying on Adobe software for everything, and it is tremendously liberating to be able to leave that world of vendor-lock-in behind (InDesign has a nasty habit of making each new version's files incompatible with previous ones). So compared to where I'm coming from, Prince (even as proprietary software) is a big step forward because it allows me to work in open formats.

[1] http://www.getty.edu/publications/romanmosaics

[2][PDF, 8MB]: http://www.getty.edu/publications/romanmosaics/assets/downlo...

jdnier• 10 years ago

I'm happy to know about Prince, but US$3800 for a server license is beyond extravagant for any but large corporate users.

tomjen32• 10 years ago

It also means your documents are locked to a proprietary standard that isn't that widely used (compared to say MS Word) and as such, may not be totally recoverable 10,20 or 30 years from now. You want to keep that in mind when you choose which typesetting system to use - for the record, I keep a digital diary that I intend should still exist 50 years from now. I use only markdown, and the occasional entry that mention really private stuff is ascii encrypted with GPG.

jnbiche• 10 years ago

It also means your documents are locked to a proprietary standard that isn't that widely used (compared to say MS Word)

HTML and CSS?

gmfawcett• 10 years ago

PrinceXML is excellent, we used it for many years for typesetting programs for conferences, etc. After a while, though, I picked up enough LaTeX to be competent at generating LaTeX from our applications (just as we would generate HTML or any other markup) and that's what I use for report generation now. While there are more templates to maintain, the quality of the results is absolutely worth the effort.

On a tangent, a very nice "win" with the (pdf)LaTeX approach is that it's fairly easy to collate other PDF documents into a larger one, with a custom table of contents, consistent running headers and footers, index of authors, etc., and intersperse them with other content from our database. (This was done by splitting the child documents into pages with pdftk, then inserting them as "PDF graphics" into the generated LaTeX document -- and as long as the original PDFs were searchable, they remained searchable in the final output.) In one of our paper-intensive processes (a proposal approval workflow), this approach cut down tremendously on manual labour that was being done to prepare proposal dossiers, and the resulting collation was much easier for the reviewers to work with (e.g., everyone could turn to page "K-34" in the dossier, and be certain they were looking at the same page of the same document).

FraaJad8• 10 years ago

As a typesetting aficionado, I have tried most of the suggestions here (Prince, Troff, Lout, Docbook XML, Asciidoc, LaTeX, ConTeXt, .. except GUI typesetting tools.)

TeX appears bad from a distance, mostly from hearsay, until you have to write an actual technical book of certain expected high production standards and you realize TeX gives you the best shot at doing so.

There is a learning curve. It is an investment of your time, but it pays off.

Also, it is not like LaTeX et al are standing still in time. There are companies and teams working on improving the ecosystem everyday. I personally like ConTeXt unless I'm doing a predefined template exists (for conferences etc.,) where it makes sense to just use their LaTeX guidelines.

baldfat6• 10 years ago

Sadly I have to work in Windows at times. Context seems to not be the most supported Windows platform. (The link to the Windows simplified package is 404) There is a New Installer http://wiki.contextgarden.net/Windows_Installation_new which does not contain a working binary. The only way to find anything for windows leads to a tar file.

I still think there is a better way to do typesetting in 2016 but all I have seen are a bunch of smaller projects without much support.

Personally I would love a markup language redefined with built in PDF-X support and a simple infrastructure. Context is Ruby, Tex, and ConText. I find LaTex to work well (for the most part) but it is not very flexible but I found the MikText Package Manager as a big step forward.

jstarks• 10 years ago

I wonder if it works under the new Linux subsystem.

xemdetia• 10 years ago

For LaTeX on Windows I enjoy http://miktex.org/.

maxerickson• 10 years ago

I thought ConText was using Lua for scripting things.

This suggests that:

http://wiki.contextgarden.net/Mark_IV

FraaJad8• 10 years ago

I wish I could answer this.. but I'm a Mac and Linux user.

with the new "Bash for windows", it will hopefully get easier in future.

groovy2shoes• 10 years ago

Both the TeX Live and MiKTeX distributions provide binaries for Context on Windows.

sevensor• 10 years ago

Care to comment on the reasons for using ConTeXt versus LaTeX? I've been curious about ConTeXt for a while, but I fall back on LaTeX out of familiarity (and the need to use provided document classes.)

FraaJad8• 10 years ago

LaTeX's selling point is, it allows someone(publisher, conference committee, school, professor) to set what is acceptable typesetting and rest of us can focus on the content.

ConTeXt is excellent if you care about design -- visual, layout, use of non-traditional typeface families, non-traditional print medium, create magazines, glossies, etc.,.

Writing a new style, class etc., for LaTeX feels like it's for "experts only". On the other hand, extending and creating new styles etc., for ConTeXt is much more natural part of creating the document.

ConTeXt is also highly programmable and extensible using Lua, XML(think machine generated catalogue).

With Metapost (graphics language), you can create highly sophistical vector diagrams. I have found pgf/tikz to be highly opinionated and it take much more upfront learning compared to metapost.

ConTeXt is also more intuitive, IMHO.

wyldfire• 10 years ago

I like LaTeX for formatting small works. But would a publisher be willing to take LaTeX input when producing a book?

coolsunglasses3• 10 years ago

They've been a pain in the ass about it IME, but you can convert it to markdown for editing, then PDF for pre-print.

FraaJad8• 10 years ago

many scientific and tech publishing houses have their own LaTeX styles (eg: MIT, nostarch).

akavel2• 10 years ago

Of recent attempts at dethroning LaTeX, there's also SILE, which actually reuses some LaTeX internals. For some examples of the output (hard to find, the author seems not great at marketing), see e.g.:

https://github.com/simoncozens/sile/blob/master/examples/ara...

https://github.com/simoncozens/sile/blob/master/examples/art...

https://github.com/simoncozens/sile/blob/master/examples/fra...

https://github.com/simoncozens/sile/blob/master/examples/par...

https://github.com/simoncozens/sile/blob/master/examples/jap...

https://github.com/simoncozens/sile/blob/master/examples/tes...

For a deeper dive, see the author's presentation at FOSDEM 2015 - I found it really amusing - from the starting line of: "I wrote a typesetting system - by mistake."

https://www.youtube.com/watch?v=5BIP_N9qQm4

FraaJad8• 10 years ago

I skimmed through their manual.. ConTeXt does all of this better, and has been doing this for longer.

jahewson• 10 years ago

It doesn't do math though.

jeblair• 10 years ago

Also, it is not like LaTeX et al are standing still in time.

Indeed -- XeTeX (and XeLaTeX) support Unicode and OpenType fonts natively (including system installed fonts). It's a pleasure to use.

x1798DE• 10 years ago

I liked TeX for typesetting my thesis, but I feel like, looking back, it would be nearly impossible to convert my document to a reflowable format. Not sure what can be done about that if you want nice typesetting in a fixed size environment, though. I'm curious to what extent the dual-format problem has been addressed.

Tomte2• 10 years ago

Heirloom troff supports OpenType and OpenType features. I still haven't found time to look into it, but it sounds awesome.

octref• 10 years ago

Ironically this page typesets Prof. Kernighan's email in a <pre> tag so mobile users could enjoy scrolling left and right 100 times to read it.

tambourine_man2• 10 years ago

Same here, to help future mobile readers:

Hi, Ramakrishnan --

Thanks for the kind words on the typography. The core formatting is just troff (groff, really) with the -ms macro package. We tried Latex briefly but neither Alan or I care for its output, and I personally find it impossible to control. Troff is the devil I know -- it's ugly and irregular in many ways but it will put characters where you want them.

The input was in XML, with a tag set of about 25 items for headings, paragraphs, index terms, program insertion, simple tables, and the like. A Go program converted this either into HTML for rapid viewing on the screen and potentially for an e-book version, or into troff for printing. Using XML was a mild nuisance when writing but the error checking was very helpful.

We wrote a variety of small Go programs and scripts to fix things up, including one that rewrites troff output to do vertical justification so all pages are the same height. There is also some fiddling with the generated postscript to put on printer's marks.

The fonts are Alan's choice, the result of a lot of work on his part to find them and get the right sizes and appearances. The handful of Asian characters were tough to get right; troff doesn't do wide Unicode characters properly, and there are a few places where we rewrote text to hide that fact.

The drawings are all Alan's work, using Google's drawing program. We toyed briefly with using pic but it wasn't really up to the job, and it would not have worked with HTML without a lot of work. We avoided mathematics beyond superscripts, and the tables are pretty limited; eqn and tbl would have been ok but again would not have dealt with HTML.

We should probably write a more organized description of what we did and make the tools available, though I think most readers are less interested in the process than you (and I) are. The other thing is that although one starts with grand ideas of being clean and orderly, by the end the process is somewhat of a mess, with a complicated makefile to keep it running.

Thanks again for writing.

Brian

ShaneOG• 10 years ago

Thank you!

NelsonMinar• 10 years ago

For anyone missing the lede here: the book is typeset in troff. This is astonishing because troff is 40+ years old and AFAIK hasn't gotten significant development in ~10 years. If it ain't broke don't fix it. http://www.troff.org/history.html

coliveira• 10 years ago

Kernighan has always used troff, so it is not a surprise that he is still using it. He can do things with troff that most people would have a hard time to do with LaTeX. On the other hand, this doesn't mean that troff is better, in fact as you said it has not been updated for a long time. For someone starting now, it is so much easier to learn and be productive with LaTeX instead of troff.

jklowden• 10 years ago

I wrote to Kernighan a few years ago, when "D is for Digital" came out, to ask him, in effect, if troff had been made obsolete by TeX. At the time I had begun using it, and wondered if it was worth investing my time to learn.

He answered that he still preferred to use groff, having tried alternatives. Since then I have used groff exclusively for writing documentation and papers. It never runs out of gas, and always produces attractive output. It offers a complete suite -- math, tables, pictures, graphs -- all in one package. It's fast, adaptable, and easy to learn. It has the highest ratio of text to markup of any typesetting system, including TeX and Markdown. It is the single most overlooked and under appreciated project on the planet.

Whether it's had "significant" development in the past 10 years depends on what that means. Current groff produces PDF format directly, without a Postscript intermediary. The PDF macros support links and TOC generation. I think that all took place in the last few years.

claystu• 10 years ago

What platform do you write on? Linux? Mac? I'm genuinely curious.

I have been looking for substitutes to Latex ever since writing my dissertation in Latex and having to debug/tweak an already horrendous Latex .sty file. I basically decided that I would either write technical documents in Tex or not at all after that. For me, Latex is great, until it isn't, and then it's Jinga-on-a-unicycle frustration.

bryogenic• 10 years ago

Life:

    The other thing is that although one starts with grand ideas of being clean and orderly, by the end the process is somewhat of a mess, with a complicated makefile to keep it running.

andy_ppp• 10 years ago

Yes! I loved that even Brian Kernighan can make a mess when trying to get something done :-) There is hope for us all!

tambourine_man2• 10 years ago

Another way of looking at it is that, if even Kernighan can make a mess when trying to get something done, there's no hope for anyone.

…

Your take on it is more uplifting

sotojuan• 10 years ago

It's a lesson that things don't have to be perfect to be able to ship.

elcapitan2• 10 years ago

We wrote a variety of small Go programs and scripts to fix things up, including one that rewrites troff output to do vertical justification so all pages are the same height.

That sounds weird, why would they want that, wouldn't that make the pages pretty irregular internally? Like varying line heights..?

pfortuny• 10 years ago

Not if the paper is opaque and there are lots of snippets (you align the space of the floats, not the line height).

elcapitan2• 10 years ago

That makes sense, thanks.

frik• 10 years ago

HTML with CSS with lend itself to generate PDFs. Are there open alternatives to Prince XML? Browsers don't support all page-layout related CSS2/3 attributes, like a page break.

noir_lord• 10 years ago

wkhtmltopdf has some extensions for forcing a page break etc.

I used it for generating some documents from a groundup rebuild of bootstrap meant purely for printing (meant we could just use a lot of the web templates with minor changes) and it worked pretty well for our uses.

PrinceXML however is in a class of its own (as you'd expect for $3.8k per server)

FraaJad8• 10 years ago

Generating PDF is not the same as typesetting.

Brian Kernighan on the Typesetting of “The Go Programming Language” Book

Related Stories