Brian Kernighan adds Unicode support to Awk

https://github.com/onetrueawk/awk/commit/9ebe940cf3c652b0e373634d2aa4a00b8395b636

by ducktective • 4 years ago

618 205 4 years ago

tialaramex4• 4 years ago

The choice to use UTF-32 (ie Unicode code points as integers, which might as well be 32-bit since your CPU definitely doesn't have a suitably sized integer type) is unexpected, as I had seen so many other systems just choose to work entirely in UTF-8 for this problem.

Now, Brian obviously has much better instincts about performance than I do and may even have tried some things and benchmarked them, but my guess would have been that you should stay in UTF-8 because it's always faster for the typical cases.

bombcar2• 4 years ago

Is UTF-32 fixed size per char? Because then it allows simple math that you can’t do on UTF-8.

moomin• 4 years ago

A “character” can be of fairly arbitrary length in Unicode, so no.

valleyer• 4 years ago

It's a fixed size per codepoint. Many clusters that appear atomic in a text editor are made up of multiple codepoints. The flag emojis are among the many examples.

simias2• 4 years ago

It's always the tradeoff, some operations are simpler on UTF-32 but they have additional memory (and therefore cache) footprint and since you typically don't want to use UTF-32 externally you have to convert back and forth which is not free.

I think these days people don't bother with UTF-32 too much because it's not even like you have a clean "one 32bit int, one character" relation anyway since some characters can be built from multiple codepoints. Since generally most code manipulating character strings are interested in characters and not codepoints, UTF-32 is effectively a variable-length encoding too...

happytoexplain4• 4 years ago

It is fixed size per code point, which are what developers (and programming languages) sometimes casually call a character, but in practice a character is a grapheme, which can be multiple code points once you're outside the ASCII range. But it can still be useful to count code points, which would be faster in UTF-32.

Edit: Mixed up code units and code points.

masklinn• 4 years ago

Because then it allows simple math that you can’t do on UTF-8.

That's not actually useful, because unicode itself is a variable length encoding.

So it mostly blows up the size of your data.

Though it might have been selected for implementation simplicity and / or backwards compatibility (e.g. same reason why Python did it, then had to invent "flexible string representation" because strings had become way too big to be acceptable).

alganet13• 4 years ago

You are right.

Unicode in UTF-8 will have variable char length. Plain ASCII will be one byte for each char, but others might have up to 4 bytes. Anything dealing with it will have to be aware of leading bytes.

UTF-32 in other hand will encode all chars, even plain ASCII ones, using 4 bytes.

Take the "length of a string" function, for example. Porting that from ASCII to UTF-32 is just dividing the length in bytes by 4. For UTF-8, you'd have to iterate over each character and figure out if there is a combination of bytes that collapse into a single character.

Ericson2314• 4 years ago

It is fixed size...for now ;)

simias2• 4 years ago

He mentions that "The amount of actual change isn't too great, so I think this might be ok" so I wonder if part of the equation has more to do with avoiding messing with legacy code rather than raw performance. If the current code expects all codepoints to have a constant-width representation, it may be complicated to add UTF-8 into the mix.

A complete guess on my part though, I never looked into AWK's source code.

xonix• 4 years ago

This sounds reasonable. When the GoAWK creator tried to add Unicode support through UTF-8 he discovered that this had drastic performance implications (rendering some algorithms to be O(N^2) instead of O(N)), if done naive https://github.com/benhoyt/goawk/issues/35. Therefore the change was reverted till the more efficient implementation can be found.

_0w8t• 4 years ago

The code only uses UTF-32 in regular expressions where I suppose it was much simpler to adopt the older code. The rest uses UTF-8.

jaxrtech• 4 years ago

It's also the case that `char` in Rust is a 32-bit primitive. Though, `String` being UTF-8 is convenient until you need to do Unicode operations on actual chars...

It is surely better than dealing with UTF-16 "surrogate pairs"[2] in most pre-UTF-8 tech where a `char` is 16-bits but can't fit all Unicode codepoints without another 16-bit `char`.

[1] https://doc.rust-lang.org/std/primitive.char.html#representa...

[2] https://stackoverflow.com/a/54922477/809572

kqr3• 4 years ago

Well, it's not like we don't have plenty of combining marks in Unicode itself, so I suspect this is something you have to deal with either way.

hgs3• 4 years ago

UTF-32 performance woes are exaggerated. UTF-8 decoding requires branching whereas UTF-32 does not and in terms of storage real world text does not consume much space.

kzrdude• 4 years ago

Utf32 has the BOM which adds complexity and at least a top level branch. Maybe it's not so bad in practice, just duplicated generated code.

But as an internal representation, some of it can be ignored by choice.

ducktective4• 4 years ago

I became aware of this while watching Professor Brailsford's interview with him (Computerphile channel): https://www.youtube.com/watch?v=GNyQxXw_oMQ

(Around 7-8 minute mark)

Update:

At 24-25 minute mark, he talks about the technologies he is inquiring to write his new book with (he mentions troff and groff).

He says he wanted to try "XeTeX" (which supports Unicode) but "...I was going to download it as an experiment and they wanted 5 gigabytes and 5 gigabytes at the particular boonies place I'm living would...mmm..not be finished yet!"

So there we go...We had the opportunity to read the mind of the developer of awk and unix and co-author of the literal "C Programming Language", confronting with the absolute state of the tooling of the modern world.

mlyle4• 4 years ago

He says he wanted to try "XeTeX" (which supports Unicode) but "...I was going to download it as an experiment and they wanted 5 gigabytes and 5 gigabytes at the particular boonies place I'm living would...mmm..not be finished yet!"

Man, I think once you're Kernighan there should be like a 1gigabit/sec symmetric circuit wherever you go just in case you use it to do something else useful.

scoot• 4 years ago

THis isn't about bandwidth, it's about the size of modern binaries.

rustqt62• 4 years ago

You put it jokingly? But distinguished people should definitely get some state managed perks like politicians do.

JdeBP• 4 years ago

... or a fast car with a load of magnetic tapes (or memory cards, per M. Munroe) in the boot fuelled up and ready to roll. (-:

asddubs• 4 years ago

on the other hand, I've done lots of excellent coding in places with awful internet. fast enough to be able to look things up when you really have to, but too annoying to meaningfully distract yourself

sargstuff11• 4 years ago

Ah, thought goto usage was considered harmful.

jfk13• 4 years ago

Sounds like he was looking at downloading a complete TeX Live distribution; XeTeX itself isn't anything like that size (by a couple orders of magnitude, at least).

ducktective4• 4 years ago

I think distros package it like TeX-full, TeX-minimal etc...The one having documentation files is a couple of GiB on Ubuntu...

I wonder what distro or editor he is using...

zimpenfish• 4 years ago

MacTeX is 4.7GB which matches the 5GB he's talking about and...

"MacTeX installs TeX Live, which contains TeX, LaTeX, AMS-TeX, and virtually every TeX-related style file and font. [...] MacTeX also installs the GUI programs TeXShop, LaTeXiT, TeX Live Utility, and BibDesk. MacTeX installs Ghostscript, an open source version of Postscript."

Which is, as you say, considerably more than just "XeTeX".

(Also those are universal binaries containing both Intel and ARM versions which probably adds some heft.)

samatman• 4 years ago

the absolute state of the tooling of the modern world

Hah, TeX Live is... not that.

It's been enormous since I installed it off a CD in the 90s. The idea, and it works, is that you can just compile anyone's stuff out of the TeX ecosystem.

There is just... a lot... in it. You don't need a package manager if you install the whole universe locally. Like I said: not what I would call a modern approach to tooling.

On the other hand, I have latex files from the mid-Noughties and, I don't even need to check: they'll compile if I want them to.

But yeah, if you want just a little piece of TeX here and there, you're off the beaten track. That's not how TUG rolls.

rdlw2• 4 years ago

TeX Live can also be configured to install the bare minimum TeX ecosystem (or just TeX+LaTeX), which only takes a few minutes to download and install but results in hunting down dependencies and manually installing them whenever you want to use a new package.

It also seems quite slow to update, and a recent (?) name change of `tools' to `latex-tools' seems to have broken multicol, which drove me to MikTeX. Internet connection required, but far less headache.

rob74• 4 years ago

Old approach: "we don't have a lot of disk space, we have to be careful what we install".

More modern approach: "hey, we have a lot of disk space, we can install everything locally".

Even more modern approach: "hey, we have fast Internet access, we only have to install stuff when we need it".

stjohnswarts• 4 years ago

He could master it in a week if he set his head to it. I don't have one doubt of that. He just doesn't really need to.

maxnoe• 4 years ago

The Problem is that TeXLive still defaults to doing a full install.

A full install means installing ~4000 packages, including their source files (tens of thousands of tex files) and built documentation (thousands of PDF files) and hundreds of free fonts (otfs, ttfs, texs own format).

This is huge (>7GB, not just the 5 GB claimed here).

However, you don't need 99 % of this for any given document.

Not installing the source files and documentation PDFs will alone reduce the size by roughly half.

Only installing the packages you really need from a minimal installation gives you a few hundred megabytes at most for even complex documents.

It's a bit annoying to get the list of packages needed though, since there is not really any working dependency management.

I wrote a python wrapper around the tex live installer [1] to make this easy for CI jobs, see e.g. [2].

On a side note: I'd recommend luatex over xetex.

- [1] https://github.com/maxnoe/texlive-batch-installation/

- [2] https://github.com/pep-dortmund/toolbox-workshop/blob/8b00f0...

jcelerier• 4 years ago

On archlinux there's the texlive-core package which does not ship the PDF docs (most of the size). It should install 500mb (most of which are fonts..) and already provide enough to build normal documents, including lualatex for unicode support

JadeNB2• 4 years ago

TeXLive also comes with installation schemes that will give you (if I remember the names correctly) bare, medium, and full installations, if you prefer not to pick packages yourself. Alternately, although I don't use it myself, I'm sure you could use MikTeX, which is much better about on-demand package installation. (Or even Overleaf, if you don't want to put anything on your local device!)

phire• 4 years ago

It's a bit annoying to get the list of packages needed though, since there is not really any working dependency management.

Sounds like someone should just create a wrapper that catches file opens of uninstalled dependencies and automatically downloads them on the fly.

dhosek• 4 years ago

I did some digging about a year ago and the biggest uses of space are the docs and, surprisingly, fonts: https://www.reddit.com/r/LaTeX/comments/oqmb3w/installed_mac...

It’s mostly a death by a thousand cuts thing.

MikTeX, which is available for Mac and Linux as well as Windows will do on-demand downloading of packages and is probably my recommendation for a new installation.

A lot of the drawbacks to TeX stem from to trying to meet the combined requirements of computing platforms of the 1970s–1990s. All input files effectively live in a single namespace so you cannot have two input files with the same name (the same applies to coding where you have a single namespace for all your commands as well).

ddmitriev• 4 years ago

Eons (>15 years) ago, I sold software that relied on having a working TeX installation, and to make it easier on my users, my installer provided a very very trimmed down version of MiKTeX as an optional download. The beautiful thing about MiKTeX is that its autoinstall feature meant that one could delete 90% of the packages, and the worst that would happen is that referencing something I deemed unessential would prompt a just-in-time download.

I recall I managed to cut MiKTeX down to about 12MB. Never heard a single complaint about it.

jhoechtl• 4 years ago

Development and re-juvenile of groff is recently taking a good path. While groff is still single-pass and doesn't do paragraph-at-once type setting, it is a much more pleasant experience than maybe like five years ago.

woile• 4 years ago

I can't believe I read this comment literally 10 minutes after installing endless GBs of xetex haha. Thanks for the info!

cfiggers2• 4 years ago

Watching this interview inspired me to start playing around with groff. It has a very steep learning curve... And being as old/niche as it is, I've found it very hard to find any active community to get newbie questions answered. If anybody knows where I could find that sort of thing, I'd be very grateful.

neutrono2• 4 years ago

NB: I'm new to HN, so please excuse any formatting errors in this comment.

Well, I've been using groff on and off for a few years, I can give it a shot. Most "macros" as in TeX, are invoked by having the first character of a line be a `.` and then as in TeX, refer to a name. Now, macros in groff are weird. Macros are not delimited by `{}` delimiters as in TeX, but either by having an environment block like so:

    .TS
    .TE

Incidentally, `.TS` and `.TE` is the environment for tbl(1), the pre-processor for tables. Or by invoking another macro, as in this example for headings and paragraphs in the -ms macro set. (The -ms macro set is called as such because that's how you get it into your document, you supply the -ms flag at build time to groff to get the -ms macros). See groff_ms(7) for more information than I will supply here.

    .NH 1
    Some heading here
    .PP
    Start of a paragraph.

Sometimes you want to start a line of input with a period. To do that you just use the character '\&' before the period. The `\&` character is a non printing character, so use it anywhere you want, really. I imagine you want to play with math typography, in that case there is EQN. EQN works by having an environment block, `.EQ` and `.EN` and in that block you type in the EQN mini-language.

    .EQ
    f ( x ) ~=~ x sup {2} - 1
    .EN

EQN does not care about precedence or anything, so use `{}` to delimit expressions. I think there is some kind of EQN document rescued from Bell Labs that tells you about how the language works. Check troff.org for it, or email the mailing list for groff, which is also pretty good for newbie questions, it is not really all that high traffic.

Groff also does pictures, so you can for example create a box with some text in it (it is capable of more, like strokes, macros, color, and I believe rotation[not sure]). To start a picture, `.PS` and `.PE` will do so, and again, a small language as input in that block.

Now, to print anything you should know the design of the original troff system was heavily based on the UNIX philosophy, which means the output you get from running the groff program prints the output to stdout. Just redirect the output to a file to get a PDF or a PS document.

Obviously, there is much I have not covered in this comment, such as macros, registers, and other exciting topics such as refer(1), but this should give you a quick start.

lr1970• 4 years ago

He says he wanted to try "XeTeX" (which supports Unicode) but "...I was going to download it as an experiment and they wanted 5 gigabytes and 5 gigabytes at the particular boonies place I'm living would...mmm..not be finished yet!"

He can try "Tectonic" [0] - a modern XeTeX based TeX/LaTeX distribution that installs a minimum system and then downloads and installs dependencies on-demand. Tectonic is written in C and Rust [1].

[0] https://tectonic-typesetting.github.io/en-US/

[1] https://github.com/tectonic-typesetting/tectonic

xvilka• 4 years ago

It's sad that Tectonic conversion to Rust[1] was never finished. For now it's just a wrapper around C and C++ code. By far, it was the most promising thing in this distribution.

[1] https://github.com/tectonic-typesetting/tectonic/issues/459

amyjess• 4 years ago

It's not just the tooling of the modern world.

I remember installing teTeX (the predecessor of TeX Live) back in 2003 and I despaired at the file size because I knew it would take days to download on the shitty dialup connection I had at the time. Yeah, it wasn't 5 GB big back then, but it was certainly big by 2003 standards.

dredmorbius2• 4 years ago

I just wanted to say: that's an excellent interview and I strongly recommend others listen to it.

ChrisRR• 4 years ago

He says he wanted to try "XeTeX" (which supports Unicode) but "...I was going to download it as an experiment and they wanted 5 gigabytes and 5 gigabytes at the particular boonies place I'm living would...mmm..not be finished yet!"

I'm struggling to understand this sentence. Who are they? What's a boonie? What's not finished?

detaro• 4 years ago

"in the boonies" = "in the wilderness/far away from everything/...". I.e. they probably have only crappy mobile internet and don't want to download large packages.

svnpenn2• 4 years ago

Looks like its not really done yet:

https://github.com/onetrueawk/awk/compare/master...unicode-s...

etaioinshrdlu• 4 years ago

I liked this interview with Brian with Lex Fridman: https://www.youtube.com/watch?v=O9upVbGSBFo

neilpanchal• 4 years ago

+1. Also, I like your username.

timakro2• 4 years ago

I believe no distro actually ships this version of awk by default. They ship GNU awk which has Unicode support anyways.

svnpenn2• 4 years ago

Debian:

https://distrowatch.com/table.php?distribution=debian&pkglis...

timakro2• 4 years ago

So it turns out the default on Debian is mawk which does NOT support Unicode. Thanks for pointing that out. This simple test gives different results for gawk and mawk.

  $ echo 'ö' | awk '{print length}'

chasil• 4 years ago

OpenBSD uses "The One True AWK."

  $ awk -V
  awk version 20211208

Kernighan's version is likely used in other places where the GPL is eschewed.

fanf2• 4 years ago

I think the other BSDs do too, including macOS.

cyocum• 4 years ago

Here is Brian Kernighan mentioning the Unicode work in an interview: https://www.youtube.com/watch?v=GNyQxXw_oMQ

nanna• 4 years ago

Apparently Kernighan is also updating his Awk book of 1988 this summer too.

https://irreal.org/blog/?p=10746

7thaccount• 4 years ago

That would be cool

wpietri6• 4 years ago

As I get old to start thinking about exactly how long I want my career to be, this is inspiring. He is coming up on 80 and he's still coding.

paledot• 4 years ago

Is it? I'm glad it's an option, but I'm also glad retiring at 40 is an option (depending on a lot of stars aligning just right of course).

My answer to "how long do I want my career to be?" will never be "until my body or mind fail me".

msravi• 4 years ago

I've found this to be THE best book for learning awk and regular expressions. Its astounding how clear and structured the material is and how easy it is to retain in your head when presented that way.

arduinomancer• 4 years ago

For context this is 37 years after it was released (1985)

lelandfe• 4 years ago

Once I figure out how… I will try to submit a pull request. I wish I understood git better, but in spite of your help, I still don't have a proper understanding, so this may take a while.

Even Kernighan struggles with git.

mordechai9000• 4 years ago

This reminds me of the relevant xkcd: https://xkcd.com/1597/

brudgers2• 4 years ago

Torvalds is a better programmer than that.

Pull requests are feature of GitHub, not a part of git.

https://docs.github.com/en/pull-requests/collaborating-with-...

nmz2• 4 years ago

I'd say its a feature of all(?) dvcs, almost all of them or the ones that I know of must use PRs/forks.

palunon• 4 years ago

Nitpick, but GitHub pull requests (which are really merge requests, you aren't pulling anything) are named after the actual pull requests used eg. between kernel maintainers. ("Please, could you pull from ...")

Git has tooling to help with those, so it kinda is a git feature: git request-pull. https://git-scm.com/docs/git-request-pull

Blikkentrekker3• 4 years ago

The culture around p.r.s is truly a high barrier of entry for many people.

Figuring out how all of this works is substantially more difficult I find in practice than fixing many longstanding trivial bugs in a great deal of software.

umanwizard• 4 years ago

What’s the alternative? The old way (which is still used by many projects) is to send patches to mailing lists, which I find more difficult: you need to learn how to generate the patch from your source code repo, send the patch as an e-mail (needing weird hacks like `git imap-send`), and then configure your MUA not to mangle it somehow. Then you also don’t have a centralized search/tracking interface.

Some good reasons not to use GitHub is because you’re familiar with standard/traditional tools, or because you prefer not to use centralized services. Both of those are fine reasons! But “the traditional way is easier” isn’t.

ducktective4• 4 years ago

[off-topic]

Following the spirit of UNIX, I did a little analysis on the upvotes this post got over time (fish-shell):

  while true; curl -sL  'https://news.ycombinator.com/item?id=32534173' | pup  '#score_32534173 text{}' | awk -F'[^0-9]*' '{print $1}' | tee -a points; sleep 15s; end

(Initially I used `grep -Po '\d+'` but switched it with an awk solution due to...context!)

I started it approx. when I posted it. Now ~2 hours have passed since. Using `gnuplot`:

  f(x) = a*x+b; fit f(x) "points" via a,b; set terminal png size 1920,1080 enhanced font "Inconsolata,20" ;set output "HN-analysis.png" ;set grid; set ylabel "points";set key bottom right ; set xlabel "sample # (15s interval)"; plot 'points' w linesp lt 7 lw 3 lc rgb "orange", f(x) lc rgb 'blue' lw 2

We generate the plot: https://i.imgur.com/pS6AaI5.png

(The jump #100 sample is due to a network error on my side)

And here are the coeffs. of a linear fit over the data (note that every 4 samples is 1 minute, so this post got ~1.52 upvotes per minute)

  a = 0.380809, b = 19.8437

wpietri6• 4 years ago

This is cool. Thanks for sharing how you did it. Nice to see awk being used in a post about awk!

defrost• 4 years ago

Fun stuff ... now correct by plotting X-axis as actual lapsed epoch time since start of sampling :-)

( For bonus fun, fit successive unbiased moving averages of a selectable width, for further fun do this for non uniform sample times .. time weighted, and look beyond simple moving averages to (say) Savitzky–Golay filter family for both uniform and non uniform sampling intervals )

sargstuff11• 4 years ago

?? nroff grap not installed ?? ;) https://n-t-roff.github.io/heirloom/doctools/grap.1.html

ducktective4• 4 years ago

I didn't get it :)

Are you suggesting another method for graphing?

stakkur• 4 years ago

It's somewhat comforting to hear even Brian K. say he doesn't understand Git well.

layer84• 4 years ago

I came to the comments to say that it’s reassuring. :)

cafard• 4 years ago

Likewise.

rustqt62• 4 years ago

Git really is a mess. The fact that commits and not diffs have hashes should be lampooned despite arguably a few small benefits. Geniuses make mistakes too and git is linuses. The only reason git is respected is because it came from Linus. If it were from Microsoft it would get all the criticism it deserves and then 20 times more

nighthawk454• 4 years ago

I never quite get this. Why would you want diffs to be the primary artifact?

I’ve always found git to be exceedingly simple in design. Though, the CLI is a quite messy I grant you.

benreesman• 4 years ago

I think it would be a little more accurate to say that git is kind of a weird default: it’s a revision control power-user tool with all the sharp corners we expect from such tools and is overkill for most users in most situations. It’s understandable that typical users resent the complexity and the foot cannons.

Whether it’s git or vi/emacs or C or LaTex or whatever, there are a bunch of us old-timers who went through the pain already and now notice the modest loss of capability on hg or VSCode or whatever.

But even as such an old-timer, I don’t think it’s a good idea to default everyone into this low-level shit. Mercurial and VSCode and Python are fine, you can do great work without “the bare metal”.

Edit: Clarified that it’s a “revision control” power-user tool, plenty of “power users” have more typical revision-control needs.

wpietri6• 4 years ago

Yeah, my thought on reading, "I still don't have a proper understanding" is that I'm not sure anybody has that. It seems like you only know too little or too much.

I have only worked with one person really who knew is backwards and forwards. That seemed helpful at first until he used his deep knowledge to try some sort of tricky fix-up and in the process lost some of my work. Given that a version control system's one job is to not lose work, I was very disappointed.

grimgrin• 4 years ago

What context do you remember about this scenario that left the reflog a rektlog

Not confined to your local machine?

formerly_proven2• 4 years ago

Local Man Deletes Files, Leaves Coworkers Very Disappointed

kqr3• 4 years ago

If he means it in the "I have decided not to take the time to learn this" then yes, it's comforting. Otherwise it's just sad -- I'm sure there are many people who would be willing to tutor him in it in exchange for only the honour of having tutored Kernighan.

ivalm• 4 years ago

i'm guessing his issue is not with git porcelain. The actually internals of git are kind of messy.

seabrookmx• 4 years ago

And to be fair he's been a professor for years, not a developer.

When I took my undergrad roughly a decade ago, github was already a thing but our courses were still using subversion.

Academia is slow to adopt new tools.

YesThatTom2• 4 years ago

Of course he did. Aho has better things to do and Weinberger is too rich to write code any more.

tkhattra• 4 years ago

i think Peter Weinberger is still writing code at google...

sargstuff11• 4 years ago

Guessing compiler stuff sells better than scripted stuff, per popularity of "Compilers: Principles, Techniques and Tools" 1st & 2nd editions.

ufo• 4 years ago

What does adding unicode support mean? Is it about making all the string indexing match codepoints instead of bytes? Or is it more than that?

sargstuff11• 4 years ago

extending ascii using bytes on the fly for type setting bounded in a utf-8 character space.

bear8642• 4 years ago

Whilst great Brian's added this, didn't Plan9's version of AWK support UTF-8?

I haven't used it, but my understanding was all the unix tools were upgraded to work with unicode - don't quite see why it's taken so long if the work was done already years ago

lizardactivist• 4 years ago

Is use of tools like awk, m4, sed etc. common and growing, or decreasing?

I've always found that for my personal use cases, writing a little Python snippet has always been much faster and simpler than understanding what CLI tool to use, and how to use it, to achieve the same thing.

MrDresden• 4 years ago

Depends on the situation.

I've had to work on environments where awk was available by default, but Python was not (and was not going to be).

It can be good to have a screwdriver for every type of screw in the toolbox.

nmz2• 4 years ago

I only use awk. sed is unnecessary when you know awk has gsub or sub. Though I would say, I'd really like lua to be in there considering its small size.

hi41• 4 years ago

K is a gift to humanity who keeps on giving. I am a big fan of his C book and the AWK book!

Long live, K!

blacklight• 4 years ago

I also became aware of this from Kernighan's interview on Computerphile. And I thought to myself "man, such an omnipresent and fundamental piece of software doesn't have UTF-8 support yet?" and then I thought that, after >20 years of using awk in my script, I never had the urge to use it to parse UTF-8 - but I guess that it's just the luck of living in a country that uses a Latin alphabet, and I couldn't imagine how people could use it on documents in e.g. Chinese or Arabic. This was really an example of how discriminatory software is when it comes to languages.

dredmorbius2• 4 years ago

I'd run across issues in handling Unicode content in a recent awk (gawk) project. This is excellent news.

sargstuff11• 4 years ago

So, has anyone seen/awk'd a unicode mul-tics TeX check box example yet?

sargstuff11• 4 years ago

Per discussion, header should be revised to "Awk kerning again a unicode mul-tics Tex thing"

sytelus• 4 years ago

Brian Kernighan is 80 years old. Donald Knuth is 84 years. Both still write code and avoid most social/non-social media in general. It's amazing given how often we get surrounded by people giving up coding at rather young age for various reasons. Who else would you include in this class?

Brian Kernighan adds Unicode support to Awk

Related Stories