The End-of-Line Story (2004)

https://www.rfc-editor.org/old/EOLstory.txt

by gabrielsroka • 2 years ago

40 15 2 years ago

I posted this last year and was invited to repost it using the second chance pool. After posting it, I noticed someone else posted it yesterday.

dang• 2 years ago

Yes, when it comes to the SCP (https://news.ycombinator.com/item?id=26998308), we try to give priority to the user who posted the article earlier and/or hasn't had a submission on the front page in a while.

gabrielsroka2• 2 years ago

Using curl in linux, I find it useful to delete \r. Makes it easier to use sed.

  while [ "$url" ]; do
    res=$(curl -i "$url" | tr -d '\r')
    headers=$(echo "$res" | sed '/^$/q')
    body=$(echo "$res" | sed '1,/^$/d')
    echo "$body"
    url=$(echo "$headers" | sed -n -E 's/link: <(.*)>; rel="next"/\1/pi')
  done

finisher2201• 2 years ago

so interesting. I've somehow always supposed that it was ms-dos that decided to be different.

pinebox• 2 years ago

When this decision was made for MS-DOS (and it may not have even been made then, but simply brought in from 86-DOS which may have cargo-culted it from CP/M-80) it is very unlikely that "what ARPAnet requires" was a consideration. Networking was not included with microcomputer operating systems at that time and network compatibility between systems from different vendors was an exotic requirement on top of that. Most microcomputers in the late 70's and early 80's either only talked to their own kind or were (more commonly) not networked at all.

People today forget or don't realize how provincial and siloed computing used to be. Even as late as the 90's it was very common for institutions to have multiple LAN's that were isolated from each other simply due to platform incompatibility.

sumtechguy• 2 years ago

Also to understand it you have to think about typewriters, printers, and then text terminals. With a typewriter a cartage return with no line feed could be desirable if you are editing or using strikethru chars. Or on a CRT going to the beginning of the line to overwrite whatever text was there could also be a desirable thing. Then add in the mix of a dozen different companies making things and it was a recipe for 'you get all the different ways'.

shermantanktop• 2 years ago

Ah, yes, brings back memories of Winsock…from the days when the OS ran the hardware inside the box, and connectivity was a peripherals problem.

jakedata• 2 years ago

I tried to post an homage to reading text that had LF without CR but the input filter helpfully corrected it. You will just have to imagine text moving down like a set of stairs descending from left to right.

Kermit was the file transfer protocol of last resort for sorting out weird translation issues. It was slow but super effective.

HN is +10 proof against ascii art.

shmeeed• 2 years ago

The irony of finding the formating of this article completely messed up on mobile is not lost on me.

imglorp• 2 years ago

This choice was designed to spread the pain equally among all operating systems of the day; each has to translate to and from the CR LF convention when text was transferred across the network.

This compromise might have made sense from a political or goodwill standpoint at the time. But it meant that everyone using these protocols would have to store, convert, or transmit the extra character forever.

chasil• 2 years ago

This goes back further.

Teletype machines needed a delay to move the printing apparatus back to the beginning of a line. The two characters provided that delay.

I never used one of these; I was too young.

https://en.m.wikipedia.org/wiki/Teletype_Model_33

https://www.revk.uk/2022/02/crlf-has-long-history.html?m=1

EvanAnderson• 2 years ago

When I got started w/ PCs in the late 80s I wrote a GW-BASIC program to take keypresses (one of those 'IF INKEY$<>""' type programs) and write each keypress out to my Okidata dot matrix printer a character at a time. Pressing keys made a physical thing move. That's always fun.

It particularly interesting to send control codes to see what they made the printer do. Pressing <ENTER> got carriage returns without advancing the platen (so you could type over and over again on the same line). <CTRL><L> made the printer advance to the next page of the fan fold. <CTRL><G> made the printer beep.

I figured out that <CTRL><H> moved the print head backward. I used this to make some "impossible to print" output (that is, impossible for my word processor to reproduce because it wouldn't send backspaces for the purposes of over-printing).

Later, during my youthful phase of probing systems on various networks, I ran into systems that, upon entry of a bad password, would send a series of backspaces, various random letters, pound signs, and asterisks before giving a new logon prompt. At the time I didn't put two and two together. (I hadn't yet read Levy's "Hackers" and the idea of using a teletype as an input device on a computer wasn't a thing I'd heard about yet.)

Years later, looking at old captures and seeing this seemingly strange series of characters, I realized what the purpose was. The characters were being sent to obliterate the password printed by your teletype's local echo.

(I want to say the OS was PRIMOS but I don't remember for sure. I saw a number of them on either Tymnet or Telenet-- maybe both.)

seoulbigchris• 2 years ago

Those teletype machines did need extra time to perform the two distinct actions associated with EOL, that is, the carriage return and line feed. Line feed seems like it would take a fixed amount of time, but CR could be pretty long if the carriage was all the way out near column 80 or 132.

So, two commands were needed on those teletypes because of how they work, not to introduce any delay. It's possible the baud rate could be slow enough such that all the mechanical movements are wrapped up during the transmission of two characters. But sometimes you need to insert a delay after an EOL, something you can set on lots of serial port terminal emulators (although that's usually done for different reasons). You can set a delay in the termcap parameters to account for this needed delay. I'm not sure if any of the common mechanical teletypes offered a handshaking line to hold off the computer until the motion stopped, but that would have been another option -- and a faster solution, since the delay time would be exactly what was needed (vs a maximum fixed delay of a software solution).

Those teletypes were originally meant to be used in teletype to teletype operations, where the maximum baud rate was fixed by the mechanical operation of the printing mechanism. If you interfaced a 60 baud TTY to a computer running at 9600 baud, you also have to delay between each character, not only at the EOL. I know this because I foolishly interfaced my Commodore C64 back in college to an old Army surplus Kleinschmidt TT-4 mechanical teletype. I had to deal with these delays, as well as converting the parallel ASCII of the C64 to the serial Baudot needed by the TT-4.

edit: I suppose they settled on CR/LF instead of LF/CR, because the former could be a little but quicker... the LF could be performed while the carriage was traveling back to the start of line. But that's just a guess.

demurgos• 2 years ago

It's pretty unfortunate that we ended up with system dependent newlines. This means that text files are not byte-for-byte the same across systems so comparisons or hashing need normalization. This also makes reliable line splitting less convenient. I was also bitten by FTP transferring binary files with text mode instead of binary mode, causing corruption.

Nowadays I tend to enforce LF everwhere. Windows can deal with it, and it ensures consistent cross platform handling. I know that Rust also uses a single LF on all platforms, not sure if there are other such languages.

nextaccountic• 2 years ago

Today it's a mistake to open files in "text mode" that translates between line endings, or otherwise save text files with any line ending that isn't LF. Even notepad opens files with LF line ending now. (while we're at it, save all text data in UTF-8 too)

Some "text protocols" have CR LF line endings but this should be relegated to a library; your own code shouldn't deal with this.

And then we can all forget this mistake

The End-of-Line Story (2004)

Related Stories