Error correction and concealment in the Compact Disc system (1982) [pdf]

https://www.pearl-hifi.com/06_Lit_Archive/02_PEARL_Arch/Vol_16/Sec_53/Philips_Tech_Review/PTechReview-40-1982-166.pdf

by mmastrac • 3 years ago

73 51 3 years ago

pearl-hifi.com

FirmwareBurner2• 3 years ago

Ah, back when Philips was at the forefront of cutting edge innovation.

mitthrowaway2• 3 years ago

Now the flag is carried by its spin-off, ASML.

FirmwareBurner2• 3 years ago

And NXP. Though it would have been nice if Philips would have evolved to be the "European iPhone" company instead of selling lights, shavers and food blenders.

OfSanguineFire2• 3 years ago

I still feel awe at the compact disc. When I was a teenager in the mid – late 90s, scanning microscopic pits with a freaking laser beam, and storing hundreds of megabytes on a storage medium, still seemed like a big deal, real cutting edge stuff. And yet that format had been invented well over a decade before.

I’m also surprised that if the CD was already invented by the early 1980s and widely sold commercially for music, it took until the early 1990s for applications in computing with CD-ROM. Was the bottleneck here CPUs, RAM, and graphics cards, which would not have been able to handle the amount of data that a CD-ROM could store?

pgeorgi• 3 years ago

Pressing CDs was expensive, which worked for the audio mass market, but not for the PC niche market. That limited the appeal of buying an (expensive) CD drive, and they weren't even using a standard protocol (so: lots of issues).

When CD creation became cheaper and the market grew, the economies started to work out in the mid-90s, until the late-90s saw computer mags with cover-CDs (a signal that everybody in that market had CD drives, and pressing CDs was cheap enough for one-off low-count productions like that)

actionfromafar3• 3 years ago

I think it was just too much data for (cheap) digital tech at the time, and the fact that Laserdisc games makers chose analogue LaserDisc over something digital on a CD is an indication.

TerrifiedMouse2• 3 years ago

Because LaserDisc could hold high quality video?

thedougd• 3 years ago

My first CD-ROM drive used a printer parallel port. It worked well enough to run my first CD-ROM software purchase, Mad Dog McCree (1993). My PC was a microchannel PS/2 and it was difficult finding reasonably priced accessories.

justsomehnguy3• 3 years ago

MPC standard gives a hint:

  The first MPC minimum standard, set in 1991, was:

    16 MHz 386SX CPU
    2 MB RAM
    30 MB hard disk
    256-color, 640×480 VGA video card
    1× (single speed) CD-ROM drive using no more than 40% of CPU to read, with < 1 second seek time
    Sound card (Creative Sound Blaster recommended as closest available to standard at the time[2]) outputting 22 kHz, 8-bit sound; and inputting 11 kHz, 8-bit sound
    Windows 3.0 with Multimedia Extensions.

Though for a CD game you really wanted something in range of DX4/100 with 16MB RAM, which surpasses MPC-II specs.

Also 4 incompatible CD interfaces on the sound cards doesn't bode well.

https://en.wikipedia.org/wiki/Multimedia_PC

https://commons.wikimedia.org/wiki/File:Mozart_16_(Oak_OTI60...

RetroTechie2• 3 years ago

Also 4 incompatible CD interfaces on the sound cards doesn't bode well.

Yeah, certainly in the beginning CD-ROM drives were marketed as a cornerstone to bring PC's into the "multimedia" age.

And thus eg. bundled with an audio card that doubled as a CD drive controller.

Only when they were more common, things like OSes, office suites etc came on CD by default.

Oh and... encyclopedia. :-)

ben77994• 3 years ago

I think most people didn't get a CD-ROM drive till the 486 was common.

386s did not come with CDs till late in their lifetime, by the time the 486 was becoming affordable most computers had a CD-ROM drive.

RetroTechie2• 3 years ago

CD drives were expensive at first.

That, and a mismatch between the huge capacity of a CD, and typical software back then.

Remember, popular games like Doom fitted on a couple of 3.5" floppies. 3D graphics was just coming. PC's with the cpu to process many MB's of game data (or decode full screen video) were often business machines with limited graphics capabilities. Homecomputers like the Amiga or Atari ST were in similar price range as a CD drive. Harddisks were like 1 or 2 GB.

CD writers were much more expensive. So out of most people's reach as backup medium.

Only when prices came down and PC's gained in cpu, storage & graphics (and software ballooned with that), it made sense to add a CD drive.

I do kind of miss the days where you'd buy a magazine with cover CD, and spend weeks playing 100s of shareware games. Then buy the full box for the best-of those. Or try a Linux distro without needing to download 100s of MB's over a 33k modem.

And it's a shame those mini-CD's (similar to GameCube discs) weren't more popular. I always liked those. Almost impossible to find these days (whether used for music, software, or as -R/RW media).

HPsquared• 3 years ago

Ah, downloading hundreds of MB on dial-up... those were the days

mcpackieh• 3 years ago

That, and a mismatch between the huge capacity of a CD, and typical software back then. Remember, popular games like Doom fitted on a couple of 3.5" floppies. 3D graphics was just coming.

I think this is why encyclopedias were one of the first "killer applications" for CD-ROMs. Wikipedia didn't yet exist, paper encyclopedia sets cost several hundred dollars at least, and floppy disks weren't really big enough for it. Having a whole encyclopedia on one or two CDs was a game changer.

bluedino• 3 years ago

The first CD-ROM connected to a computer that I ever remember was Grolier's (Compton's?) encyclopedia we had in the school library. It was a DOS program that ran on an IBM 286, external caddy-loading CD-ROM, and then a laser printer. Was probably an expensive and amazing setup at the time.

https://twitter.com/Foone/status/1308257334603337728

derefr• 3 years ago

CPU IO access speed and memory transfer delay, in a world before the advent of DMA.

There’s a reason that game consoles of the era had dedicated “Audio Processing Units” that rendered sound directly from sequencer instructions + a small set of loaded samples: trying to feed “one large continuous sample” (i.e. a raw PCM stream) through the CPU to a DAC, wouldn’t have left any time for doing actual logic. If you compressed the audio, decompressing it would take the whole CPU too.

Sure, CD-ROMs weren’t usually continuous streams of data in the way that audio CDs were; but the kinds of stuff that couldn’t just be usefully fit into a floppy — or a set of floppies installed onto a hard drive! — was usually asset files that were individually large in the same way that PCM audio was individually large, in a way that would still choke either CPUs or RAM. No point in large image files you can’t even display.

There were a few consumer use-cases for random access to individually large files — mostly public databases, of the kind we’d expect to find online today. But people who really needed these, either did use them “online” even back then (by dialing in directly to mainframe database services); or, if they were a more industrial consumer, they got these distributions on tape! These were the first things to be ported over to CD-ROM when CD-ROM drives did start becoming available.

The whole “multimedia” hype took a bit longer, only really taking off once high-quality images and low-quality video were able to actually be displayed on PC (requiring PCs to first achieve display resolutions higher than 640x480; color depths higher than 256-color; at least 2MB of VRAM; and CPUs fast enough for MPEG1 decoding of at least small 100x100 video clips.)

vel0city• 3 years ago

Speaking to the processing needed just to pass the PCM data around being a massive fraction of the overall processing power, you'll notice in a lot of early CD-ROM games they just completely sidestep this and just use the CD drive to read actual CD audio off the disk and have it decode and play back the audio directly to the sound output device instead of going through the CPU. Essentially the computer would just tell the CD drive "play track 11" when you got to the 11th level of the game.

And that was a decade past the creation of the audio CD that normal PCs (or gaming PCs!) still needed a lot of processing power just handling that data stream.

ben77994• 3 years ago

I think some older than me may remember differently, but I'm slightly older than you and I'd actually say I think for people who could afford computers there weren't that many years between getting their first CD-ROM equipped computer and their first CD Audio player.

It was within a year or two in my family, and my father was in the computer/internet industry by the mid-80s so had access to stuff pretty early. I think we got our first CD audio player in the family in 1990 and got a 486 with a CD-rom drive in late 1992.

CD Audio was expensive at first too.

ajsnigrutin• 3 years ago

To me, CDRW was the thing that changed everything... yeah sure, i could burn a CD some time before that too, but every file, every video, song, etc., was measured literally in money for blank CDs, and it was relatively expensive.

CDRW? 2MB file? Meh, just burn it, lend it to a friend, get it back, erase, 50meg file, other friend, get some other random stuff back, etc.

Then flash storage became cheap and the CDs slowly died (well, there were iomega zips and other stuff in between, but not really that popular).

francoisdevlin• 3 years ago

I had a zip disk for all my work in college. It's completely impractical for anything these days, but it was so much fun. Having 100mb in your bag in the era of floppies made you feel like a king.

PaulHoule2• 3 years ago

The first time I head sampled digital audio on a computer was on a Sun 3 workstation circa 1990 which had a high end 68k micro processor. It was a clip from "Layla" by Derek and the Dominoes

https://www.youtube.com/watch?v=TngViNw2pOo

which is featured in this television commercial

https://www.youtube.com/watch?v=2eGWW8KOQio

and got somebody in the computer lab to ask "Hey man, is that Freedom Rock?"

The thing was that computer progress was frozen in midair for a time in the early 1980s; computers like the TRS-80, Apple ][, C-64 and game consoles like the Atari 2600 all outlasted the projections of their makers because all of the parts from software to the CPU to video system (e.g. the CPU clock was usually slaved to the video refresh of NTSC television) were tightly coupled and making any substantial improvements would mean dumping everything and starting over, particularly with all new software because you didn't have real operating systems back then.

It was by the late 1980s, when the PC AT clones were established that you could really buy a computer one year that was a lot better than last year's model because architectures had advanced to the point where they could do that and keep compatibility.

So processing PCM audio went from impossible to difficult to easy to trivial and video was not far behind.

The free ride of clock rates going up and power consumption going down didn't stop until 2005, but each die shrink kept resulting in cheaper transistors until... just about now. It's not accidental at all that enthusiasts are complaining that NVIDIA 40-series GPUs aren't more cost effective than 30-series GPUs or simply greed; the economic engine of Moore's Law is dead, international competition and dirigisme might lead to another generation or two more than the free market would have delivered on its own. We've had a long time of getting better performance through more cores, wider SIMD, specialized architectures, etc. but it's going to take something big like true 3-d integration, semiconductors past silicon, or superconducting logic gates for performance to improve... Or maybe the revolution will be in "doing more with less".

quercusa• 3 years ago

There's also some clever coding at the physical level with the way pits are arranged on the disc:

https://en.wikipedia.org/wiki/Eight-to-fourteen_modulation

https://spectrum.ieee.org/kees-immink-the-man-who-put-compac...

dylan6042• 3 years ago

I don't remember which specific audio CD ripper software it was, but one of my most favoritist features ever was the "Try really hard" checkbox. This was used if you had a disc which was hard to read from either scratches or dirty or whatever. I never looked into exactly what it did, but I had assumed that a normal read just read, but try really hard would make use of the error correction which the math caused the slower reads.

To this day, I've been known to write functions called tryReallyHard() even if I haven't made a UI option explicitly for it, yet.

OfSanguineFire2• 3 years ago

cdparanoia[0], still the standard Linux command-line application for CD ripping, took its name from the fact that it goes heavy on error-correcting. However, among the filesharing community the Windows-only application Exact Audio Copy[1] always enjoyed a stronger reputation for accuracy, and several major private torrent trackers and DC++ hubs required EAC to have been used for rips, which sadly prevented Linux-only users from contributing to the scene.

[0] https://en.wikipedia.org/wiki/Cdparanoia [1] https://en.wikipedia.org/wiki/Exact_Audio_Copy

dylan6042• 3 years ago

those may be apps that use error correction, but these are not the apps I was thinking. i don't run Linux desktops. It was most likely a Mac program, but at that time I was still forced to use Windows for some work tasks. I'm thinking it was Nero Burning ROM.

mrguyorama• 3 years ago

I don't think many computer CD drives have ever exposed the error correction to the computer. It's usually handled entirely internally, as the link shows on the last page. What that program was probably doing was trying a bunch of times to read the same area from a bunch of different starting points to see if it could tease out the data with a little luck.

math caused the slower reads.

The error correction was ALWAYS being done, and that math was done on a dedicated circuit.

TacticalCoder• 3 years ago

CD rippers have progressed a lot. Now most audio CDs' hashes are stored in public databases and you can rip a CD in a 100% bit-perfect way and if a single bit is off, your checksum won't match the hash other people got in the database.

And CD rippers do check the online DB automatically.

Many people do archive their CD collection in a bit perfect way nowadays and you know when your rip is perfect (and it usually is).

ben77994• 3 years ago

Yes, and it's been this way for at least 10 years at this point.

My first wave of ripping CDs in the early 2000s this stuff wasn't around yet.

But I redid everything around 2013 and ripped them all as FLAC, and by that point EAC and the databases were very good.

tverbeure• 3 years ago

It’s really amazing how many fallback layers there are in a CD to work around errors.

There’s a first Reed-Solomon layer that can theoretically correct 2 errors per 28 byte frame, but they actually only fix 1 error and use the remaing error detection as an erasure code (instead of an error correction code) to make a second layer of Reed-Solomon correction much better due to interleaving.

If that fails, they have the audio samples swizzled in a way so that you can do interpolation between neighboring samples, and if that fails, there was yet another mechanism (that I forgot the details about) for a final effort.

I reminds me a bit of spacecraft, where there’s so much redundancy and flexibility to make things still work in case of a catastrophic failure.

The Galileo spacecraft comes to mind, where they reprogrammed the whole communication mechanism (adding compression etc) after the main antenna failed. (https://en.m.wikipedia.org/wiki/Galileo_project#High_gain_an...)

mips_r4300i• 3 years ago

The Swiss cheese safety model, but in disc form! Hah

password4321• 3 years ago

Related: yesterday's sub-discussion triggered RE:

Yamaha's old CD-Recorder, CRW-F1 really improved sound quality by abusing Red Book standard by lengthening the pits of audio CDs

https://news.ycombinator.com/item?id=37726359#37729766

ben77994• 3 years ago

I still love CDs. I started buying them around 1990-1991. I started with MP3 in 1996 but kept buying CDs. I think I got an iPod in 2003, I digitized all my CDs by 2004-2005 and I did go through a period from about 2005-2013 where I bought almost all my music as digital files.

I never got rid of my CDs though, and in recent years I started listening to them again just like Vinyl LPs. There's still something fun about listening to them, and they always sound great and work really well. I've never had a CD get damaged enough to hear any artifact. It is still a good way to support artists you like, and prices have inverted to the point CDs are affordable and Vinyl LPs are quite ridiculous a lot of the time.

lttlrck• 3 years ago

I love my CD collection, also been collecting since 1989/90, a new one arrived in the mail today and I buy used on eBay frequently. It's a musical guide to my life since then. I also went through the MP3/streaming route but it's not the same experience at all. I like the ceremony.

userbinator• 3 years ago

The picture on the last page makes a great illustration of how dense electronics integration has become over the years --- all of that, and more, are now possible in a single IC:

https://www.nxp.com/docs/en/data-sheet/SAF784X.pdf

There was an article here with a good graphical description of CDs a few years ago: https://news.ycombinator.com/item?id=21185897

Unfortunately that site is gone, and the archive doesn't have a copy.

snerbles• 3 years ago

Maybe this copy?

https://web.archive.org/web/20200322151217/https://byuu.net/...

dfox• 3 years ago

Well, it is certainly a prototype. In the early consumer CD players the whole CD decoding datapath was split into chip with the analog part (servo amplifier, which also produces an error signal that contains the read data) and digital chip with its associated DRAM that does the rest of the decoding. It is somehow surprising how empty the single layer PCB in these early CD players is. Obviously, this level of integration was required for the technology to be commercially viable. The CD somewhat coincides with the time of cost-effective VLSI ASICs, which made this possible.

On the other hand, the modern CMOS logic runs so fast and can be so densely integrated, that the 15 years old SAF784x apparently implements the “complex analog RF” part of CD player by using DSP techniques. And the process it is made in is apparently mature enough that it can accommodate various analog functions including high-powered ones so you end up with a complete CD/MP3 player on one ASIC (if you can fit what you want to do into 64kB of ARM7TDMI code and can tolerate the sound quality and EMC implications of the integrated audio output DAC)

nickdothutton• 3 years ago

Thanks to Philips, who produced such a tremendous range of R&D driven products.

Error correction and concealment in the Compact Disc system (1982) [pdf]

Related Stories