MP3 and CD-Quality Sound:
The Laws of Physics have Not Been Repealed

Walt Crawford, 1999

Since this is a long and growing article, a touch of navigation may help:

By now, youíve probably heard of MP3, the revolutionary audio format that will wipe out audio CDs and put all the big record companies out of business. An array of legal sites (particularly www.mp3.com) provide software to play and record in the MP3 format (some freeware, some shareware, some demoware). These sites also provide thousands of MP3 tracks ready for the downloading. There are free songs, either to attract you to download others by the same artist or from artists who really donít expect you to buy anything. There are for-fee downloads. In some cases, there are CD-ROMs available with as much as ten hours of music on a single disc. MP3.com is sending out thousands of free CD-ROMs with "103 of the best songs youíve never heard," to push the format and their role as a distributor.

A Little Background

If you've read Crawford's Corner in the April 1999 Library Hi Tech News, skip ahead to "The Fury and the Sound." While this is a different discussion, it's less comprehensive. (For example, I don't mention the Nyquist Limit here: the reason you need 44Khz digital sampling to represent the audio spectrum from 20Hz to 22Khz.)

MP3 is MPEG Audio Layer 3, part of the MPEG-1 format for compressed digital video. MPEG-1, used for CD-V video and within CD-ROMs, hasnít done very well in the U.S. market. It typically yields "near-VHS" video quality, which is another way of saying that the picture is mediocre. DVD uses MPEG-2, a considerably more advanced format.

Both MPEG-1 and MPEG-2 use lossy compression based on "perceptual coding," throwing away portions of the original information on the premise that you canít really hear or see whatís thrown away. So does MP3. These compression techniques can also use elimination of truly redundant information, the basis for lossless compression. For example, in two adjacent frames of a video, itís likely that 90% or more of the picture will be unchanged; MPEG takes advantage of that fact. (Similarly, within a single frame, itís likely that there will be areas of solid color, and even fairly small solid-color areas can yield substantial space savings.)

(Computer users should know that the QIC backup format and ZIP archiving format both use losslesss compression, typically averaging about a 50% savings in space for QIC, more for ZIP. In graphics, TIFF uses lossless compressionófrequently achieving a 75% savingsówhile JPEG uses lossy compression.)

The problem with lossless compression is that you can only go so far, and you canít predict how much youíll save in advance. DVD requires something like 50:1 compression on average. If a film was really a slide show (with unchanging images for a few seconds at a time), lossless compression might do the job; for most video, however, only lossy compression will work.

And work it does, at least for well-engineered DVDs. While not totally free of compression artifacts, most good DVDs still offer the best pictures available on traditional television. Incidentally, the surround-sound audio tracks on DVDs also use lossy compression.

MP3 Specifics

MP3 can be recorded at whatever bit rate is appropriate, and available MP3 tracks vary widely. Some MP3 followers assert that its perceptual coding techniques allow you to compress at an 11:1 ratio with no audible difference, and at 18:1 with differences that most people would never notice. Thatís the claim of MusicMatch JukeBox, one of the highest-rated MP3 encoding and playback programs, using compression software developed by Xing. When used to record tracks from audio CDs (called "ripping" by MP3 folk), the two "CD-quality" options are 128Kbps and 160Kbps. Audio CDs use 1.4Mbps; thus, the 128Kbps rate requires roughly 11:1 compressionóthrowing away more than 90% of the original. (The highest quality offering, 160Kbps, represents a bit less than 9:1 compression.) Their "near-CD quality" option compresses at 18:1. (Thereís a lower "FM-quality" rate as well.)

Itís possible to specify custom rates as well, either constant rates up to 320Kbps or variable rates based on a "quality" meter. The MusicMatch help file explicitly says that you shouldnít hear any quality improvement by recording at higher than 160Kbpsóor, really, 128Kbps.

At 128Kbps, only computer owners with cable modems, ISDN connections, or comparably high-speed connections can download MP3 in real time, and indeed MP3 is not designed for streaming audio. Good MP3 software also handles RealAudio, a streaming format with typically lower sound quality. But at 128Kbps, itís plausible to download songsóand itís plausible to store quite a bit of music on your computerís hard disk or some other device. Thus we have Rioís pocket-size MP3 player, which stores up to half an hour of "CD-quality" or an hour of lower-quality music on its 32MB RAM. (If you do the arithmetic, youíll see that a 128Kbps datastream takes up just under one megabyte a minute. Thus, the hour of music must be compressed at 64Kbps.)

If bands can release MP3 tracks over the Internet, getting paid for major downloads and using free songs to entice listeners, they donít need record companies with their high overhead and restrictive policies. And listeners can take a few hours of CD-quality music along on a business trip on their notebook computers, or burn ten hours of music onto a single cheap CD-R. Thatís the theory.

[top]

The Fury and the Sound

The hoopla over MP3 comes from a variety of sources. On one hand, you have the RIAA (the Recording Industry Association of America), a trade organization thatís been busy taking legal action to shut down Web sites with commercially-recorded music available for free in MP3 form. These pirate sites were widespread and well publicized; they have, by and large, disappeared. (When it comes to legal action, RIAA doesnít mess around, and internet service providers arenít going to take risks on secondary lawsuits for hosting pirate sites.)

On the other, you have PC journalists and commentators, always looking for the Hot New Thingóand frequently none too careful about evaluating that thing. MP3 is decidedly a Hot New Thing. The best journalists have carefully retained the qualifier "near-CD" for the sound quality of MP3; so, for example, a good discussion in the July 1999 Macworld consistently uses qualifiers. But that article is also headlined "So Long, CDs"óand such pundits as John Dvorak have been quick to drop the "near" term and simply claim that MP3 at 128Kbps offers full CD quality. But then, so do the software makers, or at least some of them. A recent discussion in PC Magazine sneers at those extremist audiophiles who might want encoding rates higher than 96Kbps, which seems to say that this rate is good enough for any ordinary person.

Dvorak simply claims that all recording companies are toast. Iíve already ridiculed that assertion in Crawfordís Corner, and thereís no point in discussing that issue: there are a variety of reasons that Sony, EMI, and Warner wouldnít just shrivel up in two years even if MP3 is everything itís claimed to be. Indeed, mp3.com's honcho explicitly disavows the idea that MP3 could displace recording companies.

That leaves the more interesting question: what does MP3 sound like?

When I read the first commentary dropping the "near-CD" label, I suspected it was wrong. What I know of the way sound works leads me to suspect that an 11:1 compression ratio without any audible effects is unlikely: the physics of sound donít seem to permit it, at least on a general basis. (Sony's MiniDisc compresses at a 5:1 ratio, and Sony is careful to avoid "CD quality" in connection with MiniDisc: they push it as a higher-quality and more durable alternative to audiocassettes, not as a replacement for CD.)

I had downloaded a couple of free MP3 tracks from legal sites. The sound quality varied from atrocious to roughly FM-quality, but was never anywhere near what Iíd consider to be CD-quality. But that doesnít mean much; the MP3 tracks could have been badly engineered in the first place.

I didnít think the physics made sense. But Iíve seen somewhat miraculous results in other fields, including the picture quality of DVD. Before making too much fun of MP3 extremists, it was time to do my own tests.

Thatís what the rest of this piece is about. As originally loaded (June 6, 1999), it includes comments based on the first round of testing. That original article turned out to be dead wrong--for reasons that say a lot about the difficulty of testing PC-based systems. I suspect that only one or two people ever saw that article, but you can read about the egg on my face in "Oops: MP3 and Testing Difficulties," over in the miscellaneous rants section of this site. After correcting the problem, I've revised my testing notes, which appear below.

So far, I've only tried MP3 at data rates that MusicMatch considers CD quality. I haven't tried their lower-rate "near-CD quality" options yet; nor have I tried MP3 on tougher material than the pop in the first round. I now find that, on the one hand, MP3 advocates are writing to Stereophile claiming that MP3 will do just fine when you use a 320Kbps or higher encoding rate--in other words, more than twice what's currently called CD-quality. On the other hand, you have writers sneering at those who think that they need anything more than 64Kbps.

Iíll do some more testing later, and add more comments. This is most assuredly a work in progress.

[top]

First Tests: Good News and Bad News

I downloaded the MusicMatch Jukebox and registered it for the $30 fee, since I knew that more than five recording trials would be needed. (The "free" version can play back indefinitely, I think, but stops recording after five tracksóand keeps bugging you about registration.)

My definition of "CD quality" for a new audio format is really quite simple:

If you record a track from an audio CD into the new format, the playback of that track should be audibly identical to the playback of the CD itself.

If thereís any audible difference, then itís "near-CD quality," a wildly imprecise term that Iíll charitably interpret as "better than cassette and no worse than FM."

Simon & Garfunkel at 160Kbps

Before registering the program, I gave it an easy initial test. Using the higher of two "CD-quality" rates (160Kbps), I converted two Simon & Garfunkel songs (El Condor Pasa and Bridge Over Troubled Water) to MP3 format. Pop music is less complex than classical music, and these were 30-year-old recordings, made long before digital recording. It should be a snap for the MP3 to sound as good as (or, rather, identical to) the CD.

To make the test even easier (and a good deal more practical), I was using my aging ears and the modest set of speakers attached to my home PC. I know that my high-frequency hearing is pretty well shot, and the speakers are Altec Lansing ACS400 satellites and an Altec Lansing ACS250 "subwoofer"--good PC speakers, but nothing more. (The ACS400 was one of the earliest Altec Lansing systems to offer Dolby Surround, with a side-firing speaker on each satellite along with the main midrange speaker and tweeter. I believe the ACS400/250 combination sold for $150-$200 when introduced three years ago. Itís two cuts above the cheesiest PC sound systems, but these are still low-powered plastic speakers.)

In every respect, I was making it easy for MP3. Relatively simple music recorded decades ago, high-rate MP3 recording, a modest speaker system, and a listener with mediocre hearing. I listened to sections of the MP3 version and the CD version in short bursts and longer segments, easy enough to do since I could keep both the Windows CD player and MusicMatch active on the desktop simultaneously. Note that this wasnít a case where the CDís audio output wasnít going through the sound chipís electronics: since I could control CD volume as well as MP3 volume from desktop software, CD audio definitely was going through the computerís sound circuitry. (The computer is a Gateway Celeron400, with Creative Labsí AudioPCI 64Dóthe old Ensoniq chipóas motherboard audio support. The CD drive is actually a Toshiba 5x DVD-ROM drive, and MusicMatch had no difficulty reading audio CD files digitally. Encoding usually took less than one-sixth of a trackís play time.)

Surprise, surprise: the MP3 versions did not sound the same--but they were close. Very close, in fact, when I simply listened in normal stereo mode. The timbre was a little different, and the sound in mid-treble and treble ranges seemed rougher on the MP3 side. The good news: it still sounded good, certainly better than typical LP releases or cassette versions of these songs. The bad news: it wasn't quite identical to the CD.

In surround mode, the Altec Lansing circuitry offers a slightly tougher tool for analysis. You can turn down the center channel, effectively creating a karaoke effect (since most vocals are recorded dead-center in a stereo mix, the vocals almost disappear when you do this). This mode offers a clearer picture of stereo image stability and unveils lesser problems with the sound in general. I'm afraid that the "null-center" test isn't very kind to high-rate MP3. On both songs, the sound became unstable, with audible pumping (volume going up and down when it shouldn't) and an "underwater" or warbling effect. Since doing this test, I've seen others comment on the "underwater" artifacts you can get from MP3 encoding--but usually at much lower bit rates. At about 2:25 in "Bridge Over Troubled Water," the nulled-center test shows so much warbling on the MP3 that it's unpleasant to hear.

Listening again in straight stereo, I find that there's a little more air and body on the CD version; the voices are a bit more generic in MP3 form. On "Bridge Over Troubled Water," the piano has a less distinctive sound and the massed strings are a bit shrill in MP3, with some "shimmering" that shouldn't be there.

Gordon Lightfoot at 128Kbps

The second pair of early tests was two songs from Gordon Lightfoot's Sundown album, ripped at 128Kbps. While the Simon & Garfunkel cuts were taken from Old Friends, a superbly remastered compilation of S&G songs, Sundown is a 25-year-old recording that isn't audiophile by any standards. I recorded "Circle of Steel" and "Is There Anyone Home?"

In stereo mode, I found the flute thin (almost turning into a recorder) on "Circle of Steel," with a little loss of air and depth; the oboe later in the song was edgy and distorted. Even in stereo mode, the differences were clearly audible, even on my PC speakers and with my indifferent hearing. In null-center mode, both cuts ranged from pretty good to ugly. The guitar on "Circle of Steel" seemed distorted; at 1:30 into the cut, things got bad enough to be unpleasant. In general, "Is There Anyone Home?" suffered from pumping and warbling.

At 128Kbps, even with this easy a test, I consider "CD quality" to be a serious overstatement; this wasn't even all that close. On a serious high fidelity system, I'd expect the differences to be fairly dramatic.

The mp3.com Sample CD

When the mp3.com "103 of the best songs you've never heard" CD arrived, I tried it, originally with the same problem noted in "Oops." It ranged from decent to awful.

With that problem corrected, the sample CD--which seems to be recorded at 128Kbps--ranges from very good to mediocre. For portable use, you'd probably never tell the difference between the best cuts and a CD; on a good system, you probably would.

Note that these tests were all at the highest fixed-rate options that MusicMatch offers, and don't include "serious" music, which should be more difficult to compress. I expect to do more tests when free time allows, and I'll add those tests in sections inserted before "Why Call It CD Quality?".

[top]

Eliminating Variables

Shortly after writing up the corrected version of my initial tests, I noted them to a group of colleagues whose opinions I greatly respect. Two of those colleagues, both with some MP3 experience and some background in sound, raised issues about the tests. While I could comfortably deal with one group of issues, there was another that proved difficult. To wit, "What about the sound card?"

While I know that my PC's DVD-drive CD output is going through some portion of the sound circuitry, that's not a complete answer. My "sound card" (really motherboard sound support) is the default for almost all Gateway systems, and is one of the least expensive designs that Creative Labs produces. It's not even a Creative Labs design; it's an Ensoniq chip and design (Creative Labs purchased Ensoniq).

Was it possible that the comparison between MP3 output and CD output was compromised by the fact that MP3 output is inherently limited by sound card quality? (The "ripping" shouldn't be: MusicMatch reads the CD output as a digital stream directly from the drive.)

Going through MusicMatch's options once again, I noticed one that I'd missed: recording in WAV format, which doesn't add the output to the MusicMatch catalog. If MusicMatch would convert the CD output to a WAV file, then playing back the WAV file as a comparison to the MP3 should eliminate the sound card as a variable: since MP3 playback effectively turns MP3 files into WAV output, and since both files would be coming from the same hard disk, the only variable should be MP3 compression itself. (I've been thinking about this a lot, and have yet to find any flaw in the logic.)

I recorded "Circle of Steel" in WAV format, both because it's the shortest of the four test cuts I'd been using and because it hadn't done particularly well in MP3 form: I found it tiring to listen to as background. The resulting WAV file used more than 29MB and was indeed recorded at full CD rate: 44.1KHz, 16-bit, stereo. It was 11 times as large as the MP3 file: just exactly what it should be.

There were two playback comparisons to do: comparing the WAV to the MP3 and comparing the WAV to the direct CD output. The results were revealing, but require the caveat that (here as throughout) these tests use middling speakers and are listened to with mediocre hearing. That's irrelevant for the WAV-to-MP3 results but quite relevant for the WAV-to-CD results.

To wit, I could not hear any difference whatsoever between the WAV output, played from hard disk through my PC's sound support, and the CD output, played directly from my PC's DVD drive. Whether I tried nulled-center surround mode, full stereo mode, one-channel stereo mode, or one-channel surround mode (also a tough test), I just couldn't hear any difference. That doesn't mean there isn't any (the caveat above). It means that the differences aren't audible to me, through these speakers. That result also said to me that the MP3-to-CD comparisons were valid: the quality of the sound card wasn't an issue.

Comparing the WAV output to MP3 output confirmed that. This comparison has the advantage that it's trivially easy to set identical volume levels: maximum volume from MusicMatch is identical to the output level from Windows' Media Player (which doesn't have its own volume control).

The results were, unsurprisingly, exactly the same as comparing MP3 to CD. The MP3 version had the same flaws as always, and the WAV version had none of those flaws. I didn't find the difference subtle.

I am not claiming that my PC's sound card passes CD-quality sound without any loss of quality. I suspect that's not true, and that someone with excellent hearing using a good stereo system could tell the difference. I am claiming that the differences are so much smaller than those caused by 128K MP3 encoding that they're irrelevant for these tests--and that the differences are inaudible to me on these speakers.

That clears the way for future testing rounds, including easier and more difficult music, lower bit rates, and (perhaps more interesting) MediaMatch's variable bit rate levels.

[top]

Why Call It CD Quality?

So why has the claim of CD quality appeared so often in print? Set aside what the software maker says; itís the job of PC journalists to validate such claims, not simply repeat them.

Iím a skeptic, but not a cynic. I donít believe that thereís much outright deception going on. Instead, I see a mix of four factors: hype, ignorance, equipment problems, and hearing problems.

  • Hype: People want MP3 to work miracles, and theyíre accustomed to Digital Technology as working miracles. Overstating the quality of MP3 recordings is par for the course.
  • Ignorance: Many people really donít know what stereophonic music should sound like, and I suspect quite a few PC journalists fall into that crowd. If youíve never heard anything better than a boom box, youíve never heard what a CD should sound like.
  • Equipment problems: This is a variation on "ignorance," in one sense. If your equipment doesnít offer decent reproduction, the loss of quality may not be audible. I used an inexpensive set of speakers for this testómuch better than a boombox or a typical carry-along player, but nowhere near as good as a "mid-fi" stereo system. If you canít hear the difference between a song played on FM radio and CD, you may not be bothered by MP3 sound. If you donít care about the difference between AM radio, a cheap cassette player, and CD, then you certainly wonít mind MP3 sound.
  • Hearing problems: My hearing is substantially degraded, but I do pay attention. A lot of peopleóparticularly a lot of adult menóhave significant hearing loss, and many of them (us?) really donít care. Once again: if you canít hear the difference, you wonít care about the difference.

This is not a sexist comment, but so far I havenít seen a commentary by an adult female music lover saying that MP3 offers "CD quality sound." Adult women typically retain their hearing better than men do. If a woman cares at all, itís likely that sheíll spot MP3ís defects.

So it goes. This modern miracle is a bit less miraculous than Iíd like. Thatís OKóunless, of course, the hypemeisters are right about MP3 replacing audio CDs. If this level of sound quality becomes accepted as the ideal, Iíd be unhappy. I donít see that happeningóbut Iím an optimist by nature. Right now, high-rate MP3 is good enough that I might use it for convenient background music--but no better than that.

Incidentally, while you canít play MP3-encoded CD-Rs on regular CD players today, the key word there is "today." If MP3, or some other (probably copy-protected) compressed audio format, becomes widely accepted, it would be nearly trivial for Sony and other CD-player manufacturers to add a decoding chip to their players. That would be a huge win for spoken-word recordings, where putting 10 (or even 20) hours on a single disc makes enormous sense. There are a number of other cases where commercial MP3 CDs might make sense without ruining good sound for todayís recordingsóand, to be sure, enthusiasts could collect those new obscure bands without big-label contracts.

The skeptic in me notes the problem in collecting MP3 cuts from obscure bands. While record companies may indeed be ignoring vital talents because big-company processes make them too expensive to record, thereís a good reason that most obscure bands are obscure. Namely, as with most other areas of artistry, most obscure bands just arenít very good. I've sampled every song in the "103 best songs you've never heard"--and, with a few exceptions, I'm not surprised that you've never heard them.


[Note: this essay is a work in progress, with notes being added as I do a little more investigation. A shorter version will appear in Crawfordís Corner 99.8, in the September/October 1999 Library Hi Tech News.]


Updated June 19, 1999; layout modified July 18, 1999

Return