* MP3 vs. CD: The Beer Test

Unless you’re a dog, a whale, or a computer, you’re not going to be able to tell the difference between a good MP3 and CD audio.

Think you’re an audiophile? I’ll bet you beers you can’t tell the difference between 256 Kbps VBR MP3s and CD audio in a blind test.

Track Selection

12 tracks have been selected from commercially sold, mass-produced audio CDs (i.e. “silver” discs – no CD-R, no online purchases, etc.). The primary criteria for track selection are length of track (2-4 minutes), availability (I have to own them), and pure subjectivity.

Selected tracks are:

Song	Artist	Album	Genre
Cats & Dogs	Howard Shore	The Truth About Cats & Dogs (Soundtrack)	Jazz
The Four Seasons: II. Largo	Antonio Vivaldi	Classical Christmas Favorites	Holiday
Goldberg Variations	Johann Sebastian Bach	Tune Your Brain with Bach: Heal	Classical
I Wonder As I Wander	The Night Heron Consort	A Celtic Celebration	Holiday
I’m On Fire	Bruce Springsteen	Born In The U.S.A.	Rock
No!	They Might Be Giants	No!	Alternative, Children’s Music
The Nutcracker Suite – Dance of the Sugar Plum Fairy	Peter Ilyich Tchaikovsky	Classical Christmas Favorites	Holiday
She’s Always A Woman	Billy Joel	The Stranger	Rock
Surfin’ U.S.A.	The Beach Boys	Endless Summer	Rock
Violin Concerto No. 2, K. 211, III	Itzhak Perlman,Violine; Wiener Philharmoniker, James Lavine	Mozart: Violin Concertos No. 2 & 4	Classical
We Will Rock You	Queen	ESPN Presents Stadium Anthems	Rock
Zombie Zoo	Tom Petty	Full Moon Fever	Rock

Order

Songs are ordered alphabetically by song title.

CD to MP3

MP3 tracks were imported to iTunes with these settings:

• stereo bit rate = 256 Kbps Variable Bit Rate (VBR)
• Quality = Highest
• Sample Rate = 48 kHz
• Channels = Audio
• Stereo Mode = Normal
• Smart Encoding Adjustments = Off
• Filter Frequencies Below 10 Hz = On

CD tracks were imported to iTunes using the Apple Lossless Encoder.

Randomness

To determine which track comes first in a pairing, I rolled a six-sided die. If odd, the MP3 track comes first. If even, the CD track comes first.

Sound Level

I created a playlist with the 24 tracks and sound-leveled all tracks with iVolume v3.1.4.

CD Burning

A CD-R was burned with my iMac using iTunes. The CD drive is 24x read/write, Matshita DVD-R UJ-875, firmware revision DB09.

Listening

Using any stereo components desired, subject may listen to tracks on audio CD for for as long and as often as desired switching tracks at will, making notes, etc.

Subject must work alone and not seek opinions from others.

To avoid the possibility of finding differences in the original and compressed files, subject must agree not to rip any tracks from the CD or in a any way use a computer to analyze the CD.

Subject must use his/her ears only. No visual cues (including, but not limited to, graphic equalizers) may be used.

Identification

For each pair of tracks representing the same original CD track, the subject must identify whether the A track or the B track is the original.

Scoring

Based on pure chance, the odds for getting a specific number of tracks identified correctly are:

N	Percentage
0	0.024%
1	0.293%
2	0.611%
3	5.371%
4	12.085%
5	19.336%
6	22.559%
7	19.336%
8	12.085%
9	5.371%
10	0.611%
11	0.293%
12	0.024%

What the results mean:

Number Correct	Conclusion
5, 6, or 7	No better than random chance. Sorry, you’re not an audiophile.*
3, 4, 8, or 9	Better than chance, but not significant enough to make claims about the ability to distinguish between original and compressed files without further testing.*
0, 1, or 2	Statistically significant. However, since the subject identified most tracks incorrectly, there will be much mocking.*
10 or 11	Statistically significant. Subject can reliably distinguish between original and encoded files, may claim such, and may spend time re-ripping entire CD collection at a higher bitrate. Subject will be treated to beers at a mutually agreed upon establishment.
12	Accolades of auditory excellence. Subject may also have a cheeseburger.

*Plus you owe me beers.

Credits

In May 2005, I bragged about my ability to distinguish between 128 Kbps MP3s and CD audio. My MIT friend Dave Kaffine then challenged me with this test. His test was double-blind, used an HP-48SX for ordering the tracks, and use Perl scripts for the randomness. My test is not double-blind, used alphabetical order for the tracks, and used a six-sided die for randomness, so it’s admittedly less rigorous. But I’m no Dave, nobody is. When I took Dave’s test, I got 7 of 12 tracks correct, which is statistically insignificant. However, I maintain that I was certain (and could tell within five seconds) on at least a couple of tracks. I concluded that if I could tell the difference on one track, then I must re-rip my entire CD collection from 128 Kbps CBR to 256 Kbps VBR, which I did. So I declared moral victory but lost the bet.

I think I’ll be getting free beers from anyone that takes me up on this challenge.

Related Posts

• Rolling Stone Announces More Five-Star Rated Albums And All I Got Was This Lousy Feed
  Want to know why the music business is broken? Look at how Rolling Stone Magazine mismanages its most prized asset: five-star reviews.
• Holy CD-Quality Audio, Batman! AmazonMP3.com MP3 Downloads Are 256 Kbps VBR!
  Following Apple’s lead, Amazon now offers DRM-free MP3 downloads.
• The Analog Curse
• It’s The Metadata, Stupid
  For the music that I have purchased, I want all of the metadata: the MP3s, the album art, the BPM data, the liner notes, the lyrics, the tablatures, the recording dates, the release dates, the artist’s history, etc. Today, I can’t get all of the metadata from a single source.
• The Dark (Gray) Age Of Music On The Internet
  While there are a lot of choices for consumers, the current state of music on the Internet leaves a lot to be desired.