Posted 06/04/2008 at 07:44:37am

Actually the scores are not necessarily humiliating, and they are very good when you look at favorite songs only. I would say the test indicates that people can indeed hear the difference, although not every time. Expected avg random guess score: 4 The actual scores: 3, 6, 5, 5 Avg score: (3+6+5+5) / 4 = 4.75 Favorite songs score: 3+3+0+1 = 7 4.75, the average score for the four testers, represents 19% more right answers than predicted by chance. Not bad, and it gets much better when we consider everyone's favorite song only. The single-song scores for the favorites are (3, 3, 0, 1). So the total score for the four favorites is 7, almost double the number of right answers the testers would have been likely to get by guessing. To figure the expected average random guess, consider that for each song, the testers are being asked to put the three versions in order by quality. For three things, there are six possible orders {ABC, BCA, CAB, ACB, CBA, BAC}. If "ABC" represents all three right, then the six orders give scores of {3, 0, 0, 1, 1, 1} respectively. Random guesses would randomly get one of those scores, for each song. So for each song, the expected avg random guess score is (3+0+0+1+1+1) / 6 = 1. For four songs, the random score would be four times that: 4 x 1 = 4.