Behind the Scenes at Secrets - Number 1 - June, 2008


Since a quarterly newsletter seems to be the norm for large websites, the Secrets staff has decided that it is time for us to join the trend.

The problem I see with most newsletters is that they don't deliver anything that the website doesn't deliver in one way or the other.

Secrets was the first A/V magazine to be published only on the Internet.  We have never been a trend copier. We have always done things our own way. Maybe a few mistakes here and there, but mostly, we have set standards such as the Secrets DVD Player Benchmark that is now accepted worldwide as the most comprehensive set of criteria for the evaluation of DVD players.

What we want to do here is give you info that is not available on our website, and that information we think you will find useful. Perhaps eventually, these pages will be placed in our Primer section or somewhere else, but for now, you can only get it here in the newsletter.

OK, to start, we want to explain why and how we do our bench tests. There is too much to put in one newsletter, so it will be spread out over the entire 2008 year. That way, you can send us an e-mail requesting further info on some particular aspect that you would like to see more explanation.

So, why do we perform bench tests?

The short answer is that I (John Johnson, Editor-in-Chief) have a scientific background in the field of Neuroscience. Science requires objective measurements where possible and I routinely used very sensitive instruments to obtain objective measurement.

The long answer is that listening to music through a product being reviewed and stating that the sound has a lot of air, grunge, grit, or whatever other words one wants to use which don't really mean a lot when referring to audio, does not necessarily convey the same thing to everyone who reads them, nor even to the entire staff at the magazine. If you handed that product to a different reviewer, you would very likely end up with a completely different set of words, because the field of audio is filled with subjective words to describe the sound.

There is nothing wrong with such descriptions as long as there is something objective to go along with them. That does not mean that absolutely every product has to have some objective bench tests, but at least some of them do, in order to put numbers to the way of thinking in that organization.

So, we bench test, and we do these tests on CD players, DVD players, preamplifiers, SSPs, power amplifiers, speakers, subwoofers, cables, and some accessories like earphones. In general, all the tests are performed in the room where the products are listened to. This is a matter of convenience, but for speakers in particular, it gives real world numbers.

At Secrets, we use several instruments to get our numbers. These include hardware and software.

AudioPrecision is the name that all engineers think of when bench testing audio equipment. They have been around for decades. Their first instruments were analog, and the most recent ones have both analog and digital capabilities.

Last year, we purchased an Audio Precision SYS-2722. It consists of a hardware chassis that interfaces with a PC through a card or USB port. Software controls the chassis and displays the results, but all the number crunching is performed within the hardware chassis. The 2722 is not cheap, at $25,000. But, it is a must-have component for bench testing if you want certain tests that smaller software based analysis systems just can't perform.