High-th
roughput methods based on mass spect
romet
ry (p
roteomics, metabolomics, lipidomics, etc.) p
roduce a wealth of data that cannot be analyzed without computational methods. The impact of the choice of method on the ove
rall
result of a biological study is often unde
rapp
reciated, but diffe
rent methods can
result in ve
ry diffe
rent biological findings. It is thus essential to evaluate and compa
re the co
rrectness and
relative pe
rfo
rmance of computational methods. The volume of the data as well as the complexity of the algo
rithms
rende
r unbiased compa
risons challenging. This pape
r discusses some p
roblems and challenges in testing and validation of computational methods. We discuss the diffe
rent types of data (simulated and expe
rimental validation data) as well as diffe
rent met
rics to compa
re methods. We also int
roduce a new public
reposito
ry fo
r mass spect
romet
ric
refe
rence data sets (
ref="http://compms.org/RefData" class="extLink">http://compms.org/RefData) that contains a collection of publicly available data sets for performance evaluation for a wide range of different methods.