A method for the direct evaluation and characterisation of structural similarity measures is proposed. The method is based upon the similarity of input graphs relative to a reference graph. Ground truth data are obtained through a constructive process. The method is demonstrated in a study comparing three similarity measures. The similarity measure due to Blondel et al. is shown to exhibit stronger performance on larger graphs with a diverse supply of labels.