Volumes 1993–2002 of four general medicine journals and four specialist journals were searched by hand for meta-analyses including at least five controlled trials. Characteristics were assessed using a standardized questionnaire.
A total of 272 meta-analyses, which included a median of 11 trials (range 5–195), were assessed. Most (81 % ) were published in general medicine journals. The median (range) number of databases searched increased from 1 (1–9) in 1993/1994 to 3.5 (1–21) in 2001/2002, P < 0.0001. The proportion of meta-analyses including searches by hand (10 % in 1993/1994, 25 % in 2001/2002, P = 0.005), searches of the grey literature (29 % , 51 % , P = 0.010 by chi-square test), and of trial registers (10 % , 32 % , P = 0.025) also increased. Assessments of the quality of trials also became more common (45 % , 70 % , P = 0.008), including whether allocation of patients to treatment groups had been concealed (24 % , 60 % , P = 0.001). The methodological and reporting quality was consistently higher in general medicine compared to specialist journals.
Many meta-analyses published in leading journals have important methodological limitations. The situation has improved in recent years but considerable room for further improvements remains.