Improved inter-observer agreement of an expert review panel in an oncology treatment trial - Insights from a structured interventional process

详细信息    查看全文

• 作者：Ursula Nestle^a ; ^b ; ^{ursula.nestle@uniklinik-freiburg.de" class="auth_mail" title="E-mail the corresponding author} ; Hans Christian Rischke^a ; ^c ; Susanne Martina Eschmann^d ; Gabriele Holl^e ; Marco Tosch^f ; ^g ; Matthias Miederer^h ; Michail Plotkinⁱ ; Markus Essler^j ; Cornelia Puskas^k ; Tanja Schimek-Jasch^a ; Viola Duncker-Rohr^a ; ^l ; Friederike Rü ; hl^a ; Anja Leifert^a ; Michael Mix^c ; Anca-Ligia Grosu^a ; ^b ; Jochem Kö ; nig^m ; Werner Vachⁿ
• 关键词：Expert review ; Clinical trial ; Reading criteria ; FDG ; PET/CT ; Lung cancer ; Radiotherapy
• 刊名：European Journal of Cancer
• 出版年：2015
• 出版时间：November 2015
• 年：2015
• 卷：51
• 期：17
• 页码：2525-2533
• 全文大小：970 K

文摘

Oncologic imaging is a key for successful cancer treatment. While the quality assurance (QA) of image acquisition protocols has already been focussed, QA of reading and reporting offers still room for improvement. The latter was addressed in the context of a prospective multicentre trial on fluoro-deoxyglucose (FDG)–positron-emission tomography (PET)/CT-based chemoradiotherapy for locally advanced non-small cell lung cancer (NSCLC).

Material and methods

An expert panel was prospectively installed performing blinded reviews of mediastinal NSCLC involvement in FDG–PET/CT. Due to a high initial reporting inter-observer disagreement, the independent data monitoring committee (IDMC) triggered an interventional harmonisation process, which overall involved 11 experts uttering 6855 blinded diagnostic statements. After assessing the baseline inter-observer agreement (IOA) of a blinded re-review (phase 1), a discussion process led to improved reading criteria (phase 2). Those underwent a validation study (phase 3) and were then implemented into the study routine. After 2 months (phase 4) and 1 year (phase 5), the IOA was reassessed.

Results

The initial overall IOA was moderate (kappa 0.52 CT; 0.53 PET). After improvement of reading criteria, the kappa values improved substantially (kappa 0.61 CT; 0.66 PET), which was retained until the late reassessment (kappa 0.71 CT; 0.67 PET). Subjective uncertainty was highly predictive for low IOA.

Conclusion

The IOA of an expert panel was significantly improved by a structured interventional harmonisation process which could be a model for future clinical trials. Furthermore, the low IOA in reporting nodal involvement in NSCLC may bear consequences for individual patient care.