定量评价临床研究数据质量方法的理论与实践研究
|
摘要
当今世界正由经验医学向循证医学转变。医生的任务是向病人提供最好的服务，什么是最好的，需要拿出证据，这些证据很大程度上是基于对已有的临床研究的数据与结果做分析与评价而得出的。用于系统分析与荟萃分析的临床研究，其本身的数据质量与结论的可靠性是循证医学研究结论科学性的保证。
随着西方国家寻求回归自然和对传统医学内涵的逐步了解，中医药在世界许多国家包括发达国家的使用正在日益增多。尽管中医药的很多疗法已在多个国家和地区得到应用或民间使用，但尚未得到西方医学的承认，并被归类为“缺乏有效科学证据的医学技术或方法”。
这就把科学严谨的中医药临床研究提到了重要地位，科学严谨的中医药临床研究应遵循临床研究质量管理规范(Good Clinical Practice，GCP)，有科学的医学与统计学设计，实施临床研究数据管理(Clinical Data Management，CDM)。临床研究数据管理是临床试验质量保证的关键环节，临床研究的数据管理工作贯穿于整个试验的始终，涉及到试验管理的方方面面，是中医药研究中获得数据及时性、准确性、真实性、完整性的保证。
作为中医药临床研究机构，多年来，我们跟踪国际、国内有关CDM的法规、指南与学术进展，将获得的知识应用到中医药临床研究实践当中；探索CDM的最新技术，利用这些技术研发与CDM有关的计算机系统，改进我们的操作规程，从而达到优化中医药临床研究数据管理流程、提高数据质量与工作效率的目的。
在上述经验积累的基础之上，近年来国际社会CDM界的热门话题之一--临床研究数据质量定量评价引起了我们的关注；随着中医药CDM工作的开展，我们也迫切感到量化临床研究的数据质量，不仅是评价CDM工作的量度指标，而且对于今后制定临床研究数据质量标准，供业内参照执行具有重大意义。为此我们进行了临床研究数据质量定量评价的理论与实践两方面的研究工作。
理论研究部分首先查找了国际与国内临床研究方面的法规和权威性指南，探讨与总结了有关临床研究数据质量管理方面的要求；接着对定量评价临床研究的数据质量进行全面论述，包括定量评价的原因与目的、合格数据的定义与数据错误、定量评价临床研究的数据质量的方法、数据质量检查的频次与时机、关于检出错误的处理、可接受的质量水平(AQL)，最后分析了定量评价临床研究数据质量的国内外现状。
实践研究是我在日本一家有丰富CDM经验的合同研究组织(Contract ResearchOrganization，CRO)公司内进行。背景：通常用来代表数据质量的度量标准是差错率。在临床试验实施期间，几乎所有的数据转录与操作工作程序均伴有差错率。定量测量数据质量，了解临床试验中数据存在的错误，纠正或防止对临床试验结果有影响的错误是临床研究数据管理的重要活动。然而，行业内还没有公认的数据质量定量测量方法，公开发表的差错率还不具备可比性。目的我们采用诵读核对法弄清双份数据录入后对比病例报告表的差错率，分析产生错误的来源或原因，为预防这些错误提出建议；并评价诵读核对法的错误检出能力。方法我们调查了一项临床试验的数据管理工作记录，其中诵读核对法用来保证双份数据录入后数据的质量。所有通过诵读核对法发现与纠正的错误都在这些工作记录中留有稽查轨迹。我们主要挑选出诵读核对法发现的错误，计算录入的总字段数，以计算双份录入后对比病例报告表的差错率；诵读核对法也可能漏检错误，我们浏览逻辑检查与人工逻辑检查记录，找出诵读核对法漏检的错误，以评价诵读核对法的错误要检出能力。结果全体变量的差错率是0.01631％，关键变量与非关键变量中的差错率分别为0.01429％与0.01801％，诵读核对法检出了99.66％的错误。局限性本研究所探讨的数据质量仅限于数据管理全程中的一个过程，即数据录入。临床研究数据质量最重要的是残差率(RER)，测量RER，评估其对临床试验结果的影响还需要更多的努力。结论诵读核对法可用于定量测量数据质量。使用诵读核对法时，要注意数据一览表的版面设计，既要考虑易读性与反映病例报告表与数据库中的真实数据点，还要考虑打印输出的成本。
虽然我们开展了一些中医药的CDM工作，但目前国内整体的CDM水平与国外发达国家相比还有相当大的差距。然而，即使是在国际上，临床研究数据质量的定量评价尚未形成公认的方法与可接受的数据质量标准。临床研究数据质量的定量评价有助于CDM工作规范的形成，因此，我国的CDM发展需要临床研究数据质量的定量评价来推动，同时，也要为该项工作创造环境与条件。本研究最后一个部分对实施临床研究数据质量定量评价可行性进行了探讨与分析，认为主要可从以下4个方面努力：①建立健全单位内部的，临床研究质量保证体系；②在数据管理计划中有关于数据质量定量评价的规定与计划；③建立纵向型数据库，标准化数据收集模块；④设立独立的数据质量稽查部门或职位，培养数据质量检查专业人员。
Nowadays medicine has been transforming from Experienced-based Medicine toEvidence-based Medicine. It is Doctors' mission to provide their patients with best service.What best service is determined by evidence, which largely built on analyses andevaluation of available clinical study data and outcomes? The data quality of clinicalstudies and the reliability of their conclusions which are used in systematic analysis andmeta-analysis are the basses of scientific integrity of research conclusions ofEvidence-based Medicine.
With western countries began to embrace nature and increasingly understanding theconnotation of traditional medicine, Traditional Chinese Medicine (TCM) is usedincreasingly in many countries including developed countries. Widely used, however, TCMhave yet got acknowledgement in the field of western medicine, and was classified as"medical technology or method lack of effective scientific evidence".
Scientific and precise clinical study of TCM therefore has been promoted to animportant position. TCM clinical study should not only follow Good Clinical Practice(GCP), scientifically designed in terms of both medicine and statistics, but also implementClinical Data Management (CDM). CDM is involved in all aspects of clinical trial from thebeginning to the end. CDM may guarantee the timeliness, accuracy, authenticity andintegrality of data obtained in TCM clinical study.
As an institution of TCM clinical study, we have been tracing the development ofregulations and guidelines on CDM both home and abroad and applying acquiredknowledge into our TCM clinical study practices for years. We also have explored latesttechnologies of CDM, utilized these technologies, developed CDM-related computersystems, and improved our operating procedures to streamline CDM work flow in TCMclinical studies, so as to improve data quality and work efficiency.
On the basis of above accumulated experiences, one of the hot topics in the area ofinternational CDM——quantitative evaluation of clinical study data quality arouses ourattention. With the expansion of CDM in TCM clinical study, we have realized that quantifying data quality of clinical study is not only the metrics for evaluating the CDMperformance, but also important for stipulating data quality criteria in the near future.Therefore, we conducted both theoretic and practical studies on quantitative evaluation ofdata quality of clinical study.
Theoretical part of this study begins with law and regulations, as well as authoritativeguidelines, discussing and summarizing the requirements of data quality; Overalldiscussion of quantitative evaluation of data quality of clinical study was followed,including reasons and objectives of quantification, definition of quality data and dataerrors, methods of quantitative evaluation of data quality, frequency and occasions for dataquality check, disposal of found errors, acceptable quality level(AQL), and analysis ofpresent situation on this subject both home and abroad.
Practical part of the study was conducted in a CRO company in Japan. BackgroundThe metric often used to represent data quality is error rate. During the implementation ofclinical trials, nearly all data transcription and manipulation processes are accompanied byan error rate. Measuring data quality, understanding the errors that are present in clinicaltrial data and preventing errors that have an impact on study results are important activitiesin clinical data management. However, still no established ways existed to measure dataquality, and published error rates are not comparable in the industry. Purpose we tried tomake clear of data quality after double data entry comparing with CRF withreading-through method, and assess the ability of this method to detect errors. Method weexplored data management working records in which reading-through method were usedafter double entry to assure data quality. All errors found and corrected were left with audittrails in these records. We picked up errors found mainly by reading-through method,counted fields in which keyed data are included to calculate error rates after double entrycomparing with CRF. Results We saw a 0.01631％error rate for overall variables,0.01429％for critical variables and 0.01801％for non-critical variables in our study, andreading-through method detected 99.66％of the errors. Limitation The data qualitydiscussed here is only limited to one process of data management (data entry). The moreimportant metric for clinical data quality is Residual Error Rate (RER)Measuring RER andevaluating its impact on clinical trial results need further efforts. ConclusionReading-through method can be applied to measure data quality. Attention should be givento layout design of data sheet, taking into account of legibility for readers, reflection of truedata point on CRF, and cost for printout.
Although, we have carried out CDM in TCM clinical study for several years, gap stillexists between our country and developed countries in terms of CDM. Even in abroad,there are no established method to quantify data quality of clinical study and acceptablequality level for data quality, quantitative evaluation of data quality may facilitate thenormalization of CDM process, thus the development of CMD in our country needs topromoted by quantitative evaluation of data quality; at the same time, CDM efforts shouldcreate environment and conditions for quantitative evaluation of data quality of clinicalstudy. So we discussed the feasibilities of implementing quantitative evaluation of dataquality of clinical study in our country, and concluded that the following 4 aspects needmore efforts:①Establishing and improving internal clinical study quality assurance system;②Including provision and planning for quantitative evaluation of data quality in datamanagement plan;③Establishing vertical structural database, standardizing datacollection models;④Constituting independent data quality audit department or post,cultivating professionals in this area,
随着西方国家寻求回归自然和对传统医学内涵的逐步了解，中医药在世界许多国家包括发达国家的使用正在日益增多。尽管中医药的很多疗法已在多个国家和地区得到应用或民间使用，但尚未得到西方医学的承认，并被归类为“缺乏有效科学证据的医学技术或方法”。
这就把科学严谨的中医药临床研究提到了重要地位，科学严谨的中医药临床研究应遵循临床研究质量管理规范(Good Clinical Practice，GCP)，有科学的医学与统计学设计，实施临床研究数据管理(Clinical Data Management，CDM)。临床研究数据管理是临床试验质量保证的关键环节，临床研究的数据管理工作贯穿于整个试验的始终，涉及到试验管理的方方面面，是中医药研究中获得数据及时性、准确性、真实性、完整性的保证。
作为中医药临床研究机构，多年来，我们跟踪国际、国内有关CDM的法规、指南与学术进展，将获得的知识应用到中医药临床研究实践当中；探索CDM的最新技术，利用这些技术研发与CDM有关的计算机系统，改进我们的操作规程，从而达到优化中医药临床研究数据管理流程、提高数据质量与工作效率的目的。
在上述经验积累的基础之上，近年来国际社会CDM界的热门话题之一--临床研究数据质量定量评价引起了我们的关注；随着中医药CDM工作的开展，我们也迫切感到量化临床研究的数据质量，不仅是评价CDM工作的量度指标，而且对于今后制定临床研究数据质量标准，供业内参照执行具有重大意义。为此我们进行了临床研究数据质量定量评价的理论与实践两方面的研究工作。
理论研究部分首先查找了国际与国内临床研究方面的法规和权威性指南，探讨与总结了有关临床研究数据质量管理方面的要求；接着对定量评价临床研究的数据质量进行全面论述，包括定量评价的原因与目的、合格数据的定义与数据错误、定量评价临床研究的数据质量的方法、数据质量检查的频次与时机、关于检出错误的处理、可接受的质量水平(AQL)，最后分析了定量评价临床研究数据质量的国内外现状。
实践研究是我在日本一家有丰富CDM经验的合同研究组织(Contract ResearchOrganization，CRO)公司内进行。背景：通常用来代表数据质量的度量标准是差错率。在临床试验实施期间，几乎所有的数据转录与操作工作程序均伴有差错率。定量测量数据质量，了解临床试验中数据存在的错误，纠正或防止对临床试验结果有影响的错误是临床研究数据管理的重要活动。然而，行业内还没有公认的数据质量定量测量方法，公开发表的差错率还不具备可比性。目的我们采用诵读核对法弄清双份数据录入后对比病例报告表的差错率，分析产生错误的来源或原因，为预防这些错误提出建议；并评价诵读核对法的错误检出能力。方法我们调查了一项临床试验的数据管理工作记录，其中诵读核对法用来保证双份数据录入后数据的质量。所有通过诵读核对法发现与纠正的错误都在这些工作记录中留有稽查轨迹。我们主要挑选出诵读核对法发现的错误，计算录入的总字段数，以计算双份录入后对比病例报告表的差错率；诵读核对法也可能漏检错误，我们浏览逻辑检查与人工逻辑检查记录，找出诵读核对法漏检的错误，以评价诵读核对法的错误要检出能力。结果全体变量的差错率是0.01631％，关键变量与非关键变量中的差错率分别为0.01429％与0.01801％，诵读核对法检出了99.66％的错误。局限性本研究所探讨的数据质量仅限于数据管理全程中的一个过程，即数据录入。临床研究数据质量最重要的是残差率(RER)，测量RER，评估其对临床试验结果的影响还需要更多的努力。结论诵读核对法可用于定量测量数据质量。使用诵读核对法时，要注意数据一览表的版面设计，既要考虑易读性与反映病例报告表与数据库中的真实数据点，还要考虑打印输出的成本。
虽然我们开展了一些中医药的CDM工作，但目前国内整体的CDM水平与国外发达国家相比还有相当大的差距。然而，即使是在国际上，临床研究数据质量的定量评价尚未形成公认的方法与可接受的数据质量标准。临床研究数据质量的定量评价有助于CDM工作规范的形成，因此，我国的CDM发展需要临床研究数据质量的定量评价来推动，同时，也要为该项工作创造环境与条件。本研究最后一个部分对实施临床研究数据质量定量评价可行性进行了探讨与分析，认为主要可从以下4个方面努力：①建立健全单位内部的，临床研究质量保证体系；②在数据管理计划中有关于数据质量定量评价的规定与计划；③建立纵向型数据库，标准化数据收集模块；④设立独立的数据质量稽查部门或职位，培养数据质量检查专业人员。
Nowadays medicine has been transforming from Experienced-based Medicine toEvidence-based Medicine. It is Doctors' mission to provide their patients with best service.What best service is determined by evidence, which largely built on analyses andevaluation of available clinical study data and outcomes? The data quality of clinicalstudies and the reliability of their conclusions which are used in systematic analysis andmeta-analysis are the basses of scientific integrity of research conclusions ofEvidence-based Medicine.
With western countries began to embrace nature and increasingly understanding theconnotation of traditional medicine, Traditional Chinese Medicine (TCM) is usedincreasingly in many countries including developed countries. Widely used, however, TCMhave yet got acknowledgement in the field of western medicine, and was classified as"medical technology or method lack of effective scientific evidence".
Scientific and precise clinical study of TCM therefore has been promoted to animportant position. TCM clinical study should not only follow Good Clinical Practice(GCP), scientifically designed in terms of both medicine and statistics, but also implementClinical Data Management (CDM). CDM is involved in all aspects of clinical trial from thebeginning to the end. CDM may guarantee the timeliness, accuracy, authenticity andintegrality of data obtained in TCM clinical study.
As an institution of TCM clinical study, we have been tracing the development ofregulations and guidelines on CDM both home and abroad and applying acquiredknowledge into our TCM clinical study practices for years. We also have explored latesttechnologies of CDM, utilized these technologies, developed CDM-related computersystems, and improved our operating procedures to streamline CDM work flow in TCMclinical studies, so as to improve data quality and work efficiency.
On the basis of above accumulated experiences, one of the hot topics in the area ofinternational CDM——quantitative evaluation of clinical study data quality arouses ourattention. With the expansion of CDM in TCM clinical study, we have realized that quantifying data quality of clinical study is not only the metrics for evaluating the CDMperformance, but also important for stipulating data quality criteria in the near future.Therefore, we conducted both theoretic and practical studies on quantitative evaluation ofdata quality of clinical study.
Theoretical part of this study begins with law and regulations, as well as authoritativeguidelines, discussing and summarizing the requirements of data quality; Overalldiscussion of quantitative evaluation of data quality of clinical study was followed,including reasons and objectives of quantification, definition of quality data and dataerrors, methods of quantitative evaluation of data quality, frequency and occasions for dataquality check, disposal of found errors, acceptable quality level(AQL), and analysis ofpresent situation on this subject both home and abroad.
Practical part of the study was conducted in a CRO company in Japan. BackgroundThe metric often used to represent data quality is error rate. During the implementation ofclinical trials, nearly all data transcription and manipulation processes are accompanied byan error rate. Measuring data quality, understanding the errors that are present in clinicaltrial data and preventing errors that have an impact on study results are important activitiesin clinical data management. However, still no established ways existed to measure dataquality, and published error rates are not comparable in the industry. Purpose we tried tomake clear of data quality after double data entry comparing with CRF withreading-through method, and assess the ability of this method to detect errors. Method weexplored data management working records in which reading-through method were usedafter double entry to assure data quality. All errors found and corrected were left with audittrails in these records. We picked up errors found mainly by reading-through method,counted fields in which keyed data are included to calculate error rates after double entrycomparing with CRF. Results We saw a 0.01631％error rate for overall variables,0.01429％for critical variables and 0.01801％for non-critical variables in our study, andreading-through method detected 99.66％of the errors. Limitation The data qualitydiscussed here is only limited to one process of data management (data entry). The moreimportant metric for clinical data quality is Residual Error Rate (RER)Measuring RER andevaluating its impact on clinical trial results need further efforts. ConclusionReading-through method can be applied to measure data quality. Attention should be givento layout design of data sheet, taking into account of legibility for readers, reflection of truedata point on CRF, and cost for printout.
Although, we have carried out CDM in TCM clinical study for several years, gap stillexists between our country and developed countries in terms of CDM. Even in abroad,there are no established method to quantify data quality of clinical study and acceptablequality level for data quality, quantitative evaluation of data quality may facilitate thenormalization of CDM process, thus the development of CMD in our country needs topromoted by quantitative evaluation of data quality; at the same time, CDM efforts shouldcreate environment and conditions for quantitative evaluation of data quality of clinicalstudy. So we discussed the feasibilities of implementing quantitative evaluation of dataquality of clinical study in our country, and concluded that the following 4 aspects needmore efforts:①Establishing and improving internal clinical study quality assurance system;②Including provision and planning for quantitative evaluation of data quality in datamanagement plan;③Establishing vertical structural database, standardizing datacollection models;④Constituting independent data quality audit department or post,cultivating professionals in this area,
引文
[1] 刘建平．循证医学与中医药现代化．华西医学，2002；1：133．
[2] 张鸣明，刘雪梅，李廷谦．系统评价中医药临床试验 促进中医药现代化．中国中医药信息杂志，2003；10：1～2．
[3] ICH. E6, Guideline for Good Clinical Practice. Expert Working Group; 1996.
[4] 卜擎燕，熊宁宁，邹建东，et al．从临床研究数据管理角度设计CRF．2007；16：339～343．
[5] 王怡兵，熊宁宁，卜擎燕，et al．临床试验数据采集与报告文件设计制作的技术规程．中国临床药理学与治疗学，2004；9：595～597．
[6] SFDA．药品注册管理办法．；2005．
[7] SFDA．药物临床试验质量管理规范(GCP)．；2003．
[8] Fong DY. Data management and quality assurance.Drug Information Journal,2001;35:839～844.
[9] Estabrook RW.Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report (Compass Series); Natl Academy Pr, 1999: 76.
[10] Day S, Fayers P, Harvey D. Double data entry: what value, what price?Control Clin Trials, 1998; 19:15～24.
[11] Zhang J, Hu W. Single or double data entry: considerations based on a simple binomial model.Control Clin Trials, 1998; 19:56～58.
[12] Reynolds-Haertle RA, McBride R. Single vs. double data entry in CAST.Control Clin Trials, 1992; 13:487～494.
[13] Buchele G, Och B, Bolte G, et al. Single vs. double data entry.Epidemiology,2005; 16:130～131.
[14] Kawado M, Hinotsu S, Matsuyama Y, et al. A comparison of error detection rates between the reading aloud method and the double data entry method.Control Clin Trials,2003;24:560～569.
[15] King DW, Lashley R.A quantifiable alternative to double data entry.Control Clin Trials,2000;21:94～102.
[16] Neaton JD, Duchene AG, Svendsen KH, et al. An examination of the efficiency of some quality assurance methods commonly employed in clinical trials.Star Med,1990;9:115～23124.
[17] 卜擎燕，熊宁宁，邹建东．电子数据获取：实现更加优质与高效的临床研究．中国临床药理学与治疗学，2007；356～360．
[18] Banik N.Evaluation of EDC versus Paper in a Multinational Asthma Trial.Presented at DIA European Data Management Meeting,Berlin, 1998:
[19] Spink C.Electronic Data Capture (EDC) as a means for e-clinical trial success.IBM Global Services, Pharmaceutical Clinical Development,2002:
[20] Green JA. The EDC Value proposition to the pharmaceutical industry. A Detailed Comparison of EDC Versus Paper Model Costs For Four Different Clinical Research Projects (Phase Ⅰ-Ⅲb).: Datatrak International, Inc; 2001.
[21] Forrester Research, Inc., Cambridge, Mass. For more information, visit forrester.com.
[22] Good Clinical Data Management Practices. Society for Clinical Data Management. 4~*ed; 2005.
[23] FDA. 21 CFR Part 11 Electronic Records; Electronic Signatures. the United States; 1997.
[24] FDA. Guidance for Industry: Part 11, Electronic Records; Electronic Signatures-Scope and Application. the United States; 2003.
[25] D V. Part Ⅱ Corrupt Data: Sources and Solutions. Issues in Clinical Trial Management: Research Nurse; 1999. p 12～17.
[26] Califf RM, Karnash SL, Woodlief LH. Developing systems for cost-effective auditing of clinical trials.Control Clin Trials, 1997; 18:651～660.
[27] DeMets DL. Distinctions between fraud, bias, errors, misunderstanding, and incompetence.Control Clin Trials, 1997; 18:637～650.
[28] 王怡兵，熊宁宁，卜擎燕，et al．临床试验数据质量的评估．中国临床药理学与治疗学，2004；717～720．
[29] Paul Z. Statistical issues in clinical trial data audit.Drug information journal,2004;38:371～386.
[30] KDW, Melynda H. Quantifying gains in data quality for sampling plans used in clinical trial monitoring.Drug information journal,2003;37:135～141.
[31] Brunetle R KR. A Database Quality review Process with Interim Checks.Drug Information Journa,2002;36:357～367.
[32] Sullivan EM, Gorko MA, Stellon RC CG. A statistically based process for auditing clinical data quality listings.Drug Information Journal, 1997;31:647～653.
[33] Anderson P WB.CQA Primer. W. Terre Haute.Quality Council of Indiana,1997:
[34] Gassman JJ, Owen WW, Kuntz TE, et al.Data quality assurance, monitoring, and reporting.Control Clin Trials, 1995; 16:104～136.
[35] Hosking JD, Newhouse MM, Bagniewska A, et al. Data collection and transcription.Control Clin Trials,1995;16:66～103.
[36] ML N.Optimizing Data Quality Through Statistical Process Control.15 Th. Annual US Clinical Data Management Symposium, Drug Information Association,Philadelphia, PA,2000:
[37] Shenker I. Annals of Lexicography: The Dictionary Factory.: The New Yorker; 1989. p86～100.
[38] 小関洋子．臨床试験品質保证．；2000．
[39] D H. ISO9000 quality systems handbook. Oxford: Butterworth-Heinemann Ltd; 1994.
[40] CDISC. Statistical Analysis Dataset Model: General Considerations. CDISC Analysis Dataset Modeling Team. Version 0.7~*ed; 2005.
[41] 李雪迎，康晓平，姚晨．药物临床试验安全性评价标准化数据结构的建立．中国临床药理学杂志，2006；22：158～160．
[42] 李雪迎，康晓平，姚晨．临床试验研究数据库数据结构的选择．中国临床药理学与治疗学，2005；1076～1080．
[43] 徐士良．实用数据结构，北京；清华大学出版社，2000：140～144．
[44] Prokscha S.Practical Guide to Clinical Data Management; Interpharm Press Inc, 1999: 185.
[45] K B. Legacy System: coping with Success.IEEE Software,1995;12:19～23.
[46] 刘晓建陈平 蔡希尧．遗产系统及其解决方案的综述．计算机科学，2002；29：127～129．
[2] 张鸣明，刘雪梅，李廷谦．系统评价中医药临床试验 促进中医药现代化．中国中医药信息杂志，2003；10：1～2．
[3] ICH. E6, Guideline for Good Clinical Practice. Expert Working Group; 1996.
[4] 卜擎燕，熊宁宁，邹建东，et al．从临床研究数据管理角度设计CRF．2007；16：339～343．
[5] 王怡兵，熊宁宁，卜擎燕，et al．临床试验数据采集与报告文件设计制作的技术规程．中国临床药理学与治疗学，2004；9：595～597．
[6] SFDA．药品注册管理办法．；2005．
[7] SFDA．药物临床试验质量管理规范(GCP)．；2003．
[8] Fong DY. Data management and quality assurance.Drug Information Journal,2001;35:839～844.
[9] Estabrook RW.Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report (Compass Series); Natl Academy Pr, 1999: 76.
[10] Day S, Fayers P, Harvey D. Double data entry: what value, what price?Control Clin Trials, 1998; 19:15～24.
[11] Zhang J, Hu W. Single or double data entry: considerations based on a simple binomial model.Control Clin Trials, 1998; 19:56～58.
[12] Reynolds-Haertle RA, McBride R. Single vs. double data entry in CAST.Control Clin Trials, 1992; 13:487～494.
[13] Buchele G, Och B, Bolte G, et al. Single vs. double data entry.Epidemiology,2005; 16:130～131.
[14] Kawado M, Hinotsu S, Matsuyama Y, et al. A comparison of error detection rates between the reading aloud method and the double data entry method.Control Clin Trials,2003;24:560～569.
[15] King DW, Lashley R.A quantifiable alternative to double data entry.Control Clin Trials,2000;21:94～102.
[16] Neaton JD, Duchene AG, Svendsen KH, et al. An examination of the efficiency of some quality assurance methods commonly employed in clinical trials.Star Med,1990;9:115～23124.
[17] 卜擎燕，熊宁宁，邹建东．电子数据获取：实现更加优质与高效的临床研究．中国临床药理学与治疗学，2007；356～360．
[18] Banik N.Evaluation of EDC versus Paper in a Multinational Asthma Trial.Presented at DIA European Data Management Meeting,Berlin, 1998:
[19] Spink C.Electronic Data Capture (EDC) as a means for e-clinical trial success.IBM Global Services, Pharmaceutical Clinical Development,2002:
[20] Green JA. The EDC Value proposition to the pharmaceutical industry. A Detailed Comparison of EDC Versus Paper Model Costs For Four Different Clinical Research Projects (Phase Ⅰ-Ⅲb).: Datatrak International, Inc; 2001.
[21] Forrester Research, Inc., Cambridge, Mass. For more information, visit forrester.com.
[22] Good Clinical Data Management Practices. Society for Clinical Data Management. 4~*ed; 2005.
[23] FDA. 21 CFR Part 11 Electronic Records; Electronic Signatures. the United States; 1997.
[24] FDA. Guidance for Industry: Part 11, Electronic Records; Electronic Signatures-Scope and Application. the United States; 2003.
[25] D V. Part Ⅱ Corrupt Data: Sources and Solutions. Issues in Clinical Trial Management: Research Nurse; 1999. p 12～17.
[26] Califf RM, Karnash SL, Woodlief LH. Developing systems for cost-effective auditing of clinical trials.Control Clin Trials, 1997; 18:651～660.
[27] DeMets DL. Distinctions between fraud, bias, errors, misunderstanding, and incompetence.Control Clin Trials, 1997; 18:637～650.
[28] 王怡兵，熊宁宁，卜擎燕，et al．临床试验数据质量的评估．中国临床药理学与治疗学，2004；717～720．
[29] Paul Z. Statistical issues in clinical trial data audit.Drug information journal,2004;38:371～386.
[30] KDW, Melynda H. Quantifying gains in data quality for sampling plans used in clinical trial monitoring.Drug information journal,2003;37:135～141.
[31] Brunetle R KR. A Database Quality review Process with Interim Checks.Drug Information Journa,2002;36:357～367.
[32] Sullivan EM, Gorko MA, Stellon RC CG. A statistically based process for auditing clinical data quality listings.Drug Information Journal, 1997;31:647～653.
[33] Anderson P WB.CQA Primer. W. Terre Haute.Quality Council of Indiana,1997:
[34] Gassman JJ, Owen WW, Kuntz TE, et al.Data quality assurance, monitoring, and reporting.Control Clin Trials, 1995; 16:104～136.
[35] Hosking JD, Newhouse MM, Bagniewska A, et al. Data collection and transcription.Control Clin Trials,1995;16:66～103.
[36] ML N.Optimizing Data Quality Through Statistical Process Control.15 Th. Annual US Clinical Data Management Symposium, Drug Information Association,Philadelphia, PA,2000:
[37] Shenker I. Annals of Lexicography: The Dictionary Factory.: The New Yorker; 1989. p86～100.
[38] 小関洋子．臨床试験品質保证．；2000．
[39] D H. ISO9000 quality systems handbook. Oxford: Butterworth-Heinemann Ltd; 1994.
[40] CDISC. Statistical Analysis Dataset Model: General Considerations. CDISC Analysis Dataset Modeling Team. Version 0.7~*ed; 2005.
[41] 李雪迎，康晓平，姚晨．药物临床试验安全性评价标准化数据结构的建立．中国临床药理学杂志，2006；22：158～160．
[42] 李雪迎，康晓平，姚晨．临床试验研究数据库数据结构的选择．中国临床药理学与治疗学，2005；1076～1080．
[43] 徐士良．实用数据结构，北京；清华大学出版社，2000：140～144．
[44] Prokscha S.Practical Guide to Clinical Data Management; Interpharm Press Inc, 1999: 185.
[45] K B. Legacy System: coping with Success.IEEE Software,1995;12:19～23.
[46] 刘晓建陈平 蔡希尧．遗产系统及其解决方案的综述．计算机科学，2002；29：127～129．