不同链接函数下多级评分认知诊断模型的比较及应用研究
|
摘要
认知诊断测验因具有传统测验所不具备的诊断功能而日益受到重视。当前多级评分认知诊断模型开发中,研究者采用不同的链接函数(link function)开发出不同的多级评分认知诊断模型。本研究基于局部或相邻类别链接函数(local or adjacent categories link function)的思想,开发出多级评分认知诊断模型LC-DINA。研究采用Monte Carlo模拟研究与实证应用研究相结合的方法,将新开发模型与已有模型进行比较并应用于国际数学与科学评估(TIMMS)中,为实际应用者提供借鉴。
Cognitive Diagnosis Assessment(CDA) has recently received increasing attention due to the diagnostic function that is not available in traditional measurement theory.At present,varieties of CDMs have been proposed,but these models are mostly 0-1 score that cannot be used for polytomously scored items.This greatly limits their application and development in cognitive diagnosis.Because polytomous scored items are very common in practical applications,they have higher efficacy than dichotomous scored items.To deal with polytomous items more appropriately,a few polytomous CDMs have been developed.In view of the current research,researchers often choose different link functions to develop polytomous CDMs:Dongbo Tu developed the polytomous P-DINA model based on the Global or Cumulative link function in 2010,and Ma & de la Torre developed the Seq-GDINA model based on the Continuation Ration link function in 2016.In the Seq-GDINA model,the author defines a property definition based on the item category modifying the traditional Q matrix to a category-level Q-matrix(QC-matrix).The traditional Q-matrix is based on the definition of the project level,while ignoring the different manifestations of the attributes in the category,which will result in the loss of the existing information.In this paper,a new polytomous model LC-DINA based on Local or Adjacent Categories Link Function was developed,and the MCMC algorithm was used to realize the estimation of the parameters and the properties of the new model.Then,the proposed model LC-DINA was compared with the existing model Seq-DINA and P-DINA under the QC setting by simulation study using the data from the Trends in International Mathematics and Science Study 2007(TIMMS) assessment.In conclusion,the Monte Carlo simulation and empirical application studies were used to study the logical structure and applicable conditions of the three cognitive recognition models under three different link functions,providing a practical reference for users.The results of the paper showed that:(1) The good precision of ability parameters indicated that MCMC algorithm method and new polytomous cognitive diagnosis model were feasible.(2) The comparison of LC-DINA,P-DINA and Seq-DINA showed that the estimated precision of person parameters(MMR and PRM) was influenced by the quality of the item.The attribute match ration(MMR & PMR) decreased with the increase of the slipping and guessing parameter.(3)When the real model was consistent with the fitting model,the attribute match ration was the highest.In this condition,the LC-DINA model was the best when the item quality was best,and the P-DINA model was the best when the item quality was poor,and the Seq-DINA model was the worst in all the conditions.(4) The application of using the data from the TIMSS 2007 assessment showed that:LCDINA was the most suitable model based on the Deviance Information Criterion(DIC) and the correlation coefficient between the expected score and the observed score.
Cognitive Diagnosis Assessment(CDA) has recently received increasing attention due to the diagnostic function that is not available in traditional measurement theory.At present,varieties of CDMs have been proposed,but these models are mostly 0-1 score that cannot be used for polytomously scored items.This greatly limits their application and development in cognitive diagnosis.Because polytomous scored items are very common in practical applications,they have higher efficacy than dichotomous scored items.To deal with polytomous items more appropriately,a few polytomous CDMs have been developed.In view of the current research,researchers often choose different link functions to develop polytomous CDMs:Dongbo Tu developed the polytomous P-DINA model based on the Global or Cumulative link function in 2010,and Ma & de la Torre developed the Seq-GDINA model based on the Continuation Ration link function in 2016.In the Seq-GDINA model,the author defines a property definition based on the item category modifying the traditional Q matrix to a category-level Q-matrix(QC-matrix).The traditional Q-matrix is based on the definition of the project level,while ignoring the different manifestations of the attributes in the category,which will result in the loss of the existing information.In this paper,a new polytomous model LC-DINA based on Local or Adjacent Categories Link Function was developed,and the MCMC algorithm was used to realize the estimation of the parameters and the properties of the new model.Then,the proposed model LC-DINA was compared with the existing model Seq-DINA and P-DINA under the QC setting by simulation study using the data from the Trends in International Mathematics and Science Study 2007(TIMMS) assessment.In conclusion,the Monte Carlo simulation and empirical application studies were used to study the logical structure and applicable conditions of the three cognitive recognition models under three different link functions,providing a practical reference for users.The results of the paper showed that:(1) The good precision of ability parameters indicated that MCMC algorithm method and new polytomous cognitive diagnosis model were feasible.(2) The comparison of LC-DINA,P-DINA and Seq-DINA showed that the estimated precision of person parameters(MMR and PRM) was influenced by the quality of the item.The attribute match ration(MMR & PMR) decreased with the increase of the slipping and guessing parameter.(3)When the real model was consistent with the fitting model,the attribute match ration was the highest.In this condition,the LC-DINA model was the best when the item quality was best,and the P-DINA model was the best when the item quality was poor,and the Seq-DINA model was the worst in all the conditions.(4) The application of using the data from the TIMSS 2007 assessment showed that:LCDINA was the most suitable model based on the Deviance Information Criterion(DIC) and the correlation coefficient between the expected score and the observed score.
引文
蔡艳,赵洋,刘舒畅,张淑芳,涂冬波.(2017).一种优化的多级评分认知诊断模型.心理科学,40(6),1491-1497.
康春花,任平,曾平飞.(2015).非参数认知诊断方法:多级评分的聚类分析.心理学报,47(8),1077-1088.
田伟,辛涛.(2012).基于等级反应模型的规则空间方法.心理学报,44(2),249-262.
涂冬波,蔡艳,戴海琦,丁树良.(2012).一种多策略认知诊断方法:MSCD方法的开发.心理学报,44(11),1547-1553.
涂冬波,蔡艳,戴海琦.(2013).基于HO-DINA模型的多级评分认知诊断模型的开发.心理科学,36(4),984-988.
涂冬波,蔡艳,戴海琦,丁树良.(2010).一种多级评分的认知诊断模型:P-DINA模型的开发.心理学报,42(10),1011-1020.
吴方文,涂冬波,刘明矾.(2017).分离型的多级评分认知诊断模型开发及其应用研究.心理科学,40(1),209-215.
辛涛,乐美玲,张佳慧.(2012).教育测量理论新进展及发展趋势.中国考试,5,3-11.
朱金鑫,张淑梅,辛涛.(2009).属性掌握概率分类模型--一种基于Q矩阵的认知诊断模型.北京师范大学学报(自然科学版),45(2),117-122.
祝玉芳,丁树良.(2009).基于等级反应模型的属性层级方法.心理学报,41(3),267-275.
Birenbaum,M.,&Tatsuoka,K.K.(1987).Open-ended versus multiple-choice response formats-It does make a difference for diagnostic purposes.Applied Psychological Measurement,11(4),385-395.
de la Torre,J.,&Douglas,J.(2008).Model evaluation and multiple strategies in cognitive diagnosis:An analysis of fraction subtraction data.Psychometrika,73(4),595-624.
de la Torre,J.(2011).The generalized DINA model framework.Psychometrika,76(2),179-199.
Hansen,M.P.(2013).Hierarchical item response models for cognitive diagnosis.Unpublished doctorial dissertation of University of California,Los Angeles.
Hong,H.,Wang,C.,Lim,Y.S.,&Douglas,J.(2015).Efficient models for cognitive diagnosis with continuous and mixed-type latent variables.Applied Psychological Measurement,39(1),31-43.
Kaya,Y.,&Leite,W.L.(2017).Assessing change in latent skills across time with longitudinal cognitive diagnosis modeling:An evaluation of model performance.Educational and Psychological Measurement,77(3),369-388.
Ma,W.C.,&de la Torre,J.(2016).A Sequential cognitive diagnosis model for polytomous responses.British Journal of Mathematical and Statistical Psychology,69(3),253-275.
Masters,G.N.(1982).A rasch model for partial credit scoring.Psychometrika,47(2),149-174.
Patz,R.J.,&Junker,B.W.(1999).A straightforward approach to Markov chain Monte Carlo methods for item response models.Journal of Educational and Behavioral Statistics,24(2),146-178.
Samejima,F.(1969).Estimation of latent ability using a response pattern of graded scores.Richmond,VA:Psychometric Society.
Templin,J.L.,Henson,R.A.,Rupp,A.A.,Jang,E.,&Ahmed,M.(2008).Cognitive diagnosis models for nominal response data.Paper Presented at the Annual Meeting of the National Council on Measurement in Education,New York.
von Davier,M.(2008).A general diagnostic model applied to language testing data.British Journal of Mathematical and Statistical Psychology,61(2),287-307.
康春花,任平,曾平飞.(2015).非参数认知诊断方法:多级评分的聚类分析.心理学报,47(8),1077-1088.
田伟,辛涛.(2012).基于等级反应模型的规则空间方法.心理学报,44(2),249-262.
涂冬波,蔡艳,戴海琦,丁树良.(2012).一种多策略认知诊断方法:MSCD方法的开发.心理学报,44(11),1547-1553.
涂冬波,蔡艳,戴海琦.(2013).基于HO-DINA模型的多级评分认知诊断模型的开发.心理科学,36(4),984-988.
涂冬波,蔡艳,戴海琦,丁树良.(2010).一种多级评分的认知诊断模型:P-DINA模型的开发.心理学报,42(10),1011-1020.
吴方文,涂冬波,刘明矾.(2017).分离型的多级评分认知诊断模型开发及其应用研究.心理科学,40(1),209-215.
辛涛,乐美玲,张佳慧.(2012).教育测量理论新进展及发展趋势.中国考试,5,3-11.
朱金鑫,张淑梅,辛涛.(2009).属性掌握概率分类模型--一种基于Q矩阵的认知诊断模型.北京师范大学学报(自然科学版),45(2),117-122.
祝玉芳,丁树良.(2009).基于等级反应模型的属性层级方法.心理学报,41(3),267-275.
Birenbaum,M.,&Tatsuoka,K.K.(1987).Open-ended versus multiple-choice response formats-It does make a difference for diagnostic purposes.Applied Psychological Measurement,11(4),385-395.
de la Torre,J.,&Douglas,J.(2008).Model evaluation and multiple strategies in cognitive diagnosis:An analysis of fraction subtraction data.Psychometrika,73(4),595-624.
de la Torre,J.(2011).The generalized DINA model framework.Psychometrika,76(2),179-199.
Hansen,M.P.(2013).Hierarchical item response models for cognitive diagnosis.Unpublished doctorial dissertation of University of California,Los Angeles.
Hong,H.,Wang,C.,Lim,Y.S.,&Douglas,J.(2015).Efficient models for cognitive diagnosis with continuous and mixed-type latent variables.Applied Psychological Measurement,39(1),31-43.
Kaya,Y.,&Leite,W.L.(2017).Assessing change in latent skills across time with longitudinal cognitive diagnosis modeling:An evaluation of model performance.Educational and Psychological Measurement,77(3),369-388.
Ma,W.C.,&de la Torre,J.(2016).A Sequential cognitive diagnosis model for polytomous responses.British Journal of Mathematical and Statistical Psychology,69(3),253-275.
Masters,G.N.(1982).A rasch model for partial credit scoring.Psychometrika,47(2),149-174.
Patz,R.J.,&Junker,B.W.(1999).A straightforward approach to Markov chain Monte Carlo methods for item response models.Journal of Educational and Behavioral Statistics,24(2),146-178.
Samejima,F.(1969).Estimation of latent ability using a response pattern of graded scores.Richmond,VA:Psychometric Society.
Templin,J.L.,Henson,R.A.,Rupp,A.A.,Jang,E.,&Ahmed,M.(2008).Cognitive diagnosis models for nominal response data.Paper Presented at the Annual Meeting of the National Council on Measurement in Education,New York.
von Davier,M.(2008).A general diagnostic model applied to language testing data.British Journal of Mathematical and Statistical Psychology,61(2),287-307.