基于Web的自适应考试系统的研究与设计
|
摘要
考试是衡量人们知识技能程度的重要手段,在我国,考试的存在已有两千多年的历史。但传统的纸笔考试大都以经典测验理论为基础,考试以固定的考题考核所有学生。其主要弊端是,每个特定的学生都有许多不是适应其水平的试题,高水平与低水平的考生均一律对待,考试中包含的所有难题与易题都要求作答,考分难以严格区分考生真实水平的差距。而计算机化自适应考试(computerized adaptive testing,CAT)基于项目反应理论,是一种方兴未艾的考试形式。CAT的优点是打破了传统考试“一刀切”策略,能根据考生水平差异“因人施测”,每个考题根据考生答题表现,动态抽取;考试往往时间短,结果更科学,效果亦更好。
美国研究生入学考试(Graduate Record Examination)、工商管理类研究生入学考试(Graduate for Management and Administration Test),以及全美护士国家委员会资格考试(Nurse National Committee License Test)等都已经开始使用CAT的形式。在国内,微软的MCSE考试和GRE/GMAT英语考试己经开始采用自适应考试形式。但是我国CAT使用总体情况是:CAT使用规模很小;实用的CAT、特别是B/S模式设计的CAT很少;现有的CAT中,很多能力估计和选题方法的选用相对过时。因此,CAT在我国还有很大研究和推广的空间。
本文在系统学习了自适应考试的相关理论、Web设计技术和数据库设计等知识的基础上、设计并实现了一个基于Web的自适应考试系统。CAT主要有两个核心和关键问题,能力估计方法是一个关键问题,考试的不同阶段该选用什么能力估计方法,并无固定模式。文章在对现有方法充分分析和实践探索基础上,针对不同能力估计方法的优缺点,在不同的能力估计阶段,选用了恰当的能力估计方法。选题策略是另一个核心问题,经过对多种选题方法的综合比较,按b(难度)分层是一个很好的选题方法,但针对不同的考试规模和应用背景,b应分成几层最佳?经过笔者的理论研究和实验比较,给出了针对三种考试规模下b的最佳分层数。
本文首先阐述了CAT的概念、研究现状、研究意义、CAT相关理论基础、数据库设计技术和Web设计技术;然后详细分析了系统的总体设计和各模块设计,以及在考试过程中所遇到的问题和解决方案;论文结尾通过实验给出了对CAT中能力估计和选题方法的探索结果;最后对自适应考试系统进行了总结,并且进一步展望了在线自适应考试系统今后的发展研究方向。
Examination is an important way to measure people's lever of knowledge and skill. It has existed for 2,000 years in China. But the traditional examination is based on classical testing theory and uses fixed items to exam all students. The main defect is that there is inconsistency between some item difficulty and examinee level. Treat both high-lever student and low-lever student as the same. Therefore, the testing result has its inherent high uncertainty. It is incredible to differentiate examinees only by the testing score. Computerized adaptive testing (CAT) which is ascendant examination is comparatively supported by project response theory. The advantage of CAT is that it breaks the uniformity tactics of conventional exam. CAT lies in that item difficulty is always consistent with examinees' acquaintance. Testing usually consumes less time, but the result is more scientific and better.
Graduate Record Examination, Graduate for Management and Administration Test, Nurse National Committee License Test, all of them adapt CAT. In China, MSCE of Microsoft and GRE/GMAT has used adaptive testing. But the general situation of using CAT is following: the scale of using CAT is too small; practical CAT, especially, the B/S mode is few; some estimating ability and selected topic tactics is relative out-of-date. Consequently, CAT has a large space for research and extend.
On the basis of generally studying the theory related to CAT and technology of database programming and Web design, the paper designs and implements a CAT based on Web. CAT has two core and key questions. Ability estimating method is one key question. In the different step of examination choose the available method of ability estimating. So it hasn't any fixed mode. At the basis of analyze and practical exploration, to the advantage and disadvantage of different ability estimating method, in different ability estimating period, this paper chooses the appropriate method. Item-selected tactics is the other key question. Through comparison lots of methods shows that b-stratification is a good item-selecting method. But according to different examination scale and application background, how many lever should divide is the best? After many researches and experiment comparison, the author gives the best number of item sub-database under three different testing sizes.
The article, firstly has introduced the CAT concept, current situation and significant of research, relative theory basis, database designing technology and web designing technology; and then analyze the design of whole system and each module in detail, and occur questions during examination and answering method; at the end of article, give the result for exploration of ability estimating and item-selected tactics through the experiment; finally, make a summary to adaptive testing system and look into the distance for direction of developing and researching in adaptive testing system.
美国研究生入学考试(Graduate Record Examination)、工商管理类研究生入学考试(Graduate for Management and Administration Test),以及全美护士国家委员会资格考试(Nurse National Committee License Test)等都已经开始使用CAT的形式。在国内,微软的MCSE考试和GRE/GMAT英语考试己经开始采用自适应考试形式。但是我国CAT使用总体情况是:CAT使用规模很小;实用的CAT、特别是B/S模式设计的CAT很少;现有的CAT中,很多能力估计和选题方法的选用相对过时。因此,CAT在我国还有很大研究和推广的空间。
本文在系统学习了自适应考试的相关理论、Web设计技术和数据库设计等知识的基础上、设计并实现了一个基于Web的自适应考试系统。CAT主要有两个核心和关键问题,能力估计方法是一个关键问题,考试的不同阶段该选用什么能力估计方法,并无固定模式。文章在对现有方法充分分析和实践探索基础上,针对不同能力估计方法的优缺点,在不同的能力估计阶段,选用了恰当的能力估计方法。选题策略是另一个核心问题,经过对多种选题方法的综合比较,按b(难度)分层是一个很好的选题方法,但针对不同的考试规模和应用背景,b应分成几层最佳?经过笔者的理论研究和实验比较,给出了针对三种考试规模下b的最佳分层数。
本文首先阐述了CAT的概念、研究现状、研究意义、CAT相关理论基础、数据库设计技术和Web设计技术;然后详细分析了系统的总体设计和各模块设计,以及在考试过程中所遇到的问题和解决方案;论文结尾通过实验给出了对CAT中能力估计和选题方法的探索结果;最后对自适应考试系统进行了总结,并且进一步展望了在线自适应考试系统今后的发展研究方向。
Examination is an important way to measure people's lever of knowledge and skill. It has existed for 2,000 years in China. But the traditional examination is based on classical testing theory and uses fixed items to exam all students. The main defect is that there is inconsistency between some item difficulty and examinee level. Treat both high-lever student and low-lever student as the same. Therefore, the testing result has its inherent high uncertainty. It is incredible to differentiate examinees only by the testing score. Computerized adaptive testing (CAT) which is ascendant examination is comparatively supported by project response theory. The advantage of CAT is that it breaks the uniformity tactics of conventional exam. CAT lies in that item difficulty is always consistent with examinees' acquaintance. Testing usually consumes less time, but the result is more scientific and better.
Graduate Record Examination, Graduate for Management and Administration Test, Nurse National Committee License Test, all of them adapt CAT. In China, MSCE of Microsoft and GRE/GMAT has used adaptive testing. But the general situation of using CAT is following: the scale of using CAT is too small; practical CAT, especially, the B/S mode is few; some estimating ability and selected topic tactics is relative out-of-date. Consequently, CAT has a large space for research and extend.
On the basis of generally studying the theory related to CAT and technology of database programming and Web design, the paper designs and implements a CAT based on Web. CAT has two core and key questions. Ability estimating method is one key question. In the different step of examination choose the available method of ability estimating. So it hasn't any fixed mode. At the basis of analyze and practical exploration, to the advantage and disadvantage of different ability estimating method, in different ability estimating period, this paper chooses the appropriate method. Item-selected tactics is the other key question. Through comparison lots of methods shows that b-stratification is a good item-selecting method. But according to different examination scale and application background, how many lever should divide is the best? After many researches and experiment comparison, the author gives the best number of item sub-database under three different testing sizes.
The article, firstly has introduced the CAT concept, current situation and significant of research, relative theory basis, database designing technology and web designing technology; and then analyze the design of whole system and each module in detail, and occur questions during examination and answering method; at the end of article, give the result for exploration of ability estimating and item-selected tactics through the experiment; finally, make a summary to adaptive testing system and look into the distance for direction of developing and researching in adaptive testing system.
引文
[1]漆书青.现代测量理论在考试中的应用.武汉:华中师范大学出版社,2003.4-7,192,197-221,234-235,270-274,361-363
[2]黄欣,黄维通,汤荷美,等.计算机(网络)考试的发展与自适应网络考试系统的实现.
[3]张厚粲.心理测量学的新方向—计算机化自适应测验.《自学考试研究论文集第二集》,经济科学出版社,1990.
[4]江西师大“题库理论”组.考生智能水平自适应测验,江西师范大学学报,1987,NO.2.
[5]韩布新、车宏生.计算机施测对心理测验的影响研究述评.心理与教育测量.浙江:浙江教育出版社,1997.
[6]戴海崎,丁树良,罗照盛.心理与教育统计测量专题研究文集.南昌:江西科学技术出版社,2005.6-11
[7]Lord M F.Applications of item response theory to practical testing problems.Hillsdale NJ:Erlbaum,1990.
[8]Wainer H.Computerized adaptive testing::A primer.Hillsdale NJ:Erlbraum,1990
[9]李虹,车宏生.计算机化考试的研究和发展.心理学动态,1999,7(2),24
[10]漆书青,戴海崎,丁树良.现代教育与心理测量学原理.北京:高等教育出版社,2002.125-132
[11]罗芬.项目反应理论中2PLM参数估计新方法:[硕士学位论文].南昌:江西师范大学,2003
[12]张华华,程莹.计算机化自适应测验(CAT)的发展和前景展望.考试研究,2005,1(1),20-22,32-33
[13]张华华.计算机自适应考试设计中的误区.考试研究,2002(2),35-39,156
[14]Chang,H.&Ying,Z.(1999).A-stratified multistage computered adaptive testing.Applied Psychological Measurement,23(3),211-222
[15]Chang,H.H.,Qian,J.,&Ying,Z.(2001).A-stratified multistage CAT with b-blocking.pplied Psychological Measurement,25,333-341.
[16]Leung,C.,Chang,H.,&hau,K.(2001,April).An examination of item selection rules by stratified CAT designs integrated with content balancing methods.Paper presented at the Annual Meeting of the American Educational Researcher Association,Seattle,WA.
[17]王茜娟,丁树良,谭渊.按c-分层不定长CAT的研究.江西师范大学学报(自)2005,NO.3,227-230
[18]鄢涛.基于B/S模式的通用无纸化考试系统的设计与实现:[硕士学位论文].成都:电子科技大学,2005
[19]文剑冰,杰泰.-stratified方法在不定长度CAT中的应用 第五届华人社会心理与教育学术研讨会(台北)2001
[20]廖彬山,峰霞.ASP动态网站开发教程.清华大学出版社,2000
[21]余嘉元.基于IRT和KST的网络测验.南京师大学报.1998,3,25-28
[22]沃德·切尼戴维·金凯德著薛密译.数值数学和计算.复旦大学出版社.1991
[23]何灿芝,吴俊杰.概率论与数理统计.湖南科学技术出版社,1994
[24]樊银庭等.基于客户机/服务器体系的二层与三层结构研究.计算机应用研究.2001,12
[25]范剑波,郭健康.基于Web和三层C/S结构的DBCAI系统的设计和实现.计算机应用,2001.4
[26]李虹,车宏生.计算机化考试的研究和发展.心理学动态,1999,vol7
[27]石振国.用JSP实现对Web数据库的访问.计算机应用,2001,5
[28]王昌达.基于WWW的远程辅导答疑系统的设计与实现.计算机应用,2001,6
[29]张玉连,郑小慎.智能计算机辅助教学评测系统算法分析.微计算机应用,2002,3
[2]黄欣,黄维通,汤荷美,等.计算机(网络)考试的发展与自适应网络考试系统的实现.
[3]张厚粲.心理测量学的新方向—计算机化自适应测验.《自学考试研究论文集第二集》,经济科学出版社,1990.
[4]江西师大“题库理论”组.考生智能水平自适应测验,江西师范大学学报,1987,NO.2.
[5]韩布新、车宏生.计算机施测对心理测验的影响研究述评.心理与教育测量.浙江:浙江教育出版社,1997.
[6]戴海崎,丁树良,罗照盛.心理与教育统计测量专题研究文集.南昌:江西科学技术出版社,2005.6-11
[7]Lord M F.Applications of item response theory to practical testing problems.Hillsdale NJ:Erlbaum,1990.
[8]Wainer H.Computerized adaptive testing::A primer.Hillsdale NJ:Erlbraum,1990
[9]李虹,车宏生.计算机化考试的研究和发展.心理学动态,1999,7(2),24
[10]漆书青,戴海崎,丁树良.现代教育与心理测量学原理.北京:高等教育出版社,2002.125-132
[11]罗芬.项目反应理论中2PLM参数估计新方法:[硕士学位论文].南昌:江西师范大学,2003
[12]张华华,程莹.计算机化自适应测验(CAT)的发展和前景展望.考试研究,2005,1(1),20-22,32-33
[13]张华华.计算机自适应考试设计中的误区.考试研究,2002(2),35-39,156
[14]Chang,H.&Ying,Z.(1999).A-stratified multistage computered adaptive testing.Applied Psychological Measurement,23(3),211-222
[15]Chang,H.H.,Qian,J.,&Ying,Z.(2001).A-stratified multistage CAT with b-blocking.pplied Psychological Measurement,25,333-341.
[16]Leung,C.,Chang,H.,&hau,K.(2001,April).An examination of item selection rules by stratified CAT designs integrated with content balancing methods.Paper presented at the Annual Meeting of the American Educational Researcher Association,Seattle,WA.
[17]王茜娟,丁树良,谭渊.按c-分层不定长CAT的研究.江西师范大学学报(自)2005,NO.3,227-230
[18]鄢涛.基于B/S模式的通用无纸化考试系统的设计与实现:[硕士学位论文].成都:电子科技大学,2005
[19]文剑冰,杰泰.-stratified方法在不定长度CAT中的应用 第五届华人社会心理与教育学术研讨会(台北)2001
[20]廖彬山,峰霞.ASP动态网站开发教程.清华大学出版社,2000
[21]余嘉元.基于IRT和KST的网络测验.南京师大学报.1998,3,25-28
[22]沃德·切尼戴维·金凯德著薛密译.数值数学和计算.复旦大学出版社.1991
[23]何灿芝,吴俊杰.概率论与数理统计.湖南科学技术出版社,1994
[24]樊银庭等.基于客户机/服务器体系的二层与三层结构研究.计算机应用研究.2001,12
[25]范剑波,郭健康.基于Web和三层C/S结构的DBCAI系统的设计和实现.计算机应用,2001.4
[26]李虹,车宏生.计算机化考试的研究和发展.心理学动态,1999,vol7
[27]石振国.用JSP实现对Web数据库的访问.计算机应用,2001,5
[28]王昌达.基于WWW的远程辅导答疑系统的设计与实现.计算机应用,2001,6
[29]张玉连,郑小慎.智能计算机辅助教学评测系统算法分析.微计算机应用,2002,3