汉语作文自动评价及其关键技术——来自作文自动评价(AEE)的经验
|
摘要
作文自动评价(AEE)是作文教学和考试评分领域中的一项新技术,能有效减轻人工评阅作文的负担,并提升考试评价的客观性、准确性和一致性。AEE的发展可划分为起步、快速发展和广泛应用三个阶段,近年来在应用领域和深度上不断拓展,从统计加回归逐渐过渡到结合NLP等人工智能技术。然而,AEE当前也面临了有效性验证、语义特征的提取和系统的评价方法等技术瓶颈和挑战。借鉴AEE的经验,汉语作文自动评价主要从修辞、词汇、主旨等方面进行特征选择和抽取,评价模型构建方法也有所创新。未来,汉语作文自动评价研究与开发可从中文信息处理、作文语义理解和深度学习三个方面加以改进。
Automated Essay Evaluation(AEE)is a new technology in the field of essay teaching and test scoring. It can effectively alleviate the burden of manual correction of essay and improve the objectivity,accuracy and consistency of test evaluation. The development of AEE can be divided into three stages: start-up,rapid development and wide application. In recent years,it has been expanding in application fields and depths,and gradually transitioning from statistical plus regression to combining artificial intelligence technologies such as NLP.However,AEE currently faces technical bottlenecks and challenges such as validity verification,semantic feature extraction,and systematic evaluation methods. Drawing on the experience of AEE,the automated Chinese essay evaluation mainly selects and extracts features from the aspects of rhetoric,vocabulary and subject. The evaluation model construction method is also innovative. In the future,the research and development of automated Chinese essay evaluation can be improved from three aspects: Chinese information processing,essay semantic understanding and deep learning.
Automated Essay Evaluation(AEE)is a new technology in the field of essay teaching and test scoring. It can effectively alleviate the burden of manual correction of essay and improve the objectivity,accuracy and consistency of test evaluation. The development of AEE can be divided into three stages: start-up,rapid development and wide application. In recent years,it has been expanding in application fields and depths,and gradually transitioning from statistical plus regression to combining artificial intelligence technologies such as NLP.However,AEE currently faces technical bottlenecks and challenges such as validity verification,semantic feature extraction,and systematic evaluation methods. Drawing on the experience of AEE,the automated Chinese essay evaluation mainly selects and extracts features from the aspects of rhetoric,vocabulary and subject. The evaluation model construction method is also innovative. In the future,the research and development of automated Chinese essay evaluation can be improved from three aspects: Chinese information processing,essay semantic understanding and deep learning.
引文
[1]Shermis M D,Burstein J.Introduction[C]//In Shermis M D&Burstein J(Eds.),Automated Essay Scoring:a CrossDisciplinary Perspective.Mahwah,NJ:LawrenceErlbaum Associates,2003:14-17.
[2]Johnson L B Broad.What We Really Value:Beyond Rubrics in Teaching and Assessing Writing[J].Assessing Writing,2004,9(2):181-185.
[3]Bejar I I,Flor M,Futagi Y,et al. On the Vulnerability of Automated Scoring to Construct-Irrelevant Response Strategies(CIRS):an Illustration[J]. Assessing Writing,2014,22(1):48-59.
[4]Condon W.Large-Scale Assessment,Locally-Developed Measures,and Automated Scoring of Essays:Fishing for Red Herrings?[J].Assessing Writing,2013,18(1):100-108.
[5]Ericsson P F,Haswell R H.Machine Scoring of Student Essays:Truth and Consequences[M].Logan City:Utah State University Press,2006.
[6]Page E B.The Imminence of Grading Essays by Computer[J].Phi Delta Kappan,1966,47(5):238-243.
[7]Valenti S,Neri F,Cucchiarelli A. An Overview of Current Research on Automated Essay Grading[J].Journal of Information Technology Education Research,2003,2(1):115-130.
[8]梁茂成.中国学生英语作文自动评分模型的构建[D].南京:南京大学,2005.
[9]葛诗利,陈潇潇.文本聚类在大学英语作文自动评分中应用[J].计算机工程与应用,2009,45(6):145-148.
[10]李霞,刘建达.适用于中国外语学习者的英文作文全自动集成评分算法[J].中文信息学报,2013,27(5):100-106.
[11]陈慰.基于自动评分系统的大学英语作文反馈研究——以“句酷批改网”为例[D].重庆:重庆大学,2013.
[12]冯胜利,王洁,黄梅.汉语书面语体庄雅度的自动测量[J].语言科学,2008,7(2):113-126.
[13]陈东.基于文本特征提取的汉语L2作文电子评分系统初探[D].南京:南京大学,2013.
[14]黄志娥,谢佳莉,荀恩东.HSK自动作文评分的特征选取研究[J].计算机工程与应用,2014,50(6):118-122.
[15]徐昌火,陈东,吴倩,等.汉语作为第二语言作文自动评分研究初探[J].国际汉语教学研究,2015,1(1):83-89.
[16]Grimes D,Warschauer M.Utility in a Fallible Tool:a Multi-Site Case Study of Automated Writing Evaluation[J].Journal of Technology Learning&Assessment,2010,8(6):44-56.
[17]Attali Y,Burstein J.Automated Essay Scoring With E-Rater V.2[J].The Journal of Technology,Learning and Assessment,2006,4(3):3-29.
[18]Attali Y. Validity and Reliability of Automated Essay Scoring[C]//In Handbook of Automated Essay Evaluation:Current Applications and New Directions. Rutledge:Taylor&Francis Group,2013:336-369.
[19]Hao S,Xu Y,Peng H,et al.Automated Chinese Essay Scoring from Topic Perspective Using Regularized Latent Semantic Indexing[C]. International Conference on Pattern Recognition,2014:3092-3097.
[20]Kaggle.The Hewlett Foundation:Automated Essay Scoring[EB/OL].[2019-03-20].https://www.kaggle.com/c/ASAP-AES,2018.
[21]Kaja Zupanc,Zoran Bosnic. Advances in the Field of Automated Essay Evaluation[J]. Informatica,2015,39(1):383-395.
[22]Shermis M D,Hamner B.Contrasting State-of-theArt Automated Scoring of Essays[C]//In Handbook of Automated Essay Evaluation:Current Applications and New Directions.Rutledge:Taylor&Francis Group,2013:336-369.
[23]刘挺,马金山.汉语自动句法分析的理论与方法[J].当代语言学,2009,2(1):100-112.
[24]许嘉璐.现状和设想———试论中文信息处理与现代汉语研究[J].中文信息学报,2001,2(1):1-8.
[25]Chang T,Lee C,Chang Y,et al. Enhancing Automatic Chinese Essay Scoring System from Figures-of-Speech[C].Pacific Asia Conference on Language Information and Computation,2006:28-34.
[26]梁社会,陈小荷,刘浏.先秦汉语排比句自动识别研究———以《孟子》《论语》中的排比句自动识别为例[J].计算机工程与应用,2013,49(19):222-226.
[27]刘明杨.高考作文自动评分关键技术研究[D].哈尔滨:哈尔滨工业大学,2015.
[28]巩捷甫.面向语文作文自动评阅的修辞手法识别系统的设计与实现[D].哈尔滨:哈尔滨工业大学,2016.
[29]陈一乐.基于回归分析的中文作文自动评分技术研究[D].哈尔滨:哈尔滨工业大学,2016.
[30]蔡黎,彭星源,赵军.少数民族汉语考试的作文辅助评分系统研究[J].中文信息学报,2011,25(5):120-127.
[31]彭星源,柯登峰,赵知,等.基于词汇评分的汉语作文自动评分[J].中文信息学报,2012,26(2):102-109.
[32]胡韧奋.汉语词汇测试自动命题研究[J].中文信息学报,2017,31(1):41-49.
[33]Wang Q,Xu J,Li H,et al.Regularized Latent Semantic Indexing:a New Approach to Large-Scale Topic Modeling[J].Acm Transactions on Information Systems,2013,31(1):1-44.
[34]Chang T H,Tsai P Y,Lee C H,et al.Automated Essay Scoring Using Set of Literary Sememes[C].IEEE International Conference on Natural Language Processing and Knowledge Engineering,2008,12(2):1-5.
[35]Chang T H,Lee C H.Automatic Chinese Essay Scoring Using Connections between Concepts in Paragraphs[C].International Conference on Asian Language Processing,IEEE Computer Society,2009:265-268.
[36]王耀华,李舟军,何跃鹰,等.基于文本语义离散度的自动作文评分关键技术研究[J].中文信息学报,2016,30(6):173-181.
[37]曹亦薇,杨晨.使用潜语义分析的汉语作文自动评分研究[J].考试研究,2007,1(2):65-73.
[38]Xu Y,Ke D,Su K.Contextualized Latent Semantic Indexing:a New Approach to Automated Chinese Essay Scoring[J].Journal of Intelligent Systems,2017,26(2):263-285.
[2]Johnson L B Broad.What We Really Value:Beyond Rubrics in Teaching and Assessing Writing[J].Assessing Writing,2004,9(2):181-185.
[3]Bejar I I,Flor M,Futagi Y,et al. On the Vulnerability of Automated Scoring to Construct-Irrelevant Response Strategies(CIRS):an Illustration[J]. Assessing Writing,2014,22(1):48-59.
[4]Condon W.Large-Scale Assessment,Locally-Developed Measures,and Automated Scoring of Essays:Fishing for Red Herrings?[J].Assessing Writing,2013,18(1):100-108.
[5]Ericsson P F,Haswell R H.Machine Scoring of Student Essays:Truth and Consequences[M].Logan City:Utah State University Press,2006.
[6]Page E B.The Imminence of Grading Essays by Computer[J].Phi Delta Kappan,1966,47(5):238-243.
[7]Valenti S,Neri F,Cucchiarelli A. An Overview of Current Research on Automated Essay Grading[J].Journal of Information Technology Education Research,2003,2(1):115-130.
[8]梁茂成.中国学生英语作文自动评分模型的构建[D].南京:南京大学,2005.
[9]葛诗利,陈潇潇.文本聚类在大学英语作文自动评分中应用[J].计算机工程与应用,2009,45(6):145-148.
[10]李霞,刘建达.适用于中国外语学习者的英文作文全自动集成评分算法[J].中文信息学报,2013,27(5):100-106.
[11]陈慰.基于自动评分系统的大学英语作文反馈研究——以“句酷批改网”为例[D].重庆:重庆大学,2013.
[12]冯胜利,王洁,黄梅.汉语书面语体庄雅度的自动测量[J].语言科学,2008,7(2):113-126.
[13]陈东.基于文本特征提取的汉语L2作文电子评分系统初探[D].南京:南京大学,2013.
[14]黄志娥,谢佳莉,荀恩东.HSK自动作文评分的特征选取研究[J].计算机工程与应用,2014,50(6):118-122.
[15]徐昌火,陈东,吴倩,等.汉语作为第二语言作文自动评分研究初探[J].国际汉语教学研究,2015,1(1):83-89.
[16]Grimes D,Warschauer M.Utility in a Fallible Tool:a Multi-Site Case Study of Automated Writing Evaluation[J].Journal of Technology Learning&Assessment,2010,8(6):44-56.
[17]Attali Y,Burstein J.Automated Essay Scoring With E-Rater V.2[J].The Journal of Technology,Learning and Assessment,2006,4(3):3-29.
[18]Attali Y. Validity and Reliability of Automated Essay Scoring[C]//In Handbook of Automated Essay Evaluation:Current Applications and New Directions. Rutledge:Taylor&Francis Group,2013:336-369.
[19]Hao S,Xu Y,Peng H,et al.Automated Chinese Essay Scoring from Topic Perspective Using Regularized Latent Semantic Indexing[C]. International Conference on Pattern Recognition,2014:3092-3097.
[20]Kaggle.The Hewlett Foundation:Automated Essay Scoring[EB/OL].[2019-03-20].https://www.kaggle.com/c/ASAP-AES,2018.
[21]Kaja Zupanc,Zoran Bosnic. Advances in the Field of Automated Essay Evaluation[J]. Informatica,2015,39(1):383-395.
[22]Shermis M D,Hamner B.Contrasting State-of-theArt Automated Scoring of Essays[C]//In Handbook of Automated Essay Evaluation:Current Applications and New Directions.Rutledge:Taylor&Francis Group,2013:336-369.
[23]刘挺,马金山.汉语自动句法分析的理论与方法[J].当代语言学,2009,2(1):100-112.
[24]许嘉璐.现状和设想———试论中文信息处理与现代汉语研究[J].中文信息学报,2001,2(1):1-8.
[25]Chang T,Lee C,Chang Y,et al. Enhancing Automatic Chinese Essay Scoring System from Figures-of-Speech[C].Pacific Asia Conference on Language Information and Computation,2006:28-34.
[26]梁社会,陈小荷,刘浏.先秦汉语排比句自动识别研究———以《孟子》《论语》中的排比句自动识别为例[J].计算机工程与应用,2013,49(19):222-226.
[27]刘明杨.高考作文自动评分关键技术研究[D].哈尔滨:哈尔滨工业大学,2015.
[28]巩捷甫.面向语文作文自动评阅的修辞手法识别系统的设计与实现[D].哈尔滨:哈尔滨工业大学,2016.
[29]陈一乐.基于回归分析的中文作文自动评分技术研究[D].哈尔滨:哈尔滨工业大学,2016.
[30]蔡黎,彭星源,赵军.少数民族汉语考试的作文辅助评分系统研究[J].中文信息学报,2011,25(5):120-127.
[31]彭星源,柯登峰,赵知,等.基于词汇评分的汉语作文自动评分[J].中文信息学报,2012,26(2):102-109.
[32]胡韧奋.汉语词汇测试自动命题研究[J].中文信息学报,2017,31(1):41-49.
[33]Wang Q,Xu J,Li H,et al.Regularized Latent Semantic Indexing:a New Approach to Large-Scale Topic Modeling[J].Acm Transactions on Information Systems,2013,31(1):1-44.
[34]Chang T H,Tsai P Y,Lee C H,et al.Automated Essay Scoring Using Set of Literary Sememes[C].IEEE International Conference on Natural Language Processing and Knowledge Engineering,2008,12(2):1-5.
[35]Chang T H,Lee C H.Automatic Chinese Essay Scoring Using Connections between Concepts in Paragraphs[C].International Conference on Asian Language Processing,IEEE Computer Society,2009:265-268.
[36]王耀华,李舟军,何跃鹰,等.基于文本语义离散度的自动作文评分关键技术研究[J].中文信息学报,2016,30(6):173-181.
[37]曹亦薇,杨晨.使用潜语义分析的汉语作文自动评分研究[J].考试研究,2007,1(2):65-73.
[38]Xu Y,Ke D,Su K.Contextualized Latent Semantic Indexing:a New Approach to Automated Chinese Essay Scoring[J].Journal of Intelligent Systems,2017,26(2):263-285.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |