完形填空的效度研究
|
摘要
本文旨在通过完形填空的效度研究来论证完形填空是一种高效度的和经济省力的检测综合能力的测试手段。
语言测试的历史大致分为三个阶段，前科学阶段(-至20世纪40年代)，心理测量 与结构主义阶段(20世纪50至60年代)，综合与社会语言学时期(20世纪70年代到至今)。前科学时期的特点是测试方法主要为翻译，评分方法较主观。它的优点主要是试题的设计简单易行，而缺点主要是考试结果并不能完全反映学生的水平，评分耗时，其他的如听、说等语言技能不能检测出来。心理测量与结构主义时期主要的特点是多项选择题这种离散性题型开始使用，评分方法以客观性为主。它的优点在于评分较为简单。缺点就是试题的设计难度较高。综合与社会语言学时期的主要特点是整体性测试备受推崇，测试的重点转向测试交际水平。人们较喜欢以任务为主的考试。这个时期主要的测试方法以完形填空、口试、听写和作文等为主。
综合性测试往往把各种语言技能混合在一起测试，比如完形填空就把语言的、语篇的、社会语言学的和语言策略能力放在一起测试，从而能较准确地测出考生运用语言的综合能力。其次，综合性测试十分强调语境在语言使用中的作用，因此测试结果比分离式测试更接近于考生实际运用语言的能力。第三，综合性测试在试题编写上比分离式测试更容易，评分上比传统的写作题较客观。它的不足主要是尽管综合性测试很强调语境，但仍不能视为实际运用语言的真实语境。它仍然是一种间接测试，不能完全测出学生的语言表达能力。
从以上对语言测试历史的回顾中，我们可以了解完形填空是随着综合社会语言学时期的理论发展而发展起来的。完形填空的理论基础主要有以4点：1．社会语言学的理论对完形填空的影响。社会语言学提出了语境的概念，而完形填空的完成十分依赖于语境的帮助。2．完形心理学。这主要指人们在看到不完整的物体时都有一种将缺口补全使图形完整的趋向。把一个完整的文章删去其中的一些词，读者将删去的词补全也是一种完形。这种图形的补全实质上是一样的，还可以检测人的整体认知能力。3．冗余信息。自然语言有冗余度，一篇文章即使缺失几个词，读者也可以通过语境来推断该文的意义。4．语用预期语法。语用预期语法是一个具有心理现实性的系统，包括全部与语言运用有关的认知与语言成分。这一系统是不可分的。在语言测试中应该采用综合试题或语用试题来考察这一不可分语言能力。
完形填空又叫完形程序。这一测试方法1953年由威尔逊．泰勒创设，用来测试以英语为本族语的人所阅读的文章的难度。此后，拉德尔等人多次在研究中发现完形填空是测试阅读能力的一种灵敏度很高的测试手段。他们的研究表明，完形测试和多项选择题的阅读理 解题考试有很高的相关。斯温的研究也表明完形测试和语言成绩考试有很高的相关。因此完形测试也是测试学生外语总体能力的一种手段。完形填空的研究也表明，它能测出离散
题不能测出的一些语言能力。许多研究者也相信完形填空是一种检测外语语言能力的一种
既有效又经济的手段。尽管完形填空据信是一种综合考试，但仍有一些问题值得探讨。其
中主要问题集中在完形填空到底检测的是什么能力？完形填空是否有效？怎样才能设计
有效的完形试卷？它对我们的外语教学中有哪些启示？本文将详细论述这些问题。
我们都知道一份试题有效与否意味着这个考试是否考出了它想考出的能力。如果考试
对它所想测的东西无效，那么考试的分数对它的原始目的来说毫无意义。进行效度研究可
以证明一个考试有效与否。一般来说，效度分为内在效度和外在效度两种。内在效度包括
卷面效度（卷面看上去是否像一份试卷），内容效度（内容是否代表所要测量的内容），回
应效度（学生的习惯与思维反映）。外在效度包括同期效度（与同时期的有效试卷比较看
其相关系数），预测效度（看对未来学生能力是否具有预测作用）和构卷效度（试卷是否
符合考试目的）。
卷面效度、内容效度和同期效度是本效度研究中主要研究的内容。因此，本效度研究的
内容包括：
1．自设完形填空的卷面和内容的效度分析。
2．通过对比自设完形填空与托福卷的相关系数看完形填空的同期效度。
3．本效度研究结果是否与已有的大学英语CET－4、TIM－4及国外的完形填空效度研究结
果相呼应。
本研究的步骤为：
1．设计选定考试篇章内容．
2．以每隔5词进厅删词并进行预测。
3．进行托福和雅思考试。
4．分析cloze考试本身内容分布及其他数据。
本研究分析结果为：
1．完形填空本身能检测出各种语言能力。如语言学的、社会语言学的和语篇的语言策略
能力。
2．同期效度的高度相关证明完形填空可以检测出TOEFL所要检测的能力。因此一份耗时
设计TOEFL题平时只
The thesis is intended to prove that cloze test is a highly valid and economical way to test integrative ability by making a validity research on cloze test.
In language testing history, there always believed to be three periods: pre-scientific, psychometric-structuralist, integrative-sociolinguistics periods. In pre-scientific period, the main characteristic is that the testing method is translation. It's easy to design and the scoring of the test is subjective. The disadvantages are that the results cannot scientifically reflect the students' language level and other abilities, such as listening and speaking skills cannot be tested. The main characteristics in psychometric-structuralist period are that multiple-choice items are used and scoring is objective. The results are said to reflect the students' level and the scoring of the paper is fairly easy. In integrative-sociolinguistics period, the characteristics are that holistic tests are favored. The emphasis of testing is shifted to communication and task-based tests are favored. Cloze test, oral interview and dictation are used widely.
We have known that the cloze test develops with the integrative-sociolinguistics period in the language testing history. The theoretical basis of cloze test is believed to link with: l.The sociolinguistics, which developed the idea of context, influenced cloze test much. 2. Gestalt Psychology. The notion is that people have the ability of closure, i.e. the ability to complete an incomplete pattern. The nature is the same as the process of reconstructing the passage that misses some words. 3. Redundancy Information. The redundancy information provided by natural language makes it possible to restore the missing words hi order to reconstruct textual coherence. 4. Pragmatic Expectancy Grammar. Oiler (1979) believed that it's the base of people's language ability that is internalized in people's mind. It can help people express and communicate with each other. The trait is independent of any particular test, and therefore any classical cloze test is expected to set the mental process enabling closure to take place into action. The behavior exhibited by examinees, their ability to replace the missing elements as expressed by then- scores on the test, permits conclusions to be drawn about the quality or efficiency of this internalized grammar.
The cloze test originated in the 1950s as a means of assessing the difficulty of a reading test for native speakers. Other studies soon indicated that cloze might also be a valid way of measuring native-speaker reading ability. The validity coefficient in these studies ranged from .61 to .95 between cloze tests and standardized reading examinations (Brown, J.D. 1983:238). It then was suggested that it could be used
for assessing learners' progress of second and foreign language. Many researchers believe that cloze test is a valid and economical way to test foreign language ability.
Although cloze test is believed to be an integrative test, the following questions should be understood clearly. For instance, what abilities on earth can be measured by cloze test? Is cloze test valid? How can we design a valid cloze test? What implications can we discover for our foreign language teaching?
If the test is not valid for it wants to test, then the scores of the test will mean nothing for what its original purpose. Validity includes internal and external types. Internal validity includes face, content and response validity. External validity includes concurrent, predictive and construct validity. Validity research on test items will prove whether a test is valid or not. Therefore, the thesis aims to prove cloze test is a valid and economical way for testing global ability by making the validity research on cloze test. The content of doing the research is: 1. To see face, content validity of the self-designed cloze test. 2. Check the concurrent validity by comparing the con-elation coefficient of self-designed cloze test and TOEFL and composition. 3.Quoting the other validity research
语言测试的历史大致分为三个阶段，前科学阶段(-至20世纪40年代)，心理测量 与结构主义阶段(20世纪50至60年代)，综合与社会语言学时期(20世纪70年代到至今)。前科学时期的特点是测试方法主要为翻译，评分方法较主观。它的优点主要是试题的设计简单易行，而缺点主要是考试结果并不能完全反映学生的水平，评分耗时，其他的如听、说等语言技能不能检测出来。心理测量与结构主义时期主要的特点是多项选择题这种离散性题型开始使用，评分方法以客观性为主。它的优点在于评分较为简单。缺点就是试题的设计难度较高。综合与社会语言学时期的主要特点是整体性测试备受推崇，测试的重点转向测试交际水平。人们较喜欢以任务为主的考试。这个时期主要的测试方法以完形填空、口试、听写和作文等为主。
综合性测试往往把各种语言技能混合在一起测试，比如完形填空就把语言的、语篇的、社会语言学的和语言策略能力放在一起测试，从而能较准确地测出考生运用语言的综合能力。其次，综合性测试十分强调语境在语言使用中的作用，因此测试结果比分离式测试更接近于考生实际运用语言的能力。第三，综合性测试在试题编写上比分离式测试更容易，评分上比传统的写作题较客观。它的不足主要是尽管综合性测试很强调语境，但仍不能视为实际运用语言的真实语境。它仍然是一种间接测试，不能完全测出学生的语言表达能力。
从以上对语言测试历史的回顾中，我们可以了解完形填空是随着综合社会语言学时期的理论发展而发展起来的。完形填空的理论基础主要有以4点：1．社会语言学的理论对完形填空的影响。社会语言学提出了语境的概念，而完形填空的完成十分依赖于语境的帮助。2．完形心理学。这主要指人们在看到不完整的物体时都有一种将缺口补全使图形完整的趋向。把一个完整的文章删去其中的一些词，读者将删去的词补全也是一种完形。这种图形的补全实质上是一样的，还可以检测人的整体认知能力。3．冗余信息。自然语言有冗余度，一篇文章即使缺失几个词，读者也可以通过语境来推断该文的意义。4．语用预期语法。语用预期语法是一个具有心理现实性的系统，包括全部与语言运用有关的认知与语言成分。这一系统是不可分的。在语言测试中应该采用综合试题或语用试题来考察这一不可分语言能力。
完形填空又叫完形程序。这一测试方法1953年由威尔逊．泰勒创设，用来测试以英语为本族语的人所阅读的文章的难度。此后，拉德尔等人多次在研究中发现完形填空是测试阅读能力的一种灵敏度很高的测试手段。他们的研究表明，完形测试和多项选择题的阅读理 解题考试有很高的相关。斯温的研究也表明完形测试和语言成绩考试有很高的相关。因此完形测试也是测试学生外语总体能力的一种手段。完形填空的研究也表明，它能测出离散
题不能测出的一些语言能力。许多研究者也相信完形填空是一种检测外语语言能力的一种
既有效又经济的手段。尽管完形填空据信是一种综合考试，但仍有一些问题值得探讨。其
中主要问题集中在完形填空到底检测的是什么能力？完形填空是否有效？怎样才能设计
有效的完形试卷？它对我们的外语教学中有哪些启示？本文将详细论述这些问题。
我们都知道一份试题有效与否意味着这个考试是否考出了它想考出的能力。如果考试
对它所想测的东西无效，那么考试的分数对它的原始目的来说毫无意义。进行效度研究可
以证明一个考试有效与否。一般来说，效度分为内在效度和外在效度两种。内在效度包括
卷面效度（卷面看上去是否像一份试卷），内容效度（内容是否代表所要测量的内容），回
应效度（学生的习惯与思维反映）。外在效度包括同期效度（与同时期的有效试卷比较看
其相关系数），预测效度（看对未来学生能力是否具有预测作用）和构卷效度（试卷是否
符合考试目的）。
卷面效度、内容效度和同期效度是本效度研究中主要研究的内容。因此，本效度研究的
内容包括：
1．自设完形填空的卷面和内容的效度分析。
2．通过对比自设完形填空与托福卷的相关系数看完形填空的同期效度。
3．本效度研究结果是否与已有的大学英语CET－4、TIM－4及国外的完形填空效度研究结
果相呼应。
本研究的步骤为：
1．设计选定考试篇章内容．
2．以每隔5词进厅删词并进行预测。
3．进行托福和雅思考试。
4．分析cloze考试本身内容分布及其他数据。
本研究分析结果为：
1．完形填空本身能检测出各种语言能力。如语言学的、社会语言学的和语篇的语言策略
能力。
2．同期效度的高度相关证明完形填空可以检测出TOEFL所要检测的能力。因此一份耗时
设计TOEFL题平时只
The thesis is intended to prove that cloze test is a highly valid and economical way to test integrative ability by making a validity research on cloze test.
In language testing history, there always believed to be three periods: pre-scientific, psychometric-structuralist, integrative-sociolinguistics periods. In pre-scientific period, the main characteristic is that the testing method is translation. It's easy to design and the scoring of the test is subjective. The disadvantages are that the results cannot scientifically reflect the students' language level and other abilities, such as listening and speaking skills cannot be tested. The main characteristics in psychometric-structuralist period are that multiple-choice items are used and scoring is objective. The results are said to reflect the students' level and the scoring of the paper is fairly easy. In integrative-sociolinguistics period, the characteristics are that holistic tests are favored. The emphasis of testing is shifted to communication and task-based tests are favored. Cloze test, oral interview and dictation are used widely.
We have known that the cloze test develops with the integrative-sociolinguistics period in the language testing history. The theoretical basis of cloze test is believed to link with: l.The sociolinguistics, which developed the idea of context, influenced cloze test much. 2. Gestalt Psychology. The notion is that people have the ability of closure, i.e. the ability to complete an incomplete pattern. The nature is the same as the process of reconstructing the passage that misses some words. 3. Redundancy Information. The redundancy information provided by natural language makes it possible to restore the missing words hi order to reconstruct textual coherence. 4. Pragmatic Expectancy Grammar. Oiler (1979) believed that it's the base of people's language ability that is internalized in people's mind. It can help people express and communicate with each other. The trait is independent of any particular test, and therefore any classical cloze test is expected to set the mental process enabling closure to take place into action. The behavior exhibited by examinees, their ability to replace the missing elements as expressed by then- scores on the test, permits conclusions to be drawn about the quality or efficiency of this internalized grammar.
The cloze test originated in the 1950s as a means of assessing the difficulty of a reading test for native speakers. Other studies soon indicated that cloze might also be a valid way of measuring native-speaker reading ability. The validity coefficient in these studies ranged from .61 to .95 between cloze tests and standardized reading examinations (Brown, J.D. 1983:238). It then was suggested that it could be used
for assessing learners' progress of second and foreign language. Many researchers believe that cloze test is a valid and economical way to test foreign language ability.
Although cloze test is believed to be an integrative test, the following questions should be understood clearly. For instance, what abilities on earth can be measured by cloze test? Is cloze test valid? How can we design a valid cloze test? What implications can we discover for our foreign language teaching?
If the test is not valid for it wants to test, then the scores of the test will mean nothing for what its original purpose. Validity includes internal and external types. Internal validity includes face, content and response validity. External validity includes concurrent, predictive and construct validity. Validity research on test items will prove whether a test is valid or not. Therefore, the thesis aims to prove cloze test is a valid and economical way for testing global ability by making the validity research on cloze test. The content of doing the research is: 1. To see face, content validity of the self-designed cloze test. 2. Check the concurrent validity by comparing the con-elation coefficient of self-designed cloze test and TOEFL and composition. 3.Quoting the other validity research
引文
1． Alderson, J. Charles, et, al.1995． Language Test Construction and Evaluation. London: Cambridge University Press
2． Alderson, J. Charles. 1983． " The cloze procedure and proficiency in English as a foreign language". Reprinted in Oiler, J.W. Issues in Language Testing Research. Rouley Mass: Newbury House Publisher, Inc.
3． Backer, D. 1989． Language testing: A Critical Survey and practical Guide. Edward Arnold: A Division of Hodder & Stoughton.
4． Bachman, L.F. 1990, Fundamental Consideration in Language Testing. Oxford: Oxford University Press
5． Brown, H. Douglas. 1994． Principles of Language Learning and Teaching. New Jersey: Prentice Hall Regents Englewood Cliffs.
6． Brown, J.D. 1983． " A closer look at cloze: validity and reliability" Reprinted in Oiler, J.W. Issues in Language Testing Research. Rouley Mass: Newbury House Publisher, Inc.
7． Canale, M. and Swain, M. 1980． Theoretical Basis of communicative Approached to Second Language Teaching and Testing. Applied Linguistics, I: 1-47
8． Clarke, M.A. et, al. 1998 Choice Reading Michigan: The University of Michigan Press reprinted by Beijing World Publishing Corporation
9． Ebel, R.L and D.A. Frisbie. 1991． Essentials of Educational Measurement. Englewood Cliffs, NJ; Prentice-Hall
10． Garbutt，M&O'sullivan，K．1997． 《IELTS考试指南及模拟试题》 广东：广东世界图书出版公司
11． Harmer, J. 1983． The Practice of English Language Teaching Longman: Longman Group Limited
12． Harrison, A. 1983． A Language Testing Handbook. London: Macmillan Press London.
13． Henning, G. 1987． A Guide to Language Testing. Cambridge, Mass: Newbury House.
14． Hughes, A. 1989． Testing for Language Teachers. Cambridge: Cambridge University Press.
15． Heaten, J. B. 1990． Classroom Testing. London: Longman Group UK Limit
16． Ingram, E. 1977． Basic Concepts in Testing. In J.P.B Allen and A. Davies (eds)
17． Kerlinger, F.N. 1973 Foundations of Behavioral Research. New York: Holt, Rinehart and Winston.
18.Klein Braley, C. 1985. "A cloze-up on the C-test: a study in the Construct Validation of Authentic Tests ". Language Testing.2 (1): 76-104
19.Klein Braley, C. 1983. "A cloze is a cloze is a question". Reprinted in Oller, J.W. Issues in Language Testing Research. Rouley Mass: Newbury House Publisher, Inc.
20.Madsen, H. S. 1983. Techniques in Testing. London: Oxford University Press.
21.Oller, J.W. 1979. Language Test at School. London: Longman
22.Oller, J.W. (ed). 1983. Issues in Language Testing Research. Rouley Mass Newbury House Publisher, Inc. pp203-250
23.Shohamy, E. 1983 "Interrater and intrarater reliability of the oral interview and concurrent validity with cloze procedure in Hebrew" Reprinted in Oiler, J.W. Issues in Language Testing Research. Rouley Mass: Newbury House Publisher, Inc.
24.Spolsky, Bernard. 1978. Linguists and language testers. Arlington, VA: Center for Applied Linguistics.
25．《大学英语四级考试大纲及样题》1994上海：上海外语教育出版社
26．《高校英语专业四级考试大纲(修订本)》1997上海：上海外语教育出版社
27．杨惠中&C.Weir,1998．《大学英语四．六级考试效度研究》．上海：上海外语教育出版社
28．刘润清1991．《语言测试和它的方法》北京：外语教学与研究出版社
29．<日>Sandra S.Fotos著 金木编译．“完形填空能作为整体语言能力测试手段码?”，《国外外语教学》1993年第2期
30．何国伟，华维芬，“完形填空中的评分问题”，《现代外语》1995年第2期
31．舒运祥。1999 《外语测试的理论与方法》。上海：世界图书出版公司。
32．邹申1998 《英语语言测试》—理论与操作 上海：上海外语教育出版社
2． Alderson, J. Charles. 1983． " The cloze procedure and proficiency in English as a foreign language". Reprinted in Oiler, J.W. Issues in Language Testing Research. Rouley Mass: Newbury House Publisher, Inc.
3． Backer, D. 1989． Language testing: A Critical Survey and practical Guide. Edward Arnold: A Division of Hodder & Stoughton.
4． Bachman, L.F. 1990, Fundamental Consideration in Language Testing. Oxford: Oxford University Press
5． Brown, H. Douglas. 1994． Principles of Language Learning and Teaching. New Jersey: Prentice Hall Regents Englewood Cliffs.
6． Brown, J.D. 1983． " A closer look at cloze: validity and reliability" Reprinted in Oiler, J.W. Issues in Language Testing Research. Rouley Mass: Newbury House Publisher, Inc.
7． Canale, M. and Swain, M. 1980． Theoretical Basis of communicative Approached to Second Language Teaching and Testing. Applied Linguistics, I: 1-47
8． Clarke, M.A. et, al. 1998 Choice Reading Michigan: The University of Michigan Press reprinted by Beijing World Publishing Corporation
9． Ebel, R.L and D.A. Frisbie. 1991． Essentials of Educational Measurement. Englewood Cliffs, NJ; Prentice-Hall
10． Garbutt，M&O'sullivan，K．1997． 《IELTS考试指南及模拟试题》 广东：广东世界图书出版公司
11． Harmer, J. 1983． The Practice of English Language Teaching Longman: Longman Group Limited
12． Harrison, A. 1983． A Language Testing Handbook. London: Macmillan Press London.
13． Henning, G. 1987． A Guide to Language Testing. Cambridge, Mass: Newbury House.
14． Hughes, A. 1989． Testing for Language Teachers. Cambridge: Cambridge University Press.
15． Heaten, J. B. 1990． Classroom Testing. London: Longman Group UK Limit
16． Ingram, E. 1977． Basic Concepts in Testing. In J.P.B Allen and A. Davies (eds)
17． Kerlinger, F.N. 1973 Foundations of Behavioral Research. New York: Holt, Rinehart and Winston.
18.Klein Braley, C. 1985. "A cloze-up on the C-test: a study in the Construct Validation of Authentic Tests ". Language Testing.2 (1): 76-104
19.Klein Braley, C. 1983. "A cloze is a cloze is a question". Reprinted in Oller, J.W. Issues in Language Testing Research. Rouley Mass: Newbury House Publisher, Inc.
20.Madsen, H. S. 1983. Techniques in Testing. London: Oxford University Press.
21.Oller, J.W. 1979. Language Test at School. London: Longman
22.Oller, J.W. (ed). 1983. Issues in Language Testing Research. Rouley Mass Newbury House Publisher, Inc. pp203-250
23.Shohamy, E. 1983 "Interrater and intrarater reliability of the oral interview and concurrent validity with cloze procedure in Hebrew" Reprinted in Oiler, J.W. Issues in Language Testing Research. Rouley Mass: Newbury House Publisher, Inc.
24.Spolsky, Bernard. 1978. Linguists and language testers. Arlington, VA: Center for Applied Linguistics.
25．《大学英语四级考试大纲及样题》1994上海：上海外语教育出版社
26．《高校英语专业四级考试大纲(修订本)》1997上海：上海外语教育出版社
27．杨惠中&C.Weir,1998．《大学英语四．六级考试效度研究》．上海：上海外语教育出版社
28．刘润清1991．《语言测试和它的方法》北京：外语教学与研究出版社
29．<日>Sandra S.Fotos著 金木编译．“完形填空能作为整体语言能力测试手段码?”，《国外外语教学》1993年第2期
30．何国伟，华维芬，“完形填空中的评分问题”，《现代外语》1995年第2期
31．舒运祥。1999 《外语测试的理论与方法》。上海：世界图书出版公司。
32．邹申1998 《英语语言测试》—理论与操作 上海：上海外语教育出版社