海量食品安全事件下的命名实体识别研究
|
摘要
对食品安全事件当中的实体进行分析和识别不仅有助于人们加深对食品安全事件的了解而且有利于管理者应对食品安全事件。基于食品安全事件语料库,通过系统地统计和分析人名和机构名的内部与外部特征,在制定的特征模板的基础上,基于条件随机场模型,本文完成了对机构名和人名这两类命名实体进行识别的任务。通过与最大熵模型的测试结果进行比较,实验表明条件随机场模型的整体性能比较突出,取得了较好的准确率和召回率,并说明基于条件随机场模型完全可以实现对食品安全事件文本当中实体的抽取。
It is not only helpful for people to get a deeper understanding of food safety incident but also beneficial for managers to deal with food safety incidents and analyze and identify the named entity of food safety incidents. Based on food safety incident corpus and by counting and analyzing the internal and external characteristics of the entities of organization name and the person names,the identification task of the named entities of organization name and the person names in food safety incidents using conditional random field model is completed in formulating characteristics template. The overall performance of the conditional random field model is outstanding in the experimental results comparing with the test results of maximum entropy model,and the accuracy and recall rate is very well. The experiment states the entities extraction of food safety incident is completely feasible based on conditional random field model.
It is not only helpful for people to get a deeper understanding of food safety incident but also beneficial for managers to deal with food safety incidents and analyze and identify the named entity of food safety incidents. Based on food safety incident corpus and by counting and analyzing the internal and external characteristics of the entities of organization name and the person names,the identification task of the named entities of organization name and the person names in food safety incidents using conditional random field model is completed in formulating characteristics template. The overall performance of the conditional random field model is outstanding in the experimental results comparing with the test results of maximum entropy model,and the accuracy and recall rate is very well. The experiment states the entities extraction of food safety incident is completely feasible based on conditional random field model.
引文
[1]张小衡,王玲玲.中文机构名称的识别与分析[J].中文信息学报,1997,11(4):21-32.Zhang Xiaoheng,Wang Lingling.Identification and analysis of Chinese organization and institution names[J].Journal of Chinese Information Processing,1997,11(4):21-32.
[2]Zhang Y,Zhou JF.A trainable method for extracting Chinese entity names and their relations[C].In:Proceedings of the2nd Chinese Language Processing Workshop,Hong Kong,2000:66-76.
[3]Bikel DM,Schwartz R,Weischedel RM.An algorithm that learns what’s in a name[J].Machine Learning Journal Special Issue on Natural Language Learning,1999,34(3):211-231.
[4]Rationv L,Roth D.Design challenges and misconceptions in named entity recognition[C].In:Proceedings of the 13th Conference on Computational Natural Language Learning,2009:147-155.
[5]余祖波,高庆狮,方淼.中文姓名自动识别系统的设计与实现[J].计算机工程与应用,2006,13(10):5-7.Yu Zubo,Gao Qingshi,Fang Miao.Design and realization of Chinese persons name automatic recognition system[J].Computer Engineering and Applications,2006,13(10):5-7.
[6]黄德根,杨元生,王省,张艳丽,钟万勰.基于统计方法的中文姓名识别[J].中文信息学报,2001,15(2):31-44.Huang Degen,Yang Yuansheng,Wang Xing,Zhang Yanli,Zhong Wanxie.Identification of Chinese names based on statistics[J].Journal of Chinese Information Processing,2001,15(2):31-44.
[7]张华平,刘群.基于角色标注的中国人名自动识别研究[J].计算机学报,2004,27(1):85-91.Zhang Huaping,Liu Qun.Automatic recognition of Chinese personal name based on role tagging[J].Chinese Journal of Computers,2004,27(1):85-91.
[8]孙镇,王惠临.命名实体识别研究进展综述[J].现代图书情报技术,2010,5(6):42-47.Sun Zhen,Wang Huilin.Overview on the advance of the research on named entity recognition[J].New Technology of Library and Information Service,2010,5(6):42-47.
[9]唐钊.条件随机场模型在中文人名识别中的研究与实现[J].现代计算机,2012,7(21):3-7.Tang Zhao.Research and implementation of conditional random field model in Chinese personal name recognition[J].Modern Computer,2012,7(21):3-7.
[10]郭家清,蔡东风,王智超等.一种基于条件随机场的人名识别方法[J].通讯和计算机,2007,11(2):22-25.Guo Jiaqing,Cai Dongfeng,Wang Zhichao,etc.One method of identifying name based on conditional random field model[J].Journal of Communication and Computer,2007,11(2):22-25.
[11]李双龙,刘群,王成耀.基于条件随机场的汉语分词系统[J].微计算机信息,2006,10(1):178-180.Li Shuanglong,Liu Qun,Wang Chengyao.CRF-based Chinese word segmentation research[J].Micro Computer Information,2006,10(1):178-180.
[12]陈晴.基于条件随机场的自动分词技术的研究[D].沈阳:东北大学硕士论文,2004.Chen Qing.Sutdy of automatic segmentation technique based on conditional random fields[D].Sheng Yang:Norteastern University,2004.
[13]洪铭材,张阔,唐杰.基于条件随机场(CRFs)的中文词性标注方法[J].计算机科学,2006,14(10):148-155.Hong Mingcai,Zhang Kuo,Tang Jie.A Chinese part-ofspeech tagging approach using conditional random fields[J].Computer Science,2006,14(10):148-155.
[14]廖先桃.中文命名实体识别方法研究[D].哈尔滨:哈尔滨工业大学,2007.Liao Xiantao.Research on Chinese named entity recognition[D].Harbin:Harbin Institute of Technology,2007.
[15]周俊生,戴新宇,尹存燕,陈家骏.基于层叠条件随机场模型的中文机构名自动识别[J].电子学报,2006,34(5):804-809.Zhou Junsheng,Dai Xinyu,Yin Cunyan,Chen Jiajun.Automatic recognition of Chinese organization name based on cascaded conditional random fields[J].Acta Electronica Sinica,2006,11(5):804-809.
[16]章成志.基于多层术语度的一体化术语抽取研究[J].情报学报,2011,30(3):275-285.Zhang Chengzhi.Using integration strategy and multi-level termhood to extract terminology[J].Journal of The China Society for Scientific and Technical Information,2011,30(3):275-285.
[17]许勇,宋柔.基于半CRF模型的百科全书文本段落划分[J].北京工业大学学报,2008,34(2):204-210.Xu Yong,Song Rou.A semi-markov CRF model approach to encyclopedia text topic segmentation[J].Journal of Beijing University of Technology,2008,34(2):204-210.
[2]Zhang Y,Zhou JF.A trainable method for extracting Chinese entity names and their relations[C].In:Proceedings of the2nd Chinese Language Processing Workshop,Hong Kong,2000:66-76.
[3]Bikel DM,Schwartz R,Weischedel RM.An algorithm that learns what’s in a name[J].Machine Learning Journal Special Issue on Natural Language Learning,1999,34(3):211-231.
[4]Rationv L,Roth D.Design challenges and misconceptions in named entity recognition[C].In:Proceedings of the 13th Conference on Computational Natural Language Learning,2009:147-155.
[5]余祖波,高庆狮,方淼.中文姓名自动识别系统的设计与实现[J].计算机工程与应用,2006,13(10):5-7.Yu Zubo,Gao Qingshi,Fang Miao.Design and realization of Chinese persons name automatic recognition system[J].Computer Engineering and Applications,2006,13(10):5-7.
[6]黄德根,杨元生,王省,张艳丽,钟万勰.基于统计方法的中文姓名识别[J].中文信息学报,2001,15(2):31-44.Huang Degen,Yang Yuansheng,Wang Xing,Zhang Yanli,Zhong Wanxie.Identification of Chinese names based on statistics[J].Journal of Chinese Information Processing,2001,15(2):31-44.
[7]张华平,刘群.基于角色标注的中国人名自动识别研究[J].计算机学报,2004,27(1):85-91.Zhang Huaping,Liu Qun.Automatic recognition of Chinese personal name based on role tagging[J].Chinese Journal of Computers,2004,27(1):85-91.
[8]孙镇,王惠临.命名实体识别研究进展综述[J].现代图书情报技术,2010,5(6):42-47.Sun Zhen,Wang Huilin.Overview on the advance of the research on named entity recognition[J].New Technology of Library and Information Service,2010,5(6):42-47.
[9]唐钊.条件随机场模型在中文人名识别中的研究与实现[J].现代计算机,2012,7(21):3-7.Tang Zhao.Research and implementation of conditional random field model in Chinese personal name recognition[J].Modern Computer,2012,7(21):3-7.
[10]郭家清,蔡东风,王智超等.一种基于条件随机场的人名识别方法[J].通讯和计算机,2007,11(2):22-25.Guo Jiaqing,Cai Dongfeng,Wang Zhichao,etc.One method of identifying name based on conditional random field model[J].Journal of Communication and Computer,2007,11(2):22-25.
[11]李双龙,刘群,王成耀.基于条件随机场的汉语分词系统[J].微计算机信息,2006,10(1):178-180.Li Shuanglong,Liu Qun,Wang Chengyao.CRF-based Chinese word segmentation research[J].Micro Computer Information,2006,10(1):178-180.
[12]陈晴.基于条件随机场的自动分词技术的研究[D].沈阳:东北大学硕士论文,2004.Chen Qing.Sutdy of automatic segmentation technique based on conditional random fields[D].Sheng Yang:Norteastern University,2004.
[13]洪铭材,张阔,唐杰.基于条件随机场(CRFs)的中文词性标注方法[J].计算机科学,2006,14(10):148-155.Hong Mingcai,Zhang Kuo,Tang Jie.A Chinese part-ofspeech tagging approach using conditional random fields[J].Computer Science,2006,14(10):148-155.
[14]廖先桃.中文命名实体识别方法研究[D].哈尔滨:哈尔滨工业大学,2007.Liao Xiantao.Research on Chinese named entity recognition[D].Harbin:Harbin Institute of Technology,2007.
[15]周俊生,戴新宇,尹存燕,陈家骏.基于层叠条件随机场模型的中文机构名自动识别[J].电子学报,2006,34(5):804-809.Zhou Junsheng,Dai Xinyu,Yin Cunyan,Chen Jiajun.Automatic recognition of Chinese organization name based on cascaded conditional random fields[J].Acta Electronica Sinica,2006,11(5):804-809.
[16]章成志.基于多层术语度的一体化术语抽取研究[J].情报学报,2011,30(3):275-285.Zhang Chengzhi.Using integration strategy and multi-level termhood to extract terminology[J].Journal of The China Society for Scientific and Technical Information,2011,30(3):275-285.
[17]许勇,宋柔.基于半CRF模型的百科全书文本段落划分[J].北京工业大学学报,2008,34(2):204-210.Xu Yong,Song Rou.A semi-markov CRF model approach to encyclopedia text topic segmentation[J].Journal of Beijing University of Technology,2008,34(2):204-210.