主动学习算法中采样策略研究
|
摘要
当前，文本挖掘、语音识别、生物信息发掘和视觉对象分类等应用领域面临的一个现实问题是：无标注样例数目众多，易于获得；标注样例数量稀少，难于获得。作为机器学习领域的重要研究方向之一，主动学习算法可以同时利用标注样例和无标注样例来构建高精度分类模型，因此，本文对主动学习算法中采样策略进行深入研究，并将所提出采样策略算法应用于视觉对象分类任务中。
视觉对象的语义理解是计算机视觉领域的重要问题。网络技术的快速发展使得短时间内获取大量图像成为可能，但对这些无监督或弱监督图片中包含的视觉对象进行分类成为一个艰巨、富有挑战性的任务。越来越多的科研人员致力于发展有效的机器学习算法，在已标注图像集上建立模型，继而利用学习到的知识判断和划分视觉对象所属的类别。这一做法通常需要大量标注图像用于模型训练过程，而为这些图像添加精确的标注信息需要花费大量的人力、物力，因此，迫切需要充分利用标注者资源，减少人工标注代价，旨在以尽量低的标注代价建立较高精度的模型。
为了更加有效地收集和利用图像的标注信息，主动学习算法提供了一种解决途径。算法随机选择少量图像并获取其标注信息，通过模型与标注者进行交互的形式，利用已收集标注图像中的语义信息和知识，选择部分最有助于模型训练的无标注图像提交标注者进行标注。主动学习算法的优势是通过让学习系统向标注者进行提问的方式达到减少标注者工作量的目的，这不仅充分利用了珍贵的标注者资源，而且更好的将人类知识迁移到学习系统中。因此，发掘高效的主动学习算法对视觉对象的分类与检索研究具有重要的理论价值和现实意义。
目前，虽然部分主动学习算法已经用于减少对象分类与检索任务中的标注代价，并取得了良好的效果，但是，这些算法中往往存在一些理想化的假设条件，导致其不能很好地适用于噪声或者未标注图像数据较大条件下的学习任务。鉴于这一点，本文以主动学习算法作为研究对象，在已有采样策略的基础上，结合统计学理论，发掘噪声或未标注图像数据较大条件下有效的样例选择算法。目的是以尽量低的标注和时间代价获得较高精度的分类模型，并以此为基础，构造有效的主动学习算法模型应用于视觉对象分类与检索实践当中。主要的研究工作和创新点如下：
（1）提出一种基于模型风险的加权样例选择算法
针对主动学习算法中训练数据与测试数据具有相同分布这一理想化假设条件，提出一种基于模型风险的加权样例选择算法，旨在解决因分布差异导致的采样算法效果下降，以及在给定标注代价条件下分类模型训练效果不理想的问题。算法对每个样例设置权重，使用训练数据与无标记数据上模型风险的期望误差来估计样例对应权重值，并根据该值选择最有助于分类模型训练的样例。算法与其它同类方法进行了比较，实验结果证明分类模型的精度得到有效提高。
（2）提出一种批量选择样例的训练集构造方法
针对主动学习算法面临的因视觉对象数量多，但同类别对象数量稀少而造成的正反例数量不平衡这一实际问题，提出一种批量选择样例的训练集构造方法。目的是在相同标注代价条件下，克服大量反例对分类模型的不利影响，提高分类模型的精度。算法利用分类模型风险，通过最小化模型风险的方差来构造训练分布，并依据该分布选择样例，建立训练集。算法与其它同类方法进行了比较，实验结果证明，在模型分类精度相同时，算法需要的标注代价更少。
（3）提出一种多标注者主动学习概率模型
针对主动学习算法中单一标注者必须能够提供准确无误的样例标记这一理想化假设条件，提出一种标注噪声条件下多标注者主动学习概率模型，旨在减少标注者标记质量对主动学习算法的影响。模型通过选择准确度高的标注者提供标记和估计样例对应正确标记的方式达到同时减少标记代价和提高模型精度的目的。实验结果表明，与其它同类方法相比，所提出概率模型有效减少了标注噪声的影响，提高分类模型的性能。
（4）提出一种基于Hash技术的主动学习样例选择算法
针对未标注数据数量较大条件下主动学习算法选择样例时间开销大的实际问题，提出一种基于Hash技术的样例选择算法，旨在快速返回所选样例，减少主动学习分类模型所需时间。算法通过利用Hash技术选择分类模型权重，进而，获得无标注样例与分类界面间近似距离，并依此选择样例用于训练。算法与其它同类方法进行了比较，实验结果证明，所提出算法可以有效减少训练所消耗的时间。
Currently, in fields of text mining, speech recognition, bioinformation data mining andvisual object classification, it has been a real problem that there are always lots ofunlabeled examples which are easy to be obtained, but there are a few of labeledexamples which are hard to be obtained. As one of important aspects in machinelearning, active learning techniques can utilize labeled and unlabeled examples at thesame time in order to obtain a classification model with high performance. In this paper,we make a thorough study of sampling strategy in active learning, and then we applythe proposed algorithms to real tasks of visual object classification.
It has been an important problem all along how to understand or utilize semanticinformation contained in visual objects. Due to rapid development of web techniques, itis possible to collect a lot of images in a short time, and then it becomes a challenge toclassifiy visual objects by using their semantic information which is extracted fromthese unsupervised or weak-supervised images. More and more researchers focus onmining effective algorithms of machine learning, and then judge which category avisual object belongs to according to the knowledge obtained by building a model onlabeled images. In this process, it always needs lots of precisely labeled images fortraining a model whose costs are expensive and time-consuming. In order to obtain sucha model within as few costs as possible, it needs to fully utilize the annotator resource,and then reduce the total labeling costs.
In order to collect and utilize the annotations of images, active learning algorithmsprovide effective solutions. Firstly, a small number of images are randomly chosen andtheir annotations are obtained. Then, by creating the interaction between the annotatorsand model, the learning system can freely choose some unlabeled images, which areconsidered as the most helpful images of all, for querying their annotations. The goal ofreducing annotators’ workload is achieved by making the learning system ask forannotations. Not only this method makes full use of rare annotations, but also ittransfers the knowledge of annotators into the learning system. Therefore, it is importantto mine active learning algorithms for classification and retrieval of visual objects.
Now, some active learning algorithms have been used in reducing total labeling costsof classification and retrieval of visual objects, and these works have achieved favoriteperformance on practical tasks. But there are always some idealized assumptions whichmake active learning algorithms unsuitable for noisy or big data environment. In thispaper, we focus on research of active learning algorithms. Based on existed works inactive learning, we explore sampling strategies which can be used in the condition of noise or big data, and then use them to obtain accurate classification models with as lowlabeling cost as possible. At last, we apply the proposed algorithms in the task of objectclassification and retrieval. Our main contributions are listed as follows:
(1) A sampling strategy is proposed by weighting examples based on structure risk
Aiming at the idealized assumption that training data and test data must have thesame distribution, we propose a sampling strategy by weighting some examples basedon the structure risk. Our goal is to solve the problem that the performance ofclassification model will fall, when there is difference between training and testdistribution. In our method, we use the expected error of structure risk between labeledand unlabeled data to estimate the weight value of every unlabeled example, and then,according to their corresponding weights, choose the most helpful example of all.Compared with other methods, experimental results show that the proposed method caneffectively enhance the performance of classification model.
(2) A method of constructing the training set is proposed by selecting some examples ina batch mode
Aiming at the unbalanced classification problem which is caused by lots of objects inthe whole database but few of them belong to the same category, we propose a methodof constructing training set by selecting examples in a batch mode. Our goal is to avoidthe adverse effect coming from lots of negative examples, and then enhanceclassification performance of the classifier. In our method, we estimate the trainingdistribution by minimizing the variance of structure risk, and then select a group ofexamples according to the estimated distribution. Compared with other methods, theexperimental results show that the labeling costs are fewer than other methods, whenthe classifier obtains similar performance.
(3) A multi-annotator probabilistic model of active learning is proposed
Aiming at the idealized assumption that there is only one annotator to provideaccurate annotation for the selected example, we propose a multi-annotator probabilisticmodel of active learning for noisy annotations. Our goal is to reduce the effect ofannotation quality from multiple annotators. In our probabilistic model, the totallabeling costs are reduced and the classification performance is enhanced by choosingthe most reliable annotator of all for labeling the selected example and estimating theactual annotation. Compared with other methods, the experimental results show that theproposed probabilistic model can effectively reduce the impact from noisy annotations,and then enhance the performance of classification model.
(4) A hash-based sampling strategy of active learning is proposed
Aiming at the problem that it needs a lot of time costs for selecting examples in alarge number of data, we propose the hash-based sampling strategy. Our goal is to return selected examples in a short time, and then reduce the time expense required byobtaining a classification model. In our method, the important weight elements in theparameter vector of classification model are selected, and then the approximate distancebetween the unlabeled examples and classification boundary are estimated. Comparedwith other methods, the experimental results also show that the proposed algorithm caneffectively reduce the time costs.
视觉对象的语义理解是计算机视觉领域的重要问题。网络技术的快速发展使得短时间内获取大量图像成为可能，但对这些无监督或弱监督图片中包含的视觉对象进行分类成为一个艰巨、富有挑战性的任务。越来越多的科研人员致力于发展有效的机器学习算法，在已标注图像集上建立模型，继而利用学习到的知识判断和划分视觉对象所属的类别。这一做法通常需要大量标注图像用于模型训练过程，而为这些图像添加精确的标注信息需要花费大量的人力、物力，因此，迫切需要充分利用标注者资源，减少人工标注代价，旨在以尽量低的标注代价建立较高精度的模型。
为了更加有效地收集和利用图像的标注信息，主动学习算法提供了一种解决途径。算法随机选择少量图像并获取其标注信息，通过模型与标注者进行交互的形式，利用已收集标注图像中的语义信息和知识，选择部分最有助于模型训练的无标注图像提交标注者进行标注。主动学习算法的优势是通过让学习系统向标注者进行提问的方式达到减少标注者工作量的目的，这不仅充分利用了珍贵的标注者资源，而且更好的将人类知识迁移到学习系统中。因此，发掘高效的主动学习算法对视觉对象的分类与检索研究具有重要的理论价值和现实意义。
目前，虽然部分主动学习算法已经用于减少对象分类与检索任务中的标注代价，并取得了良好的效果，但是，这些算法中往往存在一些理想化的假设条件，导致其不能很好地适用于噪声或者未标注图像数据较大条件下的学习任务。鉴于这一点，本文以主动学习算法作为研究对象，在已有采样策略的基础上，结合统计学理论，发掘噪声或未标注图像数据较大条件下有效的样例选择算法。目的是以尽量低的标注和时间代价获得较高精度的分类模型，并以此为基础，构造有效的主动学习算法模型应用于视觉对象分类与检索实践当中。主要的研究工作和创新点如下：
（1）提出一种基于模型风险的加权样例选择算法
针对主动学习算法中训练数据与测试数据具有相同分布这一理想化假设条件，提出一种基于模型风险的加权样例选择算法，旨在解决因分布差异导致的采样算法效果下降，以及在给定标注代价条件下分类模型训练效果不理想的问题。算法对每个样例设置权重，使用训练数据与无标记数据上模型风险的期望误差来估计样例对应权重值，并根据该值选择最有助于分类模型训练的样例。算法与其它同类方法进行了比较，实验结果证明分类模型的精度得到有效提高。
（2）提出一种批量选择样例的训练集构造方法
针对主动学习算法面临的因视觉对象数量多，但同类别对象数量稀少而造成的正反例数量不平衡这一实际问题，提出一种批量选择样例的训练集构造方法。目的是在相同标注代价条件下，克服大量反例对分类模型的不利影响，提高分类模型的精度。算法利用分类模型风险，通过最小化模型风险的方差来构造训练分布，并依据该分布选择样例，建立训练集。算法与其它同类方法进行了比较，实验结果证明，在模型分类精度相同时，算法需要的标注代价更少。
（3）提出一种多标注者主动学习概率模型
针对主动学习算法中单一标注者必须能够提供准确无误的样例标记这一理想化假设条件，提出一种标注噪声条件下多标注者主动学习概率模型，旨在减少标注者标记质量对主动学习算法的影响。模型通过选择准确度高的标注者提供标记和估计样例对应正确标记的方式达到同时减少标记代价和提高模型精度的目的。实验结果表明，与其它同类方法相比，所提出概率模型有效减少了标注噪声的影响，提高分类模型的性能。
（4）提出一种基于Hash技术的主动学习样例选择算法
针对未标注数据数量较大条件下主动学习算法选择样例时间开销大的实际问题，提出一种基于Hash技术的样例选择算法，旨在快速返回所选样例，减少主动学习分类模型所需时间。算法通过利用Hash技术选择分类模型权重，进而，获得无标注样例与分类界面间近似距离，并依此选择样例用于训练。算法与其它同类方法进行了比较，实验结果证明，所提出算法可以有效减少训练所消耗的时间。
Currently, in fields of text mining, speech recognition, bioinformation data mining andvisual object classification, it has been a real problem that there are always lots ofunlabeled examples which are easy to be obtained, but there are a few of labeledexamples which are hard to be obtained. As one of important aspects in machinelearning, active learning techniques can utilize labeled and unlabeled examples at thesame time in order to obtain a classification model with high performance. In this paper,we make a thorough study of sampling strategy in active learning, and then we applythe proposed algorithms to real tasks of visual object classification.
It has been an important problem all along how to understand or utilize semanticinformation contained in visual objects. Due to rapid development of web techniques, itis possible to collect a lot of images in a short time, and then it becomes a challenge toclassifiy visual objects by using their semantic information which is extracted fromthese unsupervised or weak-supervised images. More and more researchers focus onmining effective algorithms of machine learning, and then judge which category avisual object belongs to according to the knowledge obtained by building a model onlabeled images. In this process, it always needs lots of precisely labeled images fortraining a model whose costs are expensive and time-consuming. In order to obtain sucha model within as few costs as possible, it needs to fully utilize the annotator resource,and then reduce the total labeling costs.
In order to collect and utilize the annotations of images, active learning algorithmsprovide effective solutions. Firstly, a small number of images are randomly chosen andtheir annotations are obtained. Then, by creating the interaction between the annotatorsand model, the learning system can freely choose some unlabeled images, which areconsidered as the most helpful images of all, for querying their annotations. The goal ofreducing annotators’ workload is achieved by making the learning system ask forannotations. Not only this method makes full use of rare annotations, but also ittransfers the knowledge of annotators into the learning system. Therefore, it is importantto mine active learning algorithms for classification and retrieval of visual objects.
Now, some active learning algorithms have been used in reducing total labeling costsof classification and retrieval of visual objects, and these works have achieved favoriteperformance on practical tasks. But there are always some idealized assumptions whichmake active learning algorithms unsuitable for noisy or big data environment. In thispaper, we focus on research of active learning algorithms. Based on existed works inactive learning, we explore sampling strategies which can be used in the condition of noise or big data, and then use them to obtain accurate classification models with as lowlabeling cost as possible. At last, we apply the proposed algorithms in the task of objectclassification and retrieval. Our main contributions are listed as follows:
(1) A sampling strategy is proposed by weighting examples based on structure risk
Aiming at the idealized assumption that training data and test data must have thesame distribution, we propose a sampling strategy by weighting some examples basedon the structure risk. Our goal is to solve the problem that the performance ofclassification model will fall, when there is difference between training and testdistribution. In our method, we use the expected error of structure risk between labeledand unlabeled data to estimate the weight value of every unlabeled example, and then,according to their corresponding weights, choose the most helpful example of all.Compared with other methods, experimental results show that the proposed method caneffectively enhance the performance of classification model.
(2) A method of constructing the training set is proposed by selecting some examples ina batch mode
Aiming at the unbalanced classification problem which is caused by lots of objects inthe whole database but few of them belong to the same category, we propose a methodof constructing training set by selecting examples in a batch mode. Our goal is to avoidthe adverse effect coming from lots of negative examples, and then enhanceclassification performance of the classifier. In our method, we estimate the trainingdistribution by minimizing the variance of structure risk, and then select a group ofexamples according to the estimated distribution. Compared with other methods, theexperimental results show that the labeling costs are fewer than other methods, whenthe classifier obtains similar performance.
(3) A multi-annotator probabilistic model of active learning is proposed
Aiming at the idealized assumption that there is only one annotator to provideaccurate annotation for the selected example, we propose a multi-annotator probabilisticmodel of active learning for noisy annotations. Our goal is to reduce the effect ofannotation quality from multiple annotators. In our probabilistic model, the totallabeling costs are reduced and the classification performance is enhanced by choosingthe most reliable annotator of all for labeling the selected example and estimating theactual annotation. Compared with other methods, the experimental results show that theproposed probabilistic model can effectively reduce the impact from noisy annotations,and then enhance the performance of classification model.
(4) A hash-based sampling strategy of active learning is proposed
Aiming at the problem that it needs a lot of time costs for selecting examples in alarge number of data, we propose the hash-based sampling strategy. Our goal is to return selected examples in a short time, and then reduce the time expense required byobtaining a classification model. In our method, the important weight elements in theparameter vector of classification model are selected, and then the approximate distancebetween the unlabeled examples and classification boundary are estimated. Comparedwith other methods, the experimental results also show that the proposed algorithm caneffectively reduce the time costs.
引文
[1] Y. Xue, X. Liao, L. Carin, B. Krishnapuram. Multi-task learning for classificationwith Dirichlet process priors [J]. Journal of Machine Learning Research,2007,8:35-63
[2] X. Shi, W. Fan, J. Ren. Actively transfer domain knowledge. Machine Learning andKnowledge Discovery in Databases [J], Lecture Notes in Computer Science,5212:Proc of ECML PKDD. Berlin: Springer,2008:342-357
[3] Y. Yan, R. Rosales, G. Fung, J. Dy. Active learning from crowds [C]. In: Proc ofICML2011. New York: ACM,2011:1161-1168
[4] V.C. Raykar, S. Yu, L.H. Zhao, G.H.Valadez, C. Florin, L. Bogoni, L. Moy. Learningfrom crowds [J]. Journal of Machine Learning Research,2010,11:1297-1322
[5] M. Wang, X.S. Hua. Active learning in multimedia annotation and retrieval: a survey[J]. ACM Transactions on Intelligent Systems and Technology,2011,2:1-21
[6] M. Wang, X.S. Hua, Y. Song, J. Tang, L.R. Dai. Interactive video annotation bymulti-concept multi-modality active learning [C]. In: Proc of ICSC2007. NJ: IEEE,2007: pp.321-328
[7] A. Kapoor, K. Grauman, R. Urtasun, T. Darrell. Gaussian processes for objectcategorization [J]. International Journal of Computer Vision,2010,88:169-188
[8] S. Vijayanarasimhan, K. Grauman. Cost-sensitive active visual category learning [J].International Journal of Computer Vision,2011,91(1):24-44
[9] B. Robin. Conceptual indexing and active retrieval of video for interactive learningenvironments [J]. Knowledge-Based Systems,1996,9:491-499
[10] S. Vijayanarasimhan. Active Visual Category Learning [D],2011, Graduate Schoolof The University of Texas at Austin
[11] M. Everingham, L.V. Gool, C.K.I. Williams, J. Winn, A. Zisserman, The PASCALvisual object classes challenge2007[M/OL]. http://www.pascalnetwork.org/challenges/VOC/voc2007
[12] M.J. Huiskes, M.S. Lew. The MIR flickr retrieval evaluation [C]. In: Proc of ACMMIR2008. New York: ACM,2008:39-43
[13] S. Tong, D. Koller. Support vector machine active learning with applications to textclassification [J]. Journal of Machine Learning Research,2001,2:45-66
[14] H. Raghavan, O. Madani, R. Jones. Active learning with feedback on both featuresand instances [J]. Journal of Machine Learning Research,2006,7:1655-1686
[15] S.C.H. Hoi, R. Jin, M.R. Lyu, Batch mode active learning with applications to textcategorization and image retrieval [J]. IEEE Transactions on Knowledge and DataEngineering,2009,21:1233-1248
[16] J. Zhu, H. Wang, E. Hovy, M. Ma. Confidence-based stopping criteria for activelearning for data annotation [J]. ACM Transactions on Speech and LanguageProcessing,2010,6:1-24
[17] J. Zhu, H. Wang, B.K. Tsou, M. Ma. Active learning with sampling by uncertaintyand density for data annotations [J]. IEEE Transactions on Audio, Speech andLanguage Processing,2010,18:1323-1331
[18] Y. Liu. Active learning with support vector machine applied to gene expressiondata for cancer classification [J]. Journal of Chemical Information and ComputerSciences,2004,44:1936-1941
[19] B. Settles. Active learning literature survey [R]. Computer Sciences TechnicalReport1648, University of Wisconsin-Madison,2009
[20] X. Zhu. Semi-supervised learning literature survey [R]. Computer SciencesTechnical Report1530, University of Wisconsin,2005
[21] K. Tomanek, F. Olsson. A web survey on the use of active learning to supportannotation of text data [C] In: Proc of HLT-NAACL. Stroudsburg, PA: ACL,2009:45-48
[22] I. Guyon, G. Cawley, G. Dror. Design and analysis of the WCCI2010activelearning challenge [C] In: Proc of IEEE/INNS IJCNN2010. NJ: IEEE,2010:1-8
[23] D. Angluin, Queries and concept learning [J]. Machine Learning,1988,2:319-342
[24] S. Dasgupta, J. Langford. A tutorial on active learning [R]. Tutorial summary:Active learning, International Conference of Machine Learning2009
[25] A. Liu, G. Jun, J. Ghosh. A self-training approach to cost sensitive uncertaintysampling [J]. Machine Learning,2009,76(2-3):257-270
[26] E. B. Baum, K. Lang. Query learning can work poorly when a human oracle isused [C] In: Proc of IEEE IJCNN1992. NJ: IEEE,1992:335-340
[27] D. Cohn, L. Atlas, R. Ladner. Improving generalization with active learning [J].Machine Learning,1994,15(2):201-221
[28] H.S. Seung, M. Opper, H. Sompolinsky. Query by committee [C] In: Proc of COLT1992. New York: ACM,1992:287-294
[29] S. Dasgupta, A. Kalai, C. Monteleoni. Analysis of perceptron-based active learning[J]. Journal of Machine Learning Research,2009,10:281-299
[30] I. Dagan, S.P. Engelson. Committee-based sampling for training probabilisticclassifers [C] In: Proc of ICML1995. San Francisco, CA: Morgan Kaufmann,1995:150-157
[31] H. Yu. SVM selective sampling for ranking with application to data retrieval [C] In:Proc of ACM SIGKDD2005. New York: ACM,2005:354-363
[32] R. Moskovitch, N. Nissim, D. Stopel. Improving the detection of unknowncomputer worms activity using active learning [G] Lecture Notes in ComputerScience4667: Proc of30th German Conf on AI2007. Berlin: Springer,2007:489-493
[33] C.A. Thompson, M.E. Califf, R.J. Mooney. Active learning for natural languageparsing and information extraction [C] In: Proc of ICML1999. San Francisco, CA:Morgan Kaufmann,1999:406-414
[34] D. Lewis, J. Catlett. Heterogeneous uncertainty sampling for supervised learning
[C] In: Proc of ICML1994. San Francisco, CA: Morgan Kaufmann,1994:148-156
[35] A. McCallum, K. Nigam. Employing EM in pool-based active learning for textclassifcation [C] In: Proc of ICML1998. San Francisco, CA: Morgan Kaufmann,1998:359-367
[36] S. Tong, D. Koller. Support vector machine active learning with applications to textclassification[C] In: Proc of ICML2000. San Francisco, CA: Morgan Kaufmann,2000:999-1006
[37] S.C.H. Hoi, R. Jin, M.R. Lyu. Large scale text categorization by batch mode activelearning [C] In: Proc of WWW2006. New York: ACM,2006:633-642
[38] B. Settles, M. Craven. An analysis of active learning strategies for sequencelabeling tasks [C] In: Proc of EMNLP2008. Stroudsburg, PA: ACL,2008:1069-1078
[39] S. Tong, E. Chang. Support vector machine active learning for image retrieval [C]In: Proc of MULTIMEDIA2001. New York: ACM,2001:107-118
[40] C. Zhang, T. Chen. An active learning framework for content based informationretrieval [J]. IEEE Trans on Multimedia,2002,4(2):260-268
[41] R. Yan, J. Yang, A. Hauptmann. Automatically labeling video data usingmulti-class active learning [C] In: Proc of IEEE ICCV2003. NJ: IEEE,2003:516-523
[42] A. Hauptmann, W. Lin, R. Yan. Extreme video retrieval: joint maximization ofhuman and computer performance [C] In: Proc of MULTIMEDIA2006. New York:ACM,2006:385-394
[43] Y. Liu. Active learning with support vector machine applied to gene expressiondata for cancer classification [J]. Journal of Chemical Information and ComputerSciences,2004,44:1936-1941
[44] M.F. Balcan, A.Z. Broder, T. Zhang. Margin based active learning [G] LectureNotes in Computer Science4539: Proc of COLT2007. Berlin: Springer,2007:35-50
[45] S. Dasgupta, A.T. Kalai, C. Monteleoni. Analysis of perceptron-based activelearning [G] Lecture Notes in Computer Science3559: Proc of COLT2005. Berlin:Springer,2005:249-263
[46] S. Dasgupta. Coarse sample complexity bounds for active learning [C]. In: Proc ofNIPS2005. Cambridge, MA: MIT Press,2006:235-242
[47] M.F. Balcan, S. Hanneke, J. Wortman. The true sample complexity of activelearning [C] In: Proc of COLT2008. Madison, WI: Omni press,2008:45-56
[48] S. Dasgupta. Analysis of a greedy active learning strategy [C] In: Proc of NIPS2004. Cambridge, MA: MIT Press,2005:337-344
[49] Y. Freund Y, H.S. Seung, E. Shamir, N. Tishby. Selective sampling using the queryby committee algorithm [J]. Machine Learning,1997,28(2-3):133-168
[50] R. Gilad-Bachrach, A. Navot, N. Tishby. Query by committee made real [C] In:Proc of NIPS2005. Cambridge, MA: MIT Press,2006:443-450
[51] M. Balcan, A. Beygelzimer, J. Langford. Agnostic active learning [C] In: Proc ofICML2006. New York: ACM,2006:65-72
[52] S. Hanneke. A bound on the label complexity of agnostic active learning [C] In:Proc of ICML2007. New York: ACM,2007:353-360
[53] S. Dasgupta, D. Hsu, C. Monteleoni. A general agnostic active learning algorithm
[C] In: Proc of NIPS2007. Cambridge, MA: MIT Press,2008:353-360
[54] A. Beygelzimer, S. Dasgupta, J. Langford. Importance weighted active learning [C]In: Proc of ICML2009. New York: ACM,2009:49-56
[55] W. Wang, Z. Zhou. On multi-view active learning and the combination withsemi-supervised learning [C] In: Proc of ICML2008. New York: ACM,2008:1152-1159
[56] S. Hanneke. Rates of convergence in active learning [J]. The Annals of Statistics,2011,39(1):333-361
[57] R.M. Castro, R.D. Nowak. Upper and lower error bounds for active learning [C]. In:Proc of Allerton2006. Red Hook, NY: Curran Associates,2007:225-234
[58] M. Kaariainen. Active learning in the non-realizable case [G] Lecture Notes inArtificial Intelligence4264: Proc of ALT2006. Berlin: Springer,2006:63-77
[59] G. Cavallanti, N. Cesa-Bianchi,C. Gentile. Linear classification and selectivesampling under low noise conditions [C] In: Proc of NIPS2008. Cambridge, MA:MIT Press,2009:249-256
[60] R.M. Castro, R.D. Nowak. Minimax bounds for active learning [J]. IEEE Trans onInformation Theory,2008,54(5):2339-2353
[61] L. Wang. Sufficient conditions for agnostic active learnable [C] In: Proc of NIPS2009. Cambridge, MA: MIT Press,2010:1999-2007
[62] W. Wang, Z. Zhou. Multi-view active learning in the non-realizable case [C] In:Proc of NIPS2010. Cambridge, MA: MIT Press,2011:2388-2396
[63] I. Muslea, S. Minton, C.A. Knoblock. Active learning with multiple-views [J].Journal of Artificial Intelligence Research,2006,27:203-233
[64] A. Culotta, A. McCallum. Reducing labeling effort for structured prediction tasks
[C] In: Proc of AAAI2005. Menlo Park, CA: AAAI Press,2005:746-751
[65] T. Scheffer, C. Decomain, S. Wrobel. Active hidden Markov models forinformation extraction [G] Lecture Notes in Computer Science2189: Proc ofCAIDA2001. Berlin: Springer,2001:309-318
[66] S. Kim, Y. Song, K. Kim. MMR-based active machine learning for bio-namedentity recognition [C] In: Proc of HLT-NAACL2006. Stroudsburg, PA: ACL,2006:69-72
[67] G. Schohn, D. Cohn. Less is more: active learning with support vector machines [C]In: Proc of ICML2000. San Francisco, CA: Morgan Kaufmann,2000:839-846
[68] Z. Xu, K. Yu, V. Tresp. Representative sampling for text classification usingsupport vector machines [G] Lecture Notes in Computer Science2633: Proc ofECIR2003. Berlin: Springer,2003:393-407
[69] H.T. Nguyen, A. Smeulders. Active learning using pre-clustering [C] In: Proc ofICML2004. New York: ACM,2004:79-86
[70] D. Sanjoy, H. Daniel. Hierarchical sampling for active learning [C] In: Proc ofICML2008. New York: ACM,2008:208-215
[71] P. Donmez, J.G. Carbonell, P.N. Bennett. Dual strategy active learning [G] LectureNotes in Artificial Intelligence4701: Proc of ECML2007. Berlin: Springer,2007:116-127
[72] S. Huang, R. Jin, Z. Zhou. Active learning by querying informative andrepresentative examples [C] In: Proc of NIPS2010. Cambridge, MA: MIT Press,2011:892-900
[73] N. Abe, H. Mamitsuka. Query learning strategies using boosting and bagging [C]In: Proc of ICML1998. San Francisco, CA: Morgan Kaufmann,1998:1-9
[74] Y. Freund, R.E. Schapire. A decision-theoretic generalization of on-line learningand an application to boosting [J]. Journal of Computer and System Sciences,1997,55(1):119-139
[75] L. Breiman. Bagging predictors [J]. Machine Learning,1996,24(2):123-140
[76] S. Geman, E. Bienenstock, R. Doursat. Neural networks and the bias/variancedilemma [J]. Neural Computation,1992,4(1):1-58
[77] D. Cohn, Z. Ghahramani, M.I. Jordan. Active learning with statistical models [J].Artificial Intelligence Research,1996(4):129-145
[78] T. Zhang, F.J. Oles. A probability analysis on the value of unlabeled data forclassification problems [C] In: Proc of ICML2000. San Francisco, CA: MorganKaufmann,2000:1191-1198
[79] G. Paass, J. Kindermann. Bayesian query construction for neural network models
[C] In: Proc of NIPS1995. Cambridge, MA: MIT Press,1995:443-450
[80] S.C.H. Hoi, R. Jin, J. Zhu. Batch mode active learning and its application tomedical image classification [C] In: Proc of ICML2006. New York: ACM,2006:417-424
[81] N. Roy, A. McCallum. Toward optimal active learning through sampling estimationof error reduction[C] In: Proc of ICML2001. San Francisco, CA: MorganKaufmann,2001:441-448
[82] X. Zhu, J. Lafferty, Z. Ghahramani. Combining active learning andsemi-supervised learning using Gaussian fields and harmonic functions [C] In:Proc of ICML2003Workshop on the Continuum from Labeled to Unlabeled Data.Menlo Park, CA: AAAI Press,2003:58-65
[83] Y. Guo, R. Greiner. Optimistic active learning using mutual information [C] In:Proc of IJCAI2007. Menlo Park, CA: AAAI Press:823-829
[84] A. Sorokin, D. Forsyth. Utility data annotation with amazon mechanical turk [C] In:Proc of IEEE CVPR2008. NJ: IEEE,2008:1-8
[85] P. Donmez, J.G. Carbonell. Proactive learning: cost-sensitive active learning withmultiple imperfect oracles [C]. In: Proc of ACM CIKM2008, California,2008, pp.629-638
[86]邓超,郭茂祖.基于Tri-Training和数据剪辑的半监督聚类算法[J].软件学报,2008,19(3):663-673
[87]邓超,郭茂祖.基于自适应数据剪辑策略的Tri-training算法[J].计算机学报,2007,30(8):1213-1226
[88] V. N. Vapnik. Statistical learning theory [M]. John Wiley&Sons, Inc., New York,1998
[89] V.V. Fedorov. Theory of optimal experiments [M]. Academic Press,1972
[90] D. J. C. MacKay. Information-based objective functions for active data selection [J].Neural Computation,1992,4(4):590-604
[91] K. Fukumizu. Statistical active learning in multilayer perceptions [J]. IEEETransactions on Neural Networks,2000,11(1):17-26
[92] C. Sawade, N. Landwehr, S. Bickel, T. Scheffer. Active risk estimation [C]. In:Proc of ICML2010, New York: ACM,2010:1161-1168
[93] P. Donmez, G. Lebanon, K. Balasubramanian. Unsupervised supervised learning I:estimating classification and regression errors without labels [J], Journal ofMachine Learning Research,2010,11:1323-1351
[94] A. Beygelzimer, D. Hsu, J. Langford, T. Zhang. Agnostic active learning withoutconstraints [C]. In: Proc of NIPS2010. Cambridge, MA: MIT Press,2011:199-207
[95] M. Sugiyamy. Active learning in approximate linear regression based onconditional expectation of generalization error [J]. Journal of Machine LearningResearch,2006,7:141-166
[96] H. Shimodaira, Improving predictive inference under covariate shift by weightingthe log-likelihood function [J]. Journal of Statistical Planning and Inference,2000,90(2):227-244
[97] D. P. Wins. Robust weights and designs for biased regression models: least squaresand generalized m-estimation [J]. Journal of Statistical Planning and Inference,2000,83(2):395-412
[98] S. Geman, E. Bienenstock, R. Doursat. Neural networks and the bias/variancedilemma [J]. Neural Computation,1992,4(1):1-58
[99] B. Russell, A. Torralba, K. Murphy, W. T. Freeman. LabelMe: a database andweb-based tool for image annotation [J]. International Journal of ComputerVision,2008,77(1):157-173
[100] R. E. Fan, K. W. Chang, C. J. Hsieh, X. R. Wang, C. J. Lin. LIBLINEAR: Alibrary for large linear classification[J]. Journal of Machine Learning Research,2008,9:1871-1874
[101] K. Balasubramanian, P. Donmez, G. Lebanon. Unsupervised supervised learningII: training margin based classifiers without labels [J]. Journal of MachineLearning Research,2011,12:3119-3145
[102] V. Sheng, F. Provost, P.G. Ipeirotis. Get another label? improving data quality anddata mining using multiple, noisy labelers [C]. In: Proc of ACM SIGKDD2008.New York: ACM,2008:614-622
[103] P. Donmez, J.G. Carbonell, J. Schneider. Efficiently learning the accuracy oflabeling sources for selective sampling [C]. In: Proc of ACM SIGKDD2009.New York: ACM,2009:259-268
[104] Y. Yan, R. Rosales, G. Fung, M. Schmidt. Modeling annotator expertise: learningwhen everybody knows a bit of something [C]. In: Proc of AISTATS2010.Cambridge, MA: MIT Press,2010:932-939
[105] O. Dekel, O. Shamir. Good learners for evil teachers [C]. In: Proc of ICML2009.New York: ACM,2009:233-240
[106] Snow R, O’Connor B, Jurafsky D. Cheap and fast-but is it good? evaluatingnon-expert annotations for natural language tasks [C] In: Proc of EMNLP2008.Stroudsburg, PA: ACL,2008:254-263
[107] P. Donmez, J.G. Carbonell, J. Schneider. A probabilistic framework to learn frommultiple annotators with time-varying accuracy [C]. In: Proc of SDM2010.Philadelphia, PA: Soc for Industrial&Applied Math,2010:826-837
[108] B. Thiesson, C. Meek, D. Heckerman. Accelerating EM for large databases [J].Machine Learning,2001,45:279-299
[109] KDD CUP (2008)[M/OL]. http://www.sigkdd.org/kddcup/index.php?section=2008&method=info
[110] P. Jain, S. Vijayanarasimhan, K. Grauman. Hashing hyperplane queries to nearpoints with applications to large-scale active learning [C] In: Proc of NIPS2010.Cambridge, MA: MIT Press,2011:928-936
[111] L-J. Li, H. Su, E. Xing, Fei-Fei Li. Object bank: a high-level image representationfor scene classification&semantic feature sparsification [J]. In: Proc of NIPS2010. Cambridge, MA: MIT Press,2011:1378-1386
[112] S. Vijayanarasimhan, K. Grauman. Large-scale live active learning: trainingobject detectors with crawled data and crowds [C]. In: Proc of CVPR2011. NJ:IEEE,2011:1449-1456
[113] N. Panda, K. Goh, E.Y. Chang. Active learning in very large databases [J]. Journalof Multimedia Tools and Applications,2006,31:249-267
[114] P. Jain, S. Vijayanarasimhan, K. Grauman. Hashing hyperplane queries to nearpoints with applications to large-scale active learning [C] In: Proc of NIPS2010.Cambridge, MA: MIT Press,2011:928-936
[115] Y. Weiss, A. Torralba, R. Fergus. Spectral hashing [C] In: Proc of NIPS2009.Cambridge, MA: MIT Press,2010:1753-1760
[116] B. Kulis, K. Grauman. Kernelized locality-sensitive hashing [J]. IEEE Transactionon Pattern Analysis Machine Intelligence,2012,34:1092-1104
[117] R. Lin, D. Ross, J. Yagnik. SPEC hashing: similarity preserving algorithm forentropy-based coding [C]. In: Proc of CVPR2010. NJ: IEEE,2010:848-854
[118] J. Yagnik, D. Strelow, D. Ross, R.S. Lin. The power of comparative reasoning [C].In: Proc of ICCV2011. NJ: IEEE,2011:2431-2438
[119] Y. Gong, S. Lazebnik, A. Gordo, F. Perronnin. Iterative quantization: aprocrustean approach to learning binary codes for large-scale image retrieval[J]. Accepted by IEEE Transaction on Pattern Analysis and Machine Intelligence,2012
[120] G.X. Yuan, K.W. Chang, C.J. Hsieh, C.J. Lin. A comparison of optimizationmethods and software for large-scale L1-regularized linear classification [J].Journal of Machine Learning Research,2010,11:3183-3234
[121] M. Rastegari, C. Fang, L. Torresani. Scalable object-class retrieval withapproximate and top-k ranking [C]. In: Proc of ICCV2011. NJ: IEEE,2011:2659-2666
[122] G. Griffin, A. Holub, P. Perona. Caltech-256object category dataset [R].Computer Sciences Technical Report7694, California Institute of Technology,2007
[123] L. Torresani, M. Szummer, A. Fitzgibbon. Efficient object category recognitionusing classemes [J]. Lecture Notes in Computer Science,6311: Proc of ECCV2010. Berlin: Springer,2010:776-789
[124] A. Bergamo, L. Torresani, A. Fitzgibbon. PiCoDes: learning a compact code fornovel-category recognition [C]. In: Proc of NIPS2011. Cambridge, MA: MITPress,2012:2088-2096
[2] X. Shi, W. Fan, J. Ren. Actively transfer domain knowledge. Machine Learning andKnowledge Discovery in Databases [J], Lecture Notes in Computer Science,5212:Proc of ECML PKDD. Berlin: Springer,2008:342-357
[3] Y. Yan, R. Rosales, G. Fung, J. Dy. Active learning from crowds [C]. In: Proc ofICML2011. New York: ACM,2011:1161-1168
[4] V.C. Raykar, S. Yu, L.H. Zhao, G.H.Valadez, C. Florin, L. Bogoni, L. Moy. Learningfrom crowds [J]. Journal of Machine Learning Research,2010,11:1297-1322
[5] M. Wang, X.S. Hua. Active learning in multimedia annotation and retrieval: a survey[J]. ACM Transactions on Intelligent Systems and Technology,2011,2:1-21
[6] M. Wang, X.S. Hua, Y. Song, J. Tang, L.R. Dai. Interactive video annotation bymulti-concept multi-modality active learning [C]. In: Proc of ICSC2007. NJ: IEEE,2007: pp.321-328
[7] A. Kapoor, K. Grauman, R. Urtasun, T. Darrell. Gaussian processes for objectcategorization [J]. International Journal of Computer Vision,2010,88:169-188
[8] S. Vijayanarasimhan, K. Grauman. Cost-sensitive active visual category learning [J].International Journal of Computer Vision,2011,91(1):24-44
[9] B. Robin. Conceptual indexing and active retrieval of video for interactive learningenvironments [J]. Knowledge-Based Systems,1996,9:491-499
[10] S. Vijayanarasimhan. Active Visual Category Learning [D],2011, Graduate Schoolof The University of Texas at Austin
[11] M. Everingham, L.V. Gool, C.K.I. Williams, J. Winn, A. Zisserman, The PASCALvisual object classes challenge2007[M/OL]. http://www.pascalnetwork.org/challenges/VOC/voc2007
[12] M.J. Huiskes, M.S. Lew. The MIR flickr retrieval evaluation [C]. In: Proc of ACMMIR2008. New York: ACM,2008:39-43
[13] S. Tong, D. Koller. Support vector machine active learning with applications to textclassification [J]. Journal of Machine Learning Research,2001,2:45-66
[14] H. Raghavan, O. Madani, R. Jones. Active learning with feedback on both featuresand instances [J]. Journal of Machine Learning Research,2006,7:1655-1686
[15] S.C.H. Hoi, R. Jin, M.R. Lyu, Batch mode active learning with applications to textcategorization and image retrieval [J]. IEEE Transactions on Knowledge and DataEngineering,2009,21:1233-1248
[16] J. Zhu, H. Wang, E. Hovy, M. Ma. Confidence-based stopping criteria for activelearning for data annotation [J]. ACM Transactions on Speech and LanguageProcessing,2010,6:1-24
[17] J. Zhu, H. Wang, B.K. Tsou, M. Ma. Active learning with sampling by uncertaintyand density for data annotations [J]. IEEE Transactions on Audio, Speech andLanguage Processing,2010,18:1323-1331
[18] Y. Liu. Active learning with support vector machine applied to gene expressiondata for cancer classification [J]. Journal of Chemical Information and ComputerSciences,2004,44:1936-1941
[19] B. Settles. Active learning literature survey [R]. Computer Sciences TechnicalReport1648, University of Wisconsin-Madison,2009
[20] X. Zhu. Semi-supervised learning literature survey [R]. Computer SciencesTechnical Report1530, University of Wisconsin,2005
[21] K. Tomanek, F. Olsson. A web survey on the use of active learning to supportannotation of text data [C] In: Proc of HLT-NAACL. Stroudsburg, PA: ACL,2009:45-48
[22] I. Guyon, G. Cawley, G. Dror. Design and analysis of the WCCI2010activelearning challenge [C] In: Proc of IEEE/INNS IJCNN2010. NJ: IEEE,2010:1-8
[23] D. Angluin, Queries and concept learning [J]. Machine Learning,1988,2:319-342
[24] S. Dasgupta, J. Langford. A tutorial on active learning [R]. Tutorial summary:Active learning, International Conference of Machine Learning2009
[25] A. Liu, G. Jun, J. Ghosh. A self-training approach to cost sensitive uncertaintysampling [J]. Machine Learning,2009,76(2-3):257-270
[26] E. B. Baum, K. Lang. Query learning can work poorly when a human oracle isused [C] In: Proc of IEEE IJCNN1992. NJ: IEEE,1992:335-340
[27] D. Cohn, L. Atlas, R. Ladner. Improving generalization with active learning [J].Machine Learning,1994,15(2):201-221
[28] H.S. Seung, M. Opper, H. Sompolinsky. Query by committee [C] In: Proc of COLT1992. New York: ACM,1992:287-294
[29] S. Dasgupta, A. Kalai, C. Monteleoni. Analysis of perceptron-based active learning[J]. Journal of Machine Learning Research,2009,10:281-299
[30] I. Dagan, S.P. Engelson. Committee-based sampling for training probabilisticclassifers [C] In: Proc of ICML1995. San Francisco, CA: Morgan Kaufmann,1995:150-157
[31] H. Yu. SVM selective sampling for ranking with application to data retrieval [C] In:Proc of ACM SIGKDD2005. New York: ACM,2005:354-363
[32] R. Moskovitch, N. Nissim, D. Stopel. Improving the detection of unknowncomputer worms activity using active learning [G] Lecture Notes in ComputerScience4667: Proc of30th German Conf on AI2007. Berlin: Springer,2007:489-493
[33] C.A. Thompson, M.E. Califf, R.J. Mooney. Active learning for natural languageparsing and information extraction [C] In: Proc of ICML1999. San Francisco, CA:Morgan Kaufmann,1999:406-414
[34] D. Lewis, J. Catlett. Heterogeneous uncertainty sampling for supervised learning
[C] In: Proc of ICML1994. San Francisco, CA: Morgan Kaufmann,1994:148-156
[35] A. McCallum, K. Nigam. Employing EM in pool-based active learning for textclassifcation [C] In: Proc of ICML1998. San Francisco, CA: Morgan Kaufmann,1998:359-367
[36] S. Tong, D. Koller. Support vector machine active learning with applications to textclassification[C] In: Proc of ICML2000. San Francisco, CA: Morgan Kaufmann,2000:999-1006
[37] S.C.H. Hoi, R. Jin, M.R. Lyu. Large scale text categorization by batch mode activelearning [C] In: Proc of WWW2006. New York: ACM,2006:633-642
[38] B. Settles, M. Craven. An analysis of active learning strategies for sequencelabeling tasks [C] In: Proc of EMNLP2008. Stroudsburg, PA: ACL,2008:1069-1078
[39] S. Tong, E. Chang. Support vector machine active learning for image retrieval [C]In: Proc of MULTIMEDIA2001. New York: ACM,2001:107-118
[40] C. Zhang, T. Chen. An active learning framework for content based informationretrieval [J]. IEEE Trans on Multimedia,2002,4(2):260-268
[41] R. Yan, J. Yang, A. Hauptmann. Automatically labeling video data usingmulti-class active learning [C] In: Proc of IEEE ICCV2003. NJ: IEEE,2003:516-523
[42] A. Hauptmann, W. Lin, R. Yan. Extreme video retrieval: joint maximization ofhuman and computer performance [C] In: Proc of MULTIMEDIA2006. New York:ACM,2006:385-394
[43] Y. Liu. Active learning with support vector machine applied to gene expressiondata for cancer classification [J]. Journal of Chemical Information and ComputerSciences,2004,44:1936-1941
[44] M.F. Balcan, A.Z. Broder, T. Zhang. Margin based active learning [G] LectureNotes in Computer Science4539: Proc of COLT2007. Berlin: Springer,2007:35-50
[45] S. Dasgupta, A.T. Kalai, C. Monteleoni. Analysis of perceptron-based activelearning [G] Lecture Notes in Computer Science3559: Proc of COLT2005. Berlin:Springer,2005:249-263
[46] S. Dasgupta. Coarse sample complexity bounds for active learning [C]. In: Proc ofNIPS2005. Cambridge, MA: MIT Press,2006:235-242
[47] M.F. Balcan, S. Hanneke, J. Wortman. The true sample complexity of activelearning [C] In: Proc of COLT2008. Madison, WI: Omni press,2008:45-56
[48] S. Dasgupta. Analysis of a greedy active learning strategy [C] In: Proc of NIPS2004. Cambridge, MA: MIT Press,2005:337-344
[49] Y. Freund Y, H.S. Seung, E. Shamir, N. Tishby. Selective sampling using the queryby committee algorithm [J]. Machine Learning,1997,28(2-3):133-168
[50] R. Gilad-Bachrach, A. Navot, N. Tishby. Query by committee made real [C] In:Proc of NIPS2005. Cambridge, MA: MIT Press,2006:443-450
[51] M. Balcan, A. Beygelzimer, J. Langford. Agnostic active learning [C] In: Proc ofICML2006. New York: ACM,2006:65-72
[52] S. Hanneke. A bound on the label complexity of agnostic active learning [C] In:Proc of ICML2007. New York: ACM,2007:353-360
[53] S. Dasgupta, D. Hsu, C. Monteleoni. A general agnostic active learning algorithm
[C] In: Proc of NIPS2007. Cambridge, MA: MIT Press,2008:353-360
[54] A. Beygelzimer, S. Dasgupta, J. Langford. Importance weighted active learning [C]In: Proc of ICML2009. New York: ACM,2009:49-56
[55] W. Wang, Z. Zhou. On multi-view active learning and the combination withsemi-supervised learning [C] In: Proc of ICML2008. New York: ACM,2008:1152-1159
[56] S. Hanneke. Rates of convergence in active learning [J]. The Annals of Statistics,2011,39(1):333-361
[57] R.M. Castro, R.D. Nowak. Upper and lower error bounds for active learning [C]. In:Proc of Allerton2006. Red Hook, NY: Curran Associates,2007:225-234
[58] M. Kaariainen. Active learning in the non-realizable case [G] Lecture Notes inArtificial Intelligence4264: Proc of ALT2006. Berlin: Springer,2006:63-77
[59] G. Cavallanti, N. Cesa-Bianchi,C. Gentile. Linear classification and selectivesampling under low noise conditions [C] In: Proc of NIPS2008. Cambridge, MA:MIT Press,2009:249-256
[60] R.M. Castro, R.D. Nowak. Minimax bounds for active learning [J]. IEEE Trans onInformation Theory,2008,54(5):2339-2353
[61] L. Wang. Sufficient conditions for agnostic active learnable [C] In: Proc of NIPS2009. Cambridge, MA: MIT Press,2010:1999-2007
[62] W. Wang, Z. Zhou. Multi-view active learning in the non-realizable case [C] In:Proc of NIPS2010. Cambridge, MA: MIT Press,2011:2388-2396
[63] I. Muslea, S. Minton, C.A. Knoblock. Active learning with multiple-views [J].Journal of Artificial Intelligence Research,2006,27:203-233
[64] A. Culotta, A. McCallum. Reducing labeling effort for structured prediction tasks
[C] In: Proc of AAAI2005. Menlo Park, CA: AAAI Press,2005:746-751
[65] T. Scheffer, C. Decomain, S. Wrobel. Active hidden Markov models forinformation extraction [G] Lecture Notes in Computer Science2189: Proc ofCAIDA2001. Berlin: Springer,2001:309-318
[66] S. Kim, Y. Song, K. Kim. MMR-based active machine learning for bio-namedentity recognition [C] In: Proc of HLT-NAACL2006. Stroudsburg, PA: ACL,2006:69-72
[67] G. Schohn, D. Cohn. Less is more: active learning with support vector machines [C]In: Proc of ICML2000. San Francisco, CA: Morgan Kaufmann,2000:839-846
[68] Z. Xu, K. Yu, V. Tresp. Representative sampling for text classification usingsupport vector machines [G] Lecture Notes in Computer Science2633: Proc ofECIR2003. Berlin: Springer,2003:393-407
[69] H.T. Nguyen, A. Smeulders. Active learning using pre-clustering [C] In: Proc ofICML2004. New York: ACM,2004:79-86
[70] D. Sanjoy, H. Daniel. Hierarchical sampling for active learning [C] In: Proc ofICML2008. New York: ACM,2008:208-215
[71] P. Donmez, J.G. Carbonell, P.N. Bennett. Dual strategy active learning [G] LectureNotes in Artificial Intelligence4701: Proc of ECML2007. Berlin: Springer,2007:116-127
[72] S. Huang, R. Jin, Z. Zhou. Active learning by querying informative andrepresentative examples [C] In: Proc of NIPS2010. Cambridge, MA: MIT Press,2011:892-900
[73] N. Abe, H. Mamitsuka. Query learning strategies using boosting and bagging [C]In: Proc of ICML1998. San Francisco, CA: Morgan Kaufmann,1998:1-9
[74] Y. Freund, R.E. Schapire. A decision-theoretic generalization of on-line learningand an application to boosting [J]. Journal of Computer and System Sciences,1997,55(1):119-139
[75] L. Breiman. Bagging predictors [J]. Machine Learning,1996,24(2):123-140
[76] S. Geman, E. Bienenstock, R. Doursat. Neural networks and the bias/variancedilemma [J]. Neural Computation,1992,4(1):1-58
[77] D. Cohn, Z. Ghahramani, M.I. Jordan. Active learning with statistical models [J].Artificial Intelligence Research,1996(4):129-145
[78] T. Zhang, F.J. Oles. A probability analysis on the value of unlabeled data forclassification problems [C] In: Proc of ICML2000. San Francisco, CA: MorganKaufmann,2000:1191-1198
[79] G. Paass, J. Kindermann. Bayesian query construction for neural network models
[C] In: Proc of NIPS1995. Cambridge, MA: MIT Press,1995:443-450
[80] S.C.H. Hoi, R. Jin, J. Zhu. Batch mode active learning and its application tomedical image classification [C] In: Proc of ICML2006. New York: ACM,2006:417-424
[81] N. Roy, A. McCallum. Toward optimal active learning through sampling estimationof error reduction[C] In: Proc of ICML2001. San Francisco, CA: MorganKaufmann,2001:441-448
[82] X. Zhu, J. Lafferty, Z. Ghahramani. Combining active learning andsemi-supervised learning using Gaussian fields and harmonic functions [C] In:Proc of ICML2003Workshop on the Continuum from Labeled to Unlabeled Data.Menlo Park, CA: AAAI Press,2003:58-65
[83] Y. Guo, R. Greiner. Optimistic active learning using mutual information [C] In:Proc of IJCAI2007. Menlo Park, CA: AAAI Press:823-829
[84] A. Sorokin, D. Forsyth. Utility data annotation with amazon mechanical turk [C] In:Proc of IEEE CVPR2008. NJ: IEEE,2008:1-8
[85] P. Donmez, J.G. Carbonell. Proactive learning: cost-sensitive active learning withmultiple imperfect oracles [C]. In: Proc of ACM CIKM2008, California,2008, pp.629-638
[86]邓超,郭茂祖.基于Tri-Training和数据剪辑的半监督聚类算法[J].软件学报,2008,19(3):663-673
[87]邓超,郭茂祖.基于自适应数据剪辑策略的Tri-training算法[J].计算机学报,2007,30(8):1213-1226
[88] V. N. Vapnik. Statistical learning theory [M]. John Wiley&Sons, Inc., New York,1998
[89] V.V. Fedorov. Theory of optimal experiments [M]. Academic Press,1972
[90] D. J. C. MacKay. Information-based objective functions for active data selection [J].Neural Computation,1992,4(4):590-604
[91] K. Fukumizu. Statistical active learning in multilayer perceptions [J]. IEEETransactions on Neural Networks,2000,11(1):17-26
[92] C. Sawade, N. Landwehr, S. Bickel, T. Scheffer. Active risk estimation [C]. In:Proc of ICML2010, New York: ACM,2010:1161-1168
[93] P. Donmez, G. Lebanon, K. Balasubramanian. Unsupervised supervised learning I:estimating classification and regression errors without labels [J], Journal ofMachine Learning Research,2010,11:1323-1351
[94] A. Beygelzimer, D. Hsu, J. Langford, T. Zhang. Agnostic active learning withoutconstraints [C]. In: Proc of NIPS2010. Cambridge, MA: MIT Press,2011:199-207
[95] M. Sugiyamy. Active learning in approximate linear regression based onconditional expectation of generalization error [J]. Journal of Machine LearningResearch,2006,7:141-166
[96] H. Shimodaira, Improving predictive inference under covariate shift by weightingthe log-likelihood function [J]. Journal of Statistical Planning and Inference,2000,90(2):227-244
[97] D. P. Wins. Robust weights and designs for biased regression models: least squaresand generalized m-estimation [J]. Journal of Statistical Planning and Inference,2000,83(2):395-412
[98] S. Geman, E. Bienenstock, R. Doursat. Neural networks and the bias/variancedilemma [J]. Neural Computation,1992,4(1):1-58
[99] B. Russell, A. Torralba, K. Murphy, W. T. Freeman. LabelMe: a database andweb-based tool for image annotation [J]. International Journal of ComputerVision,2008,77(1):157-173
[100] R. E. Fan, K. W. Chang, C. J. Hsieh, X. R. Wang, C. J. Lin. LIBLINEAR: Alibrary for large linear classification[J]. Journal of Machine Learning Research,2008,9:1871-1874
[101] K. Balasubramanian, P. Donmez, G. Lebanon. Unsupervised supervised learningII: training margin based classifiers without labels [J]. Journal of MachineLearning Research,2011,12:3119-3145
[102] V. Sheng, F. Provost, P.G. Ipeirotis. Get another label? improving data quality anddata mining using multiple, noisy labelers [C]. In: Proc of ACM SIGKDD2008.New York: ACM,2008:614-622
[103] P. Donmez, J.G. Carbonell, J. Schneider. Efficiently learning the accuracy oflabeling sources for selective sampling [C]. In: Proc of ACM SIGKDD2009.New York: ACM,2009:259-268
[104] Y. Yan, R. Rosales, G. Fung, M. Schmidt. Modeling annotator expertise: learningwhen everybody knows a bit of something [C]. In: Proc of AISTATS2010.Cambridge, MA: MIT Press,2010:932-939
[105] O. Dekel, O. Shamir. Good learners for evil teachers [C]. In: Proc of ICML2009.New York: ACM,2009:233-240
[106] Snow R, O’Connor B, Jurafsky D. Cheap and fast-but is it good? evaluatingnon-expert annotations for natural language tasks [C] In: Proc of EMNLP2008.Stroudsburg, PA: ACL,2008:254-263
[107] P. Donmez, J.G. Carbonell, J. Schneider. A probabilistic framework to learn frommultiple annotators with time-varying accuracy [C]. In: Proc of SDM2010.Philadelphia, PA: Soc for Industrial&Applied Math,2010:826-837
[108] B. Thiesson, C. Meek, D. Heckerman. Accelerating EM for large databases [J].Machine Learning,2001,45:279-299
[109] KDD CUP (2008)[M/OL]. http://www.sigkdd.org/kddcup/index.php?section=2008&method=info
[110] P. Jain, S. Vijayanarasimhan, K. Grauman. Hashing hyperplane queries to nearpoints with applications to large-scale active learning [C] In: Proc of NIPS2010.Cambridge, MA: MIT Press,2011:928-936
[111] L-J. Li, H. Su, E. Xing, Fei-Fei Li. Object bank: a high-level image representationfor scene classification&semantic feature sparsification [J]. In: Proc of NIPS2010. Cambridge, MA: MIT Press,2011:1378-1386
[112] S. Vijayanarasimhan, K. Grauman. Large-scale live active learning: trainingobject detectors with crawled data and crowds [C]. In: Proc of CVPR2011. NJ:IEEE,2011:1449-1456
[113] N. Panda, K. Goh, E.Y. Chang. Active learning in very large databases [J]. Journalof Multimedia Tools and Applications,2006,31:249-267
[114] P. Jain, S. Vijayanarasimhan, K. Grauman. Hashing hyperplane queries to nearpoints with applications to large-scale active learning [C] In: Proc of NIPS2010.Cambridge, MA: MIT Press,2011:928-936
[115] Y. Weiss, A. Torralba, R. Fergus. Spectral hashing [C] In: Proc of NIPS2009.Cambridge, MA: MIT Press,2010:1753-1760
[116] B. Kulis, K. Grauman. Kernelized locality-sensitive hashing [J]. IEEE Transactionon Pattern Analysis Machine Intelligence,2012,34:1092-1104
[117] R. Lin, D. Ross, J. Yagnik. SPEC hashing: similarity preserving algorithm forentropy-based coding [C]. In: Proc of CVPR2010. NJ: IEEE,2010:848-854
[118] J. Yagnik, D. Strelow, D. Ross, R.S. Lin. The power of comparative reasoning [C].In: Proc of ICCV2011. NJ: IEEE,2011:2431-2438
[119] Y. Gong, S. Lazebnik, A. Gordo, F. Perronnin. Iterative quantization: aprocrustean approach to learning binary codes for large-scale image retrieval[J]. Accepted by IEEE Transaction on Pattern Analysis and Machine Intelligence,2012
[120] G.X. Yuan, K.W. Chang, C.J. Hsieh, C.J. Lin. A comparison of optimizationmethods and software for large-scale L1-regularized linear classification [J].Journal of Machine Learning Research,2010,11:3183-3234
[121] M. Rastegari, C. Fang, L. Torresani. Scalable object-class retrieval withapproximate and top-k ranking [C]. In: Proc of ICCV2011. NJ: IEEE,2011:2659-2666
[122] G. Griffin, A. Holub, P. Perona. Caltech-256object category dataset [R].Computer Sciences Technical Report7694, California Institute of Technology,2007
[123] L. Torresani, M. Szummer, A. Fitzgibbon. Efficient object category recognitionusing classemes [J]. Lecture Notes in Computer Science,6311: Proc of ECCV2010. Berlin: Springer,2010:776-789
[124] A. Bergamo, L. Torresani, A. Fitzgibbon. PiCoDes: learning a compact code fornovel-category recognition [C]. In: Proc of NIPS2011. Cambridge, MA: MITPress,2012:2088-2096
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |