认知无线电中智能学习技术研究
|
摘要
在当今世界经济和社会发展信息化的潮流中，无线通信技术得到了迅速发展，各种无线业务和应用以惊人迅速增长，新的无线通信及网络技术不断涌现。无线通信和网络蓬勃发展的同时，其高速和宽带化特征使得对频谱资源的需求日益加大，同时也使得频谱资源的供需矛盾问题日益突出。认知无线电提出的从固定频谱使用到动态频谱使用的变革可有效地提高频谱利用率；同时，认知无线电强调的高度智能性符合无线通信系统和网络的发展方向。本论文关注认知无线电智能性体现的最重要环节——学习，在离线学习和在线学习两个方面开展研究，主要完成了以下具有创新性的研究成果：
1.提出一种通用的CR离线学习与决策框架，在此框架下研究神经网络（NN）和最小二乘支持向量机（LSSVM）两种学习方法的具体应用。在NN方面，提出了“直接的”径向基函数神经网络（RBF-NN）学习与决策方法；与传统“间接的”方法相比，通过在学习训练前增加优化案例搜索处理，减少了输入及输出神经元数量，降低了训练的复杂度；并可直接完成配置参数决策，提高决策实时性。在LLSVM方面，从复杂度、性能等方面对比研究了CR场景下的几种多分类LSSVM；研究了非支配排序遗传算法完成LSSVM超参数搜索以提高学习算法的普适性。仿真结果表明，RBF-NN和LSSVM方法都能提升认知无线电系统性能，非支配排序的遗传算法能够在较少进化代数内搜索出合适的超参数，基于LSSVM的离线学习具有更好的决策性能和泛化性能。
2.提出一种能促进多用户学习收敛的，基于用户聚类和可变学习速率的多Agent强化学习方法，以解决多用户在下垫式频谱共享下的信道选择与功率分配问题。首先使用分层处理分离信道选择与功率控制，采用快速最优信道搜索结合基于性能预测的用户数均衡调节方法实现信道分配；其次，使用随机博弈框架对多用户功率控制问题进行建模，引入K均值用户聚类减少博弈参与用户数量和降低单个用户的环境复杂度，并提出可变Q学习速率和策略学习速率的方法进一步促进多Agent强化学习的收敛。仿真结果表明，该方法能使多个用户的功率状态和总收益有效收敛，并且获得整体性能达到次优。
3.提出了一种在总功率资源受限条件下基于纳什议价解的方法，用于多信道多用户的信道选择与功率分配。设计了合理的纳什议价效用函数，使纳什乘积能明确表征认知无线电系统的性能指标，并证明了纳什议价解的存在性与唯一性。提出基于梯度下降思想和性能变化预测的具体迭代议价过程完成信道与功率的分配。理论分析与仿真结果表明，基于纳什议价的功率分配满足全局比例公平，信道与功率分配的迭代算法能够得到较好性能，达到系统总性能的次优解。
4.提出协作去耦合方法和跨层联合方法解决多跳认知无线网络的多层资源分配问题。协作去耦合方法首先单独完成路径选择任务，随后进行信道与功率的博弈分配；跨层联合方法则通过博弈直接对路径、信道、功率三层资源进行同时分配。两种方法都综合考虑网络层、MAC层、物理层的启发原则，引入了节点被干扰度信息和节点主动干扰度信息来辅助路径的选择；设计了基于功率允许宽度信息的Boltzmann探索来完成信道与功率选择；设计了长链路和瓶颈链路替换消除手段以进一步提高网络性能。从促进收敛角度，选用序贯博弈方法，并设计了具体的博弈过程；此外还分析了博弈的纳什均衡，讨论了两种算法的复杂度。仿真结果表明，协作去耦合方法和跨层联合方法在成功流数量、流可达速率、发射功耗性能指标上均优于简单去耦合的链路博弈、流博弈方法。
5.提出一种无需信息交互的多用户自主Q学习方法，用于实现认知无线电中的多用户动态频谱访问。该方法采用自学习方案，每个认知无线电用户不需要耗费通信资源与其他用户进行信息交互，仅通过观察自己的回报进行强化学习，定义的回报值能反应信道优劣以及信道冲突状态；设计了充分探索、倾向优势信道、冲突惩罚的学习策略，实现多用户多信道的动态频谱访问。对2用户2信道的场景，提出一种快速学习算法并证明了它能够收敛到整体回报最大。仿真结果表明，该方法能使认知无线电多用户多信道选择以大概率收敛到纳什均衡，且得到高的整体回报性能。
Along with the informational progress in current world and society, the wirelesscommunication technology has experienced rapid development. Various wirelessservices and applications increase at an amazing speed, and novel techniques in wirelesscommunication and networking emerge continuously. Simultaneously, the characteristicof wireless networks in high data rate and wide band leads to the increasing requirementfor spectrum resource, so that the contradiction between the supply and the demand ofspectrum resource appears to be more and more extrusive.
Cognitive radio (CR) proposes a reform strategy with a dynamic spectrum utilizinginstead of the fixed one, which can effectively raise the frequency utilizing efficiency.Moreover, the high intelligence emphasized by CR accords with the developmentdirection of wireless communication systems and networks.
This dissertation focuses on the learning part in CR, which should be the best oneto exhibit its intelligence. The author researches on off-line learning and on-linelearning, and the major contributions are as follows:
1. A universal learning and decision making framework for CR is proposed. Theconcrete applications of two learning methods based on the framework, i.e. aneural network (NN) method and the least square support vector machine(LSSVM) method, are investigated. In the aspect of NN, a “direct” radial basisfunction neural network (RBF-NN) based learning and decision making method isproposed. Compared with the traditional “indirect” methods, the “direct” one hasan additional process for optimal cases searching before training stage, which canreduce the number of input neurons and output neurons to decrease the trainingcomplexity, and can improve its real-time performance by means of direct decisionon parameter configuration. In the aspect of LSSVM, several multiclassclassification approaches are discussed and compared in term of complexity andperformance under the constructed CR scenario. The non-dominated sortinggenetic algorithm is adopted to implement the hyper-parameters searching ofLLSVM, so as to enhance its universality. Simulation results show that both theRBF-NN method and the LSSVM method can improve the performance of CRsystems, and that the non-dominated sorting genetic algorithm can search outsuitable hyper-parameters within a few evolutional generations, and that theLSSVM method behaves better decision performance and generalization property.
2. For solving the problem of channel allocation and power control in spectrumunderlay cognitive radios, a multi-agent enforcement learning method based onuser clustering and a variable learning rate is proposed, which can effectivelyimprove convergence of multiuser learning. Firstly, a hierarchy processing methodis used to separate channel selection and power control, and channel allocation isimplemented by fast optimal search combined with user-number balance based onperformance prediction. Secondly, stochastic game framework is adopted to modelthe multiuser power control issue. In subsequent multi-agent enforcement learning,K-means user clustering method is employed to reduce the user number in gameand single user’s environment complexity, and a variable learning rate scheme forQ learning and policy learning is proposed to promote the convergence ofmultiuser learning. Simulation results show that the method can make themultiuser’s power status and global reward converging effectively, and moreoverthe whole performance can reach sub-optimal.
3. A Nash bargaining solution based method is proposed for multi-channel multi-userchannel selection and power allocation under the condition of limited total powerin CR. A reasonable Nash bargaining utility function is designed, which makesNash product be able to explicitly express the performance of CR systems;furthermore, the existence and uniqueness of Nash bargaining solution is proved.An iteration bargaining procedure based on the idea of gradient descent andprediction of performance variation is proposed to implement allocation ofchannel and power. The theoretical analysis and experimental simulation show thatthe power allocation method based on Nash bargaining solution conforms toproportional fairness, and the iteration algorithms for channel and powerallocation can achieve sub-optimal performance of the whole systems.
4. A cooperative decoupling method and a cross-layer joint method are proposed formulti-layer resource allocation in multi-hop cognitive radio networks. In thecooperative decoupling method, the task of path choosing is accomplishedindependently, and then is the game allocation of channel and power. In thecross-layer joint method, the three-layer resource of path, channel and power issimultaneously allocated by a game process. The both methods syntheticallyemploy the heuristic principles of network layer, MAC layer and physical layer;they assist the path choosing by using the information of interference receivingdegree and interference sending degree. The Boltzmann exploration based onwidth of permitting power is designed to execute the selections of channel and power. A method of replacement and elimination of long link or bottleneck link isused to further enhance the network performance. For improving convergence, asequential game process instead of simultaneous game process is chosen and itsconcrete implement process is provided, since the former has better behavior incurrent scenario. Besides, the Nash equilibrium of the games and the complexityof two related algorithms are analyzed and discussed. Simulation results show thatthe cooperative decoupling method and the cross-layer joint method have betterperformance in the number of success flows, the achievable data transmission rateand power consumption than the cooperative link game and the local flow gamewith simple decoupling.
5. A multiuser independent Q-learning method without information interaction isproposed for multiuser dynamic spectrum accessing in cognitive radios. Themethod adopts self-learning paradigm, where each CR user performsreinforcement learning only through observing individual performance reward, sothat it can save the communication resource in exchanging information with others,and where the reward is suitably defined according to the present channel qualityand channel conflict status. A learning strategy to implement multiuser dynamicspectrum accessing is designed, which performs sufficient exploration based on acriterion of trending to the better channel while punishing the conflict channel. Fortwo users two channels scenario, a fast learning algorithm is proposed and it isproved that the algorithm can converge to the maximal total reward. Thesimulation results show that the CR system based on the proposed method inmulti-user and multi-channel selection can converge to Nash equilibrium withlarge probability and acquire high performance in the whole reward.
1.提出一种通用的CR离线学习与决策框架，在此框架下研究神经网络（NN）和最小二乘支持向量机（LSSVM）两种学习方法的具体应用。在NN方面，提出了“直接的”径向基函数神经网络（RBF-NN）学习与决策方法；与传统“间接的”方法相比，通过在学习训练前增加优化案例搜索处理，减少了输入及输出神经元数量，降低了训练的复杂度；并可直接完成配置参数决策，提高决策实时性。在LLSVM方面，从复杂度、性能等方面对比研究了CR场景下的几种多分类LSSVM；研究了非支配排序遗传算法完成LSSVM超参数搜索以提高学习算法的普适性。仿真结果表明，RBF-NN和LSSVM方法都能提升认知无线电系统性能，非支配排序的遗传算法能够在较少进化代数内搜索出合适的超参数，基于LSSVM的离线学习具有更好的决策性能和泛化性能。
2.提出一种能促进多用户学习收敛的，基于用户聚类和可变学习速率的多Agent强化学习方法，以解决多用户在下垫式频谱共享下的信道选择与功率分配问题。首先使用分层处理分离信道选择与功率控制，采用快速最优信道搜索结合基于性能预测的用户数均衡调节方法实现信道分配；其次，使用随机博弈框架对多用户功率控制问题进行建模，引入K均值用户聚类减少博弈参与用户数量和降低单个用户的环境复杂度，并提出可变Q学习速率和策略学习速率的方法进一步促进多Agent强化学习的收敛。仿真结果表明，该方法能使多个用户的功率状态和总收益有效收敛，并且获得整体性能达到次优。
3.提出了一种在总功率资源受限条件下基于纳什议价解的方法，用于多信道多用户的信道选择与功率分配。设计了合理的纳什议价效用函数，使纳什乘积能明确表征认知无线电系统的性能指标，并证明了纳什议价解的存在性与唯一性。提出基于梯度下降思想和性能变化预测的具体迭代议价过程完成信道与功率的分配。理论分析与仿真结果表明，基于纳什议价的功率分配满足全局比例公平，信道与功率分配的迭代算法能够得到较好性能，达到系统总性能的次优解。
4.提出协作去耦合方法和跨层联合方法解决多跳认知无线网络的多层资源分配问题。协作去耦合方法首先单独完成路径选择任务，随后进行信道与功率的博弈分配；跨层联合方法则通过博弈直接对路径、信道、功率三层资源进行同时分配。两种方法都综合考虑网络层、MAC层、物理层的启发原则，引入了节点被干扰度信息和节点主动干扰度信息来辅助路径的选择；设计了基于功率允许宽度信息的Boltzmann探索来完成信道与功率选择；设计了长链路和瓶颈链路替换消除手段以进一步提高网络性能。从促进收敛角度，选用序贯博弈方法，并设计了具体的博弈过程；此外还分析了博弈的纳什均衡，讨论了两种算法的复杂度。仿真结果表明，协作去耦合方法和跨层联合方法在成功流数量、流可达速率、发射功耗性能指标上均优于简单去耦合的链路博弈、流博弈方法。
5.提出一种无需信息交互的多用户自主Q学习方法，用于实现认知无线电中的多用户动态频谱访问。该方法采用自学习方案，每个认知无线电用户不需要耗费通信资源与其他用户进行信息交互，仅通过观察自己的回报进行强化学习，定义的回报值能反应信道优劣以及信道冲突状态；设计了充分探索、倾向优势信道、冲突惩罚的学习策略，实现多用户多信道的动态频谱访问。对2用户2信道的场景，提出一种快速学习算法并证明了它能够收敛到整体回报最大。仿真结果表明，该方法能使认知无线电多用户多信道选择以大概率收敛到纳什均衡，且得到高的整体回报性能。
Along with the informational progress in current world and society, the wirelesscommunication technology has experienced rapid development. Various wirelessservices and applications increase at an amazing speed, and novel techniques in wirelesscommunication and networking emerge continuously. Simultaneously, the characteristicof wireless networks in high data rate and wide band leads to the increasing requirementfor spectrum resource, so that the contradiction between the supply and the demand ofspectrum resource appears to be more and more extrusive.
Cognitive radio (CR) proposes a reform strategy with a dynamic spectrum utilizinginstead of the fixed one, which can effectively raise the frequency utilizing efficiency.Moreover, the high intelligence emphasized by CR accords with the developmentdirection of wireless communication systems and networks.
This dissertation focuses on the learning part in CR, which should be the best oneto exhibit its intelligence. The author researches on off-line learning and on-linelearning, and the major contributions are as follows:
1. A universal learning and decision making framework for CR is proposed. Theconcrete applications of two learning methods based on the framework, i.e. aneural network (NN) method and the least square support vector machine(LSSVM) method, are investigated. In the aspect of NN, a “direct” radial basisfunction neural network (RBF-NN) based learning and decision making method isproposed. Compared with the traditional “indirect” methods, the “direct” one hasan additional process for optimal cases searching before training stage, which canreduce the number of input neurons and output neurons to decrease the trainingcomplexity, and can improve its real-time performance by means of direct decisionon parameter configuration. In the aspect of LSSVM, several multiclassclassification approaches are discussed and compared in term of complexity andperformance under the constructed CR scenario. The non-dominated sortinggenetic algorithm is adopted to implement the hyper-parameters searching ofLLSVM, so as to enhance its universality. Simulation results show that both theRBF-NN method and the LSSVM method can improve the performance of CRsystems, and that the non-dominated sorting genetic algorithm can search outsuitable hyper-parameters within a few evolutional generations, and that theLSSVM method behaves better decision performance and generalization property.
2. For solving the problem of channel allocation and power control in spectrumunderlay cognitive radios, a multi-agent enforcement learning method based onuser clustering and a variable learning rate is proposed, which can effectivelyimprove convergence of multiuser learning. Firstly, a hierarchy processing methodis used to separate channel selection and power control, and channel allocation isimplemented by fast optimal search combined with user-number balance based onperformance prediction. Secondly, stochastic game framework is adopted to modelthe multiuser power control issue. In subsequent multi-agent enforcement learning,K-means user clustering method is employed to reduce the user number in gameand single user’s environment complexity, and a variable learning rate scheme forQ learning and policy learning is proposed to promote the convergence ofmultiuser learning. Simulation results show that the method can make themultiuser’s power status and global reward converging effectively, and moreoverthe whole performance can reach sub-optimal.
3. A Nash bargaining solution based method is proposed for multi-channel multi-userchannel selection and power allocation under the condition of limited total powerin CR. A reasonable Nash bargaining utility function is designed, which makesNash product be able to explicitly express the performance of CR systems;furthermore, the existence and uniqueness of Nash bargaining solution is proved.An iteration bargaining procedure based on the idea of gradient descent andprediction of performance variation is proposed to implement allocation ofchannel and power. The theoretical analysis and experimental simulation show thatthe power allocation method based on Nash bargaining solution conforms toproportional fairness, and the iteration algorithms for channel and powerallocation can achieve sub-optimal performance of the whole systems.
4. A cooperative decoupling method and a cross-layer joint method are proposed formulti-layer resource allocation in multi-hop cognitive radio networks. In thecooperative decoupling method, the task of path choosing is accomplishedindependently, and then is the game allocation of channel and power. In thecross-layer joint method, the three-layer resource of path, channel and power issimultaneously allocated by a game process. The both methods syntheticallyemploy the heuristic principles of network layer, MAC layer and physical layer;they assist the path choosing by using the information of interference receivingdegree and interference sending degree. The Boltzmann exploration based onwidth of permitting power is designed to execute the selections of channel and power. A method of replacement and elimination of long link or bottleneck link isused to further enhance the network performance. For improving convergence, asequential game process instead of simultaneous game process is chosen and itsconcrete implement process is provided, since the former has better behavior incurrent scenario. Besides, the Nash equilibrium of the games and the complexityof two related algorithms are analyzed and discussed. Simulation results show thatthe cooperative decoupling method and the cross-layer joint method have betterperformance in the number of success flows, the achievable data transmission rateand power consumption than the cooperative link game and the local flow gamewith simple decoupling.
5. A multiuser independent Q-learning method without information interaction isproposed for multiuser dynamic spectrum accessing in cognitive radios. Themethod adopts self-learning paradigm, where each CR user performsreinforcement learning only through observing individual performance reward, sothat it can save the communication resource in exchanging information with others,and where the reward is suitably defined according to the present channel qualityand channel conflict status. A learning strategy to implement multiuser dynamicspectrum accessing is designed, which performs sufficient exploration based on acriterion of trending to the better channel while punishing the conflict channel. Fortwo users two channels scenario, a fast learning algorithm is proposed and it isproved that the algorithm can converge to the maximal total reward. Thesimulation results show that the CR system based on the proposed method inmulti-user and multi-channel selection can converge to Nash equilibrium withlarge probability and acquire high performance in the whole reward.
引文
[1]张平,冯志勇,认知无线电网络,北京:科学出版社,2010年.
[2]National Telecommunications and Information Administration,“United statesfrequency allocation chart,” Available: http://www.ntia.doc.gov/osmhome/allochrt.html
[3]Federal Communications Commission,“Spectrum policy task force,” ET Docket02-135, Nov.2002.
[4]J. Mitola and G. Q. Maguire,“Cognitive radio: Making software radios morepersonal,” IEEE Personal Communications, vol.6, no.4, pp.13-18,1999.
[5]J. Mitola,“Cognitive radio: An integrated agent architecture for software definedradio,” Ph.D. dissertation, Royal Inst. of Technol., Stockholm, Sweden,2000.
[6]S. Haykin,“Cognitive radio: Brain-empowered wireless communications,” IEEEJournal on Selected Areas in Communications, vol.23, no.2, pp.201-220, Feb.2005.
[7]Federal Communications Commission. Notice of proposed rule making and order.FCC ET Docket,2003(03).
[8]T. W. Rondeau et al.,“Optimization, learning, and decision making in a cognitiveengine,” Proc. SDR Forum06Technical Conference, Orlando, FL, USA,2006.
[9]J. Strassner, N. Agoulmine and E. Lehtihet,“FOCALE-A Novel AutonomicNetworking Architecture,” TSSA Journal, vol.3, no.1, pp.64-69, May2007.
[10]R. W. Brodersen, A. Wolisz and D. Cabiric, et al.,“CORVUS: A cognitive radioapproach for usage of virtual unlicensed spectrum,” White Paper, BerkeleyWireless Research Center, UC Berkeley, Jul.2004.
[11]王金龙,吴启晖,龚玉萍,宋绯,认知无线电网络,北京:机械工业出版社,2010年.
[12]http://isisn.nsfc.gov.cn/egrantindex/funcindex/prjsearch-list
[13]I. F. Akyildiz, W. Lee, M. C. Vuran, and S. Mohanty,“NeXt generation/dynamicspectrum access/cognitive radio wireless networks: A survey,” Computer Networks,vol.50, no.13, pp.2127-2159,2006.
[14]Y. Lu, W. Guo, X. Wang, and W. Wang,“Distributed streaming compressivespectrum sensing for wide-band cognitive radio networks,” IEEE73rd VehicularTechnology Conference, Yokohama, Japan, pp.1-5,2011.
[15]Z. Bao, Q. Huang, Y. Zhai, and G. Lu,“Fast blind spectrum sensing method basedon determinant of covariance matrix,” International Conference on ComputerScience and Information Processing, Xi'an, China, pp.439-443,2012.
[16]J. Tang, S. Misra, and G. Xue,“Joint spectrum allocation and scheduling for fairspectrum sharing in cognitive radio wireless networks,” Computer Networks, vol.52, no.11, pp.2148-2158,2008.
[17]W. Lee, and I. F. Akyldiz,“A spectrum decision framework for cognitive radionetworks,” IEEE Transactions on Mobile Computing, vol.10, no.2, pp.161-174,2011.
[18]Q. Liang, G. Sun, and X. Wang,“Random leader: a distributed-centralizedspectrum sharing scheme in cognitive radios,” IEEE International Conference onCommunications, Ottawa, ON, pp.1806-1810,2012.
[19]J. M. Peha,“Sharing spectrum through spectrum policy reform and cognitiveradio,” Proceedings of the IEEE, vol.97, no.4, pp.708-719,2009.
[20]R. Blasco-Serrano, L. Jing, R. Thobaben, E. Jorsieck, A. Kliks, and M. Skoglund,“Comparison of underlay and overlay spectrum sharing strategies in MISOcognitive channels,”7th International ICST Conference on Cognitive RadioOriented Wireless Networks and Communications, Stockholm, pp.224-229,2012.
[21]J. E. Suris, L. A. Dasilva, H. Zhu, and A. B. Mackenzie,“Cooperative game theoryfor distributed spectrum sharing,” IEEE International Conference onCommunications, Glasgow, Scotland, pp.5282-5287,2007.
[22]J. Neel,“Analysis and design of cognitive radio networks and distributed radioresource management algorithms,” Ph.D. dissertation, Virginia Tech., Virginia,USA, Sep.2006.
[23]T. W. Rondeau, B. Le, C. J. Rieser, and C. W. Bostian,“Cognitive radios withgenetic algorithms: Intelligent control of software defined radios,” in SDR’04Forum Technical Conference, Phoenix, AZ, pp. C3–C8,2004.
[24]S. K. Park, Y. Shin, and W. C. Lee,“Goal-pareto based nsga for optimalinreconfiguration of cognitive radio systems,”2nd International Conference onCognitive Radio Oriented Wireless Networks and Communications, Orlando, FL,USA, pp.147–153, Aug.2007
[25]赵知劲,郑仕链,尚俊娜,孔宪正,“基于量子遗传算法的认知无线电决策引擎研究,”物理学报, vol.56, no.11, pp.6760-6766,2007.
[26]J. M. Kim, S. H. Sohn, N. Han, G. Zheng, Y. M. Kim, and J. K. Lee,“Cognitiveradio software testbed using dual optimization in genetic algorithm,”3rdInternational Conference on Cognitive Radio Oriented Wireless Networks andCommunications, Singapore, pp.1–6, May2008.
[27]C. J. Rieser,“Biologically inspired cognitive radio engine model utilizingdistributed genetic algorithms for secure and robust wireless communications andnetworking,” Ph.D. dissertation, Virginia Tech., Virginia, USA, Aug.2004
[28]Z. Song, B. Shen, Z. Zhou, and K. S. Kwak,“Improved ant routing algorithm incognitive radio networks,”9th International Symposium on Communications andInformation Technology, Icheon, pp.110-114, Sept.2009.
[29]N. Zhao, S. Li, and Z. Wu,“Cognitive radio engine design based on ant colonyoptimization,” Wireless Personal Communications, vol.65, no.1, pp.15-24, Jul.2012.
[30]Z. Zhao, S. Xu and S. Zheng et al.,“Cognitive radio adaptation using particleswarm optimization,” Wireless Communications and Mobile Computing, vol.9, no.7, pp.875-881, Jul.2009.
[31]T. W. Rondeau,“Application of artificial intelligence to wireless communications,”Ph.D. dissertation, Virginia Tech., Virginia, USA, Sept.2007.
[32]A. He, J. Gaeddert, K. Bae, T. R. Newman, J. H. Reed, et al,“Development of acase-based reasoning cognitive engine for IEEE802.22WRAN applications,”ACM SIGMOBILE Mobile Computing and Communications Review, vol.13, no.2,pp.37-48,2009.
[33]L. Morales-Tirado, J. E. Suris-Pietri, and J. H. Reed,“A hybrid cognitive enginefor improving coverage in3G wireless networks,” IEEE International Conferenceon Communications Workshops, Dresden, pp.1-5, Jun.2009.
[34]T. Yucek, and H. Arslan,“Spectrum characterization for opportunistic cognitiveradio systems,” Military Communications Conference, Washington, DC, pp.1-5,Oct.2006.
[35]E. Adamopoulou, K. Demestichas, P. Demestichas and M. Theologou,“Enhancingcognitive radio systems with robust reasoning，” International Journal ofCommunication Systems, vol.21, no.3, pp.311-330, Mar.2008.
[36]P. Demestichas, A. Katidiotis, K. A. Tsagkaris, E. F. Adamopoulou, and K. P.Demestichas,“Enhancing Channel Estimation in Cognitive Radio Systems bymeans of Bayesian Networks,” Wireless Personal Communications, vol.49, no.1,pp.87-105, Apr.2009.
[37]Z. Zhang and X. Xie,“Intelligent cognitive radio: Research on learning andevaluation of cr based on neural network,” ITI5th International Conference onInformation and Communications Technology, Dhaka, Bangladesh, pp.33-37, Dec.2007.
[38]K. Tsagkaris, A. Katidiotis, and P. Demestichas,“Neural network-based learningschemes for cognitive radio systems,” Computer Communications, vol.31, pp.3394–3404, Sept.2008.
[39]N. Baldo, and M. Zorzi,“Learning and adaptation in cognitive radios using neuralnetworks,”5th IEEE Consumer Communications and Networking Conference, LasVegas, NV, pp.998-1003, Feb2008.
[40]W. Troy, C. S. Douglas, and G. Dirk,“A statistical method for reconfiguration ofcognitive radios,” IEEE Wireless Communications, vol.14, no.4, pp.34-40,2007.
[41]D. Fudenberg, and T. Jean, Game Theory, MIT Press, Cambridge, MA,1991.
[42]B. Wang, Y. Wu, and K. J. R. Liu,“Game theory for cognitive radio networks: Anoverview,” Computer Networks, vol.54, no.14, pp.2537-2561,2011.
[43]L. Busoniu, R. Babuska, and B. D. Schutter,“A comprehensive survey ofmultiagent reinforcement learning,” IEEE Transactions on Systems, Man, andCybernetics, vol.38, no.2, pp.156-172,2008.
[44]赵志宏,高阳,骆斌,陈世福,“多Agent系统中强化学习的研究现状和发展趋势,”计算机科学, vol.31, no.3, pp.23-27,2004.
[45]C. Claus, and C. Boutilier,“The dynamic of reinforcement learning in cooperativemultiagent systems,” Proceedings of the15th National Conference on ArtificialIntelligence, Madison, USA, pp.746-762,1998.
[46]A. Greenwald, and K. Hall,“Correlated-Q learning,” Proceedings of the20thInternational Conference on Machine Learning, Washington DC, pp.242-249,2003.
[47]R. Etkin, A. Parekh, and D. Tse,“Spectrum sharing for unlicensed bands,” IEEEJournal on Selected Areas in Communications, vol.25, no.3, pp.517-528, Apr.2007.
[48]M. Schaar, and F. Fu,“Spectrum access games and strategic learning in cognitiveradio networks for delay-critical applications,” Proceedings of the IEEE, vol.97,no.4, pp.720-740, Apr.2009.
[49]H. Liu, A. B. MacKenzie, and B. Krishnamachari,“Bargaining to improve channelsharing between selfish cognitive radios,” IEEE Global TelecommunicationsConference, Honolulu, HI, pp.1-7, Nov.2009.
[50]C. U. Saraydar, N. B. Mandayam, and D. J. Goodman,"Efficient power control viapricing in wireless data networks," IEEE Transactions on Communications, vol.50,no.2, pp.291-303, Feb.2002
[51]B. Latifa, Z. Gao, and S. Liu,“No-regret learning for simultaneous power controland channel allocation in cognitive radio networks,” Computing, Communicationsand Applications Conference, Hong Kong, pp.267-271, Jan.2012.
[52]C. Yang, J. Li, and Z. Tian,“Optimal Power Control for Cognitive Radio NetworksUnder Coupled Interference Constraints: A Cooperative Game-TheoreticPerspective,” IEEE Transactions on Vehicular Technology, vol.59, no.4, pp.1696-1706, May2010.
[53]N. Omidvar, and B. H. Khalaj,“A game theoretic approach for power allocation inthe downlink of cognitive radio networks,” IEEE16th International Workshop onComputer Aided Modeling and Design of Communication Links and Networks,Kyoto, Japan, pp.158-162,2011.
[54]J. W. Lee, A. Tang, J. Huang, M. Chiang, and A. R. Calderbank,“ReverseEngineering MAC: A Non-Cooperative Game Model,” IEEE Journal on SelectedAreas in Communications, vol.24, no.6, pp.1135-1147, Aug.2007.
[55]C. Zhou, and C. Chigan,“A game theoretic DSA-driven MAC framework forcognitive radio networks,” IEEE International Conference on Communications, pp.4165–4169, May2008.
[56]L. M. Law, J. Huang, M. Liu, and S. Y. R. Li,“Price of anarchy for cognitive MACgames,” IEEE Global Telecommunications Conference, Honolulu, HI, pp.1-6, Nov.2009.
[57]R. La and V. Anantharam,“Optimal routing control: repeated game approach,”IEEE Transactions on Automatic Control, vol.47, no.3, pp.437-450, Mar.2002.
[58]K. Wang, W. Wong, and T. Y. Chai,“A MANET routing protocol using Q-learningmethod integrated with Bayesian network,” IEEE International Conference onCommunication Systems, Singapore, pp.270-274, Nov.2012.
[59]A. Tang, J. Wang, S. H. Low, and M. Chiang,“Equilibrium of heterogeneouscongestion control protocols: Existence and Uniqueness,” IEEE/ACMTransactions on Networking, vol.15no.4, pp.824-837, Oct.2007.
[60]C. George, and K. Elias,“The price of anarchy of finite congestion games,”Proceedings of the thirty-seventh annual ACM symposium on Theory of computing,New York, USA, pp.67-73,2005.
[61]B. Wang, Z. Ji, and K.J.R. Liu,“Self-learning repeated game framework fordistributed primary-prioritized dynamic spectrum access,”2nd IEEE Workshop onNetworking Technologies for Software Define Radio Networks, San Diego, USA pp.631–638,2007.
[62]Y. Wu, B. Wang, K.J.R. Liu, and T.C. Clancy,“Repeated open spectrum sharinggame with cheat-proof strategies,” IEEE Transactions on WirelessCommunications, vol.8, no.4, pp.1922–1933,2009.
[63]M. Canales, J. Ortin, and J. R. Gallego,“Game theoretic approach for end-to-endresource allocation in multihop cognitive radio networks,” IEEE CommunicationsLetters, vol.16, no.5, pp.654-657,2012.
[64]F. Fu, and M. Schaar,“Learning to compete for resources in wireless stochasticgames,” IEEE Transactions on Vehicular Technology, vol.58, no.4, pp.1904-1919,May2009.
[65]Q. Ni, and C. C. Zarakovitis,“Nash bargaining game theoretic scheduling for jointchannel and power allocation in cognitive radio systems,” IEEE Journal onSelected Areas in Communications, vol.30, no.1, pp.70-81, Jan.2012.
[66]A. A. Daoud, T. Alpcan, S. Agarwal, and M. Alanyali,“A stackelberg game forpricing uplink power in wide-band cognitive radio networks,”47th IEEEConference on Decision and Control, Cancun, pp.1422-1427, Dec.2008.
[67]Y. Teng, Y. Zhang, F. Niu, C. Dai and M. Song,“Reinforcement learning basedauction algorithm for dynamic spectrum access in cognitive radio networks,” IEEE72nd Vehicular Technology Conference Fall, Ottawa, pp.1-5, Sept.2010.
[68]H. Zhu, C. Pandana, and K. J. R. Liu,“Distributive opportunistic spectrum accessfor cognitive radio using correlated equilibrium and no-regret learning,” IEEEWireless Communications and Networking Conference, Kowloon, pp.11-15, Mar.2007.
[69] H. LI,“Multi-agent Q-learning for Aloha-like spectrum access in cognitive radiosystems,” Fifth IEEE Workshop on Networking Technologies for Software DefinedRadio Networks, Boston, USA, pp.1-6, Jun.2010.
[70]杨春刚,认知无线网络中基于博弈论的功率控制研究,博士论文,西安电子科技大学,Apr.2011.
[71]Y. Zhao, S. Mao, J. O. Neel, and J. H. Reed,“Performance evaluation of cognitiveradios: metrics, utility functions, and methodology,” Proceedings of the IEEE, vol.97, no.4, pp.642-659, Apr.2009.
[72]P. Zhou, Y. Chang, and J. A. Copeland,“Learning through reinforcement forrepeated power control game in cognitive radio networks,” IEEE GlobalCommunications Conference, Piscataway, pp.1-6, Dec.2010.
[73]P. Zhou, Y. Chang, and J. A. Copeland,“Reinforcement learning for repeatedpower control game in cognitive radio networks,” IEEE Journal on Selected Areasin Communications, vol.30, no.1, pp.54-69,2012.
[74]Q. Wang, and H. Zheng,“Route and spectrum selection in dynamic spectrumnetworks,” IEEE Consumer Communications and Networking Conference,LasVegas, pp.1-5,2006.
[75]T. Nadkar, V. Thumar, G. P. S. Tej, S. N. Merchant, and U. B. Desai,“Distributedpower allocation for secondary users in a cognitive radio scenario,” IEEETransactions on Wireless Communications, vol.11, no.4, pp.1576-1586, Apr.2012.
[76]Q. H. QH Mahmoud, Cognitive networks, West Sussex, England: Wiley,2007.
[77]M. Zeidenberg, Neural networks in artificial intelligence. NJ, USA: Ellis HorwoodLimited.,1990.
[78]A. Fehske, J. Gaeddert, and J. Reed,“A new approach to signal classification usingspectral correlation and neural networks,” First IEEE International Symposium onNew Frontiers in Dynamic Spectrum Access Networks, Baltimore, USA, pp.144-150, Nov.2005.
[79]V. K. Tumuluru, P. Wang, and D. Niyato,“A neural network based spectrumprediction scheme for cognitive radio,” IEEE International Conference onCommunications, Cape Town, pp.1-5, May2010.
[80]G. Ganesan, and Y. Li,“Cooperative spectrum sensing in cognitive radio, part I:Two user networks,” IEEE Transactions on Wireless Communications, vol.6, no.6,pp.2204-2213, Jun.2007.
[81]S. Haykin, Neural networks and learning machines, New Jersey: Prentice-Hall，2008.
[82]V. N. Vapnik, The nature of statistical learning theory, New York: Springer-Verlag,1995.
[83]C. Cortes and V. N. Vapnik,“Support-Vector networks,” Machine Learning, vol.40, no.3, pp.273-297, Sept.1995.
[84]J. A. K. Suykens and J. Vandewalle,“Least squares support vector machineclassifiers,” Neural Processing Letters, vol.9, no.3, pp.293-300, Jun.1999.
[85]B. Rozovskii and M. Yor, Stochastic approximation and recursive algorithms andapplications, New York, USA: Springer,2003.
[86]L. Shapley,“Stochastic games,” Proceedings of the National Academy of Sciencesof the United States of America, vol.39, no.10, pp.1095-1100,1953.
[87]A. Neyman，and S. Sorin, Stochastic games and applications, Dordrecht: KluwerAcademic Press,2003.
[88] J. Filar, K. Vrieze, and O. Vrieze, Competitive markov decision processes, Berlin:Springer,1997.
[89]M. L. Minsky,“Theory of Neural-Analog Reinforcement Systems and itsApplication to the Brain-Model Problem,” Ph.D. dissertation, PrincetonUniversity,1954.
[90]M. Waltz, and K. S. Fu,“A heuristic approach to reinforcement learning controlsystems,” IEEE Transactions on Automatic Control, vol.10, no.4, pp.390-398,Oct.1965.
[91]C. J. C. H. Watkins,“Learning from delayed rewards,” Ph.D. dissertation,University of Cambridge, England,1989.
[92]C. J. C. H. Watkin, and P. Dayan,“Q-learning,” Machine learning, vol.8, no.3-4, pp.279-292, May1992.
[93]江虹,伍春,马景辉,“最小二乘SVM的CR认知引擎设计,”电子科技大学学报, vol.40, no.1, pp.41-46,2011.
[94]A. Ginsberg, J. D. Poston, and W. D. Horne,“Experiments in Cognitive Radio andDynamic Spectrum Access using An Ontology-Rule Hybrid Architecture,” SecondInternational Conference on Rules and Rule Markup Languages for the SemanticWeb, athens, GA, USA, Nov.2006.
[95] D. L. McGuinness, and F. Harmelen,“OWL Web Ontology Language Overview,”Available: http://www.w3.org/TR/owl-features/.
[96]S. B. Kotsiantis, D. Kanellopoulos and P. E. Pintelas,“Data preprocessing forsupervised leaning,” International Journal of Computer Science, vol.1, no.2, pp.111-117,2006.
[97]Weston J, Watkins C,“Multi-class support vector machines,” Technical Report,Royal Holloway, University of London,1998.
[98]T. V. Gestel, J. A. K. Suykens, B. Baesens, S. Viaene, J. Vanthienen, G. Dedene, B.D. Moor, and J. Vandewalle,“Benchmarking least squares support vector machineclassifiers,” Machine Learning, vol.54, no.1, pp.5-32, Nov.2004.
[99]C. Demirkesen and C. Hocine,“A comparison of multiclass SVM methods for realworld natural scenes,” Advanced Concepts for Intelligent Vision Systems LectureNotes in Computer Science, vol.5259, pp.752-763, Oct.2008.
[100]P. Murugan, S. Kannan and S. Baskar,“Application of NSGA-II Algorithm toSingle-Objective Transmission Constrained Generation Expansion Planning,”IEEE Transactions on Power Systems, vol.24, no.4, pp.1790-1797,2009.
[101]A. Goldsmith, S.A. Jafar, I. Maric, and S. Srinivasa,“Breaking spectrum gridlockwith cognitive radios: An information theoretic perspective,” Proceedings of theIEEE, vol.97, no.5, pp.894-914, May2009.
[102]R. Blasco-Serrano, J. Lv, R. Thobaben, E. Jorswieck, and M. Skoglund,“Comparison of underlay and overlay spectrum sharing strategies in MISOcognitive channels,”7th International ICST Conference on Cognitive RadioOriented Wireless Networks and Communications, Stockholm, pp.224-229, Jun.2012.
[103]M. Maskery, V. Krishnamurthy, and Q. Zhao,“Decentralized dynamic spectrumcccess for cognitive radios: cooperative design of a non-cooperative game,” IEEETransactions on Communications, vol.57, no.2, pp.459-469, Feb.2009.
[104]A. Attar, M. R. Nakhai, and A. H. Aghvami,“Cognitive radio game for secondaryspectrum access problem,” IEEE Transactions on Wireless Communications, vol.8, no.4, pp.2121-2131,2009.
[105]C. Luo, F. R. Yu, H. Ji, and V. C. M. Leung,“Optimal channel access for TCPperformance improvement in cognitive radio networks,” Wireless Networks, vol.17, no.2, pp.479-492, Feb.2011.
[106]C. Yang, and J. Li,“Mixed-strategy based discrete power control approach forcognitive radios: a matrix game-theoretic framework,”2nd InternationalConference on Future Computer and Communication, Wuhan, vol.3, pp.806-810, May2010.
[107]H. Yu, L. Gao, Z. Li, X. Wang, and E. Hossain,“Pricing for uplink power controlin cognitive radio networks,” IEEE Transactions on Vehicular Technology, vol.59, no.4, pp.1769-1778, May2010.
[108]L. Busoniu, R. Babuska, B. Schutter,“A comprehensive survey of multiagentreinforcement learning,” IEEE Transactions on Systems, vol.38, no.2, pp.156-172,2008.
[109]J. Wang, and X. Su,“An improved K-Means clustering algorithm,” IEEE3rdInternational Conference on Communication Software and Networks, Xuzhou, pp.44-46,2011.
[110]C. Claus and C. Boutilier,“The dynamics of reinforcement learning incooperative multiagent systems,”15th National Conference on ArtificialIntelligence and Tenth Innovative Applications of Artificial IntelligenceConference, Madison, WI, pp.746-752, Jul.1998.
[111]M. L. Littman,“Markov games as a framework for multi-agent reinforcementlearning,”11th International Conference on Machine Learning, New Brunswick,NJ, pp.157–163. Jul.1994.
[112]J. Hu and M. P. Wellman,“Multiagent reinforcement learning: Theoreticalframework and an algorithm,” in Proc.15th International Conference onMachine Learning, Madison, WI, pp.242-250, Jul.1998.
[113]A. Greenwald and K. Hall,“Correlated-Q learning,”20th InternationalConference on Machine Learning, Washington, DC, pp.242–249, Aug.2003.
[114]M. Bowling, and M. Veloso,“Multiagent learning using a variable learning rate,”Artificial Intelligence, vol.136, no.2, pp.215-220,2002.
[115]C. Goochul, S. Sridharan, S. Vishwanath, and S. H. Chan,“On the capacity ofoverlay cognitive radios with partial cognition,” IEEE Transactions onInformation Theory, vol.58, no.5, pp.2935-2949,2012.
[116]R. Zhang, Y. Liang, and S. Cui,“Dynamic resource allocation in cognitive radionetworks,” IEEE Signal Processing Magazine, vol.27, no.3, pp.102-114,2010.
[117]N. Nie, and C. Comaniciu,“Adaptive channel allocation spectrum etiquette forcognitive radio networks,” First IEEE International Symposium on NewFrontiers in Dynamic Spectrum Access Networks, Baltimore, pp.269-278,2005.
[118]X. Yong, B. Guoan, and D. Niyato,“A simple distributed power control algorithmfor cognitive radio networks,” IEEE Transactions on Wireless Communications,vol.10, no.11, pp.3594-3600,2011.
[119]C. Yang, J. Li, and Z. Tian,“Optimal power control for cognitive radio networksunder coupled interference constraints: a cooperative game-theoreticperspective,” IEEE Transactions on Vehicular Technology, vol.59, no.4, pp.1696-1706,2010.
[120]S. Yang, Z. Chi, and F. Yuguang,“Joint channel and power allocation in wirelessmesh networks: a game theoretical perspective,” IEEE Journal on Selected Areasin Communications, vol.26, no.7, pp.1149-1159,2008.
[121]Y. E. Lin, K. H. Liu, and H. Y. Hsieh,“Design of power control protocols forspectrum sharing In cognitive radio networks: a game theoretic perspective,”IEEE International Conference on Communications, Cape Town, pp.23-27,2010.
[122]Z. Han, Z. Ji, and K. J. R. Liu,“Non-cooperative resource competition game byvirtual referee in multi-cell OFDMA networks,” IEEE Journal on Selected Areasin Communications, vol.25, no.6, pp.1079–1090,2007.
[123]A. Muthoo, Bargaining Theory with Applications, Cambridge, UK: CambridgeUniversity Press,1999.
[124]J. Nash,“The Bargaining Problem,” Econometrica, vol.18, no.2, pp.155-162,Apr.1950.
[125]M. Canales, and J. R. Gallego,“Distributed channel allocation and power controlin cognitive radio networks using game theory,” IEEE Vehicular TechnologyConference, San Francisco, CA, pp.1-5,2011.
[126]I. Ganchev, Z. Ji, and M. O'Droma,“A realization of cognitive pilot channelsthrough wireless billboard channel infrastructure for cognitive radio,”2nd BalticCongress on Future Internet Communications, Vilnius, pp.19-25,2012.
[127]V. Stavroulaki, K. Tsagkaris, P. Demestichas, J. Gebert, M. Mueck, A. Schmidt, R.Ferrus, O. Sallent, M. Filo, C. Mouton, and L. Rakotoharison,“Cognitive controlchannels: from concept to identification of implementation options,” IEEECommunications Magazine, vol.50, no.7, pp.96-108,2012.
[128]P. Gupta, and P. R. KumarThe,“Capacity of Wireless Networks,” IEEETransactions on Information Theory, vol.46, no.2, pp.388-404,2000.
[129]M. Canales, J. R. Gallego, and R. Ciria,“Distributed channel allocation andpower control in cognitive radio networks using game theory,”[C].74th IEEEVehicular Technology Conference, San Francisco, pp.1-5,2011.
[130]A. Abrardo, A. Alessio, P. Detti, and M. Moretti,“Centralized radio resourceallocation for OFDMA cellular systems,” IEEE International Conference onCommunications, Glasgow, UK, pp.5738-2743, Jun.2007.
[131]J. Nie and S. Haykin,“A Q-learning-based dynamic channel assignmenttechnique for mobile communication systems,” IEEE Transactions on VehicularTechnology, vol.48, no.5, pp.1676–1687,1999.
[132]S. Xergias, N. Passas, and A. K. Salkintzis,“Centralized resource allocation for
multimedia traffic in IEEE802.16mesh networks,” Proceedings of the IEEE, vol.
96, no.1, pp.54-63, Jan.2008.
[2]National Telecommunications and Information Administration,“United statesfrequency allocation chart,” Available: http://www.ntia.doc.gov/osmhome/allochrt.html
[3]Federal Communications Commission,“Spectrum policy task force,” ET Docket02-135, Nov.2002.
[4]J. Mitola and G. Q. Maguire,“Cognitive radio: Making software radios morepersonal,” IEEE Personal Communications, vol.6, no.4, pp.13-18,1999.
[5]J. Mitola,“Cognitive radio: An integrated agent architecture for software definedradio,” Ph.D. dissertation, Royal Inst. of Technol., Stockholm, Sweden,2000.
[6]S. Haykin,“Cognitive radio: Brain-empowered wireless communications,” IEEEJournal on Selected Areas in Communications, vol.23, no.2, pp.201-220, Feb.2005.
[7]Federal Communications Commission. Notice of proposed rule making and order.FCC ET Docket,2003(03).
[8]T. W. Rondeau et al.,“Optimization, learning, and decision making in a cognitiveengine,” Proc. SDR Forum06Technical Conference, Orlando, FL, USA,2006.
[9]J. Strassner, N. Agoulmine and E. Lehtihet,“FOCALE-A Novel AutonomicNetworking Architecture,” TSSA Journal, vol.3, no.1, pp.64-69, May2007.
[10]R. W. Brodersen, A. Wolisz and D. Cabiric, et al.,“CORVUS: A cognitive radioapproach for usage of virtual unlicensed spectrum,” White Paper, BerkeleyWireless Research Center, UC Berkeley, Jul.2004.
[11]王金龙,吴启晖,龚玉萍,宋绯,认知无线电网络,北京:机械工业出版社,2010年.
[12]http://isisn.nsfc.gov.cn/egrantindex/funcindex/prjsearch-list
[13]I. F. Akyildiz, W. Lee, M. C. Vuran, and S. Mohanty,“NeXt generation/dynamicspectrum access/cognitive radio wireless networks: A survey,” Computer Networks,vol.50, no.13, pp.2127-2159,2006.
[14]Y. Lu, W. Guo, X. Wang, and W. Wang,“Distributed streaming compressivespectrum sensing for wide-band cognitive radio networks,” IEEE73rd VehicularTechnology Conference, Yokohama, Japan, pp.1-5,2011.
[15]Z. Bao, Q. Huang, Y. Zhai, and G. Lu,“Fast blind spectrum sensing method basedon determinant of covariance matrix,” International Conference on ComputerScience and Information Processing, Xi'an, China, pp.439-443,2012.
[16]J. Tang, S. Misra, and G. Xue,“Joint spectrum allocation and scheduling for fairspectrum sharing in cognitive radio wireless networks,” Computer Networks, vol.52, no.11, pp.2148-2158,2008.
[17]W. Lee, and I. F. Akyldiz,“A spectrum decision framework for cognitive radionetworks,” IEEE Transactions on Mobile Computing, vol.10, no.2, pp.161-174,2011.
[18]Q. Liang, G. Sun, and X. Wang,“Random leader: a distributed-centralizedspectrum sharing scheme in cognitive radios,” IEEE International Conference onCommunications, Ottawa, ON, pp.1806-1810,2012.
[19]J. M. Peha,“Sharing spectrum through spectrum policy reform and cognitiveradio,” Proceedings of the IEEE, vol.97, no.4, pp.708-719,2009.
[20]R. Blasco-Serrano, L. Jing, R. Thobaben, E. Jorsieck, A. Kliks, and M. Skoglund,“Comparison of underlay and overlay spectrum sharing strategies in MISOcognitive channels,”7th International ICST Conference on Cognitive RadioOriented Wireless Networks and Communications, Stockholm, pp.224-229,2012.
[21]J. E. Suris, L. A. Dasilva, H. Zhu, and A. B. Mackenzie,“Cooperative game theoryfor distributed spectrum sharing,” IEEE International Conference onCommunications, Glasgow, Scotland, pp.5282-5287,2007.
[22]J. Neel,“Analysis and design of cognitive radio networks and distributed radioresource management algorithms,” Ph.D. dissertation, Virginia Tech., Virginia,USA, Sep.2006.
[23]T. W. Rondeau, B. Le, C. J. Rieser, and C. W. Bostian,“Cognitive radios withgenetic algorithms: Intelligent control of software defined radios,” in SDR’04Forum Technical Conference, Phoenix, AZ, pp. C3–C8,2004.
[24]S. K. Park, Y. Shin, and W. C. Lee,“Goal-pareto based nsga for optimalinreconfiguration of cognitive radio systems,”2nd International Conference onCognitive Radio Oriented Wireless Networks and Communications, Orlando, FL,USA, pp.147–153, Aug.2007
[25]赵知劲,郑仕链,尚俊娜,孔宪正,“基于量子遗传算法的认知无线电决策引擎研究,”物理学报, vol.56, no.11, pp.6760-6766,2007.
[26]J. M. Kim, S. H. Sohn, N. Han, G. Zheng, Y. M. Kim, and J. K. Lee,“Cognitiveradio software testbed using dual optimization in genetic algorithm,”3rdInternational Conference on Cognitive Radio Oriented Wireless Networks andCommunications, Singapore, pp.1–6, May2008.
[27]C. J. Rieser,“Biologically inspired cognitive radio engine model utilizingdistributed genetic algorithms for secure and robust wireless communications andnetworking,” Ph.D. dissertation, Virginia Tech., Virginia, USA, Aug.2004
[28]Z. Song, B. Shen, Z. Zhou, and K. S. Kwak,“Improved ant routing algorithm incognitive radio networks,”9th International Symposium on Communications andInformation Technology, Icheon, pp.110-114, Sept.2009.
[29]N. Zhao, S. Li, and Z. Wu,“Cognitive radio engine design based on ant colonyoptimization,” Wireless Personal Communications, vol.65, no.1, pp.15-24, Jul.2012.
[30]Z. Zhao, S. Xu and S. Zheng et al.,“Cognitive radio adaptation using particleswarm optimization,” Wireless Communications and Mobile Computing, vol.9, no.7, pp.875-881, Jul.2009.
[31]T. W. Rondeau,“Application of artificial intelligence to wireless communications,”Ph.D. dissertation, Virginia Tech., Virginia, USA, Sept.2007.
[32]A. He, J. Gaeddert, K. Bae, T. R. Newman, J. H. Reed, et al,“Development of acase-based reasoning cognitive engine for IEEE802.22WRAN applications,”ACM SIGMOBILE Mobile Computing and Communications Review, vol.13, no.2,pp.37-48,2009.
[33]L. Morales-Tirado, J. E. Suris-Pietri, and J. H. Reed,“A hybrid cognitive enginefor improving coverage in3G wireless networks,” IEEE International Conferenceon Communications Workshops, Dresden, pp.1-5, Jun.2009.
[34]T. Yucek, and H. Arslan,“Spectrum characterization for opportunistic cognitiveradio systems,” Military Communications Conference, Washington, DC, pp.1-5,Oct.2006.
[35]E. Adamopoulou, K. Demestichas, P. Demestichas and M. Theologou,“Enhancingcognitive radio systems with robust reasoning，” International Journal ofCommunication Systems, vol.21, no.3, pp.311-330, Mar.2008.
[36]P. Demestichas, A. Katidiotis, K. A. Tsagkaris, E. F. Adamopoulou, and K. P.Demestichas,“Enhancing Channel Estimation in Cognitive Radio Systems bymeans of Bayesian Networks,” Wireless Personal Communications, vol.49, no.1,pp.87-105, Apr.2009.
[37]Z. Zhang and X. Xie,“Intelligent cognitive radio: Research on learning andevaluation of cr based on neural network,” ITI5th International Conference onInformation and Communications Technology, Dhaka, Bangladesh, pp.33-37, Dec.2007.
[38]K. Tsagkaris, A. Katidiotis, and P. Demestichas,“Neural network-based learningschemes for cognitive radio systems,” Computer Communications, vol.31, pp.3394–3404, Sept.2008.
[39]N. Baldo, and M. Zorzi,“Learning and adaptation in cognitive radios using neuralnetworks,”5th IEEE Consumer Communications and Networking Conference, LasVegas, NV, pp.998-1003, Feb2008.
[40]W. Troy, C. S. Douglas, and G. Dirk,“A statistical method for reconfiguration ofcognitive radios,” IEEE Wireless Communications, vol.14, no.4, pp.34-40,2007.
[41]D. Fudenberg, and T. Jean, Game Theory, MIT Press, Cambridge, MA,1991.
[42]B. Wang, Y. Wu, and K. J. R. Liu,“Game theory for cognitive radio networks: Anoverview,” Computer Networks, vol.54, no.14, pp.2537-2561,2011.
[43]L. Busoniu, R. Babuska, and B. D. Schutter,“A comprehensive survey ofmultiagent reinforcement learning,” IEEE Transactions on Systems, Man, andCybernetics, vol.38, no.2, pp.156-172,2008.
[44]赵志宏,高阳,骆斌,陈世福,“多Agent系统中强化学习的研究现状和发展趋势,”计算机科学, vol.31, no.3, pp.23-27,2004.
[45]C. Claus, and C. Boutilier,“The dynamic of reinforcement learning in cooperativemultiagent systems,” Proceedings of the15th National Conference on ArtificialIntelligence, Madison, USA, pp.746-762,1998.
[46]A. Greenwald, and K. Hall,“Correlated-Q learning,” Proceedings of the20thInternational Conference on Machine Learning, Washington DC, pp.242-249,2003.
[47]R. Etkin, A. Parekh, and D. Tse,“Spectrum sharing for unlicensed bands,” IEEEJournal on Selected Areas in Communications, vol.25, no.3, pp.517-528, Apr.2007.
[48]M. Schaar, and F. Fu,“Spectrum access games and strategic learning in cognitiveradio networks for delay-critical applications,” Proceedings of the IEEE, vol.97,no.4, pp.720-740, Apr.2009.
[49]H. Liu, A. B. MacKenzie, and B. Krishnamachari,“Bargaining to improve channelsharing between selfish cognitive radios,” IEEE Global TelecommunicationsConference, Honolulu, HI, pp.1-7, Nov.2009.
[50]C. U. Saraydar, N. B. Mandayam, and D. J. Goodman,"Efficient power control viapricing in wireless data networks," IEEE Transactions on Communications, vol.50,no.2, pp.291-303, Feb.2002
[51]B. Latifa, Z. Gao, and S. Liu,“No-regret learning for simultaneous power controland channel allocation in cognitive radio networks,” Computing, Communicationsand Applications Conference, Hong Kong, pp.267-271, Jan.2012.
[52]C. Yang, J. Li, and Z. Tian,“Optimal Power Control for Cognitive Radio NetworksUnder Coupled Interference Constraints: A Cooperative Game-TheoreticPerspective,” IEEE Transactions on Vehicular Technology, vol.59, no.4, pp.1696-1706, May2010.
[53]N. Omidvar, and B. H. Khalaj,“A game theoretic approach for power allocation inthe downlink of cognitive radio networks,” IEEE16th International Workshop onComputer Aided Modeling and Design of Communication Links and Networks,Kyoto, Japan, pp.158-162,2011.
[54]J. W. Lee, A. Tang, J. Huang, M. Chiang, and A. R. Calderbank,“ReverseEngineering MAC: A Non-Cooperative Game Model,” IEEE Journal on SelectedAreas in Communications, vol.24, no.6, pp.1135-1147, Aug.2007.
[55]C. Zhou, and C. Chigan,“A game theoretic DSA-driven MAC framework forcognitive radio networks,” IEEE International Conference on Communications, pp.4165–4169, May2008.
[56]L. M. Law, J. Huang, M. Liu, and S. Y. R. Li,“Price of anarchy for cognitive MACgames,” IEEE Global Telecommunications Conference, Honolulu, HI, pp.1-6, Nov.2009.
[57]R. La and V. Anantharam,“Optimal routing control: repeated game approach,”IEEE Transactions on Automatic Control, vol.47, no.3, pp.437-450, Mar.2002.
[58]K. Wang, W. Wong, and T. Y. Chai,“A MANET routing protocol using Q-learningmethod integrated with Bayesian network,” IEEE International Conference onCommunication Systems, Singapore, pp.270-274, Nov.2012.
[59]A. Tang, J. Wang, S. H. Low, and M. Chiang,“Equilibrium of heterogeneouscongestion control protocols: Existence and Uniqueness,” IEEE/ACMTransactions on Networking, vol.15no.4, pp.824-837, Oct.2007.
[60]C. George, and K. Elias,“The price of anarchy of finite congestion games,”Proceedings of the thirty-seventh annual ACM symposium on Theory of computing,New York, USA, pp.67-73,2005.
[61]B. Wang, Z. Ji, and K.J.R. Liu,“Self-learning repeated game framework fordistributed primary-prioritized dynamic spectrum access,”2nd IEEE Workshop onNetworking Technologies for Software Define Radio Networks, San Diego, USA pp.631–638,2007.
[62]Y. Wu, B. Wang, K.J.R. Liu, and T.C. Clancy,“Repeated open spectrum sharinggame with cheat-proof strategies,” IEEE Transactions on WirelessCommunications, vol.8, no.4, pp.1922–1933,2009.
[63]M. Canales, J. Ortin, and J. R. Gallego,“Game theoretic approach for end-to-endresource allocation in multihop cognitive radio networks,” IEEE CommunicationsLetters, vol.16, no.5, pp.654-657,2012.
[64]F. Fu, and M. Schaar,“Learning to compete for resources in wireless stochasticgames,” IEEE Transactions on Vehicular Technology, vol.58, no.4, pp.1904-1919,May2009.
[65]Q. Ni, and C. C. Zarakovitis,“Nash bargaining game theoretic scheduling for jointchannel and power allocation in cognitive radio systems,” IEEE Journal onSelected Areas in Communications, vol.30, no.1, pp.70-81, Jan.2012.
[66]A. A. Daoud, T. Alpcan, S. Agarwal, and M. Alanyali,“A stackelberg game forpricing uplink power in wide-band cognitive radio networks,”47th IEEEConference on Decision and Control, Cancun, pp.1422-1427, Dec.2008.
[67]Y. Teng, Y. Zhang, F. Niu, C. Dai and M. Song,“Reinforcement learning basedauction algorithm for dynamic spectrum access in cognitive radio networks,” IEEE72nd Vehicular Technology Conference Fall, Ottawa, pp.1-5, Sept.2010.
[68]H. Zhu, C. Pandana, and K. J. R. Liu,“Distributive opportunistic spectrum accessfor cognitive radio using correlated equilibrium and no-regret learning,” IEEEWireless Communications and Networking Conference, Kowloon, pp.11-15, Mar.2007.
[69] H. LI,“Multi-agent Q-learning for Aloha-like spectrum access in cognitive radiosystems,” Fifth IEEE Workshop on Networking Technologies for Software DefinedRadio Networks, Boston, USA, pp.1-6, Jun.2010.
[70]杨春刚,认知无线网络中基于博弈论的功率控制研究,博士论文,西安电子科技大学,Apr.2011.
[71]Y. Zhao, S. Mao, J. O. Neel, and J. H. Reed,“Performance evaluation of cognitiveradios: metrics, utility functions, and methodology,” Proceedings of the IEEE, vol.97, no.4, pp.642-659, Apr.2009.
[72]P. Zhou, Y. Chang, and J. A. Copeland,“Learning through reinforcement forrepeated power control game in cognitive radio networks,” IEEE GlobalCommunications Conference, Piscataway, pp.1-6, Dec.2010.
[73]P. Zhou, Y. Chang, and J. A. Copeland,“Reinforcement learning for repeatedpower control game in cognitive radio networks,” IEEE Journal on Selected Areasin Communications, vol.30, no.1, pp.54-69,2012.
[74]Q. Wang, and H. Zheng,“Route and spectrum selection in dynamic spectrumnetworks,” IEEE Consumer Communications and Networking Conference,LasVegas, pp.1-5,2006.
[75]T. Nadkar, V. Thumar, G. P. S. Tej, S. N. Merchant, and U. B. Desai,“Distributedpower allocation for secondary users in a cognitive radio scenario,” IEEETransactions on Wireless Communications, vol.11, no.4, pp.1576-1586, Apr.2012.
[76]Q. H. QH Mahmoud, Cognitive networks, West Sussex, England: Wiley,2007.
[77]M. Zeidenberg, Neural networks in artificial intelligence. NJ, USA: Ellis HorwoodLimited.,1990.
[78]A. Fehske, J. Gaeddert, and J. Reed,“A new approach to signal classification usingspectral correlation and neural networks,” First IEEE International Symposium onNew Frontiers in Dynamic Spectrum Access Networks, Baltimore, USA, pp.144-150, Nov.2005.
[79]V. K. Tumuluru, P. Wang, and D. Niyato,“A neural network based spectrumprediction scheme for cognitive radio,” IEEE International Conference onCommunications, Cape Town, pp.1-5, May2010.
[80]G. Ganesan, and Y. Li,“Cooperative spectrum sensing in cognitive radio, part I:Two user networks,” IEEE Transactions on Wireless Communications, vol.6, no.6,pp.2204-2213, Jun.2007.
[81]S. Haykin, Neural networks and learning machines, New Jersey: Prentice-Hall，2008.
[82]V. N. Vapnik, The nature of statistical learning theory, New York: Springer-Verlag,1995.
[83]C. Cortes and V. N. Vapnik,“Support-Vector networks,” Machine Learning, vol.40, no.3, pp.273-297, Sept.1995.
[84]J. A. K. Suykens and J. Vandewalle,“Least squares support vector machineclassifiers,” Neural Processing Letters, vol.9, no.3, pp.293-300, Jun.1999.
[85]B. Rozovskii and M. Yor, Stochastic approximation and recursive algorithms andapplications, New York, USA: Springer,2003.
[86]L. Shapley,“Stochastic games,” Proceedings of the National Academy of Sciencesof the United States of America, vol.39, no.10, pp.1095-1100,1953.
[87]A. Neyman，and S. Sorin, Stochastic games and applications, Dordrecht: KluwerAcademic Press,2003.
[88] J. Filar, K. Vrieze, and O. Vrieze, Competitive markov decision processes, Berlin:Springer,1997.
[89]M. L. Minsky,“Theory of Neural-Analog Reinforcement Systems and itsApplication to the Brain-Model Problem,” Ph.D. dissertation, PrincetonUniversity,1954.
[90]M. Waltz, and K. S. Fu,“A heuristic approach to reinforcement learning controlsystems,” IEEE Transactions on Automatic Control, vol.10, no.4, pp.390-398,Oct.1965.
[91]C. J. C. H. Watkins,“Learning from delayed rewards,” Ph.D. dissertation,University of Cambridge, England,1989.
[92]C. J. C. H. Watkin, and P. Dayan,“Q-learning,” Machine learning, vol.8, no.3-4, pp.279-292, May1992.
[93]江虹,伍春,马景辉,“最小二乘SVM的CR认知引擎设计,”电子科技大学学报, vol.40, no.1, pp.41-46,2011.
[94]A. Ginsberg, J. D. Poston, and W. D. Horne,“Experiments in Cognitive Radio andDynamic Spectrum Access using An Ontology-Rule Hybrid Architecture,” SecondInternational Conference on Rules and Rule Markup Languages for the SemanticWeb, athens, GA, USA, Nov.2006.
[95] D. L. McGuinness, and F. Harmelen,“OWL Web Ontology Language Overview,”Available: http://www.w3.org/TR/owl-features/.
[96]S. B. Kotsiantis, D. Kanellopoulos and P. E. Pintelas,“Data preprocessing forsupervised leaning,” International Journal of Computer Science, vol.1, no.2, pp.111-117,2006.
[97]Weston J, Watkins C,“Multi-class support vector machines,” Technical Report,Royal Holloway, University of London,1998.
[98]T. V. Gestel, J. A. K. Suykens, B. Baesens, S. Viaene, J. Vanthienen, G. Dedene, B.D. Moor, and J. Vandewalle,“Benchmarking least squares support vector machineclassifiers,” Machine Learning, vol.54, no.1, pp.5-32, Nov.2004.
[99]C. Demirkesen and C. Hocine,“A comparison of multiclass SVM methods for realworld natural scenes,” Advanced Concepts for Intelligent Vision Systems LectureNotes in Computer Science, vol.5259, pp.752-763, Oct.2008.
[100]P. Murugan, S. Kannan and S. Baskar,“Application of NSGA-II Algorithm toSingle-Objective Transmission Constrained Generation Expansion Planning,”IEEE Transactions on Power Systems, vol.24, no.4, pp.1790-1797,2009.
[101]A. Goldsmith, S.A. Jafar, I. Maric, and S. Srinivasa,“Breaking spectrum gridlockwith cognitive radios: An information theoretic perspective,” Proceedings of theIEEE, vol.97, no.5, pp.894-914, May2009.
[102]R. Blasco-Serrano, J. Lv, R. Thobaben, E. Jorswieck, and M. Skoglund,“Comparison of underlay and overlay spectrum sharing strategies in MISOcognitive channels,”7th International ICST Conference on Cognitive RadioOriented Wireless Networks and Communications, Stockholm, pp.224-229, Jun.2012.
[103]M. Maskery, V. Krishnamurthy, and Q. Zhao,“Decentralized dynamic spectrumcccess for cognitive radios: cooperative design of a non-cooperative game,” IEEETransactions on Communications, vol.57, no.2, pp.459-469, Feb.2009.
[104]A. Attar, M. R. Nakhai, and A. H. Aghvami,“Cognitive radio game for secondaryspectrum access problem,” IEEE Transactions on Wireless Communications, vol.8, no.4, pp.2121-2131,2009.
[105]C. Luo, F. R. Yu, H. Ji, and V. C. M. Leung,“Optimal channel access for TCPperformance improvement in cognitive radio networks,” Wireless Networks, vol.17, no.2, pp.479-492, Feb.2011.
[106]C. Yang, and J. Li,“Mixed-strategy based discrete power control approach forcognitive radios: a matrix game-theoretic framework,”2nd InternationalConference on Future Computer and Communication, Wuhan, vol.3, pp.806-810, May2010.
[107]H. Yu, L. Gao, Z. Li, X. Wang, and E. Hossain,“Pricing for uplink power controlin cognitive radio networks,” IEEE Transactions on Vehicular Technology, vol.59, no.4, pp.1769-1778, May2010.
[108]L. Busoniu, R. Babuska, B. Schutter,“A comprehensive survey of multiagentreinforcement learning,” IEEE Transactions on Systems, vol.38, no.2, pp.156-172,2008.
[109]J. Wang, and X. Su,“An improved K-Means clustering algorithm,” IEEE3rdInternational Conference on Communication Software and Networks, Xuzhou, pp.44-46,2011.
[110]C. Claus and C. Boutilier,“The dynamics of reinforcement learning incooperative multiagent systems,”15th National Conference on ArtificialIntelligence and Tenth Innovative Applications of Artificial IntelligenceConference, Madison, WI, pp.746-752, Jul.1998.
[111]M. L. Littman,“Markov games as a framework for multi-agent reinforcementlearning,”11th International Conference on Machine Learning, New Brunswick,NJ, pp.157–163. Jul.1994.
[112]J. Hu and M. P. Wellman,“Multiagent reinforcement learning: Theoreticalframework and an algorithm,” in Proc.15th International Conference onMachine Learning, Madison, WI, pp.242-250, Jul.1998.
[113]A. Greenwald and K. Hall,“Correlated-Q learning,”20th InternationalConference on Machine Learning, Washington, DC, pp.242–249, Aug.2003.
[114]M. Bowling, and M. Veloso,“Multiagent learning using a variable learning rate,”Artificial Intelligence, vol.136, no.2, pp.215-220,2002.
[115]C. Goochul, S. Sridharan, S. Vishwanath, and S. H. Chan,“On the capacity ofoverlay cognitive radios with partial cognition,” IEEE Transactions onInformation Theory, vol.58, no.5, pp.2935-2949,2012.
[116]R. Zhang, Y. Liang, and S. Cui,“Dynamic resource allocation in cognitive radionetworks,” IEEE Signal Processing Magazine, vol.27, no.3, pp.102-114,2010.
[117]N. Nie, and C. Comaniciu,“Adaptive channel allocation spectrum etiquette forcognitive radio networks,” First IEEE International Symposium on NewFrontiers in Dynamic Spectrum Access Networks, Baltimore, pp.269-278,2005.
[118]X. Yong, B. Guoan, and D. Niyato,“A simple distributed power control algorithmfor cognitive radio networks,” IEEE Transactions on Wireless Communications,vol.10, no.11, pp.3594-3600,2011.
[119]C. Yang, J. Li, and Z. Tian,“Optimal power control for cognitive radio networksunder coupled interference constraints: a cooperative game-theoreticperspective,” IEEE Transactions on Vehicular Technology, vol.59, no.4, pp.1696-1706,2010.
[120]S. Yang, Z. Chi, and F. Yuguang,“Joint channel and power allocation in wirelessmesh networks: a game theoretical perspective,” IEEE Journal on Selected Areasin Communications, vol.26, no.7, pp.1149-1159,2008.
[121]Y. E. Lin, K. H. Liu, and H. Y. Hsieh,“Design of power control protocols forspectrum sharing In cognitive radio networks: a game theoretic perspective,”IEEE International Conference on Communications, Cape Town, pp.23-27,2010.
[122]Z. Han, Z. Ji, and K. J. R. Liu,“Non-cooperative resource competition game byvirtual referee in multi-cell OFDMA networks,” IEEE Journal on Selected Areasin Communications, vol.25, no.6, pp.1079–1090,2007.
[123]A. Muthoo, Bargaining Theory with Applications, Cambridge, UK: CambridgeUniversity Press,1999.
[124]J. Nash,“The Bargaining Problem,” Econometrica, vol.18, no.2, pp.155-162,Apr.1950.
[125]M. Canales, and J. R. Gallego,“Distributed channel allocation and power controlin cognitive radio networks using game theory,” IEEE Vehicular TechnologyConference, San Francisco, CA, pp.1-5,2011.
[126]I. Ganchev, Z. Ji, and M. O'Droma,“A realization of cognitive pilot channelsthrough wireless billboard channel infrastructure for cognitive radio,”2nd BalticCongress on Future Internet Communications, Vilnius, pp.19-25,2012.
[127]V. Stavroulaki, K. Tsagkaris, P. Demestichas, J. Gebert, M. Mueck, A. Schmidt, R.Ferrus, O. Sallent, M. Filo, C. Mouton, and L. Rakotoharison,“Cognitive controlchannels: from concept to identification of implementation options,” IEEECommunications Magazine, vol.50, no.7, pp.96-108,2012.
[128]P. Gupta, and P. R. KumarThe,“Capacity of Wireless Networks,” IEEETransactions on Information Theory, vol.46, no.2, pp.388-404,2000.
[129]M. Canales, J. R. Gallego, and R. Ciria,“Distributed channel allocation andpower control in cognitive radio networks using game theory,”[C].74th IEEEVehicular Technology Conference, San Francisco, pp.1-5,2011.
[130]A. Abrardo, A. Alessio, P. Detti, and M. Moretti,“Centralized radio resourceallocation for OFDMA cellular systems,” IEEE International Conference onCommunications, Glasgow, UK, pp.5738-2743, Jun.2007.
[131]J. Nie and S. Haykin,“A Q-learning-based dynamic channel assignmenttechnique for mobile communication systems,” IEEE Transactions on VehicularTechnology, vol.48, no.5, pp.1676–1687,1999.
[132]S. Xergias, N. Passas, and A. K. Salkintzis,“Centralized resource allocation for
multimedia traffic in IEEE802.16mesh networks,” Proceedings of the IEEE, vol.
96, no.1, pp.54-63, Jan.2008.