认知无线电系统中自适应跨层优化机制的研究
|
摘要
未来无线通信系统中频谱资源的稀缺以及复杂通信系统的运维成为无法回避的重要问题,为此学术界提出了一种基于认知技术的智能通信系统,即认知无线电系统。随着数字信号处理、计算机网络、人工智能等技术的快速发展,高度综合这些技术来实现认知无线电系统已成为可能。如何让认知无线电系统感知环境,并利用人工智能技术从环境中学习,通过实时重配某些跨层运行参数使系统适应环境的变化,从而实现复杂通信系统中满足用户需求的灵活可靠通信以及频谱资源的有效利用。已成为未来无线通信系统的研究热点,也是本文研究的出发点。
本文主要对认知无线电系统的频谱感知与接入、传输与调度等自适应跨层无线资源管理方面的技术进行了研究,主要研究内容和取得的创新性成果如下：
1、首先提出了认知无线电系统中动态频谱感知、接入、传输、调度的跨层优化框架。在此框架下,根据频谱可用性和频谱感知结果的变化规律,将授权系统对频谱的占用过程建模为交替更新过程。并由此推导出频谱可用性的统计规律,将频谱感知结果建模为离散时间马尔科夫链。由于基于该优化框架的约束马尔科夫决策过程具有特殊性,即认知无线电用户的行动不会对系统状态产生影响,根据这一特点,对传统的线性规划解法作了修改,简化了求解过程。为了解决马尔科夫决策过程因变量过多而导致的“维灾”问题,采用策略分离分别求解最优接入策略和最优传输策略,减少变量数。同时提出了不需要求解线性规划问题的启发式算法。
2、在前面研究的基础上,将认知无线电系统应用于环境参数未知的情况,即系统缺乏马尔科夫决策过程状态转移的先验概率信息。为此采用一种强化学习算法,即R学习,使得系统能够自适应地从环境中学习近似最优策略。近似最优策略考虑了由缓存器内分组数量决定的QoS与长期平均功率效率之间的平衡问题。为解决R学习只适合于无约束的马尔科夫决策过程的问题,引入拉格朗日乘子法将约束问题转换为非约束问题。利用拉格朗日乘子的单调特性,提出了一种基于黄金分割算法的拉格朗日乘子搜索算法。此外还提出了一种状态空间压缩法和行动集缩减法以减少R学习所需的内存资源,并提高R学习的收敛速度。同时还证明了在某些合理的假设条件下,该压缩法和缩减法不会影响R学习收敛到最优策略。
3、前面的研究工作针对的是单用户、单链路的情况,将其扩展到多用户竞争频谱资源的认知无线电网络中。利用微观经济学中的拍卖理论,提出了一种基于重复多标拍卖的频谱分配机制。在此机制下,网络用户是竞标者,接入点或基站在一次拍卖中充当拍卖人。竞标者为满足自身效用给频谱资源投标,由拍卖人根据最大化网络收益原则确定胜利者。为迎合不同的QoS标准,制定了三种不同的频谱分配优化目标,以及相对应的三种竞标者的价值函数。和其它分配机制相比较,本文提出的基于重复多标拍卖的机制具有几点优势：具有分散的特点,而且所需的信令交互以及计算开销都很小。
The scarcity of spectrum and operation of complex networks become inevitable crucial problems in future wireless communication systems, therefore, a kind of intelligent communication system based on cognitive science, namely, cognitive radio system is proposed by academia. For the rapid development of digital signal processing, computer networks and artificial intelligence, it is possible to realize cognitive radio systems by high integration of these technologies. How to make cognitive radio systems sense environment, learn from environment by artificial intelligence and adapt to evolution of environment by reconfiguration of cross-layer operational parameters so as to achieve flexible and reliable communication meeting the requirements of subscribers in complex communication systems and efficient utilization of spectrum has ever been one important research topic of modern wireless communication systems, and also been the objective of this paper.
This dissertation mainly focuses on the adaptive cross-layer radio resource management of cognitive radio systems, such as sensing and access of spectrum, transmission and scheduling. More specifically, the main research contents and innovations are as follows:
1. Firstly, a cross-layer optimization framework of dynamic spectrum sensing and access, transmission and scheduling in cognitive radio systems is proposed. According to the evolution of spectrum availability and sensing results, spectrum occupancy of licensed users is model as an alternative renewal process in the framework. Accordingly, statistical characteristic of spectrum availability is induced and spectrum sensing results is formulated as a discrete time Markov chain. In the proposed constrained Markov decision process (CMDP) for optimization framework, the evolution of state does not affected by actions. Consequently, traditional LP is modified to simplify solution of the CMDP. To cope with the curse of dimensionality incurred by the overmany number of variables belonging to the CMDP, policy separation approach is employed to work out optimal access police and optimal transmission police respectively. Moreover, heuristic algorithms by which it is no use for solving LP are proposed.
2. Based on the previous study, the cognitive radio systems are considered to be deployed in the scenario that environment parameters are undiscovered, that is, the systems do not have a priori knowledge about state transition probabilities of CMDP. Therefore, a kind of reinforcement learning, namely, R learning is employed by the systems to learning nearly optimal policy from the environment. The tradeoff between the QoS depending on buffer occupancy and the long-term average power efficiency is involved in the nearly optimal policy. Since R learning only adapts to unconstrained MDP (UMDP), Lagrangian multiplier approach is utilized to convert the CMDP to a corresponding UMDP. The nice monotone property of Lagrangian multiplier facilitates the search of the proper multiplier by proposed golden section search method. In addition, state space compaction and action set reduction are proposed respectively to reduce the storage cost and accelerate the convergence of R Learning. Meanwhile, the proposition that R learning policy employing the state space compaction and action set reduction converges to optimal policy under some reasonable assumption is proved.
3. The work described above focuses on single user and single link, and is extended to the cognitive radio networks in which multiple users compete for spectrum. By auction theory of microeconomic, a spectrum allocation mechanism based on repeated multi-bid auction is proposed. In this mechanism, users of the system are the bidders and the access point or base station acts as an auctioneer in an auction round. Each bidder gives a multi-bid for spectrum to satisfy the utility of it. The winner determination made by the auctioneer can work out the allocation result and improve system efficiency by maximizing the revenue of the network. To satisfy different QoS criteria, three different optimization objectives of spectrum allocation and corresponding value functions of bidders are defined respectively. Compared with other allocation mechanisms, repeated multi-bid auction mechanism has some advantages:it is decentralized in nature and requires little signaling exchange and computational expense.
本文主要对认知无线电系统的频谱感知与接入、传输与调度等自适应跨层无线资源管理方面的技术进行了研究,主要研究内容和取得的创新性成果如下：
1、首先提出了认知无线电系统中动态频谱感知、接入、传输、调度的跨层优化框架。在此框架下,根据频谱可用性和频谱感知结果的变化规律,将授权系统对频谱的占用过程建模为交替更新过程。并由此推导出频谱可用性的统计规律,将频谱感知结果建模为离散时间马尔科夫链。由于基于该优化框架的约束马尔科夫决策过程具有特殊性,即认知无线电用户的行动不会对系统状态产生影响,根据这一特点,对传统的线性规划解法作了修改,简化了求解过程。为了解决马尔科夫决策过程因变量过多而导致的“维灾”问题,采用策略分离分别求解最优接入策略和最优传输策略,减少变量数。同时提出了不需要求解线性规划问题的启发式算法。
2、在前面研究的基础上,将认知无线电系统应用于环境参数未知的情况,即系统缺乏马尔科夫决策过程状态转移的先验概率信息。为此采用一种强化学习算法,即R学习,使得系统能够自适应地从环境中学习近似最优策略。近似最优策略考虑了由缓存器内分组数量决定的QoS与长期平均功率效率之间的平衡问题。为解决R学习只适合于无约束的马尔科夫决策过程的问题,引入拉格朗日乘子法将约束问题转换为非约束问题。利用拉格朗日乘子的单调特性,提出了一种基于黄金分割算法的拉格朗日乘子搜索算法。此外还提出了一种状态空间压缩法和行动集缩减法以减少R学习所需的内存资源,并提高R学习的收敛速度。同时还证明了在某些合理的假设条件下,该压缩法和缩减法不会影响R学习收敛到最优策略。
3、前面的研究工作针对的是单用户、单链路的情况,将其扩展到多用户竞争频谱资源的认知无线电网络中。利用微观经济学中的拍卖理论,提出了一种基于重复多标拍卖的频谱分配机制。在此机制下,网络用户是竞标者,接入点或基站在一次拍卖中充当拍卖人。竞标者为满足自身效用给频谱资源投标,由拍卖人根据最大化网络收益原则确定胜利者。为迎合不同的QoS标准,制定了三种不同的频谱分配优化目标,以及相对应的三种竞标者的价值函数。和其它分配机制相比较,本文提出的基于重复多标拍卖的机制具有几点优势：具有分散的特点,而且所需的信令交互以及计算开销都很小。
The scarcity of spectrum and operation of complex networks become inevitable crucial problems in future wireless communication systems, therefore, a kind of intelligent communication system based on cognitive science, namely, cognitive radio system is proposed by academia. For the rapid development of digital signal processing, computer networks and artificial intelligence, it is possible to realize cognitive radio systems by high integration of these technologies. How to make cognitive radio systems sense environment, learn from environment by artificial intelligence and adapt to evolution of environment by reconfiguration of cross-layer operational parameters so as to achieve flexible and reliable communication meeting the requirements of subscribers in complex communication systems and efficient utilization of spectrum has ever been one important research topic of modern wireless communication systems, and also been the objective of this paper.
This dissertation mainly focuses on the adaptive cross-layer radio resource management of cognitive radio systems, such as sensing and access of spectrum, transmission and scheduling. More specifically, the main research contents and innovations are as follows:
1. Firstly, a cross-layer optimization framework of dynamic spectrum sensing and access, transmission and scheduling in cognitive radio systems is proposed. According to the evolution of spectrum availability and sensing results, spectrum occupancy of licensed users is model as an alternative renewal process in the framework. Accordingly, statistical characteristic of spectrum availability is induced and spectrum sensing results is formulated as a discrete time Markov chain. In the proposed constrained Markov decision process (CMDP) for optimization framework, the evolution of state does not affected by actions. Consequently, traditional LP is modified to simplify solution of the CMDP. To cope with the curse of dimensionality incurred by the overmany number of variables belonging to the CMDP, policy separation approach is employed to work out optimal access police and optimal transmission police respectively. Moreover, heuristic algorithms by which it is no use for solving LP are proposed.
2. Based on the previous study, the cognitive radio systems are considered to be deployed in the scenario that environment parameters are undiscovered, that is, the systems do not have a priori knowledge about state transition probabilities of CMDP. Therefore, a kind of reinforcement learning, namely, R learning is employed by the systems to learning nearly optimal policy from the environment. The tradeoff between the QoS depending on buffer occupancy and the long-term average power efficiency is involved in the nearly optimal policy. Since R learning only adapts to unconstrained MDP (UMDP), Lagrangian multiplier approach is utilized to convert the CMDP to a corresponding UMDP. The nice monotone property of Lagrangian multiplier facilitates the search of the proper multiplier by proposed golden section search method. In addition, state space compaction and action set reduction are proposed respectively to reduce the storage cost and accelerate the convergence of R Learning. Meanwhile, the proposition that R learning policy employing the state space compaction and action set reduction converges to optimal policy under some reasonable assumption is proved.
3. The work described above focuses on single user and single link, and is extended to the cognitive radio networks in which multiple users compete for spectrum. By auction theory of microeconomic, a spectrum allocation mechanism based on repeated multi-bid auction is proposed. In this mechanism, users of the system are the bidders and the access point or base station acts as an auctioneer in an auction round. Each bidder gives a multi-bid for spectrum to satisfy the utility of it. The winner determination made by the auctioneer can work out the allocation result and improve system efficiency by maximizing the revenue of the network. To satisfy different QoS criteria, three different optimization objectives of spectrum allocation and corresponding value functions of bidders are defined respectively. Compared with other allocation mechanisms, repeated multi-bid auction mechanism has some advantages:it is decentralized in nature and requires little signaling exchange and computational expense.
引文
[1]N. K. Hoven. On the feasibility of cognitive radio:[Thesis of M.S.], Berkeley:University of California,2005
[2]R. W. Broderson, A. Wolisz, D. Cabric, et al. CORVUS:a cognitive radio approach for usage of virtual unlicensed spectrum:[White Paper], Berkeley:UC Berkeley Wireless Research Center,2004
[3]Matthias Wellens, Jin Wu, Petri M"ah"onen. Evaluation of spectrum occupancy in indoor and outdoor scenario in the context of cognitive radio. IEEE International Conference on Cognitive Radio Oriented Wireless Networks and Communications,2007,420-427
[4]Qusay H. Mahmoud. Cognitive networks:towards self-aware networks. New Jersey:John Wiley & Sons Inc.,2007,2-128
[5]Frank H.P. Fitzek, Marcos D. Katz. Cognitive wireless networks:concepts, methodologies and visions inspiring:the age of enlightenment of wireless communications. Netherlands:Springer, 2007,12-56
[6]Ekram Hossain, Vijay Bhargava. Cognitive wireless communication networks. New York: Springer,2007,23-36
[7]Joseph Mitola Ⅲ, G. Q. Maguire. Cognitive radio:making software radios more personal. IEEE Personal Communications,1999,6(4):13-18
[8]Joseph Mitola Ⅲ. Cognitive radio for flexible mobile multimedia communications. Sixth International Workshop on Mobile Multimedia Communications,1999,3-10
[9]Joseph Mitola Ⅲ. Cognitive radio:an integrated agent architecture for software defined radio: [Ph.D. Dissertation], Stockholm:Royal Institute of Technology,2000
[10]T. A. Weiss and F. K. Jondral. Spectrum pooling:an innovative strategy for the enhancement of spectrum efficiency. IEEE Radio Communications,2004,42(3):8-14
[11]T. Weiss, A. Krohn, J. Hillenbrand, et al. Efficient signaling of spectral resources in spectrum pooling systems. Proc. of the 10th Symposium on Communications and Vehicular Technology, 2003,1:1-6
[12]T. Weiss, A. Krohn, F. Jondral. Synchronization algorithms and preamble concepts for spectrum pooling systems. Proc. of the Mobile & Wireless Communications Summit,2003,1: 788-792
[13]F. Capar, I. Martoyo, T. Weiss, et al. Comparison of bandwidth utilization for controlled and uncontrolled channel assignment in a spectrum pooling system. IEEE Vehicular Technology Conference,2007,3:1069-1073
[14]M. Muck, D. Bourse, K. Moessner, et al. End-to-end reconfigurability in heterogeneous wireless systems-software and cognitive radio solutions enriched by policy-and context-based decision making.16th IST Mobile and Wireless Communications Summit,2007,1-5
[15]Bruce A. Fette. Cognitive radio technology. Oxford:Elsevier Inc.,2006,1-25
[16]Narayan Mandayam, Christopher Rose, Predrag Spasojevic, et al. Nets-prowin:cognitive radios for open access to spectrum:[Project Summary], New Jersey:Rutgers University,2005
[17]B. Ackland, M. Bushnell, C. Rose, et al. Nets-prowin:high performance cognitive radio platform with integrated physical and network layer capabilities:[Project Summary], New Jersey:Rutgers University,2005
[18]Jun Zhao, Haitao Zheng, Guang Hua Yang. Distributed coordination in dynamic sspectrum allocation networks. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,259-268
[19]J. Zhao, H. Zheng, G. Yang. Spectrum sharing through distributed coordination in dynamic spectrum access networks. Wireless Communications and Mobile Computing,2007,7(9): 1061-1075
[20]George Dimitrakopoulos, Panagiotis Demestichas, David Grandblaise, et al. Congnitive radio, spectrum and radio resource management:[White Paper], Wireless World Research Forum Working Group 6,2004
[21]C. Cordeiro, K. Challapali, D. Birru, et al. IEEE 802.22:the first worldwide wireless standard based on cognitive radio. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,328-337
[22]XG Working Group. The XG vision:[RFC], Massachusetts:BBN Technologies,2004
[23]XG Working Group. The XG architecture framework:[RFC], Massachusetts:BBN Technologies,2004
[24]XG Working Group. XG policy language framework:[RFC], Massachusetts:BBN Technologies,2004
[25]Joseph Mitola III. Cognitive radio architecture:the engineering foundations of radio XML. New Jersey:John Wiley & Sons Inc.,2006,236-275
[26]Simon Haykin. Cognitive radio:brain-empowered wireless communications. IEEE Journal on Selected Areas in Communications,2005,23(2):201-220
[27]Ryan W. Thomas, Daniel H. Friend, Luiz A. Dasilva, et al. Cognitive networks:adaptation and learning to achieve end-to-end performance objectives. IEEE Communications Magazine, 2006,44(12):51-57
[28]Gerhard Weiss. Multiagent systems:a modern approach to distributed artificial intelligence. Cambridge:MIT Press,1999,121-146
[29]Q. Zhao, L. Tong, A. Swami, Y. Chen. Decentralized cognitive mac for opportunistic spectrum access in ad hoc networks:a POMDP framework. IEEE Journal Selected Areas in Communications,2007,25(3):589-600
[30]Q. Zhao, L. Tong, and A. Swamiz. Decentralized cognitive mac for dynamic spectrum access. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005,224-232
[31]Yunxia Chen, Qing Zhao, A. Swami. Joint PHY-MAC design for opporunistic spectrum access with multi-channel. IEEE International Workshop on Signal Processing Advances in Wireless Communications,2007,1-5
[32]Jiang Zhu, Ning Wei, Bingyang Xu, et al. Optimal and suboptimal access and transmission polices for dynamic spectrum access over fading channels in cognitive radio networks. Chinese Journal of Electronics.2008,17(4):726-732
[33]Hyoil Kim, K. G. Shin. Efficient discovery of spectrum opportunities with MAC-layer sensing in cognitive radio networks. IEEE Transactions on Mobile Computing,2008,7(5):533-545
[34]Q. C. Zhao, S. Geirhofer, L. Tong, and B. M. Sadler, Optimal dynamic spectrum access via periodic channel sensing, IEEE Wireless Communications and Networking Conference,2007, 33-37
[35]S. Geirhofer, L. Tong, B. M. Sadler. Cognitive medium access:constraining interference based on experimental models. IEEE Journal Selected Areas in Communications,2008,26 (1): 95-105
[36]C. T. Chou, Hyoil Kim, Sai Shankar, et al. What and how much to gain by spectral agility. IEEE Journal Selected Areas in Communications,2007,25 (3):576-588
[37]C. Clancy, J. Hecker, E. Stuntebeck, et al. Applications of machine learning to cognitive radio networks. IEEE Wireless Communications Magzine,2007,14 (4):47-52
[38]FCC. Establishment of interference temperature metric to quantify and manage interference and to expand available unlicensed operation in certain fixed mobile and satellite frequency bands. Notice of Inquiry and Proposed Rulemaking, ET Docket No.03-289,2003
[39]FCC. In the matter of establishment of an interference temperature metric to quantify and management interference and to expand available unlicensed operation in certain fixed, mobile and satellite frequency bands. Notice of Inquiry and Notice of Proposed Rule Making, ET Docket No.03-237, November 13,2003
[40]T. Clancy, W. Arbaugh. Measuring interference temperature:[Technical Report], Virginia: Virginia Tech,2006
[41]T. Clancy. Formalizing the interference temperature model. Wiley Journal on Wireless Communications and Mobile Computing,2007,7(9):1077-1086
[42]J. Bater, H. P. Tan, K. Brown, et al. Modelling interference temperature constraints for spectrum access in cognitive radio network. IEEE International Conference on Communications,2007,6493-6498
[43]Y. Xing, C. N. Mathur, M. A. Haleem, et al. Dynamic spectrum access with QOS and interference temperature constraints. IEEE Transactions on Mobile Computing,2007,6(4): 423-433
[44]Jie Chen, Chulin Liao, Chuan Han, et al. Extending available spectrum in cognitive radio: hierarchical spectrum sharing network. Elsevier Computer Networks,2008,52(4):850-863
[45]J. Huang, R. Berry, and M. L. Honig. Auction-based spectrum sharing. ACM/Springer Mobile Networks and Applications,2006,11(3):405-418
[46]Ramin Rezaiifar, M. Makowski, Srikanta Kumar. Stochastic control of handoffs in cellular networks. IEEE Journal on Selected Areas in Communications,1995,13(7):1348-1362
[47]H. S. Wang and N. Moayeri. Finite-state Markov channel-a useful model for radio communication channels. IEEE Transactions on Vehicular Technology,1995,44 (1):163-171
[48]D. V. Djonin, V. Krishnamurthy. Q-learning algorithms for constrained Markov decision processes with randomized monotone policies:application to MIMO transmission control. IEEE Transactions on Signal Processing,2007,55(5):2170-2181
[49]D. V. Djonin, V. Krishnamurthy. MIMO transmission control in fading channels:a constrained Markov decision process formulation with monotone randomized policies. IEEE Transactions on Signal Processing,2007,55(10):5069-5083
[50]Md. J. Hossain, D. V. Djonin, V. K. Bhargava. Delay limited optimal and suboptimal power and bit loading algorithms for OFDM systems over correlated fading. IEEE Global Communications Conference,2005,3448-3453
[51]朱江,黄海洋,李少谦.一种基于认知无线电技术的能效传输控制方案.计算机工程与应用,2008,44(31)：28-32
[52]Yunxia Chen, Qing Zhao, A. Swami. Distributed cognitive MAC for energy-constrained opportunistic spectrum access. IEEE Military Communications Conference,2006,1-7
[53]Yunxia Chen, Qing Zhao. Bursty traffic in energy-constrained opportunistic spectrum access. IEEE Global Communications Conference,2007,4641-4646
[54]Charles Pandana, K. J. Ray Liu. Near-optimal reinforcement learning framework for energy-aware sensor communications. IEEE Transactions on Wireless Communications,2005, 23(4):788-797
[55]Jianying Li, Binyang Xu, Zhangjing Xu, et al. adaptive packet scheduling algorithm for cognitive radio system. IEEE International Conference on Communication Technology,2006, 111-115.
[56]徐斌阳,李健颖,李少谦.认知无线电系统频谱自适应调度算法设计.电子科技大学学报,2008,37(6)：815-818
[57]F. Fu, M. Van Der Schaar. Dynamic spectrum sharing using learning for delay-sensitive applications. IEEE International Conference on Communications,2008,2825-2829
[58]Amir Ghasemi, E. S. Sousa. Collaborative spectrum sensing for opportunistic access in fading environment. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,131-136
[59]E. Visotsky, S. Kuffner, R. Peterson. On collaborative detection of TV transmissions in support of dynamic spectrum sharing. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,338-345
[60]D. Grandblaise, C. Kloeck, K. Moessner, et al. Techno-economic of collaborative based secondary spectrum usage-E2R research project outcomes overview.2005 IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,318-327
[61]S. M. Mishra, A. Sahai and R. W. Brodersen. Cooperative sensing among cognitive radios. IEEE International Conference on communications,2006,1658-1663
[62]H. Zheng, C. Peng. Collaboration and fairness in opportunistic spectrum access. IEEE International Conference on Communication,2005,3132-3136
[63]Savo G. Glisic. Advanced wireless communications:4G cognitive and cooperative broadband technology. Chichester:John Wiley & Sons Ltd.,2007,537-587
[64]J. Neel, J. H. Reed, R. P. Gilles. The role of game theory in the analysis of software radio networks. SDR Forum Technical Conference,2002, NP-3-02
[65]J. Neel, J. H. Reed, R. P. Gilles. Convergence of cognitive radio networks. Wireless Communications and Networking Conference,2004,2250-2255.
[66]C. Kloeck, H. Jaekel, F. K. Jondral. Dynamic and local combined pricing, allocation and billing system with cognitive radios. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,73-81
[67]Han Zhu, Zhu (James) Ji, K. J. Ray Liu. Fair multiuser channel allocation for OFDMA networks using Nash bargaining solutions and coalitions. IEEE Transactions on Communications,2005,53(8):1366-1376
[68]Suhas Mathur, Lalitha Sankar, Narayan B. Mandayam. Coalitions in cooperative wireless networks. IEEE Journal on Se lected Areas in Communications,2008,26(7):1104-1115
[69]A. R. Fattahi, F. Fu, M. Van Der Schaar, et al. Mechanism-based resource allocation for multimedia transmission over spectrum agile wireless networks,2007,25(3):601-612
[70]S. C. Su, M. Van Der Schaar. On the application of game-theoretic mechanism design for resource allocation in multimedia systems. IEEE Transactions on Multimedia,2008,10(6): 1197-1207
[71]F. Fu, M. Van Der Schaar. Noncollaborative resource management for wireless multim edia applications using mechanism design. IEEE Transactions on Multimedia,2007,9(4):851-868,
[72]F. Fu, T. M. Stoenescu, M. Van Der Schaar. A pricing mechanism for resource allocation in wireless multimedia a pplications. IEEE Journal of Selected Topics in Signal Process,2007, 1(2):264-279
[73]Y. Wu, B. Wang, K. J. R. Liu. Repeated spectrum sharing game with self-enforcing truth-telling mechanism. IEEE International Conference on Communications,2008, 3583-3587
[74]Y. Wu, B. Wang, K. J. R. Liu, et al. A multi-winner cognitive spectrum auction framework with collusion-resistant mechanisms. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2008,1-9
[75]B. Wang, Y. Wu, Z. Ji, et al. Game theoretical mechanism design methods. IEEE Signal Processing Magazine.2008,25(6):74-84
[76]Ulrich Derigs. Optimization and operations reseach. Oxford:EOLSS Publishers,2003:111-156
[77]F. Akyildiz, W. Lee, M. C. Vuran, et al. Next generation/dynamic spectrum access/cognitive radio wireless networks:a survey. Elsevier Computer Networks,2006,50(13):2127-2159
[78]Zhimei Jiang, Ge Ye, Ye (Geoffrey) Li. Max-utility wireless resource management for best-effort traffic. IEEE Transactions on Wirless Communications.2005,4(1):100-111
[79]Guocong Song, Ye (Geoffrey) Li. Utility-based resource allocation and scheduling in OFDM-based wireless broadband networks. IEEE Communications Magazine.2005,43(12): 127-134
[80]Guocong Song, Ye (Geoffrey) Li. Cross-layer optimization for OFDM wireless networks—part Ⅰ:theoretical framework.2005,4(2):614-624
[81]Guocong Song, Ye (Geoffrey). Li Cross-layer optimization for OFDM wireless networks—part Ⅱ:algorithm development.2005,4(2):614-624
[82]F. Yu, V. W. S. Wong, V. C. M. Leung. Efficient QoS provisioning for adaptive multimedia in mobile communication networks by reinforcement learning. ACM/Springer Mobile Networks and Applications Journal.2006,11(1):101-110
[83]Hui Tong, Timothy X. Brown. Adaptive call admission control under quality of service constraints:a reinforcement learning solution. IEEE Journal on Selected Areas in Communications.2000,18(2):209-221
[84]J. Nie, S. Haykin. A dynamic channel assignment policy through Q-learning. IEEE Transactions on Information Neural Networks,1999,10(6):1443-1455
[85]C.Y. Liao, F. Yu, V. C. M. Leung, et al. A novel dynamic cell configuration scheme in next-generation situation-aware CDMA networks. IEEE Transactions on Wireless Communications,2006,24(1):16-25
[86]C. J. Chang, B. W. Chen, T. Y. Liu, et al. Fuzzy/neural congestion control for integrated voice and data DS-CDMA/FRMA cellular networks. IEEE Transactions on Wireless Communications,18(2):283-293
[87]Nie Nie, C. Comaniciu. Adaptive channel allocation spectrum etiquette for cognitive radio networks. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, Baltimore,2005,269-278
[88]B. Moore, E. Ellesson, J. Strassner, et al. Policy core information model:[RFC], The Internet Society,2001
[89]D. P. Bertsekas, J. Tsitsiklis, Neuro-dynamic programming. Belmont, Massachusetts:Athena Scientific,1996:132-146
[90]Eric Rasmusen. Game and information:an introduction to game theory. Cambridge:Blackwell Publisher,1994:11-21
[91]D. Fudenberg, J. Tirole. Game theory. Cambridge:The MIT Press,1991:13-16
[92]Allen B. MacKenzie, Luiz A. DaSilva. Game theory for wireless engineers. NewYork:Morgan & Claypool Publishers,2006:1-77
[93]D. Niyato, E Hossain. Spectrum trading:an economics of radio resource sharing in cognitive radio. IEEE Wireless Communications Magazine,2008,15(6):71-80
[94]D. Niyato, E. Hossain. Market-equilibrium, competitive, and cooperative pricing for spectrum sharing in cognitive radio networks:analysis and comparison. IEEE Transactions on Wireless Communications,2008,7(11):4273-4283
[95]D. Niyato, E. Hossain. Competitive spectrum sharing in cognitive radio networks:a dynamic game approach. IEEE Transactions on Wireless Communications,2008,7(7):2651-2660
[96]D. Niyato, E. Hossain. Competitive pricing for spectrum sharing in cognitive radio networks: dynamic game, inefficiency of Nash equilibrium, and collusion. IEEE Journal on Selected Areas in Communications,2008,26(1):192-202
[97]Jerzy Filar, Koos Vrieze. Competitive Markov decision processes. NewYork:Springer, 1996:1-12
[98]Yalin Evren Sagduyu, Anthony Ephremides. Power control and rate adaptation as stochastic games for random access. IEEE International Conference on Decision and Control,2003, 4202-4207
[99]Y. E. Sagduyu, A. Ephremides. A game-theoretic look at simple relay channel. ACM/Kluwer Journal of Wireless Networks,2006,12(5):545-560
[100]E. Altman. Flow control using the theory of zero-sum Markov games. IEEE Transactions Automatic Control,1994,39(4):814-818
[101]Beibei Wang, Zhu Ji, K. J. Ray Liu. Self-learning repeated game framework for distributed primary-prioritized dynamic spectrum access. IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks,2007,631-638
[102]H. Zhu, C. Pandana, K. J. R. Liu. A self-learning repeated game framework for optimizing packet forwarding networks. IEEE Wireless Communications and Networking Conference, 2005,2131-2136
[103]Yanxiang He, Jun Xiao. Multi-agent based selfish routing for multi-channel wireless mesh networks. Autonomic and Trusted Computing,2006,4088:71-83
[104]A. Forster. Machine learning techniques applied to wireless ad-hoc networks:guide and survey. IEEE International Conference on Intelligent Sensors, Sensor Networks and Information, 2007,365-370
[105]Niklas Wirstrom. Optimization of wireless sensor networks using machine learning:[Thesis of M.S.], Stockholm:Royal Institute of Technology,2006
[106]C. Santivanez, R. Ramanathan, C. Partridge, et al. Opportunistic spectrum access:challenges, architecture, protocols. ACM Proceedings of the 2nd Annual International Workshop on Wireless Internet,2006,10-370
[107]Y. Xing, R. Chandramouli, S. Mangold. Dynamic spectrum access in open spectrum wireless networks. IEEE Journal on Selected Areas in Communications,2006,24(3):626-637
[108]Heng Wang, B. Narayan Mandayam. A simple packet-transmission scheme for wireless data over fading channels. IEEE Transactions on Communications,2004,52(7):1055-1059
[109]M. K. Simon, M. Alouini. Digital communication over fading channels. New York:John Wiley & Sons,2001,98-125
[110]E. Altman. Constrained Markov decision process:stochastic modeling. London:Chapman and Hall/CRC,1999,23-98
[111]M. L. Puterman. Markov decision process:discrete stochastic dynamic programming. New York:John Wiley & Sons,1994,12-32
[112]A. Karmokar, D. V. Djonin, V. K. Bhargava. Optimal and suboptimal packet scheduling over time-varying flat fading channels. IEEE Transactions on Wireless Communications,2006,5 (2):446-457
[113]A. Karmokar, D. V. Djonin, V. K. Bhargava. POMDP-based coding rate adaptation for type-ⅰ hybrid ARQ systems over fading channels with memory. IEEE Transactions on Wireless Communications,2006,5 (12):3512-3523
[114]Gunter Bolch, Stefan Greiner, Hermann de Meer. Queueing networks and Markov chains: modeling and performance evaluation with computer science applications. New York:John Wiley & Sons,2006,185-206
[115]林闯.计算机网络和计算机系统的性能评价.北京：清华大学出版社,2001,37-38
[116]S. Shakkottai, A. L. Stolyar. Scheduling for multiple flows sharing a time-varying channel:the exponential rule. American Mathematical Society Translations,2002,12(2):185-202
[117]Sridhar Mahadevan. Average reward reinforcement learning:foundations, algorithms, and empirical results. Machine Learning,1996,22(1):159-195
[118]A. Schwartz. A reinforcement learning method for maximizing undiscounted rewards. The Tenth International Conference on Machine Learning,1993,298-305
[119]Wayne L. Winston. Operations research:mathematical programming. New York:Wadsworth Inc.,1994,135-136
[120]F. J. Beutler and K. W. Ross. Optimal policies for controlled Markov chains with a constraint. Journal of Mathematical Analysis and Application,1985,112(1):236-252
[121]S. Gandhi, C. Buragohain, L. Cao, H. Zheng, et al. Towards real-time dynamic spectrum auctions. Elsevier Computer Networks,2008,52(5):879-897
[122]Xia Zhou, Sorabh Gandi, Subhash Suri, et al. Ebay in the sky:strategy-proof wireless spectrum auctions. ACM International Conference on Mobile Computing and Networking, 2008,2-13
[123]Xia Zhou, Shravan Mettu, Haitao Zheng, et al. Traffic-driven dynamic spectrum auctions. IEEE Workshop on Networking Technologies for Software Defined Radio Networks,2008, 1-6
[2]R. W. Broderson, A. Wolisz, D. Cabric, et al. CORVUS:a cognitive radio approach for usage of virtual unlicensed spectrum:[White Paper], Berkeley:UC Berkeley Wireless Research Center,2004
[3]Matthias Wellens, Jin Wu, Petri M"ah"onen. Evaluation of spectrum occupancy in indoor and outdoor scenario in the context of cognitive radio. IEEE International Conference on Cognitive Radio Oriented Wireless Networks and Communications,2007,420-427
[4]Qusay H. Mahmoud. Cognitive networks:towards self-aware networks. New Jersey:John Wiley & Sons Inc.,2007,2-128
[5]Frank H.P. Fitzek, Marcos D. Katz. Cognitive wireless networks:concepts, methodologies and visions inspiring:the age of enlightenment of wireless communications. Netherlands:Springer, 2007,12-56
[6]Ekram Hossain, Vijay Bhargava. Cognitive wireless communication networks. New York: Springer,2007,23-36
[7]Joseph Mitola Ⅲ, G. Q. Maguire. Cognitive radio:making software radios more personal. IEEE Personal Communications,1999,6(4):13-18
[8]Joseph Mitola Ⅲ. Cognitive radio for flexible mobile multimedia communications. Sixth International Workshop on Mobile Multimedia Communications,1999,3-10
[9]Joseph Mitola Ⅲ. Cognitive radio:an integrated agent architecture for software defined radio: [Ph.D. Dissertation], Stockholm:Royal Institute of Technology,2000
[10]T. A. Weiss and F. K. Jondral. Spectrum pooling:an innovative strategy for the enhancement of spectrum efficiency. IEEE Radio Communications,2004,42(3):8-14
[11]T. Weiss, A. Krohn, J. Hillenbrand, et al. Efficient signaling of spectral resources in spectrum pooling systems. Proc. of the 10th Symposium on Communications and Vehicular Technology, 2003,1:1-6
[12]T. Weiss, A. Krohn, F. Jondral. Synchronization algorithms and preamble concepts for spectrum pooling systems. Proc. of the Mobile & Wireless Communications Summit,2003,1: 788-792
[13]F. Capar, I. Martoyo, T. Weiss, et al. Comparison of bandwidth utilization for controlled and uncontrolled channel assignment in a spectrum pooling system. IEEE Vehicular Technology Conference,2007,3:1069-1073
[14]M. Muck, D. Bourse, K. Moessner, et al. End-to-end reconfigurability in heterogeneous wireless systems-software and cognitive radio solutions enriched by policy-and context-based decision making.16th IST Mobile and Wireless Communications Summit,2007,1-5
[15]Bruce A. Fette. Cognitive radio technology. Oxford:Elsevier Inc.,2006,1-25
[16]Narayan Mandayam, Christopher Rose, Predrag Spasojevic, et al. Nets-prowin:cognitive radios for open access to spectrum:[Project Summary], New Jersey:Rutgers University,2005
[17]B. Ackland, M. Bushnell, C. Rose, et al. Nets-prowin:high performance cognitive radio platform with integrated physical and network layer capabilities:[Project Summary], New Jersey:Rutgers University,2005
[18]Jun Zhao, Haitao Zheng, Guang Hua Yang. Distributed coordination in dynamic sspectrum allocation networks. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,259-268
[19]J. Zhao, H. Zheng, G. Yang. Spectrum sharing through distributed coordination in dynamic spectrum access networks. Wireless Communications and Mobile Computing,2007,7(9): 1061-1075
[20]George Dimitrakopoulos, Panagiotis Demestichas, David Grandblaise, et al. Congnitive radio, spectrum and radio resource management:[White Paper], Wireless World Research Forum Working Group 6,2004
[21]C. Cordeiro, K. Challapali, D. Birru, et al. IEEE 802.22:the first worldwide wireless standard based on cognitive radio. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,328-337
[22]XG Working Group. The XG vision:[RFC], Massachusetts:BBN Technologies,2004
[23]XG Working Group. The XG architecture framework:[RFC], Massachusetts:BBN Technologies,2004
[24]XG Working Group. XG policy language framework:[RFC], Massachusetts:BBN Technologies,2004
[25]Joseph Mitola III. Cognitive radio architecture:the engineering foundations of radio XML. New Jersey:John Wiley & Sons Inc.,2006,236-275
[26]Simon Haykin. Cognitive radio:brain-empowered wireless communications. IEEE Journal on Selected Areas in Communications,2005,23(2):201-220
[27]Ryan W. Thomas, Daniel H. Friend, Luiz A. Dasilva, et al. Cognitive networks:adaptation and learning to achieve end-to-end performance objectives. IEEE Communications Magazine, 2006,44(12):51-57
[28]Gerhard Weiss. Multiagent systems:a modern approach to distributed artificial intelligence. Cambridge:MIT Press,1999,121-146
[29]Q. Zhao, L. Tong, A. Swami, Y. Chen. Decentralized cognitive mac for opportunistic spectrum access in ad hoc networks:a POMDP framework. IEEE Journal Selected Areas in Communications,2007,25(3):589-600
[30]Q. Zhao, L. Tong, and A. Swamiz. Decentralized cognitive mac for dynamic spectrum access. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005,224-232
[31]Yunxia Chen, Qing Zhao, A. Swami. Joint PHY-MAC design for opporunistic spectrum access with multi-channel. IEEE International Workshop on Signal Processing Advances in Wireless Communications,2007,1-5
[32]Jiang Zhu, Ning Wei, Bingyang Xu, et al. Optimal and suboptimal access and transmission polices for dynamic spectrum access over fading channels in cognitive radio networks. Chinese Journal of Electronics.2008,17(4):726-732
[33]Hyoil Kim, K. G. Shin. Efficient discovery of spectrum opportunities with MAC-layer sensing in cognitive radio networks. IEEE Transactions on Mobile Computing,2008,7(5):533-545
[34]Q. C. Zhao, S. Geirhofer, L. Tong, and B. M. Sadler, Optimal dynamic spectrum access via periodic channel sensing, IEEE Wireless Communications and Networking Conference,2007, 33-37
[35]S. Geirhofer, L. Tong, B. M. Sadler. Cognitive medium access:constraining interference based on experimental models. IEEE Journal Selected Areas in Communications,2008,26 (1): 95-105
[36]C. T. Chou, Hyoil Kim, Sai Shankar, et al. What and how much to gain by spectral agility. IEEE Journal Selected Areas in Communications,2007,25 (3):576-588
[37]C. Clancy, J. Hecker, E. Stuntebeck, et al. Applications of machine learning to cognitive radio networks. IEEE Wireless Communications Magzine,2007,14 (4):47-52
[38]FCC. Establishment of interference temperature metric to quantify and manage interference and to expand available unlicensed operation in certain fixed mobile and satellite frequency bands. Notice of Inquiry and Proposed Rulemaking, ET Docket No.03-289,2003
[39]FCC. In the matter of establishment of an interference temperature metric to quantify and management interference and to expand available unlicensed operation in certain fixed, mobile and satellite frequency bands. Notice of Inquiry and Notice of Proposed Rule Making, ET Docket No.03-237, November 13,2003
[40]T. Clancy, W. Arbaugh. Measuring interference temperature:[Technical Report], Virginia: Virginia Tech,2006
[41]T. Clancy. Formalizing the interference temperature model. Wiley Journal on Wireless Communications and Mobile Computing,2007,7(9):1077-1086
[42]J. Bater, H. P. Tan, K. Brown, et al. Modelling interference temperature constraints for spectrum access in cognitive radio network. IEEE International Conference on Communications,2007,6493-6498
[43]Y. Xing, C. N. Mathur, M. A. Haleem, et al. Dynamic spectrum access with QOS and interference temperature constraints. IEEE Transactions on Mobile Computing,2007,6(4): 423-433
[44]Jie Chen, Chulin Liao, Chuan Han, et al. Extending available spectrum in cognitive radio: hierarchical spectrum sharing network. Elsevier Computer Networks,2008,52(4):850-863
[45]J. Huang, R. Berry, and M. L. Honig. Auction-based spectrum sharing. ACM/Springer Mobile Networks and Applications,2006,11(3):405-418
[46]Ramin Rezaiifar, M. Makowski, Srikanta Kumar. Stochastic control of handoffs in cellular networks. IEEE Journal on Selected Areas in Communications,1995,13(7):1348-1362
[47]H. S. Wang and N. Moayeri. Finite-state Markov channel-a useful model for radio communication channels. IEEE Transactions on Vehicular Technology,1995,44 (1):163-171
[48]D. V. Djonin, V. Krishnamurthy. Q-learning algorithms for constrained Markov decision processes with randomized monotone policies:application to MIMO transmission control. IEEE Transactions on Signal Processing,2007,55(5):2170-2181
[49]D. V. Djonin, V. Krishnamurthy. MIMO transmission control in fading channels:a constrained Markov decision process formulation with monotone randomized policies. IEEE Transactions on Signal Processing,2007,55(10):5069-5083
[50]Md. J. Hossain, D. V. Djonin, V. K. Bhargava. Delay limited optimal and suboptimal power and bit loading algorithms for OFDM systems over correlated fading. IEEE Global Communications Conference,2005,3448-3453
[51]朱江,黄海洋,李少谦.一种基于认知无线电技术的能效传输控制方案.计算机工程与应用,2008,44(31)：28-32
[52]Yunxia Chen, Qing Zhao, A. Swami. Distributed cognitive MAC for energy-constrained opportunistic spectrum access. IEEE Military Communications Conference,2006,1-7
[53]Yunxia Chen, Qing Zhao. Bursty traffic in energy-constrained opportunistic spectrum access. IEEE Global Communications Conference,2007,4641-4646
[54]Charles Pandana, K. J. Ray Liu. Near-optimal reinforcement learning framework for energy-aware sensor communications. IEEE Transactions on Wireless Communications,2005, 23(4):788-797
[55]Jianying Li, Binyang Xu, Zhangjing Xu, et al. adaptive packet scheduling algorithm for cognitive radio system. IEEE International Conference on Communication Technology,2006, 111-115.
[56]徐斌阳,李健颖,李少谦.认知无线电系统频谱自适应调度算法设计.电子科技大学学报,2008,37(6)：815-818
[57]F. Fu, M. Van Der Schaar. Dynamic spectrum sharing using learning for delay-sensitive applications. IEEE International Conference on Communications,2008,2825-2829
[58]Amir Ghasemi, E. S. Sousa. Collaborative spectrum sensing for opportunistic access in fading environment. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,131-136
[59]E. Visotsky, S. Kuffner, R. Peterson. On collaborative detection of TV transmissions in support of dynamic spectrum sharing. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,338-345
[60]D. Grandblaise, C. Kloeck, K. Moessner, et al. Techno-economic of collaborative based secondary spectrum usage-E2R research project outcomes overview.2005 IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,318-327
[61]S. M. Mishra, A. Sahai and R. W. Brodersen. Cooperative sensing among cognitive radios. IEEE International Conference on communications,2006,1658-1663
[62]H. Zheng, C. Peng. Collaboration and fairness in opportunistic spectrum access. IEEE International Conference on Communication,2005,3132-3136
[63]Savo G. Glisic. Advanced wireless communications:4G cognitive and cooperative broadband technology. Chichester:John Wiley & Sons Ltd.,2007,537-587
[64]J. Neel, J. H. Reed, R. P. Gilles. The role of game theory in the analysis of software radio networks. SDR Forum Technical Conference,2002, NP-3-02
[65]J. Neel, J. H. Reed, R. P. Gilles. Convergence of cognitive radio networks. Wireless Communications and Networking Conference,2004,2250-2255.
[66]C. Kloeck, H. Jaekel, F. K. Jondral. Dynamic and local combined pricing, allocation and billing system with cognitive radios. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2005,73-81
[67]Han Zhu, Zhu (James) Ji, K. J. Ray Liu. Fair multiuser channel allocation for OFDMA networks using Nash bargaining solutions and coalitions. IEEE Transactions on Communications,2005,53(8):1366-1376
[68]Suhas Mathur, Lalitha Sankar, Narayan B. Mandayam. Coalitions in cooperative wireless networks. IEEE Journal on Se lected Areas in Communications,2008,26(7):1104-1115
[69]A. R. Fattahi, F. Fu, M. Van Der Schaar, et al. Mechanism-based resource allocation for multimedia transmission over spectrum agile wireless networks,2007,25(3):601-612
[70]S. C. Su, M. Van Der Schaar. On the application of game-theoretic mechanism design for resource allocation in multimedia systems. IEEE Transactions on Multimedia,2008,10(6): 1197-1207
[71]F. Fu, M. Van Der Schaar. Noncollaborative resource management for wireless multim edia applications using mechanism design. IEEE Transactions on Multimedia,2007,9(4):851-868,
[72]F. Fu, T. M. Stoenescu, M. Van Der Schaar. A pricing mechanism for resource allocation in wireless multimedia a pplications. IEEE Journal of Selected Topics in Signal Process,2007, 1(2):264-279
[73]Y. Wu, B. Wang, K. J. R. Liu. Repeated spectrum sharing game with self-enforcing truth-telling mechanism. IEEE International Conference on Communications,2008, 3583-3587
[74]Y. Wu, B. Wang, K. J. R. Liu, et al. A multi-winner cognitive spectrum auction framework with collusion-resistant mechanisms. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks,2008,1-9
[75]B. Wang, Y. Wu, Z. Ji, et al. Game theoretical mechanism design methods. IEEE Signal Processing Magazine.2008,25(6):74-84
[76]Ulrich Derigs. Optimization and operations reseach. Oxford:EOLSS Publishers,2003:111-156
[77]F. Akyildiz, W. Lee, M. C. Vuran, et al. Next generation/dynamic spectrum access/cognitive radio wireless networks:a survey. Elsevier Computer Networks,2006,50(13):2127-2159
[78]Zhimei Jiang, Ge Ye, Ye (Geoffrey) Li. Max-utility wireless resource management for best-effort traffic. IEEE Transactions on Wirless Communications.2005,4(1):100-111
[79]Guocong Song, Ye (Geoffrey) Li. Utility-based resource allocation and scheduling in OFDM-based wireless broadband networks. IEEE Communications Magazine.2005,43(12): 127-134
[80]Guocong Song, Ye (Geoffrey) Li. Cross-layer optimization for OFDM wireless networks—part Ⅰ:theoretical framework.2005,4(2):614-624
[81]Guocong Song, Ye (Geoffrey). Li Cross-layer optimization for OFDM wireless networks—part Ⅱ:algorithm development.2005,4(2):614-624
[82]F. Yu, V. W. S. Wong, V. C. M. Leung. Efficient QoS provisioning for adaptive multimedia in mobile communication networks by reinforcement learning. ACM/Springer Mobile Networks and Applications Journal.2006,11(1):101-110
[83]Hui Tong, Timothy X. Brown. Adaptive call admission control under quality of service constraints:a reinforcement learning solution. IEEE Journal on Selected Areas in Communications.2000,18(2):209-221
[84]J. Nie, S. Haykin. A dynamic channel assignment policy through Q-learning. IEEE Transactions on Information Neural Networks,1999,10(6):1443-1455
[85]C.Y. Liao, F. Yu, V. C. M. Leung, et al. A novel dynamic cell configuration scheme in next-generation situation-aware CDMA networks. IEEE Transactions on Wireless Communications,2006,24(1):16-25
[86]C. J. Chang, B. W. Chen, T. Y. Liu, et al. Fuzzy/neural congestion control for integrated voice and data DS-CDMA/FRMA cellular networks. IEEE Transactions on Wireless Communications,18(2):283-293
[87]Nie Nie, C. Comaniciu. Adaptive channel allocation spectrum etiquette for cognitive radio networks. IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, Baltimore,2005,269-278
[88]B. Moore, E. Ellesson, J. Strassner, et al. Policy core information model:[RFC], The Internet Society,2001
[89]D. P. Bertsekas, J. Tsitsiklis, Neuro-dynamic programming. Belmont, Massachusetts:Athena Scientific,1996:132-146
[90]Eric Rasmusen. Game and information:an introduction to game theory. Cambridge:Blackwell Publisher,1994:11-21
[91]D. Fudenberg, J. Tirole. Game theory. Cambridge:The MIT Press,1991:13-16
[92]Allen B. MacKenzie, Luiz A. DaSilva. Game theory for wireless engineers. NewYork:Morgan & Claypool Publishers,2006:1-77
[93]D. Niyato, E Hossain. Spectrum trading:an economics of radio resource sharing in cognitive radio. IEEE Wireless Communications Magazine,2008,15(6):71-80
[94]D. Niyato, E. Hossain. Market-equilibrium, competitive, and cooperative pricing for spectrum sharing in cognitive radio networks:analysis and comparison. IEEE Transactions on Wireless Communications,2008,7(11):4273-4283
[95]D. Niyato, E. Hossain. Competitive spectrum sharing in cognitive radio networks:a dynamic game approach. IEEE Transactions on Wireless Communications,2008,7(7):2651-2660
[96]D. Niyato, E. Hossain. Competitive pricing for spectrum sharing in cognitive radio networks: dynamic game, inefficiency of Nash equilibrium, and collusion. IEEE Journal on Selected Areas in Communications,2008,26(1):192-202
[97]Jerzy Filar, Koos Vrieze. Competitive Markov decision processes. NewYork:Springer, 1996:1-12
[98]Yalin Evren Sagduyu, Anthony Ephremides. Power control and rate adaptation as stochastic games for random access. IEEE International Conference on Decision and Control,2003, 4202-4207
[99]Y. E. Sagduyu, A. Ephremides. A game-theoretic look at simple relay channel. ACM/Kluwer Journal of Wireless Networks,2006,12(5):545-560
[100]E. Altman. Flow control using the theory of zero-sum Markov games. IEEE Transactions Automatic Control,1994,39(4):814-818
[101]Beibei Wang, Zhu Ji, K. J. Ray Liu. Self-learning repeated game framework for distributed primary-prioritized dynamic spectrum access. IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks,2007,631-638
[102]H. Zhu, C. Pandana, K. J. R. Liu. A self-learning repeated game framework for optimizing packet forwarding networks. IEEE Wireless Communications and Networking Conference, 2005,2131-2136
[103]Yanxiang He, Jun Xiao. Multi-agent based selfish routing for multi-channel wireless mesh networks. Autonomic and Trusted Computing,2006,4088:71-83
[104]A. Forster. Machine learning techniques applied to wireless ad-hoc networks:guide and survey. IEEE International Conference on Intelligent Sensors, Sensor Networks and Information, 2007,365-370
[105]Niklas Wirstrom. Optimization of wireless sensor networks using machine learning:[Thesis of M.S.], Stockholm:Royal Institute of Technology,2006
[106]C. Santivanez, R. Ramanathan, C. Partridge, et al. Opportunistic spectrum access:challenges, architecture, protocols. ACM Proceedings of the 2nd Annual International Workshop on Wireless Internet,2006,10-370
[107]Y. Xing, R. Chandramouli, S. Mangold. Dynamic spectrum access in open spectrum wireless networks. IEEE Journal on Selected Areas in Communications,2006,24(3):626-637
[108]Heng Wang, B. Narayan Mandayam. A simple packet-transmission scheme for wireless data over fading channels. IEEE Transactions on Communications,2004,52(7):1055-1059
[109]M. K. Simon, M. Alouini. Digital communication over fading channels. New York:John Wiley & Sons,2001,98-125
[110]E. Altman. Constrained Markov decision process:stochastic modeling. London:Chapman and Hall/CRC,1999,23-98
[111]M. L. Puterman. Markov decision process:discrete stochastic dynamic programming. New York:John Wiley & Sons,1994,12-32
[112]A. Karmokar, D. V. Djonin, V. K. Bhargava. Optimal and suboptimal packet scheduling over time-varying flat fading channels. IEEE Transactions on Wireless Communications,2006,5 (2):446-457
[113]A. Karmokar, D. V. Djonin, V. K. Bhargava. POMDP-based coding rate adaptation for type-ⅰ hybrid ARQ systems over fading channels with memory. IEEE Transactions on Wireless Communications,2006,5 (12):3512-3523
[114]Gunter Bolch, Stefan Greiner, Hermann de Meer. Queueing networks and Markov chains: modeling and performance evaluation with computer science applications. New York:John Wiley & Sons,2006,185-206
[115]林闯.计算机网络和计算机系统的性能评价.北京：清华大学出版社,2001,37-38
[116]S. Shakkottai, A. L. Stolyar. Scheduling for multiple flows sharing a time-varying channel:the exponential rule. American Mathematical Society Translations,2002,12(2):185-202
[117]Sridhar Mahadevan. Average reward reinforcement learning:foundations, algorithms, and empirical results. Machine Learning,1996,22(1):159-195
[118]A. Schwartz. A reinforcement learning method for maximizing undiscounted rewards. The Tenth International Conference on Machine Learning,1993,298-305
[119]Wayne L. Winston. Operations research:mathematical programming. New York:Wadsworth Inc.,1994,135-136
[120]F. J. Beutler and K. W. Ross. Optimal policies for controlled Markov chains with a constraint. Journal of Mathematical Analysis and Application,1985,112(1):236-252
[121]S. Gandhi, C. Buragohain, L. Cao, H. Zheng, et al. Towards real-time dynamic spectrum auctions. Elsevier Computer Networks,2008,52(5):879-897
[122]Xia Zhou, Sorabh Gandi, Subhash Suri, et al. Ebay in the sky:strategy-proof wireless spectrum auctions. ACM International Conference on Mobile Computing and Networking, 2008,2-13
[123]Xia Zhou, Shravan Mettu, Haitao Zheng, et al. Traffic-driven dynamic spectrum auctions. IEEE Workshop on Networking Technologies for Software Defined Radio Networks,2008, 1-6