Dynamic federation of grid and cloud storage
详细信息 查看全文
- 作者：Fabrizio Furano ; Oliver Keeble ; Laurence Field
- 关键词：Cloud storage ; Data Bridge ; authentication ; authorization ; performance ; scalability ; Dynafed ; Dynamic Federation ; S3 ; Amazon ; Web ; Azure ; HTTP ; WebDAV
- 刊名：Physics of Particles and Nuclei Letters
- 出版年：2016
- 出版时间：September 2016
- 年：2016
- 卷：13
- 期：5
- 页码：629-633
- 全文大小：292 KB
- 刊物类别：Physics and Astronomy
- 刊物主题：Physics
Elementary Particles and Nuclei
Russian Library of Science - 出版者：MAIK Nauka/Interperiodica distributed exclusively by Springer Science+Business Media LLC.
- ISSN：1531-8567
- 卷排序：13
文摘
The Dynamic Federations project (“Dynafed”) enables the deployment of scalable, distributed storage systems composed of independent storage endpoints. While the Uniform Generic Redirector at the heart of the project is protocol-agnostic, we have focused our effort on HTTP-based protocols, including S3 and WebDAV. The system has been deployed on testbeds covering the majority of the ATLAS and LHCb data, and supports geography-aware replica selection. The work done exploits the federation potential of HTTP to build systems that offer uniform, scalable, catalogue-less access to the storage and metadata ensemble and the possibility of seamless integration of other compatible resources such as those from cloud providers. Dynafed can exploit the potential of the S3 delegation scheme, effectively federating on the fly any number of S3 buckets from different providers and applying a uniform authorization to them. This feature has been used to deploy in production the BOINC Data Bridge, which uses the Uniform Generic Redirector with S3 buckets to harmonize the BOINC authorization scheme with the Grid/X509. The Data Bridge has been deployed in production with good results. We believe that the features of a loosely coupled federation of open-protocolbased storage elements open many possibilities of smoothly evolving the current computing models and of supporting new scientific computing projects that rely on massive distribution of data and that would appreciate systems that can more easily be interfaced with commercial providers and can work natively with Web browsers and clients.