Parallel inference for massive distributed spatial data using low-rank models

详细信息    查看全文

• 作者：Matthias Katzfuss ; Dorit Hammerling
• 关键词：Distributed computing ; Gaussian process ; Particle filter ; Predictive process ; Spatial random effects model ; Spatio ; temporal statistics
• 刊名：Statistics and Computing
• 出版年：2017
• 出版时间：March 2017
• 年：2017
• 卷：27
• 期：2
• 页码：363-375
• 全文大小：
• 刊物类别：Mathematics and Statistics
• 刊物主题：Statistics and Computing/Statistics Programs; Artificial Intelligence (incl. Robotics); Statistical Theory and Methods; Probability and Statistics in Computer Science;
• 出版者：Springer US
• ISSN：1573-1375
• 卷排序：27

文摘

Due to rapid data growth, statistical analysis of massive datasets often has to be carried out in a distributed fashion, either because several datasets stored in separate physical locations are all relevant to a given problem, or simply to achieve faster (parallel) computation through a divide-and-conquer scheme. In both cases, the challenge is to obtain valid inference that does not require processing all data at a single central computing node. We show that for a very widely used class of spatial low-rank models, which can be written as a linear combination of spatial basis functions plus a fine-scale-variation component, parallel spatial inference and prediction for massive distributed data can be carried out exactly, meaning that the results are the same as for a traditional, non-distributed analysis. The communication cost of our distributed algorithms does not depend on the number of data points. After extending our results to the spatio-temporal case, we illustrate our methodology by carrying out distributed spatio-temporal particle filtering inference on total precipitable water measured by three different satellite sensor systems.