Streamlining Access to Joint Polar Satellite System (JPSS) Data Products for Weather Forecasting (358)
John Evans, , ,
Distributing data products from the Joint Polar Satellite System (JPSS) and other satellites requires a highly efficient, flexible, resilient, and scalable information infrastructure. The National Weather Service (NWS) currently delivers these data to forecasters over its Satellite Broadcast Network via standing subscriptions; but the data streams from JPSS, GOES-R, and other satellite families are quickly outgrowing what this mechanism can deliver. To ensure effective access to and use of these data, the National Environmental Satellite, Data, and Information Service (NESDIS) Environmental Satellite Product Distribution Service (ESPDS) is testing an on-demand Web service capability based on the Open Geospatial Consortium Web Coverage Service standard, with data transfer via fiber-optic TCP/IP networks, for near-real-time users nationwide.The volume, velocity, and variety of JPSS data products impose several challenges to such a service. It must be efficient to handle large volumes of complex, frequently updated data, and to fulfill many concurrent requests. It must offer flexible data handling and delivery, to work with a diverse and changing collection of data, and to tailor its outputs on-demand into products that users need, including spatiotemporal subsetting and aggregation, cartographic projection, and data compression. Filtering by variable (/ band / layer) may also boost user flexibility and alleviate network loads; to that end we are investigating how best to provide random access by location, time, and parameter into large data collectins -- in particular the suite of JPSS Enterprise NetCDF data products. The service must scale to ensure near-real-time performance despite rapid changes in data volume, variety, and demand as new satellites launch, as more products come online, and as users rely increasingly on the service. This requires the use of Big Data methods such as spatial indexing, geolocation approximations, parallelization, tiling, and caching. Finally, the service must interoperate with other complex systems, including the Advanced Weather Information Processing System (AWIPS) used by operational NWS forecasters.The poster sketches the goals, challenges, and current progress towards widespread access to the full suite of JPSS data products for operational weather forecasting.