β Data Sources#
Thanks to the NOAA Big Data Project, more meteorological data is more easily accessible than ever before. Herbie enables you to download NWP model output from different data sources, wherever the data you are interested in is available. Because NWP data can be hosted on different remote servers, Herbie will look at different sources to find where a file exists. This is important when working with real-time versus archived data. For instance, model output on NOAMDS is only there for 2-14 days, depending on the model.
Herbie sets a default βpriorityβ that tells Herbie the order to search different sources for a file. The available data sources is defined for each model by the model template file and additional sources can be added. The default search order is aws, nomads, google, azure, pando, pando2.
For example, the following are the most popular data sources:
- AWS: Amazon Web Services
Herbie string,
'aws'
.NOAA Big Data Program
Slight latency for real-time products.
- NOMADS: NOAA Operational Model Archive and Distribution System
Herbie string,
'nomads'
.Original data source. All available data included.
Newest forecast products.
Available for todayβs and yesterdayβs runs for HRRR.
π¨ Download limits.
- Google: Google Cloud Platform Earth
Herbie string,
'google'
.NOAA Big Data Program
Slight latency for real-time products.
- Azure: Microsoft Azure
Herbie string,
'azure'
.- NOAA Big Data Program
Only recent HRRR and RAP data? Subset?
ECMWF open data forecast and wave products
- Pando: The University of Utah HRRR archive
Herbie string,
'pando'
and'pando2'
.Research archive. Older files being removed.
- ECMWF Open Data
Herbie string,
'ecmwf'
ECMWF open data forecast and wave products