☁ Data Sources#

Thanks to the NOAA Big Data Project, more meteorological data is more easily accessible than ever before. Herbie enables you to download NWP model output from different data sources, wherever the data you are interested in is available. Because NWP data can be hosted on different remote servers, Herbie will look at different sources to find where a file exists. This is important when working with real-time versus archived data. For instance, model output on NOAMDS is only there for 2-14 days, depending on the model.

Herbie sets a default β€œpriority” that tells Herbie the order to search different sources for a file. The available data sources is defined for each model by the model template file and additional sources can be added. The default search order is aws, nomads, google, azure, pando, pando2.

For example, the following are the most popular data sources:

  1. AWS: Amazon Web Services
    • Herbie string, 'aws'.

    • NOAA Big Data Program

    • Slight latency for real-time products.

  2. NOMADS: NOAA Operational Model Archive and Distribution System
    • Herbie string, 'nomads'.

    • Original data source. All available data included.

    • Newest forecast products.

    • Available for today’s and yesterday’s runs for HRRR.

    • 🚨 Download limits.

  3. Google: Google Cloud Platform Earth
    • Herbie string, 'google'.

    • NOAA Big Data Program

    • Slight latency for real-time products.

  4. Azure: Microsoft Azure
    • Herbie string, 'azure'.

    • NOAA Big Data Program
      • Only recent HRRR and RAP data? Subset?

    • ECMWF open data forecast and wave products

  5. Pando: The University of Utah HRRR archive
    • Herbie string, 'pando' and 'pando2'.

    • Research archive. Older files being removed.

  6. ECMWF Open Data
    • Herbie string, 'ecmwf'

    • ECMWF open data forecast and wave products