πͺ Subset with searchString
#
Subsetting is done using the GRIB2 index files. Index files define the grib variables/parameters of each message (sometimes it is useful to think of a grib message as a βlayerβ of the file) and define the byte range of the message.
Herbie can subset a file by grib message by downloading a byte range of the file. This way, instead of downloading the full file, you can download just the βlayerβ of the file you want. The searchString method implemented in Herbie to do a partial download is similar to what is explained here: https://www.cpc.ncep.noaa.gov/products/wesley/fast_downloading_grib.html
Herbie supports reading two different types of index files
Index files output by the wgrib2 command-line utility. These index files are common for forecast models provided by NCEP.
Index files output by the ecCodes/grib_ls command-line utlity. These index files are common for forecast models provided by ECMWF.
You can use regular expression to search for lines in the index file. If H
is a Herbie object, the regex search is performed on the H.read_idx().search_this
column of the DataFrame
Tip
If you need help with regular expression, search the web or look at this cheatsheet. Check regular expressions with regexr or regex101.
wgrib2-style index files#
Here are some examples you can use for the searchString
argument for the wgrib2-style index files.
|
GRIB messages that will be downloaded |
---|---|
|
Temperature at 2 m. |
|
Temperature fields at all levels. |
|
U Wind at all pressure levels. |
|
All variables on the 500 mb level. |
|
All accumulated precipitation fields. |
|
Accumulated precip since initialization time |
|
Accumulated precip over last hour |
|
U wind component at 10 meters. |
|
U and V wind component at 10 and 80 m. |
|
U and V wind component at all levels. |
|
U and V wind components at all hybrid levels |
|
U and V wind components at all pressure levels |
|
(Same as above) |
|
Temperature and Dew Point for all levels . |
|
TMP, DPT, and Relative Humidity for all levels. |
|
Composite Reflectivity |
|
All variables at the surface. |
|
Beginning of string (^), end of string ($) wildcard (.*) |
Hint
The NCEP Parameters & Units Table is a useful resource to help you identify wgrib2-style GRIB variable abbreviations and their meanings.
ecCodes-style index files#
Here are some examples you can use for the searchString
argument for the grib_ls-style index files.
Look at the ECMWF GRIB Parameter Database https://apps.ecmwf.int/codes/grib/param-db
This table is for the operational forecast product (and ensemble product):
searchString (oper/enso) |
Messages that will be downloaded |
---|---|
β:2t:β |
2-m temperature |
β:10u:β |
10-m u wind vector |
β:10v:β |
10-m v wind vector |
β:10(u|v): |
10m u and 10m v wind |
β:d:β |
Divergence (all levels) |
β:gh:β |
geopotential height (all levels) |
β:gh:500β |
geopotential height only at 500 hPa |
β:st:β |
soil temperature |
β:tp:β |
total precipitation |
β:msl:β |
mean sea level pressure |
β:q:β |
Specific Humidity |
β:r:β |
relative humidity |
β:ro:β |
Runn-off |
β:skt:β |
skin temperature |
β:sp:β |
surface pressure |
β:t:β |
temperature |
β:tcwv:β |
Total column vertically integrated water vapor |
β:vo:β |
Relative vorticity |
β:v:β |
v wind vector |
β:u:β |
u wind vector |
β:(t|u|v|r):β |
Temp, u/v wind, RH (all levels) |
β:500:β |
All variables on the 500 hPa level |
This table is for the wave product (and ensemble wave product):
searchString (wave/waef) |
Messages that will be downloaded |
---|---|
β:swh:β |
Significant height of wind waves + swell |
β:mwp:β |
Mean wave period |
β:mwd:β |
Mean wave direction |
β:pp1d:β |
Peak wave period |
β:mp2:β |
Mean zero-crossing wave period |
Hint
The ECMWF Parameter Database is a useful resource to help you identify ecCodes-style GRIB variable abbreviations and their meanings.