HRRR Download Scripting Tips

If you just want to download a few files, the easiest way is with the web download interface which makes downloading files as easy as clicking a button.

Most users, however, need to download a bunch of HRRR files. You will need to write a script for that. The most simple way to download a file from Pando is with urllib.request. Writing a script is as simple as changing the URL to get the files you want.

import urllib.request
download_this = 'https://pando-rgw01.chpc.utah.edu/hrrr/sfc/20200624/hrrr.t01z.wrfsfcf17.grib2'
save_as = 'my_file.grib2'
urllib.request.urlretrieve(download_this, save_as)

I made this Jupyter Notebook for you. It demonstrates how to use my download_HRRR function to download many files from the Pando (or NOMADS) HRRR archive. All you need to give the function is a list of datetimes and forecast hours.

Jupyter Notebook: Part 1

Full HRRR files are very large (~150 MB) and that adds up quick if you need a lot of days and forecasts. Often, you only need some of the data in the file. Remember that GRIB2 data is based on messages made up of binary concatenated together. It is possible to download only the parts of the file for specific variables or "fields" that you want.

You will save a lot of disk space and improve download time if you download just the variables you need. The size of a single HRRR grid is about 1 MB.

The challenge to downloading parts of the full GRIB2 file finding the byte range for a variable you want. The beginning byte of each variable is given in the index, or .idx, file.

Partial downloads with cURL require a known byte range. The grbi2.idx ( sfc example , prs example) files are metadata text files that contain the beginning byte of each field in the file. Each grib2 file has a unique index file. To find the byte range for a variable, the above function searches for the line that contains the specified variable abbreviation and level.

My Jupyter Notebook Part 3 shows you how I do this and provide a function to download a set of variables from a HRRR GRIB2 file. Part 3 combines the functions for downloading many files and downloading subsets.

Jupyter Notebook: Part 2 Jupyter Notebook: Part 3

When you have a GRIB2 file, there are two key packages that read GRIB2 files. Both can be installed via conda-forge.

1. pygrib is what I started to learn and still use sometimes. | Video Demo | Documentation.
2. cfgrib works well reading GRIB2 data as xarray datasets. Make sure you have the latest version (>0.9.8) | Documentation

Getting Started with cfgrib

import cfgrib

a = cfgrib.open_datasets('hrrr.t00z.wrfsfcf08.grib2')

The loaded data is a list of xarray dataset objects. Each item in the list is an xarray dataset for a different level. For example, entire atmosphere, height above ground, isobaric level, surface, etc.

This is a good method to use if you don't know the file names on Pando (such as if you are looking at GOES files), but the HRRR file names are very predictable. The following examples show how to download the files with the straightforward urllib.request and how to download a subset of a files (single variable grid) by exploiting the byte range feature of cURL.

Latitude and longitude for every HRRR grid point is defined as part of each grib message. The values for each grid are not stored for each grid box (that would take a lot of memory), but are calculated by the wgrib2 utility with the stored projection information.

If you are using pygrib, you can get the variable data, latitude, and longitude like this: value, lat, lon = grbs[1].data()

For convenience and some unique applications, I created an HDF5 file that contains just the HRRR latitude and longitude grids. HRRR_latlon.h5

If you are dealing with a vector quantity, like wind direction, you need to convert the U and V wind component from grid-relative to earth-relative to correctly orient the wind vectors.

Convert winds Grid-relative to Earth-Relative