Bulk Data Download

This page walks you through Bulk Data Download: pulling water level, rainfall, climate, model-ready files, and photos out of the Hyfi portal across many sites and long date ranges. Exports are built in the background and delivered by email.

Table of Contents

• Access Requirements
• What You Can Export
• Size Limits and Delivery
• Creating an Export
• What's Inside the ZIP
• Reading the QC Column
• Time Step, Gaps, and Estimation
• Rainfall and Climate
• USGS and NOAA Gauges
• Timezones and Daylight Saving
• Ice-Obstructed Readings
• Photo Exports
• Troubleshooting
• Need Help?

Access Requirements

To use Bulk Data Download, you need ADMIN access on your team. If you don't see the download icon in the navigation bar, email support@hyfi.io and we'll add the access for you.

Exports are always emailed to the address you are signed in with. You cannot send an export to someone else's address. Forward them the download link instead.

What You Can Export

Start by telling the page what the data is for. Each option sets sensible defaults for everything below it, and you can still change any individual setting afterwards.

CSV / Excel

Depth, elevation, and precipitation as spreadsheets you can open in Excel or Google Sheets.

SWMM

The CSVs plus native .dat timeseries, evenly spaced and ready to import. A daily climate file is optional.

SWAT+

A daily weather pack in native .pcp, .tmp, .hmd, .wnd, and .slr files, plus your water level as CSV to calibrate against. The button on the page reads SWAT.

Photos

Trail camera stills and field photo uploads, without the numbers.

There is no Custom button to press. Change any setting after picking one of the four above and the page moves to a custom mix on its own, deselecting the preset. Nothing you have chosen is lost; the receipt on the right always lists exactly what the export contains.

SWAT+ has no water-level input. It is driven by weather, so the water level comes along as CSV for you to check the model against rather than to feed into it.

Size Limits and Delivery

Large exports take a few minutes to build. You do not need to keep the page open. Close the tab and wait for the email.

Site count on its own is not what makes an export too big. What matters is the total number of rows, which is your sites multiplied by your measurements multiplied by how many time steps fit in the date range. The receipt estimates this as you go. For a large export it tells you how long to expect. For a very large one it warns that the export may not finish in one run. You can still request it, and if it does not finish, the email says so and what to change: a shorter range, fewer sites or measurements, or a coarser interval.

Creating an Export

1. Click the download icon in the top navigation bar. Hovering over it shows Bulk Data Download.
   



2. Choose what you're downloading for. The panel underneath tells you exactly what you'll get.
   



3. Select your sites. Search by site name, such as "Telegraph over River Rouge", or by Hyfi site code, the short plus code that identifies each location, such as "86JR8P2H+H93F". The site code also appears in every exported filename, so it is worth recognising even if you normally search by name. To take every site in the list at once, use All listed next to the search box. Icons in the list mark sites that have a trail camera or field photos.
   



4. Set your date range with a preset (7 days up to 3 years, or All available) or with custom start and end times. All available uses each site's install and shutdown dates when we know them, and otherwise the most recent 1095 days.
   



5. Pick what to include. Depth, elevation, and precipitation are on by default. Field photos can be added here too, and Trail camera photos appears once you have picked a site with a camera. Each one says how many of your selected sites actually have photos. More measurements opens temperature, humidity, wind, solar radiation, and evapotranspiration, which are weather data at the site rather than readings from a Hyfi sensor.
   



6. Optional. Open Advanced options to add model files, or to change the time step, timezone, gap handling, or estimation method. Most people never need to. The (?) icons explain what each model file contains.
   



7. Check the summary on the right. It lists every file you're about to receive and every setting that produced them. Where does this data come from? at the bottom opens the source of each measurement for the sites you picked.
   



8. Click Email me the ZIP. You'll see a confirmation, and the email arrives when the files are built.
   

What's Inside the ZIP

Every export arrives as a single ZIP, named for the day it was built, for example hyfi_export_2026_10_02.zip. Depending on what you asked for, it contains:

• csv/: one file per site and measurement, named with the site code, the measurement, and the timezone, for example 86JR8P2H+H93F_depth_UTC.csv
• swmm/: .dat timeseries and climate files, if you selected SWMM
• swat/: .pcp, .tmp, .hmd, .wnd, and .slr station files, plus the matching .cli index files, if you selected SWAT+
• cameras/: trail camera stills, one folder per site
• field_photos/: field photo uploads, one folder per site
• qa_report.json: a machine-readable summary of coverage, with expected and actual point counts, percent estimated, percent missing, and the largest gap for every site and measurement

Each CSV also opens with a header block recording the site, coordinates, datum, agency, data source, time range, timezone, units, time step, estimation method, and maximum gap. If you hand the file to a colleague or a reviewer, those settings travel with it.

Reading the QC Column

Every row in every CSV carries a qc column saying where that value came from. This is the fastest way to tell real readings from estimated ones.

With a 10-minute interval and a 2-hour maximum gap, a four-hour outage looks like this in the CSV. Nothing marks the gap in the rows themselves, so it is worth knowing to expect it:

date,time,value,qc
06/14/2026,13:50,3.180,measured
06/14/2026,14:00,3.210,measured
06/14/2026,18:00,2.940,measured     <-- four hours later, no rows in between
06/14/2026,18:10,2.930,measured

The same outage in qa_report.json, where it is recorded explicitly:

{
  "site_code": "86JR8P2H+H93F",
  "parameter": "depth",
  "expected_points": 52560,
  "actual_points": 51705,
  "pct_interpolated": 5.52,
  "pct_missing": 1.63,
  "largest_gap_minutes": 240,
  "gaps_exceeding_max_gap": [
    { "start": "2026-06-14T14:00:00Z", "end": "2026-06-14T18:00:00Z", "gapMinutes": 240 }
  ]
}

Time Step, Gaps, and Estimation

Monitors do not all report at the same moment, so an export resamples every site onto one even time step. That is what lets the files line up in a model without a cleanup pass first.

• Output interval: every 5, 10, 15, or 30 minutes, or hourly.
• Estimation method: how a value between two readings is worked out, either linear, nearest, or previous.
• Maximum gap: how long an outage can be before the export stops estimating across it. Anything longer is left out entirely rather than given an invented shape, as shown in the example above.
• Rainfall gaps: leave blank, or fill with zero. This is the difference between "we don't know" and "it didn't rain," and it changes a model's totals, so choose it deliberately.

SWAT+ files are daily, so the interval, maximum gap, and estimation settings do not apply to them.

Rainfall and Climate

Rainfall is always hourly, whatever output interval you choose. Each row is the total inches that fell in that hour, which is the same number as inches per hour at a one-hour step. Rainfall is never split across the shorter interval or estimated between readings, so a 10-minute export has water level every 10 minutes and rainfall once an hour.

Each rainfall row covers the hour that ends at its timestamp. A row stamped 21:00 is the rain that fell from 20:00 to 21:00.

In SWMM, set the rain gage to Rain Format = INTENSITY with a 1:00 recording interval. The .dat file states this in its header.

Where the numbers come from:

• Hyfi sites: NOAA MRMS radar rainfall (radar plus rain gauges), for the site's location. A few sites have no rainfall on record, and their rainfall file shows the whole range as missing.
• USGS and NOAA gauges: ERA5 weather reanalysis at the gauge's location, marked reanalysis.
• Climate measurements (temperature, humidity, wind, solar radiation, evapotranspiration): ERA5 at every site.

USGS and NOAA Gauges

Public USGS and NOAA gauges can go in the same export as your Hyfi sites. The receipt notes when your selection includes one.

• Full date range. Water level covers the whole range you ask for, back to when each gauge's online record begins. That start date varies by gauge. Anything earlier shows as missing in qa_report.json.
• Gaps come from the gauge. Agency gauges go offline for repairs, ice, or floods, sometimes for weeks. Those outages show up as gaps exactly as they do in the agency's own data.
• Recent data. USGS and NOAA publish readings with a short delay, often around an hour, so the last reading may be a little before the end of your range.
• USGS depth and elevation. Depth is the gauge height the USGS reports. Elevation adds the height of the gauge's own reference point, in the vertical datum USGS records for that gauge, usually NAVD88. The CSV header names the datum.
• NOAA water level. NOAA reports water level already measured against a datum, so depth and elevation contain the same values. We use NAVD88. A station with no NAVD88 datum uses MLLW instead, for the whole range, and the CSV header says which one it is.
• Great Lakes NOAA stations report only in the IGLD datum and are not supported yet.

Timezones and Daylight Saving

You can export in UTC or in any fixed US offset: Eastern, Central, Mountain, Pacific, Alaska, or Hawaii. Standard and daylight are listed separately, so you choose EST or EDT, CST or CDT, and so on.

Ice-Obstructed Readings

When ice or snow sits over a Hyfi monitor, the water level it reports is not the water level. You mark those periods yourself, on the site chart: open a site during ice season, click the snowflake button above the chart, and drag across the affected stretch of the timeline. Hyfi staff can mark them too, but in practice the people who know a site is iced over are the people watching it.

Any team member can mark ice, on any Hyfi site, whenever the charted date range overlaps October through May. It is not limited to administrators. Ice marking is not available on USGS or NOAA gauges.

Once a window is marked, the Exclude data within ice obstruction periods box under Advanced options decides what an export does with it:

• Ticked (the default): those rows are left out of the file.
• Unticked: the values stay, marked ice_affected in the qc column, so you can judge them yourself.

The qa_report.json records how many ice windows applied to each site and how many points each one affected.

Photo Exports

Photo exports come in two kinds, and you can include either or both:

• Trail camera stills: automatic images from the trail camera at the site, sampled at roughly one per hour. They land in the cameras/ folder.
• Field photos: images uploaded by people visiting the site. They land in the field_photos/ folder.

The site list shows which sites have a trail camera and which have field uploads, so you can narrow the selection before you commit. Photo exports get large quickly, and the page warns you before you request one that will take a while.

If you ask for photos only and none of the sites you picked has any, the page says so and holds the submit button rather than emailing you an empty ZIP.

Troubleshooting

Need Help?

If you're having trouble downloading data, or you need an export larger than what one run can build, email support@hyfi.io and we'll help.