Name: il-dmr
Owner: Data Science for Social Good
Description: Illinois EPA Discharge Monitoring Reports download and import
Created: 2015-10-09 22:44:37.0
Updated: 2017-03-15 22:44:52.0
Pushed: 2017-03-15 22:44:51.0
Homepage: null
Size: 3
Language: Python
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
This repository contains a couple of scripts for downloading the Illinois EPA Discharge Monitoring Reports (DMR) and importing them into a database.
The main script is dmr.py which downloads the csv files from the EPA website. Some useful examples:
To download all dmr csv files (to il-dmr/csv directory) use
ownload and import csv files (serially, which is good in case of failure) first run `create.csv` and then
The import is done using psql -c '\copy so place the relevant PostgreSQL credentials in environment variables. To download a particular year
ownload a particular NPDES id (all years, unless `--year` is specified)
By default existing csv files will not be re-downloaded. To re-download (useful for updating a year in progress)
e are two additional scripts:
ate.sql` for creating the target dmr table
ort.sh` for bulk import of csvs after they have been downloaded.