English
资源说明：manipulating I/O API metadata with R netCDF tools
This project is intended to illustrate the hacking of "I/O API metadata" (aka IOAPI metadata) in netCDF files. (For more info on those standards, see
http://www.baronams.com/products/ioapi/
http://www.unidata.ucar.edu/software/netcdf/
) The top-level drivers are two bash scripts:
1 computeCropSum.sh: this takes an input of valid IOAPI (but which is semantically crippled), and
* removes all but the data variable of interest
* fixes the problem with its data
* assimilates some BELD data to a separate layer
* integrates the data over each gridcell
* "windows" the data to a subdomain
* plots the data, for each layer, recording metadata
2 mergeWithCmaqEmissions.sh: this takes as input
* the output of computeCropSum.sh (aka "EPIC emissions")
* emissions files from specified CMAQ-5 evaluation dates (aka "CMAQ emissions")
* MCIP output, containing map scale factors (MSFs), from one of those dates (since the MSFs are presumed time-invariant)
and
* windows each of the CMAQ emissions to match the EPIC emissions' domain
* adds to each CMAQ emissions file a specie=N2O for contain the integration of the EPIC emissions
* converts the EPIC emissions' units to match those of the CMAQ emissions
while ensuring IOAPI-validity is conserved (e.g., the output can be
* loaded by VERDI, which is notoriously "hard to please"
* consumed by CCTM (aka CMAQ)
). The top-level drivers mostly drive two sets of netCDF-aware tools:
1 NCO
http://nco.sourceforge.net/
2 R, specifically packages=ncdf4, M3
http://cirrus.ucsd.edu/~pierce/ncdf/
http://cran.r-project.org/web/packages/M3/
Note that package=M3 requires (among others) package=ncdf4, which requires netCDF 4.
For IOAPI testing, this project also uses
3 m3tools, specifically m3stat and m3wndw
www.baronams.com/products/ioapi/M3STAT.html
www.baronams.com/products/ioapi/M3WNDW.html
For display, this project also uses
4 R package=fields
http://www.image.ucar.edu/Software/Fields/
5 a commandline PDF viewer of your choice (edit the script, or comment out)
Since the code requires bash and netCDF 4, you may have difficulty running on windows: cygwin may be helpful. (If you're only interested in reading the most-IOAPI-aware code, see ioapi.r.) To run the example,
0 Ensure you have proper dependencies. Check your
* NCO: should be uplevel (I have version=4.0.5, latest=4.1.0) and in the path
* R: it should be uplevel (I have version=2.14.0, latest=2.15.0), in the path, and include the above packages
* bash: should be uplevel (I have version=3.2.25, latest=4.2) and your shell (always :-)
* m3tools: should be uplevel and in path
* PDF viewer: edit the driver script to call yours (or comment it out)
1 Fork the git repository, which will create a folder for following activity.
E.g. (in bash),
$ rm -fr /tmp/ioapi-hack-R/
$ pushd /tmp
/tmp $ env GIT_SSL_NO_VERIFY=true git clone https://TomRoche@github.com/TomRoche/ioapi-hack-R.git
> Cloning into 'ioapi-hack-R'...
> remote: Counting objects: 29, done.
> remote: Compressing objects: 100% (22/22), done.
> remote: Total 29 (delta 9), reused 27 (delta 7)
> Unpacking objects: 100% (29/29), done.
/tmp $ popd ; pushd /tmp/ioapi-hack-R/
The folder contents should resemble
> drwxr-xr-x you yours 4.0K date time .
> -rwxr-xr-x you yours 6.6K date time beldRead.r
> -rwxr-xr-x you yours 963 date time beldToRDS.r
> -rw-r--r-- you yours 15K date time computeCropSum.r
> -rwxr-xr-x you yours 13K date time computeCropSum.sh
> -rw-r--r-- you yours 6.2K date time demonotonicizeVar.r
> drwxr-xr-x you yours 4.0K date time .git
> -rw-r--r-- you yours 120 date time .gitignore
> -rw-r--r-- you yours 16K date time ioapi.r
> -rwxr-xr-x you yours 3.8K date time plotLayersForTimestep.r
> -rw-r--r-- you yours 6.5K date time processLayers.r
> -rw-r--r-- you yours 6.1K date time processVars.r
> -rw-r--r-- you yours 1.7K date time README
> -rw-r--r-- you yours 1.8K date time tlrRutilities.r
> -rw-r--r-- you yours 8.4K date time windowEmissions.r
> -rw-r--r-- you yours 8.1K date time writeBELDlayer.r
2 Download the input data from
https://github.com/TomRoche/ioapi-hack-R/downloads
E.g. (in bash),
URI_ROOT="https://github.com/downloads/TomRoche/ioapi-hack-R"
DIR="/tmp/ioapi-hack-R"
for FN in \
"map.CMAQkm.world.dat" \
"epic_site_crops_0529_USA_2Ellen.csv" \
"epic_site_crops_0529_USA_2Ellen.rds" \
"5yravg.test.nc" \
; do
URI="${URI_ROOT}/${FN}"
FP="${DIR}/${FN}" # file path
# curl can't handle the redirects, so use wget
# ARGS="-c -O ${FP}"
# for broken-cert environments (like EPA terrae) use '--no-check-certificate'
ARGS="--no-check-certificate -c -O ${FP}"
for CMD in \
"wget ${ARGS} ${URI}" \
; do
echo -e "$ ${CMD}"
eval "${CMD}"
done
done
The folder contents should resemble
-rwxr-xr-x you yours 6.6K date time beldRead.r
-rwxr-xr-x you yours 963 date time beldToRDS.r
-rw-r--r-- you yours 15K date time computeCropSum.r
-rwxr-xr-x you yours 13K date time computeCropSum.sh
-rw-r--r-- you yours 6.2K date time demonotonicizeVar.r
drwxr-xr-x you yours 4.0K date time .git
-rw-r--r-- you yours 120 date time .gitignore
-rw-r--r-- you yours 16K date time ioapi.r
-rwxr-xr-x you yours 3.8K date time plotLayersForTimestep.r
-rw-r--r-- you yours 6.5K date time processLayers.r
-rw-r--r-- you yours 6.1K date time processVars.r
-rw-r--r-- you yours 1.7K date time README
-rw-r--r-- you yours 1.8K date time tlrRutilities.r
-rw-r--r-- you yours 8.4K date time windowEmissions.r
-rw-r--r-- you yours 8.1K date time writeBELDlayer.r
-rw-r--r-- you yours 1.1M date time map.CMAQkm.world.dat
-rw-r--r-- you yours 1.5M date time epic_site_crops_0529_USA_2Ellen.rds
-rw-r--r-- you yours 5.5M date time epic_site_crops_0529_USA_2Ellen.csv
-rw-r--r-- you yours 638M date time 5yravg.test.nc
3 Run it: e.g.,
/tmp/ioapi-hack-R $ ./computeCropSum.sh
It should produce
3.1 a ginormous console trace, at times paused
3.2 several new netCDF files, e.g.,
-rw-r--r-- you yours 400K date time 5yravg.DN2windowed.nc
-rw-r--r-- you yours 24M date time 5yravg.DN2summed.nc
-rw-r--r-- you yours 24M date time 5yravg.DN2belded.nc
-rw-r--r-- you yours 24M date time 5yravg.DN2layers_fixed.nc
-rw-r--r-- you yours 24M date time 5yravg.DN2layered.nc
-rw-r--r-- you yours 1.1M date time temp.extend.nc
-rw-r--r-- you yours 24M date time temp.full.nc
-rw-r--r-- you yours 22M date time 5yravg.DN2demonotonicized.nc
-rw-r--r-- you yours 22M date time 5yravg.DN2vars_fixed.nc
-rw-r--r-- you yours 22M date time 5yravg.DN2stripped.nc
The main output is '5yravg.DN2windowed.nc', which you should be able to open with VERDI (the point of the IOAPI part of this exercise :-) Compared to the source NetCDF (5yravg.test.nc), that output
* covers a smaller horizontal extent
* has many fewer datavars: in fact, only one (DN2) other than the IOAPI "metadatavar" TFLAG.
3.3 a plot file containing plots of each data layer in the window, plus
* a plot of the sum of the BELD proportions in each gridcell (penultimate layer)
* a plot of the integrated DN2 emissions over each gridcell (final layer)
After you run it, improve it! The code is currently quite crude, so feel free to fix/extend--and then send me pull request!
HTH, Tom Roche
本源码包内暂不包含可直接显示的源代码文件，请下载源码包。
