AirNow and CAM-chem
First we need to import the driver.
from melodies_monet import driver
Please install h5py to open files from the Amazon S3 servers.
Please install h5netcdf to open files from the Amazon S3 servers.
# Needed if you want to make changes to `melodies_monet` and don't want to restart kernel:
%load_ext autoreload
%autoreload 2
Initiate the analysis class
Now lets create an instance of the melodies_monet.driver
analysis
class.
It consists of 4 main parts: model instances, observation instances, a paired instance of both.
This will allow us to move things around the plotting function for spatial and overlays and more complex plots.
an = driver.analysis()
an
analysis(
control='control.yaml',
control_dict=None,
models={},
obs={},
paired={},
start_time=None,
end_time=None,
download_maps=True,
output_dir=None,
debug=False,
)
Control File
Read in the required yaml control file that sets up all the definitions of what we want to pair and plot.
an.control = 'control_camchem.yaml'
an.read_control()
an.control_dict
Show code cell output
{'analysis': {'start_time': '2019-09-01-00:00:00',
'end_time': '2019-09-09-00:00:00',
'output_dir': './output/camchem',
'download_maps': False,
'debug': False},
'model': {'cam-chem': {'files': 'example:camchem:fv',
'mod_type': 'cesm_fv',
'radius_of_influence': 150000.0,
'mapping': {'airnow': {'O3': 'OZONE'}},
'projection': 'None',
'plot_kwargs': {'color': 'dodgerblue', 'marker': '+', 'linestyle': '-.'}}},
'obs': {'airnow': {'use_airnow': True,
'filename': 'example:airnow:2019-09',
'obs_type': 'pt_sfc',
'variables': {'PM2.5': {'unit_scale': 1,
'unit_scale_method': '*',
'nan_value': -1.0,
'ylabel_plot': 'PM2.5 (ug/m3)',
'ty_scale': 2.0,
'vmin_plot': 0.0,
'vmax_plot': 22.0,
'vdiff_plot': 15.0,
'nlevels_plot': 23},
'OZONE': {'unit_scale': 1,
'unit_scale_method': '*',
'nan_value': -1.0,
'ylabel_plot': 'Ozone (ppbv)',
'vmin_plot': 15.0,
'vmax_plot': 55.0,
'vdiff_plot': 20.0,
'nlevels_plot': 21}}}},
'plots': {'plot_grp1': {'type': 'timeseries',
'fig_kwargs': {'figsize': [12, 6]},
'default_plot_kwargs': {'linewidth': 2.0, 'markersize': 10.0},
'text_kwargs': {'fontsize': 18.0},
'domain_type': ['all', 'epa_region'],
'domain_name': ['CONUS', 'R1'],
'data': ['airnow_cam-chem'],
'data_proc': {'rem_obs_nan': True,
'ts_select_time': 'time',
'ts_avg_window': 'H',
'set_axis': False}},
'plot_grp2': {'type': 'taylor',
'fig_kwargs': {'figsize': [8, 8]},
'default_plot_kwargs': {'linewidth': 2.0, 'markersize': 10.0},
'text_kwargs': {'fontsize': 16.0},
'domain_type': ['all', 'epa_region'],
'domain_name': ['CONUS', 'R1'],
'data': ['airnow_cam-chem'],
'data_proc': {'rem_obs_nan': True, 'set_axis': True}},
'plot_grp3': {'type': 'spatial_bias',
'fig_kwargs': {'states': True, 'figsize': [10, 5]},
'text_kwargs': {'fontsize': 16.0},
'domain_type': ['all', 'epa_region'],
'domain_name': ['CONUS', 'R1'],
'data': ['airnow_cam-chem'],
'data_proc': {'rem_obs_nan': True, 'set_axis': True}},
'plot_grp4': {'type': 'spatial_overlay',
'fig_kwargs': {'states': True, 'figsize': [10, 5]},
'text_kwargs': {'fontsize': 16.0},
'domain_type': ['all', 'epa_region'],
'domain_name': ['CONUS', 'R1'],
'data': ['airnow_cam-chem'],
'data_proc': {'rem_obs_nan': True, 'set_axis': True}},
'plot_grp5': {'type': 'boxplot',
'fig_kwargs': {'figsize': [8, 6]},
'text_kwargs': {'fontsize': 20.0},
'domain_type': ['all', 'epa_region'],
'domain_name': ['CONUS', 'R1'],
'data': ['airnow_cam-chem'],
'data_proc': {'rem_obs_nan': True, 'set_axis': False}}}}
Load the model data
The driver will automatically loop through the “models” found in the model section of the control file and create model classes for each. Classes include the label, mapping information, and xarray object as well as the filenames. Note it can open multiple files easily by including wildcards. Here we are only opening one CAM-chem file.
an.open_models()
Downloading data from 'https://csl.noaa.gov/groups/csl4/modeldata/melodies-monet/data/example_model_data/cesmfv_example/CAM_chem_merra2_FCSD_1deg_QFED_world_201909-01-09_small_sfc.nc' to file 'C:\Users\zmoon\AppData\Local\pooch\pooch\Cache\f57b114fb465b23ee998209b50df7d9d-CAM_chem_merra2_FCSD_1deg_QFED_world_201909-01-09_small_sfc.nc'.
cam-chem
{'files': 'example:camchem:fv', 'mod_type': 'cesm_fv', 'radius_of_influence': 150000.0, 'mapping': {'airnow': {'O3': 'OZONE'}}, 'projection': 'None', 'plot_kwargs': {'color': 'dodgerblue', 'marker': '+', 'linestyle': '-.'}}
example:camchem:fv
**** Reading CESM model output...
an.models
{'cam-chem': model(
model='cesm_fv',
radius_of_influence=150000.0,
mod_kwargs={},
file_str='example:camchem:fv',
label='cam-chem',
obj=...,
mapping={'airnow': {'O3': 'OZONE'}},
label='cam-chem',
...
)}
an.models['cam-chem'].obj
<xarray.Dataset> Dimensions: (time: 36, z: 1, lat: 192, lon: 288) Coordinates: * lat (lat) float64 -90.0 -89.06 -88.12 -87.17 ... 87.17 88.12 89.06 90.0 * z (z) float64 992.5 * lon (lon) float64 0.0 1.25 2.5 3.75 5.0 ... 355.0 356.2 357.5 358.8 * time (time) datetime64[ns] 2019-09-01T06:00:00 ... 2019-09-10 Data variables: O3 (time, z, lat, lon) float32 dask.array<chunksize=(36, 1, 192, 288), meta=np.ndarray> PM25 (time, z, lat, lon) float32 dask.array<chunksize=(36, 1, 192, 288), meta=np.ndarray> Attributes: Conventions: CF-1.0 source: CAM case: fmerra.2.1003.FCSD.f09.qfedcmip.56L.001.branch02 logname: buchholz host: cheyenne3 initial_file: /glade/p/cesmdata/cseg/inputdata/atm/cam/inic/fv/f.e20... topography_file: /glade/p/cesmdata/cseg/inputdata/atm/cam/met/MERRA2/0.... model_doi_url: https://doi.org/10.5065/D67H1H0V time_period_freq: hour_6 history: Mon Feb 28 16:25:23 2022: ncks -7 -L 1 --baa=4 --ppc d... NCO: netCDF Operators version 5.0.6 (Homepage = http://nco....
# All the info in the model class can be called here.
print(an.models['cam-chem'].label)
print(an.models['cam-chem'].mapping)
cam-chem
{'airnow': {'O3': 'OZONE'}}
# All the info in the analysis class can also be called.
print(an.start_time)
print(an.end_time)
print(an.download_maps)
2019-09-01 00:00:00
2019-09-09 00:00:00
True
Open Obs
Now for monet-analysis we will open preprocessed data in either netcdf icartt or some other format. We will not be retrieving data like monetio does for some observations (ie aeronet, airnow, etc….). Instead we will provide utitilies to do this so that users can add more data easily.
Like models we list all obs objects in the yaml file and it will loop through and create driver.observation instances that include the model type, file, objects (i.e. data object) and label
an.control_dict['obs']
{'airnow': {'use_airnow': True,
'filename': 'example:airnow:2019-09',
'obs_type': 'pt_sfc',
'variables': {'PM2.5': {'unit_scale': 1,
'unit_scale_method': '*',
'nan_value': -1.0,
'ylabel_plot': 'PM2.5 (ug/m3)',
'ty_scale': 2.0,
'vmin_plot': 0.0,
'vmax_plot': 22.0,
'vdiff_plot': 15.0,
'nlevels_plot': 23},
'OZONE': {'unit_scale': 1,
'unit_scale_method': '*',
'nan_value': -1.0,
'ylabel_plot': 'Ozone (ppbv)',
'vmin_plot': 15.0,
'vmax_plot': 55.0,
'vdiff_plot': 20.0,
'nlevels_plot': 21}}}}
an.open_obs()
# All the info in the observation class can also be called.
an.obs['airnow'].obj
<xarray.Dataset> Dimensions: (x: 3786, time: 2091, y: 1) Coordinates: * x (x) int64 0 1 2 3 4 5 6 7 ... 3779 3780 3781 3782 3783 3784 3785 * time (time) datetime64[ns] 2019-09-01 ... 2019-09-30T00:30:00 latitude (y, x) float64 47.65 47.51 49.02 53.3 ... 36.92 47.93 41.37 longitude (y, x) float64 -52.82 -52.79 -55.8 -60.36 ... -94.84 106.9 69.27 siteid (y, x) object '000010102' '000010401' ... 'UZB010001' Dimensions without coordinates: y Data variables: (12/30) BARPR (time, y, x) float64 ... BC (time, y, x) float64 ... CO (time, y, x) float64 ... NO (time, y, x) float64 ... NO2 (time, y, x) float64 ... NO2Y (time, y, x) float64 ... ... ... cmsa_name (y, x) float64 -1.0 -1.0 -1.0 -1.0 -1.0 ... -1.0 -1.0 -1.0 -1.0 msa_code (y, x) float64 -1.0 -1.0 -1.0 -1.0 ... -1.0 3.306e+04 -1.0 -1.0 msa_name (y, x) object '' '' '' '' '' '' ... '' '' '' ' Miami, OK ' '' '' state_name (y, x) object 'CC' 'CC' 'CC' 'CC' 'CC' 'CC' ... '' '' '' '' '' epa_region (y, x) object 'CA' 'CA' 'CA' 'CA' 'CA' ... '' 'R6' 'DSMG' 'DSUZ' time_local (time, y, x) datetime64[ns] ... Attributes: title: format: NetCDF-4 date_created: 2021-06-07
Pair model and obs data
%%time
an.pair_data()
Show code cell output
[########################################] | 100% Completed | 0.1s
[########################################] | 100% Completed | 0.2s
[########################################] | 100% Completed | 0.2s
[########################################] | 100% Completed | 0.3s
Wall time: 29.2 s
an.paired
{'airnow_cam-chem': pair(
type='pt_sfc',
radius_of_influence=1000000.0,
obs='airnow',
model='cam-chem',
model_vars=['O3'],
obs_vars=['OZONE'],
filename='airnow_cam-chem.nc',
)}
an.paired['airnow_cam-chem'].obj
<xarray.Dataset> Dimensions: (time: 2091, x: 3786) Coordinates: * time (time) datetime64[ns] 2019-09-01 ... 2019-09-30T00:30:00 * x (x) int32 0 1 2 3 4 5 6 7 ... 3779 3780 3781 3782 3783 3784 3785 Data variables: (12/34) BARPR (time, x) float64 -1.0 -1.0 -1.0 -1.0 ... -1.0 -1.0 -1.0 -1.0 BC (time, x) float64 -1.0 -1.0 -1.0 -1.0 ... -1.0 -1.0 -1.0 -1.0 CO (time, x) float64 -1.0 -1.0 -1.0 -1.0 ... -1.0 -1.0 -1.0 -1.0 NO (time, x) float64 -1.0 -1.0 -1.0 -1.0 ... -1.0 -1.0 -1.0 -1.0 NO2 (time, x) float64 -1.0 -1.0 -1.0 -1.0 ... -1.0 -1.0 -1.0 -1.0 NO2Y (time, x) float64 -1.0 -1.0 -1.0 -1.0 ... -1.0 -1.0 -1.0 -1.0 ... ... cmsa_name (x) float64 -1.0 -1.0 -1.0 -1.0 -1.0 ... -1.0 -1.0 -1.0 -1.0 msa_code (x) float64 -1.0 -1.0 -1.0 -1.0 ... -1.0 3.306e+04 -1.0 -1.0 msa_name (x) object '' '' '' '' '' '' '' ... '' '' '' ' Miami, OK ' '' '' state_name (x) object 'CC' 'CC' 'CC' 'CC' 'CC' 'CC' ... '' '' '' '' '' '' epa_region (x) object 'CA' 'CA' 'CA' 'CA' 'CA' ... '' 'R6' 'DSMG' 'DSUZ' siteid (x) object '000010102' '000010401' ... 'UB1010001' 'UZB010001'
Generate plots
%%time
an.plotting()
Show code cell output
Warning: ty_scale not specified for OZONE, so default used.
Reference std: 16.085098346255723
Warning: ty_scale not specified for OZONE, so default used.
Reference std: 10.063694443664746
C:\tools\miniconda3\envs\melodies-monet-docs\lib\site-packages\cartopy\io\__init__.py:241: DownloadWarning: Downloading: https://naturalearth.s3.amazonaws.com/10m_cultural/ne_10m_admin_0_boundary_lines_land.zip
warnings.warn(f'Downloading: {url}', DownloadWarning)
[########################################] | 100% Completed | 0.1s
[########################################] | 100% Completed | 0.1s
[########################################] | 100% Completed | 0.2s
[########################################] | 100% Completed | 0.3s
[########################################] | 100% Completed | 0.1s
[########################################] | 100% Completed | 0.1s
[########################################] | 100% Completed | 0.1s
[########################################] | 100% Completed | 0.1s
Wall time: 33.4 s
10 Figures









