Idealized Synthetic Data

Under development

import numpy as np
import pandas as pd
import xarray as xr
from IPython.display import display  # so can run as script too

from melodies_monet import driver

Please install h5py to open files from the Amazon S3 servers.
Please install h5netcdf to open files from the Amazon S3 servers.

an = driver.analysis()
an.control = "control_idealized.yaml"
an.read_control()
an

analysis(
    control='control_idealized.yaml',
    control_dict=...,
    models={},
    obs={},
    paired={},
    start_time=Timestamp('2019-09-09 00:00:00'),
    end_time=Timestamp('2019-09-10 00:00:00'),
    time_intervals=None,
    download_maps=True,
    output_dir='./output/idealized',
    output_dir_save='./output/idealized',
    output_dir_read='./output/idealized',
    debug=True,
    save={'paired': {'method': 'netcdf', 'prefix': 'asdf', 'data': 'all'}},
    read={'paired': {'method': 'netcdf', 'filenames': {'test_obs_test_model': 'asdf_test_obs_test_model.nc4'}}},
)

Note: This is the complete file that was loaded.

control_idealized.yaml

analysis:
  start_time: "2019-09-09 00:00"
  end_time: "2019-09-10 00:00"
  output_dir: ./output/idealized
  # output_dir_save:  # defaults to `output_dir`
  # output_dir_read:  # defaults to `output_dir`
  debug: True
  save:
    paired:
      method: 'netcdf' # 'netcdf' or 'pkl'
      prefix: 'asdf' # use only with method=netcdf; don't set if you don't want a fn prefix
      # output_name: '0905.pkl' # use only with method=pkl
      data: 'all'
      # ^ 'all' to save out all pairs or
      #   ['pair1','pair2',...] to save out specific pairs.
      #   With method='pkl' this is ignored and always saves all.
    # models:
    # obs:
  read:
    paired:
      method: 'netcdf' # 'netcdf' or 'pkl'
      filenames:
        test_obs_test_model: 'asdf_test_obs_test_model.nc4'
      # filenames: ['0904.pkl','0905.pkl'] # example for pkl method, uses str or iterable of filenames
    # models:
    # obs:

model:
  test_model:
    files: test_model.nc
    mod_type: random
    variables:
      A:
        units: "Units of A"
        unit_scale: 1
        unit_scale_method: "*"
      B:
        units: "Units of B"
        unit_scale: 1
        unit_scale_method: "*"
    mapping:
      test_obs:
        A: "A_obs"
        B: "B_obs"
    projection: ~  # unused

obs:
  test_obs:
    # use_airnow: True
    filename: test_obs.nc
    obs_type: pt_sfc

plots:
  plot_grp1:
    type: 'timeseries'
    default_plot_kwargs:  # required (with at least one key)
      linewidth: 2.0
    domain_type: ['all']  # required
    domain_name: ['CONUS']  # required
    data: ['test_obs_test_model']  # required
    data_proc:  # These four seem to be required for time series
      rem_obs_nan: True  # True: Remove all points where model or obs variable is NaN. False: Remove only points where model variable is NaN.
      ts_select_time: 'time'  # 'time' for UTC or 'time_local'
      ts_avg_window: '3H'  # Options: None for no averaging or list pandas resample rule (e.g., 'H', 'D')
      # ^ TODO: null setting seems not working
      set_axis: False  # If True, add vmin_plot and vmax_plot for each variable in obs.

  plot_grp2:
    type: 'spatial_overlay'
    fig_kwargs:
      states: True
      figsize: [10, 5]
    domain_type: ['all']  # required
    domain_name: ['CONUS']  # required
    data: ['test_obs_test_model']  # required
    data_proc:
      rem_obs_nan: True
      set_axis: True

Plot

%%time

an.plotting()

Warning: variables dict for A_obs not provided, so defaults used

C:\Users\zmoon\git\MELODIES-MONET\melodies_monet\plots\surfplots.py:671: FutureWarning: The default value of numeric_only in DataFrameGroupBy.mean is deprecated. In a future version, numeric_only will default to False. Either specify numeric_only or select only columns which should be valid for the function.
  df_mean=df.groupby(['siteid'],as_index=False).mean()

Warning: variables dict for B_obs not provided, so defaults used

C:\Users\zmoon\git\MELODIES-MONET\melodies_monet\plots\surfplots.py:671: FutureWarning: The default value of numeric_only in DataFrameGroupBy.mean is deprecated. In a future version, numeric_only will default to False. Either specify numeric_only or select only columns which should be valid for the function.
  df_mean=df.groupby(['siteid'],as_index=False).mean()

CPU times: total: 5.3 s
Wall time: 5.31 s

../_images/8e26c80e89ce493670851de39fe5a4a6b35310b410c33b2fc4a5cd147b261c4f.png

../_images/376995bee195941ccb64c237a792be79a25bc2160c690f329bbc3e22863fbaa6.png

../_images/1cb0497772e4398656558e24d37bd97c49893d2cfe77d730216edf797c812048.png

../_images/6f461bf68058abb326cbc169ba629472bd97145796c6b13b4472e3ba3edf812f.png

Read the Docs v: latest

Versions: latest; develop

Downloads

On Read the Docs: Project Home; Builds