{ "cells": [ { "cell_type": "markdown", "id": "f9507824", "metadata": {}, "source": [ "# ISH and UFS-CHEM\n", "\n", "Our first example will demonstrate the basics available in MELODIES MONET to \n", "compare UFS-CHEM results against ISH surface observations for temperature, wind speed, and dewpoint.\n", "\n", "ISH is a particularly useful for model evaluations because it is a global dataset. As you will see in the YAML file, we can create plots globally, for individual countries, and states. NOTE: MELODIES-MONET examples that use ISH will take longer to finish running due to the larger amount of data.\n", "\n", "First, we import the {mod}`melodies_monet.driver` module." ] }, { "cell_type": "code", "execution_count": 1, "id": "3d43faf7", "metadata": {}, "outputs": [], "source": [ "from melodies_monet import driver" ] }, { "cell_type": "markdown", "id": "94ac7f4a", "metadata": {}, "source": [ "## Analysis driver class\n", "\n", "Now, lets create an instance of the analysis driver class, {class}`melodies_monet.driver.analysis`.\n", "It consists of these main parts:\n", "* model instances\n", "* observation instances\n", "* a paired instance of both" ] }, { "cell_type": "code", "execution_count": 2, "id": "7530b948", "metadata": {}, "outputs": [], "source": [ "an = driver.analysis()" ] }, { "cell_type": "markdown", "id": "5986249b", "metadata": {}, "source": [ "Initially, most of our {class}`~melodies_monet.driver.analysis` object's attributes\n", "are set to ``None``, though some have meaningful defaults:" ] }, { "cell_type": "code", "execution_count": 3, "id": "dbed05e8", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "analysis(\n", " control='control.yaml',\n", " control_dict=None,\n", " models={},\n", " obs={},\n", " paired={},\n", " start_time=None,\n", " end_time=None,\n", " time_intervals=None,\n", " download_maps=True,\n", " output_dir=None,\n", " output_dir_save=None,\n", " output_dir_read=None,\n", " debug=False,\n", " save=None,\n", " read=None,\n", " regrid=False,\n", ")" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "an" ] }, { "cell_type": "markdown", "id": "2e875aab", "metadata": {}, "source": [ "## Control file\n", "\n", "We set the YAML control file and begin by reading the file.\n", "\n", "```{note}\n", "Check out the {doc}`/appendix/yaml` for info on how to create\n", "and modify these files.\n", "```" ] }, { "cell_type": "code", "execution_count": 4, "id": "367c092f", "metadata": { "tags": [ "hide-output" ] }, "outputs": [ { "data": { "text/plain": [ "{'analysis': {'start_time': '2017-07-01-00:00:00',\n", " 'end_time': '2017-07-03-00:00:00',\n", " 'output_dir': './output/ish_ufschem',\n", " 'debug': True},\n", " 'model': {'ufschem_v1': {'files': 'example:ufschem:2017-07',\n", " 'files_surf': 'example:ufschem:2017-07_sfc',\n", " 'mod_type': 'ufs',\n", " 'is_global': True,\n", " 'mod_kwargs': {'surf_only': True,\n", " 'sfc_varlist': ['tmp2m', 'spfh2m', 'ugrd10m', 'vgrd10m']},\n", " 'radius_of_influence': 100000,\n", " 'extra_calc': {'dewpoint': {'pres_calc': 'surfpres_pa',\n", " 'specific_hum': 'spfh2m'},\n", " 'rel_hum': {'pres_calc': 'surfpres_pa',\n", " 'specific_hum': 'spfh2m',\n", " 'temp_calc': 'tmp2m'},\n", " 'windspeed': {'u_comp': 'ugrd10m', 'v_comp': 'vgrd10m'},\n", " 'winddir': {'u_comp': 'ugrd10m', 'v_comp': 'vgrd10m'},\n", " 'wind_barb': {'u_comp': 'ugrd10m', 'v_comp': 'vgrd10m'},\n", " 'rose_plot': {'model_wdir': 'winddir', 'model_wspd': 'windspeed'}},\n", " 'mapping': {'ish': {'tmp2m': 't', 'windspeed': 'ws', 'dewpoint': 'dpt'}},\n", " 'projection': None,\n", " 'plot_kwargs': {'color': 'magenta', 'marker': 's', 'linestyle': '-'}}},\n", " 'obs': {'ish': {'filename': 'example:ish:2017-07',\n", " 'obs_type': 'pt_sfc',\n", " 'variables': {'t': {'unit_scale': 273.15,\n", " 'unit_scale_method': '+',\n", " 'nan_value': -1.0,\n", " 'ylabel_plot': 'Temperature 2m (K)',\n", " 'xlabel_plot': 'Temperature 2m (K)',\n", " 'vmin_plot': 273.15,\n", " 'vmax_plot': 313,\n", " 'vdiff_plot': 20.0,\n", " 'nlevels_plot': 21},\n", " 'ws': {'unit_scale': 1,\n", " 'unit_scale_method': '*',\n", " 'nan_value': -1.0,\n", " 'ylabel_plot': 'Windspeed 10m (m/s)',\n", " 'xlabel_plot': 'Windspeed 10m (m/s)',\n", " 'vmin_plot': 0,\n", " 'vmax_plot': 12,\n", " 'vdiff_plot': 4,\n", " 'nlevels_plot': 18},\n", " 'wdir': {'unit_scale': 1,\n", " 'unit_scale_method': '*',\n", " 'nan_value': -1.0,\n", " 'ylabel_plot': 'Wind Direction 10m (deg)',\n", " 'xlabel_plot': 'Wind Direction 10m (deg)',\n", " 'vmin_plot': 0,\n", " 'vmax_plot': 370,\n", " 'vdiff_plot': 4,\n", " 'nlevels_plot': 18},\n", " 'dpt': {'unit_scale': 273.15,\n", " 'unit_scale_method': '+',\n", " 'nan_value': -1.0,\n", " 'ylabel_plot': 'Dewpoint 2m (K)',\n", " 'xlabel_plot': 'Dewpoint 2m (K)',\n", " 'vmin_plot': 273.15,\n", " 'vmax_plot': 300,\n", " 'vdiff_plot': 4,\n", " 'nlevels_plot': 18}},\n", " 'extra_calc': {'rose_plot': {'obs_wdir': 'wdir', 'obs_wspd': 'ws'}}}},\n", " 'plots': {'plot_grp1': {'type': 'timeseries',\n", " 'fig_kwargs': {'figsize': [12, 6]},\n", " 'default_plot_kwargs': {'linewidth': 2.0, 'markersize': 10.0},\n", " 'text_kwargs': {'fontsize': 24.0},\n", " 'domain_type': ['country', 'state'],\n", " 'domain_name': ['US', 'MI'],\n", " 'data': ['ish_ufschem_v1'],\n", " 'data_proc': {'rem_obs_nan': True,\n", " 'ts_select_time': 'time_local',\n", " 'ts_avg_window': 'h',\n", " 'set_axis': True,\n", " 'filter_string': '24 < latitude < 50 and -130 < longitude < -60'}},\n", " 'plot_grp1a': {'type': 'timeseries',\n", " 'fig_kwargs': {'figsize': [12, 6]},\n", " 'default_plot_kwargs': {'linewidth': 2.0, 'markersize': 10.0},\n", " 'text_kwargs': {'fontsize': 24.0},\n", " 'domain_type': ['country', 'state'],\n", " 'domain_name': ['US', 'MI'],\n", " 'data': ['ish_ufschem_v1'],\n", " 'data_proc': {'rem_obs_nan': True,\n", " 'ts_select_time': 'time_local',\n", " 'ts_avg_window': 'h',\n", " 'set_axis': True,\n", " 'filter_string': '24 < latitude < 50 and -130 < longitude < -60 and ws != 0 and wdir != 0'}},\n", " 'plot_grp2': {'type': 'spatial_bias',\n", " 'fig_kwargs': {'states': True, 'figsize': [10, 5]},\n", " 'text_kwargs': {'fontsize': 16.0},\n", " 'domain_type': ['state'],\n", " 'domain_name': ['MI'],\n", " 'data': ['ish_ufschem_v1'],\n", " 'data_proc': {'rem_obs_nan': True, 'set_axis': True, 'wind_barb': True}},\n", " 'plot_grp3': {'type': 'spatial_overlay',\n", " 'fig_kwargs': {'states': True, 'figsize': [10, 5]},\n", " 'text_kwargs': {'fontsize': 16.0},\n", " 'domain_type': ['country', 'state'],\n", " 'domain_name': ['US', 'MI'],\n", " 'data': ['ish_ufschem_v1'],\n", " 'data_proc': {'rem_obs_nan': True,\n", " 'set_axis': True,\n", " 'wind_barb': True,\n", " 'wind_barb_step': 2,\n", " 'filter_string': '24 < latitude < 50 and -130 < longitude < -60'}},\n", " 'plot_grp3a': {'type': 'spatial_overlay',\n", " 'fig_kwargs': {'states': True, 'figsize': [10, 5]},\n", " 'text_kwargs': {'fontsize': 16.0},\n", " 'domain_type': ['country', 'state'],\n", " 'domain_name': ['US', 'MI'],\n", " 'data': ['ish_ufschem_v1'],\n", " 'data_proc': {'rem_obs_nan': True,\n", " 'set_axis': True,\n", " 'wind_barb': True,\n", " 'wind_barb_step': 2,\n", " 'filter_string': '24 < latitude < 50 and -130 < longitude < -60 and ws != 0 and wdir != 0'}},\n", " 'plot_grp3b': {'type': 'spatial_overlay',\n", " 'fig_kwargs': {'states': True, 'figsize': [10, 5]},\n", " 'text_kwargs': {'fontsize': 16.0},\n", " 'domain_type': ['all'],\n", " 'domain_name': ['Filter_Box_Midwest'],\n", " 'data': ['ish_ufschem_v1'],\n", " 'data_proc': {'rem_obs_nan': True,\n", " 'set_axis': True,\n", " 'wind_barb': True,\n", " 'wind_barb_step': 2,\n", " 'wind_barb_kwargs': {'length': 6, 'linewidth': 0.85, 'color': 'magenta'},\n", " 'filter_string': '37 < latitude < 49 and -104 < longitude < -80'}},\n", " 'plot_grp4': {'type': 'boxplot',\n", " 'fig_kwargs': {'figsize': [8, 6]},\n", " 'text_kwargs': {'fontsize': 20.0},\n", " 'domain_type': ['state'],\n", " 'domain_name': ['MI'],\n", " 'data': ['ish_ufschem_v1'],\n", " 'data_proc': {'rem_obs_nan': True, 'set_axis': False}},\n", " 'plot_grp4a': {'type': 'boxplot',\n", " 'fig_kwargs': {'figsize': [8, 6]},\n", " 'text_kwargs': {'fontsize': 20.0},\n", " 'gridlines': True,\n", " 'domain_type': ['all'],\n", " 'domain_name': ['Filter_Box_Midwest'],\n", " 'data': ['ish_ufschem_v1'],\n", " 'data_proc': {'rem_obs_nan': True,\n", " 'set_axis': False,\n", " 'filter_string': '37 < latitude < 49 and -104 < longitude < -80',\n", " 'set_stat_sig': True}},\n", " 'plot_grp5': {'type': 'multi_boxplot',\n", " 'fig_kwargs': {'figsize': [10, 8]},\n", " 'text_kwargs': {'fontsize': 20.0},\n", " 'gridlines': True,\n", " 'xlabel': 'Binned by Observed Temperature (K)',\n", " 'domain_type': ['state'],\n", " 'domain_name': ['MI'],\n", " 'interval_list': [273.15, 281.15, 289.15, 297.15, 305.15],\n", " 'interval_labels': ['[273.15, 281.15)',\n", " '[281.15, 289.15)',\n", " '[289.15, 297.15)',\n", " '[297.15, 305.15)'],\n", " 'interval_var': 't',\n", " 'model_name_list': ['ISH', 'ufschem_v1'],\n", " 'data': ['ish_ufschem_v1'],\n", " 'data_proc': {'rem_obs_nan': True, 'set_axis': False}},\n", " 'plot_grp6': {'type': 'rose_plot',\n", " 'text_kwargs': {'fontsize': 16.0},\n", " 'domain_type': ['state', 'siteid'],\n", " 'domain_name': ['MI', '72539614815'],\n", " 'data': ['ish_ufschem_v1'],\n", " 'color_map': 'cividis',\n", " 'data_proc': {'rem_obs_nan': True, 'set_axis': True, 'wr_calm_limit': 0.5}},\n", " 'plot_grp7': {'type': 'diurnal',\n", " 'fig_kwargs': {'figsize': [12, 6]},\n", " 'default_plot_kwargs': {'linewidth': 2.0, 'markersize': 10.0},\n", " 'text_kwargs': {'fontsize': 24.0},\n", " 'domain_type': ['state', 'siteid'],\n", " 'domain_name': ['MI', '72539614815'],\n", " 'data': ['ish_ufschem_v1'],\n", " 'data_proc': {'rem_obs_nan': True,\n", " 'ts_select_time': 'time_local',\n", " 'ts_avg_window': 'h',\n", " 'set_axis': True}}},\n", " 'stats': {'stat_list': ['MB', 'MdnB', 'R2', 'RMSE', 'MAE', 'MSE'],\n", " 'round_output': 2,\n", " 'output_table': True,\n", " 'output_table_kwargs': {'figsize': [7, 3],\n", " 'fontsize': 12.0,\n", " 'xscale': 1.4,\n", " 'yscale': 1.4,\n", " 'edges': 'horizontal'},\n", " 'domain_type': ['state'],\n", " 'domain_name': ['MI'],\n", " 'data': ['ish_ufschem_v1']}}" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "an.control = \"control_ish_ufschem-example.yaml\"\n", "an.read_control()\n", "an.control_dict" ] }, { "cell_type": "markdown", "id": "e1ee480a", "metadata": {}, "source": [ "````{admonition} Note: This is the complete file that was loaded.\n", ":class: dropdown\n", "\n", "```{literalinclude} control_wrfchem_mech-0905_2.yaml\n", ":caption:\n", ":linenos:\n", "```\n", "````\n", "\n", "Now, some of our {class}`~melodies_monet.driver.analysis` object's attributes are populated:" ] }, { "cell_type": "code", "execution_count": 5, "id": "66cff826", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "analysis(\n", " control='control_ish_ufschem-example.yaml',\n", " control_dict=...,\n", " models={},\n", " obs={},\n", " paired={},\n", " start_time=Timestamp('2017-07-01 00:00:00'),\n", " end_time=Timestamp('2017-07-03 00:00:00'),\n", " time_intervals=None,\n", " download_maps=True,\n", " output_dir='./output/ish_ufschem',\n", " output_dir_save='./output/ish_ufschem',\n", " output_dir_read='./output/ish_ufschem',\n", " debug=True,\n", " save=None,\n", " read=None,\n", " regrid=False,\n", ")" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "an" ] }, { "cell_type": "markdown", "id": "f6a98d98", "metadata": {}, "source": [ "## Load the model data\n", "\n", "The driver will automatically loop through the \"models\" found in the `model` section\n", "of the YAML file and create an instance of {class}`melodies_monet.driver.model` for each\n", "that includes the\n", "* label\n", "* mapping information\n", "* file names (can be expressed using a glob expression)\n", "* xarray object\n", "\n", "````{admonition} Note: Relevant control file section.\n", ":class: dropdown\n", "\n", "```{literalinclude} control_wrfchem_mech-0905_2.yaml\n", ":caption:\n", ":linenos:\n", ":lines: 24-56\n", "```\n", "````" ] }, { "cell_type": "code", "execution_count": 6, "id": "9a1ed373", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ufs\n", "example:ufschem:2017-07\n", "**** Reading UFS-AQM or UFS-Chem model output...\n", "Performing extra model calculations...\n", "Calculating modeled Dewpoint...\n", "Calculating modeled relative humidity...\n", "Calculating modeled windspeed...\n", "Calculating modeled wind direction...\n" ] } ], "source": [ "an.open_models()" ] }, { "cell_type": "markdown", "id": "487d1357", "metadata": {}, "source": [ "Applying {meth}`~melodies_monet.driver.analysis.open_models`\n", "populates the {attr}`~melodies_monet.driver.analysis.models` attribute." ] }, { "cell_type": "code", "execution_count": 7, "id": "18e908f6", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'ufschem_v1': model(\n", " model='ufs',\n", " is_global=True,\n", " radius_of_influence=100000,\n", " mod_kwargs={'surf_only': True, 'sfc_varlist': ['tmp2m', 'spfh2m', 'ugrd10m', 'vgrd10m'], 'fname_sfc': ['/home/rschwantes/.cache/pooch/5207e0e702072b5e37fd507da111f78d-2017_07_01_03_ufschemv1_sfc.nc'], 'var_list': ['lat', 'lon', 'phalf', 'tmp', 'pressfc', 'dpres', 'hgtsfc', 'delz']},\n", " file_str='example:ufschem:2017-07',\n", " label='ufschem_v1',\n", " obj=...,\n", " extra_calc={'dewpoint': {'pres_calc': 'surfpres_pa', 'specific_hum': 'spfh2m'}, 'rel_hum': {'pres_calc': 'surfpres_pa', 'specific_hum': 'spfh2m', 'temp_calc': 'tmp2m'}, 'windspeed': {'u_comp': 'ugrd10m', 'v_comp': 'vgrd10m'}, 'winddir': {'u_comp': 'ugrd10m', 'v_comp': 'vgrd10m'}, 'wind_barb': {'u_comp': 'ugrd10m', 'v_comp': 'vgrd10m'}, 'rose_plot': {'model_wdir': 'winddir', 'model_wspd': 'windspeed'}},\n", " mapping={'ish': {'tmp2m': 't', 'windspeed': 'ws', 'dewpoint': 'dpt'}},\n", " variable_dict={'surfpres_pa': 'None', 'spfh2m': 'None', 'tmp2m': 'None', 'ugrd10m': 'None', 'vgrd10m': 'None', 'winddir': 'None', 'windspeed': 'None'},\n", " label='ufschem_v1',\n", " ...\n", " )}" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "an.models" ] }, { "cell_type": "markdown", "id": "67fee521", "metadata": {}, "source": [ "We can access the underlying dataset with the\n", "{attr}`~melodies_monet.driver.model.obj` attribute." ] }, { "cell_type": "code", "execution_count": 8, "id": "c27a6f8e", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
<xarray.Dataset> Size: 320MB\n",
"Dimensions: (y: 192, x: 384, time: 72, z: 1)\n",
"Coordinates:\n",
" latitude (y, x) float64 590kB dask.array<chunksize=(192, 384), meta=np.ndarray>\n",
" longitude (y, x) float64 590kB dask.array<chunksize=(192, 384), meta=np.ndarray>\n",
" * time (time) datetime64[ns] 576B 2017-07-01T01:00:00 ... 2017-07-04\n",
" * x (x) float64 3kB 0.0 0.9375 1.875 2.812 ... 357.2 358.1 359.1\n",
" * y (y) float64 2kB 89.28 88.36 87.42 ... -87.42 -88.36 -89.28\n",
"Dimensions without coordinates: z\n",
"Data variables: (12/14)\n",
" temperature_k (time, z, y, x) float32 21MB dask.array<chunksize=(1, 1, 192, 384), meta=np.ndarray>\n",
" surfpres_pa (time, z, y, x) float32 21MB dask.array<chunksize=(1, 1, 192, 384), meta=np.ndarray>\n",
" dp_pa (time, z, y, x) float32 21MB dask.array<chunksize=(1, 1, 192, 384), meta=np.ndarray>\n",
" surfalt_m (time, z, y, x) float32 21MB dask.array<chunksize=(1, 1, 192, 384), meta=np.ndarray>\n",
" dz_m (time, z, y, x) float32 21MB dask.array<chunksize=(1, 1, 192, 384), meta=np.ndarray>\n",
" pres_pa_mid (time, z, y, x) float64 42MB dask.array<chunksize=(1, 1, 192, 384), meta=np.ndarray>\n",
" ... ...\n",
" ugrd10m (time, z, y, x) float32 21MB dask.array<chunksize=(1, 1, 192, 384), meta=np.ndarray>\n",
" vgrd10m (time, z, y, x) float32 21MB dask.array<chunksize=(1, 1, 192, 384), meta=np.ndarray>\n",
" dewpoint (time, z, y, x) float32 21MB dask.array<chunksize=(1, 1, 192, 384), meta=np.ndarray>\n",
" rel_hum (time, z, y, x) float32 21MB dask.array<chunksize=(1, 1, 192, 384), meta=np.ndarray>\n",
" windspeed (time, z, y, x) float32 21MB dask.array<chunksize=(1, 1, 192, 384), meta=np.ndarray>\n",
" winddir (time, z, y, x) float32 21MB 243.3 244.2 ... 352.7 351.9\n",
"Attributes:\n",
" grid: gaussian\n",
" grid_id: 1\n",
" hydrostatic: non-hydrostatic\n",
" im: 384\n",
" jm: 192\n",
" ncnsto: 139\n",
" source: FV3GFS\n",
" NCO: netCDF Operators version 5.0.7 (Homepage = http://nco.sf.ne...\n",
" ak: [0. 0.]\n",
" bk: [0.99467119 1. ]\n",
" history: Fri Feb 27 09:53:22 2026: ncrcat 20170701_dynf001.nc 201707...<xarray.Dataset> Size: 48MB\n",
"Dimensions: (time: 49, y: 1, x: 12321)\n",
"Coordinates:\n",
" * time (time) datetime64[ns] 392B 2017-07-01 ... 2017-07-03\n",
" siteid (x) <U11 542kB ...\n",
" latitude (x) float64 99kB ...\n",
" longitude (x) float64 99kB ...\n",
" * x (x) int64 99kB 0 1 2 3 4 5 ... 12316 12317 12318 12319 12320\n",
"Dimensions without coordinates: y\n",
"Data variables: (12/19)\n",
" varlength (time, y, x) float64 5MB ...\n",
" wdir (time, y, x) float64 5MB nan nan 320.0 80.0 ... nan nan nan\n",
" ws (time, y, x) float64 5MB nan nan 4.0 2.0 ... nan nan nan nan\n",
" ceiling (time, y, x) float64 5MB ...\n",
" vsb (time, y, x) float64 5MB ...\n",
" t (time, y, x) float64 5MB nan nan 271.1 275.1 ... nan nan nan\n",
" ... ...\n",
" icao (y, x) <U4 197kB ...\n",
" elevation (y, x) float64 99kB ...\n",
" utcoffset (y, x) float64 99kB ...\n",
" begin (y, x) datetime64[ns] 99kB ...\n",
" end (y, x) datetime64[ns] 99kB ...\n",
" time_local (time, y, x) datetime64[ns] 5MB ...