{ "cells": [ { "cell_type": "markdown", "id": "f9507824", "metadata": {}, "source": [ "# Reading Paired Data\n", "\n", "First let's just import the driver." ] }, { "cell_type": "code", "execution_count": 1, "id": "3d43faf7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Please install h5py to open files from the Amazon S3 servers.\n", "Please install h5netcdf to open files from the Amazon S3 servers.\n" ] } ], "source": [ "from melodies_monet import driver" ] }, { "cell_type": "markdown", "id": "24cc3a8c-dafa-4dd7-9328-04a0f8b0aada", "metadata": {}, "source": [ "## Read model and observations" ] }, { "cell_type": "code", "execution_count": 2, "id": "65671ca7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "example:wrfchem:racm_esrl\n", "**** Reading WRF-Chem model output...\n", "example:wrfchem:racm_esrl_vcp\n", "**** Reading WRF-Chem model output...\n" ] } ], "source": [ "an = driver.analysis()\n", "an.control = \"control_wrfchem_saveandread.yaml\"\n", "an.read_control()\n", "an.control_dict\n", "\n", "an.open_models()\n", "an.open_obs()" ] }, { "cell_type": "markdown", "id": "ddc902a4-7885-4c3e-b820-7096d00dddc0", "metadata": {}, "source": [ "## Read saved data using control file\n", "\n", "The driver will read the data based on the information included in the control file by calling {func}`an.read_analysis()`.\n", "\n", "In the control file analysis section, setting method to `'netcdf'` for a given attribute of the analysis class (e.g., paired, models, obs) will read NetCDF-4 files and set the appropriate attribute with the data. Filenames must be specified as a dict, with the keys being the pair name and the values being either a string with the filename to be read, or an iterable with multiple filenames to be read. If multiple files (such as several different days) are specified they will be joined by coordinates with [xarray's merge function](https://docs.xarray.dev/en/stable/generated/xarray.merge.html).\n", "\n", "In the control file analysis section, setting method to `'pkl'` for a given attribute of the analysis class (e.g., paired, models, obs) will read .pkl files and set the appropriate attribute with the data. Filenames must be specified as either a string or an iterable. If multiple files (such as several different days) are specified, they will be joined by coordinates with xarray's merge function." ] }, { "cell_type": "code", "execution_count": 3, "id": "94e9281e-0845-411b-a59d-96456ccb5a6b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Reading: ./output/save_and_read/0905_airnow_RACM_ESRL.nc4\n", "Reading: ./output/save_and_read/0905_airnow_RACM_ESRL_VCP.nc4\n" ] } ], "source": [ "an.read_analysis()" ] }, { "cell_type": "code", "execution_count": 4, "id": "774455e6-6dc1-4e65-995c-3ac75cd0a9d7", "metadata": { "tags": [ "hide-output" ] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:     (time: 2091, x: 3786)\n",
       "Coordinates:\n",
       "  * time        (time) datetime64[ns] 2019-09-01 ... 2019-09-30T00:30:00\n",
       "  * x           (x) int64 0 1 2 3 4 5 6 7 ... 3779 3780 3781 3782 3783 3784 3785\n",
       "Data variables: (12/35)\n",
       "    BARPR       (time, x) float64 ...\n",
       "    BC          (time, x) float64 ...\n",
       "    CO          (time, x) float64 ...\n",
       "    NO          (time, x) float64 ...\n",
       "    NO2         (time, x) float64 ...\n",
       "    NO2Y        (time, x) float64 ...\n",
       "    ...          ...\n",
       "    cmsa_name   (x) float64 -1.0 -1.0 -1.0 -1.0 -1.0 ... -1.0 -1.0 -1.0 -1.0\n",
       "    msa_code    (x) float64 -1.0 -1.0 -1.0 -1.0 ... -1.0 3.306e+04 -1.0 -1.0\n",
       "    msa_name    (x) object '' '' '' '' '' '' '' ... '' '' '' ' Miami, OK ' '' ''\n",
       "    state_name  (x) object 'CC' 'CC' 'CC' 'CC' 'CC' 'CC' ... '' '' '' '' '' ''\n",
       "    epa_region  (x) object 'CA' 'CA' 'CA' 'CA' 'CA' ... '' 'R6' 'DSMG' 'DSUZ'\n",
       "    siteid      (x) object '000010102' '000010401' ... 'UB1010001' 'UZB010001'\n",
       "Attributes:\n",
       "    title:         \n",
       "    format:        NetCDF-4\n",
       "    date_created:  2022-12-05\n",
       "    dict_json:     {\\n    "type": "pt_sfc",\\n    "radius_of_influence": 10000...\n",
       "    group_name:    airnow_RACM_ESRL
" ], "text/plain": [ "\n", "Dimensions: (time: 2091, x: 3786)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2019-09-01 ... 2019-09-30T00:30:00\n", " * x (x) int64 0 1 2 3 4 5 6 7 ... 3779 3780 3781 3782 3783 3784 3785\n", "Data variables: (12/35)\n", " BARPR (time, x) float64 ...\n", " BC (time, x) float64 ...\n", " CO (time, x) float64 ...\n", " NO (time, x) float64 ...\n", " NO2 (time, x) float64 ...\n", " NO2Y (time, x) float64 ...\n", " ... ...\n", " cmsa_name (x) float64 ...\n", " msa_code (x) float64 ...\n", " msa_name (x) object ...\n", " state_name (x) object ...\n", " epa_region (x) object ...\n", " siteid (x) object ...\n", "Attributes:\n", " title: \n", " format: NetCDF-4\n", " date_created: 2022-12-05\n", " dict_json: {\\n \"type\": \"pt_sfc\",\\n \"radius_of_influence\": 10000...\n", " group_name: airnow_RACM_ESRL" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "an.paired['airnow_RACM_ESRL'].obj" ] }, { "cell_type": "code", "execution_count": 5, "id": "57ff229b-8f12-4997-84c4-e68618f46806", "metadata": { "tags": [ "hide-output" ] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:     (time: 2091, x: 3786)\n",
       "Coordinates:\n",
       "  * time        (time) datetime64[ns] 2019-09-01 ... 2019-09-30T00:30:00\n",
       "  * x           (x) int64 0 1 2 3 4 5 6 7 ... 3779 3780 3781 3782 3783 3784 3785\n",
       "Data variables: (12/35)\n",
       "    BARPR       (time, x) float64 ...\n",
       "    BC          (time, x) float64 ...\n",
       "    CO          (time, x) float64 ...\n",
       "    NO          (time, x) float64 ...\n",
       "    NO2         (time, x) float64 ...\n",
       "    NO2Y        (time, x) float64 ...\n",
       "    ...          ...\n",
       "    cmsa_name   (x) float64 -1.0 -1.0 -1.0 -1.0 -1.0 ... -1.0 -1.0 -1.0 -1.0\n",
       "    msa_code    (x) float64 -1.0 -1.0 -1.0 -1.0 ... -1.0 3.306e+04 -1.0 -1.0\n",
       "    msa_name    (x) object '' '' '' '' '' '' '' ... '' '' '' ' Miami, OK ' '' ''\n",
       "    state_name  (x) object 'CC' 'CC' 'CC' 'CC' 'CC' 'CC' ... '' '' '' '' '' ''\n",
       "    epa_region  (x) object 'CA' 'CA' 'CA' 'CA' 'CA' ... '' 'R6' 'DSMG' 'DSUZ'\n",
       "    siteid      (x) object '000010102' '000010401' ... 'UB1010001' 'UZB010001'\n",
       "Attributes:\n",
       "    title:         \n",
       "    format:        NetCDF-4\n",
       "    date_created:  2022-12-05\n",
       "    dict_json:     {\\n    "type": "pt_sfc",\\n    "radius_of_influence": 10000...\n",
       "    group_name:    airnow_RACM_ESRL_VCP
" ], "text/plain": [ "\n", "Dimensions: (time: 2091, x: 3786)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2019-09-01 ... 2019-09-30T00:30:00\n", " * x (x) int64 0 1 2 3 4 5 6 7 ... 3779 3780 3781 3782 3783 3784 3785\n", "Data variables: (12/35)\n", " BARPR (time, x) float64 ...\n", " BC (time, x) float64 ...\n", " CO (time, x) float64 ...\n", " NO (time, x) float64 ...\n", " NO2 (time, x) float64 ...\n", " NO2Y (time, x) float64 ...\n", " ... ...\n", " cmsa_name (x) float64 ...\n", " msa_code (x) float64 ...\n", " msa_name (x) object ...\n", " state_name (x) object ...\n", " epa_region (x) object ...\n", " siteid (x) object ...\n", "Attributes:\n", " title: \n", " format: NetCDF-4\n", " date_created: 2022-12-05\n", " dict_json: {\\n \"type\": \"pt_sfc\",\\n \"radius_of_influence\": 10000...\n", " group_name: airnow_RACM_ESRL_VCP" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "an.paired['airnow_RACM_ESRL_VCP'].obj" ] }, { "cell_type": "markdown", "id": "72514743-b2cb-453c-9ff6-7e4873a90b20", "metadata": {}, "source": [ "## Read data without using control file\n", "\n", "Alternatively, the same can be achieved by calling {func}`melodies_monet.util.read_util.read_saved_data` directly. The object to set must be an attribute of the instance of the analysis class (e.g., {attr}`an.paired `, {attr}`an.models `, {attr}`an.obs `)." ] }, { "cell_type": "markdown", "id": "9b9f6c4e-ee74-4ebb-9a26-3a0dcda3faf5", "metadata": {}, "source": [ "```python\n", "# For netCDF files \n", "from melodies_monet.util.read_util import read_saved_data\n", "\n", "read_saved_data(\n", " analysis=an,\n", " filenames={'airnow_wrfchem_v4.2': ['0905_airnow_wrfchem_v4.2.nc4']},\n", " method='netcdf',\n", " attr='paired')\n", "```" ] }, { "cell_type": "markdown", "id": "90908ece-b080-4057-ad45-0b4b4d89fd8e", "metadata": {}, "source": [ "```python\n", "# For pickle files \n", "from melodies_monet.util.read_util import read_saved_data\n", "\n", "read_saved_data(analysis=an, filenames=['0905.pkl'], method='pkl', attr='paired')\n", "```" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.15" } }, "nbformat": 4, "nbformat_minor": 5 }