{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Datasets\n",
    "\n",
    "ObsPlus includes a few interesting datasets which are used for testing purposes, but a [template with instructions](https://github.com/niosh-mining/opsdata) is provided if you would like to create and distribute your own.\n",
    "\n",
    "\n",
    "The datasets are \"lazy\" in that all but the most essential information will be downloaded only when some code requests the dataset. This helps keep the size of ObsPlus small, a network connection will be needed the first time each dataset is used. Here are a few examples of things that can be done with datasets:\n",
    "\n",
    "## Dataset basics\n",
    "Loading a dataset only requires knowing its name (and having installed it, more on that later)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-20T08:37:55.639210Z",
     "iopub.status.busy": "2026-03-20T08:37:55.639033Z",
     "iopub.status.idle": "2026-03-20T08:37:56.819119Z",
     "shell.execute_reply": "2026-03-20T08:37:56.818358Z"
    }
   },
   "outputs": [],
   "source": [
    "import obspy\n",
    "\n",
    "import obsplus\n",
    "\n",
    "ds = obsplus.load_dataset(\"crandall_test\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best way to access the data in a dataset is by using the desired client:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-20T08:37:56.821739Z",
     "iopub.status.busy": "2026-03-20T08:37:56.821313Z",
     "iopub.status.idle": "2026-03-20T08:37:58.000697Z",
     "shell.execute_reply": "2026-03-20T08:37:57.999831Z"
    }
   },
   "outputs": [],
   "source": [
    "wave_client = ds.waveform_client\n",
    "station_client = ds.station_client\n",
    "event_client = ds.event_client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These clients behave the same as any `Client` in ObsPy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-20T08:37:58.003182Z",
     "iopub.status.busy": "2026-03-20T08:37:58.002953Z",
     "iopub.status.idle": "2026-03-20T08:37:58.273528Z",
     "shell.execute_reply": "2026-03-20T08:37:58.272662Z"
    }
   },
   "outputs": [],
   "source": [
    "st = wave_client.get_waveforms()\n",
    "assert isinstance(st, obspy.Stream)\n",
    "inv = station_client.get_stations()\n",
    "assert isinstance(inv, obspy.Inventory)\n",
    "cat = event_client.get_events()\n",
    "assert isinstance(cat, obspy.Catalog)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A 'Fetcher' can be used for \"dataset aware\" querying."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-20T08:37:58.275651Z",
     "iopub.status.busy": "2026-03-20T08:37:58.275455Z",
     "iopub.status.idle": "2026-03-20T08:37:58.478609Z",
     "shell.execute_reply": "2026-03-20T08:37:58.477765Z"
    }
   },
   "outputs": [],
   "source": [
    "fetcher = ds.get_fetcher()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each dataset is a just a directory of files whose path is stored as the `data_path` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-20T08:37:58.480513Z",
     "iopub.status.busy": "2026-03-20T08:37:58.480326Z",
     "iopub.status.idle": "2026-03-20T08:37:58.485738Z",
     "shell.execute_reply": "2026-03-20T08:37:58.484973Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PosixPath('/home/runner/opsdata/crandall_test')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds.data_path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The *included* data files are found in the `source_path`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-20T08:37:58.516873Z",
     "iopub.status.busy": "2026-03-20T08:37:58.516652Z",
     "iopub.status.idle": "2026-03-20T08:37:58.520582Z",
     "shell.execute_reply": "2026-03-20T08:37:58.519840Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PosixPath('/home/runner/work/obsplus/obsplus/src/obsplus/datasets/crandall_test')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds.source_path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Datasets can be copied with the `copy_dataset` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-20T08:37:58.522260Z",
     "iopub.status.busy": "2026-03-20T08:37:58.522079Z",
     "iopub.status.idle": "2026-03-20T08:37:58.568287Z",
     "shell.execute_reply": "2026-03-20T08:37:58.567588Z"
    }
   },
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "obsplus.copy_dataset(\"crandall_test\", \".\")\n",
    "path = Path(\".\") / \"crandall_test\"\n",
    "assert path.exists() and path.is_dir()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data path\n",
    "By default, all datasets are stored in the user's home directory in a directory called 'opsdata'. Each dataset is contained by a subdirectory with the same name as the dataset. The environmental variable `OPSDATA_PATH` can be used to change the default dataset location."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Included Test Datasets\n",
    "\n",
    "1. TA_test:\n",
    "    A small dataset with two stations from the TA with channels that have very low sampling rates. \n",
    "\n",
    "2. Crandall_test:\n",
    "    Event waveforms for the [Crandall Canyon Mine collapse](https://en.wikipedia.org/wiki/Crandall_Canyon_Mine) and associated aftershocks. The dataset also includes a catalog of the events and a station inventory.\n",
    "    \n",
    "3. Bingham_test:\n",
    "    Event waveforms associated with the [Bingham Canyon Landslide](https://en.wikipedia.org/wiki/Bingham_Canyon_Mine#Landslides), one of the largest anthropogenic landslides ever recorded. The dataset also includes a catalog of the events and a station inventory. \n",
    "    \n",
    "Each of these data sets is accessed via `obsplus.load_dataset` function which takes the name of the dataset as the only argument and returns a `DataSet` instance. This will take a few minutes if the datasets have not yet been downloaded, otherwise it should be very quick."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-03-20T08:37:58.570671Z",
     "iopub.status.busy": "2026-03-20T08:37:58.570484Z",
     "iopub.status.idle": "2026-03-20T08:37:58.582198Z",
     "shell.execute_reply": "2026-03-20T08:37:58.581155Z"
    }
   },
   "outputs": [],
   "source": [
    "# cleanup temporary directory\n",
    "import shutil\n",
    "from pathlib import Path\n",
    "\n",
    "path = Path(\"crandall_test\")\n",
    "if path.exists():\n",
    "    shutil.rmtree(path)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}