{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# Vera C. Rubin Observatory Scheduler Reports\n",
    "\n",
    "The reports here are indexed below by night (dayobs) and instrument, with links to the following reports in different columns:\n",
    "\n",
    "- **prenight** is a report describing a simulation of the night completed the previous morning. It includes basic statistics of the visits from the simulation (total science visits taken, etc.), a dynamic map of the visits, and tools for plotting various visit parameters against time. In addition, the report includes visualizationn of the reward functions that led to each FBS selection, which can be helpful in understading what led to the FBS selecting any given visit.\n",
    "- **multiprenight** is a report that compares the nominal pre-night simulation described above and a few variants of it (e.g. simulating a late start). I includes figures like tables showing what fraction of visits that occur on one simulation also occur in other variants.  (Those interested in more details for the variants can use the Times Square interface to generate prenight reports like the one described above for any of the simulations completed; see the index [here](https://usdf-rsp.slac.stanford.edu/times-square/github/lsst/schedview_notebooks/prenight/prenight_sims): fill in the night you are interested in on the left, then select the desired sim from the resultant table.)\n",
    "- **nightsum** is a scheduler-focused summary of the night, including basic paramaters of the night (twilight times, etc.), basic statistics for the visits of the night (e.g. number of science visits, mean gap times between visits, etc.), a tool for plotting assorted visit parameters against time, an interactive visit map, a handful of maps for different metrics such as photometric depth, a DDF cadence plot showing when DDFs have been observed over the past year and how deep each night's observations went, and a text table of exposures.\n",
    "- **compareprenight** is a report that compares the set of science visits actually taken during a night with the pre-night simulations of that night, including plots in which parameter values are plotted over time, with simulation visits overplotted on completed visits, statistics on the numbers of visits each simulation has in common with the completed visits and the offsets in corresponding visit's start times, and a plot of completed vs. simulated visit start times for visits they have in common.\n",
    "\n",
    "`eff_time` as reported in the statistics in the table is a measure of the photometric depth, as described in section 7 of [DMTN-296](https://dmtn-296.lsst.io/), using reference m5 values from [SMTN-002](https://smtn-002.lsst.io/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "# To render this notebook to html, in the current directory with the current python environment\n",
    "\n",
    "jupyter nbconvert \\\n",
    "    --to html \\\n",
    "    --execute \\\n",
    "    --no-input \\\n",
    "    --ExecutePreprocessor.kernel_name=python3 \\\n",
    "    --ExecutePreprocessor.startup_timeout=3600 \\\n",
    "    --ExecutePreprocessor.timeout=3600 \\\n",
    "    schedview_reports_toc.ipynb\n",
    "\n",
    "# The public USDF page served at\n",
    "#    https://s3df.slac.stanford.edu/data/rubin/sim-data/schedview/reports\n",
    "# is found at\n",
    "#    /sdf/group/rubin/web_data/sim-data/schedview/reports/index.html\n",
    "\n",
    "# You can set up the conda environment used to automatically generate these pages with\n",
    "#    conda activate /sdf/data/rubin/shared/scheduler/envs/prenight\n",
    "\"\"\"\n",
    "pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import HTML\n",
    "import sys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3",
   "metadata": {},
   "outputs": [],
   "source": [
    "#sched_source = 'env'\n",
    "sched_source = 'shared'\n",
    "#sched_source = 'devel'\n",
    "match sched_source:\n",
    "    case 'shared':\n",
    "        if os.path.exists('/sdf/data/rubin/shared/scheduler/packages'):\n",
    "            sys.path.insert(0, \"/sdf/data/rubin/shared/scheduler/packages/rubin_scheduler\")\n",
    "            sys.path.insert(0, \"/sdf/data/rubin/shared/scheduler/packages/rubin_sim\")\n",
    "            sys.path.insert(0, \"/sdf/data/rubin/shared/scheduler/packages/schedview\")\n",
    "    case 'devel':\n",
    "        if os.path.exists('/sdf/data/rubin/user/neilsen/devel'):\n",
    "            sys.path.insert(0, \"/sdf/data/rubin/user/neilsen/devel/rubin_scheduler\")\n",
    "            sys.path.insert(0, \"/sdf/data/rubin/user/neilsen/devel/rubin_sim\")\n",
    "            sys.path.insert(0, \"/sdf/data/rubin/user/neilsen/devel/schedview\")\n",
    "    case _:\n",
    "        # Use whatever is in the kernel python environment\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "from schedview.util import config_logging_for_reports\n",
    "import logging\n",
    "config_logging_for_reports(logging.ERROR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "from rubin_scheduler.site_models import Almanac\n",
    "\n",
    "from rubin_sim import maf\n",
    "from rubin_sim.sim_archive.prenightindex import get_prenight_index, select_latest_prenight_sim\n",
    "from rubin_sim.sim_archive import vseqarchive\n",
    "\n",
    "from schedview.reports import find_reports, make_report_link_table, make_report_rss_feed\n",
    "from schedview.collect.visits import cached_read_visits, NIGHT_STACKERS\n",
    "from schedview.compute import smallsum\n",
    "from schedview import DayObs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6",
   "metadata": {},
   "outputs": [],
   "source": [
    "schedview_cache_dir = \"/sdf/group/rubin/web_data/sim-data/schedview/cache\"\n",
    "schedview_report_base = \"/sdf/group/rubin/web_data/sim-data/schedview/reports\"\n",
    "schedview_url_base = \"https://s3df.slac.stanford.edu/data/rubin/sim-data/schedview/reports\"\n",
    "dayobs_today = DayObs.from_date('today')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7",
   "metadata": {},
   "outputs": [],
   "source": [
    "visits = cached_read_visits(dayobs_today, 'lsstcam', schedview_cache_dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8",
   "metadata": {},
   "outputs": [],
   "source": [
    "if \"can_see_sky\" in visits.columns:\n",
    "    visits = visits.loc[visits.can_see_sky.fillna(True).astype(bool), :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9",
   "metadata": {},
   "outputs": [],
   "source": [
    "reports = find_reports(schedview_report_base, url_base=schedview_url_base)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10",
   "metadata": {},
   "outputs": [],
   "source": [
    "latest_prenight_dayobs = DayObs.from_date(\n",
    "    reports.reset_index().set_index(['instrument', 'report']).loc[('lsstcam', 'prenight'), 'dayobs'].max()\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11",
   "metadata": {},
   "outputs": [],
   "source": [
    "prenight_sims = get_prenight_index(latest_prenight_dayobs.yyyymmdd, telescope=\"simonyi\")\n",
    "best_prenight_sim = select_latest_prenight_sim(prenight_sims)\n",
    "prenight_visits = vseqarchive.get_visits(\n",
    "    best_prenight_sim[\"visitseq_url\"],\n",
    "    stackers=NIGHT_STACKERS + [maf.TeffStacker()],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12",
   "metadata": {},
   "outputs": [],
   "source": [
    "report_columns = (\"prenight\", \"multiprenight\", \"nightsum\", \"compareprenight\")\n",
    "report_link_table =  make_report_link_table(reports, report_columns=report_columns, visits=visits)\n",
    "display(HTML(report_link_table))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13",
   "metadata": {},
   "outputs": [],
   "source": [
    "instruments = reports.index.get_level_values(level='instrument').unique()\n",
    "for instrument in instruments:\n",
    "    inst_reports = reports.loc[instrument, :]\n",
    "    report_names = inst_reports['report'].unique()\n",
    "    for report_name in report_names:\n",
    "        latest_dayobs = inst_reports.reset_index().set_index('report').loc[report_name, 'dayobs'].max()\n",
    "        latest_url = reports.set_index('report', append=True).loc[(instrument, latest_dayobs, report_name), 'url']\n",
    "        latest_file = schedview_report_base + latest_url[len(schedview_url_base):]\n",
    "        latest_file_link = Path(schedview_report_base).joinpath(report_name, instrument, 'latest.html')\n",
    "        try:\n",
    "            try:\n",
    "                latest_file_link.hardlink_to(latest_file)\n",
    "            except FileExistsError:\n",
    "                latest_file_link.unlink()\n",
    "                latest_file_link.hardlink_to(latest_file)\n",
    "        except (PermissionError, OSError):\n",
    "            print(\"Could not update link to latest versions\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14",
   "metadata": {},
   "outputs": [],
   "source": [
    "rss_feed_base = '/sdf/group/rubin/web_data/sim-data/schedview'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Writes the rss feed to a publicly readable USDF site.\n",
    "for report in 'nightsum', 'prenight':\n",
    "    for instrument in 'lsstcam', 'latiss':\n",
    "        rss_feed_fname = f'{rss_feed_base}/{instrument}_{report}.rss'\n",
    "        these_reports = reports.reset_index().query(f'instrument==\"{instrument}\" and report==\"{report}\"').set_index(['instrument', 'dayobs'])\n",
    "        rss_tree = make_report_rss_feed(these_reports, fname=rss_feed_fname, visits=visits, prenight_visits=prenight_visits, max_days=2, title=f\"{instrument} {report}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Writes the rss feed to a publicly readable USDF site.\n",
    "rss_feed_fname = f'{rss_feed_base}/public_schedview_reports.rss'\n",
    "rss_tree = make_report_rss_feed(reports, fname=rss_feed_fname, max_days=1, visits=visits, prenight_visits=prenight_visits)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "LSST",
   "language": "python",
   "name": "lsst"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}