{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AutoCNet\n",
    "AutoCnet is a set of tools for generating, analyzing, and improving control networks of various sizes. In this tutorial, we will take a look at the basic components of AutoCNet and use it to analyze the initial Galileo Europa network."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Required for autocnet imports\n",
    "import os\n",
    "os.environ['ISISROOT'] = '/usgs/cpkgs/anaconda3_linux/envs/isis4.2.0'\n",
    "os.environ['PROJ_LIB'] = '/usgs/cpkgs/anaconda3_linux/envs/autocnet/share/proj'\n",
    "\n",
    "# Autocnet Imports\n",
    "from autocnet.graph.network import NetworkCandidateGraph\n",
    "from autocnet.graph.edge import NetworkEdge\n",
    "from autocnet.io.db.model import Matches, Points\n",
    "from autocnet.transformation.roi import Roi\n",
    "from autocnet.config_parser import parse_config\n",
    "\n",
    "# Helpful Python Modules\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import getpass\n",
    "from pathlib import Path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Configuration\n",
    "AutoCNet leverages services for cluster processing and data presistance that require configuration parameters to setup properly. For the cluster processing, AutoCNet uses a module called [redis](https://redis.io/) to create queues for the cluster jobs waiting to be dispatched to the cluster, a cluster for the computations, and conda environments activated during the cluster jobs to give the cluster access to the appropriate packages. Then for the database presistence AutoCNet uses a PostGIS database that requires spatial information.\n",
    "\n",
    "In summary, AutoCNet needs configuration parameters for the following servives:\n",
    "- redis\n",
    "- cluster\n",
    "- env (short for conda environment)\n",
    "- database\n",
    "- spatial\n",
    "\n",
    "## Setting up the Configuration\n",
    "You can configure AutoCNet with either a configuration file or a Python dictionary. If you are going to be doing continuing work on a network we recommend that you use a config file and treat it like the project file for your network. For this workshop, we're going to be using a dictionary because we can programmatically fill it in to ensure each participant has their own sand-box to work in.\n",
    "\n",
    "The configuration parameters are typically held in a configuration yaml file. A configuration file has been compiled for use internal to the USGS ASC facilities leveraging a shared cluster and database. Use AutoCNet's function 'parse_config' to read in the yaml file and output a dictionary variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Grab our template config and parse it\n",
    "template_config = '/scratch/jmapel/europa/config_template.yml'\n",
    "config_dict = parse_config(template_config)\n",
    "\n",
    "# Populate some fields with unique names using your username\n",
    "uid = getpass.getuser()\n",
    "\n",
    "# Setup directories on scratch to store data and logs\n",
    "workshop_scratch = os.path.join('/scratch/jmapel/europa/workshop_scratch', uid)\n",
    "workshop_logs = os.path.join(workshop_scratch, \"logs\")\n",
    "workshop_temp = os.path.join(workshop_scratch, \"temp\")\n",
    "Path(workshop_logs).mkdir(parents=True, exist_ok=True)\n",
    "Path(workshop_temp).mkdir(parents=True, exist_ok=True)\n",
    "config_dict[\"cluster\"][\"cluster_log_dir\"] = workshop_logs\n",
    "config_dict[\"cluster\"][\"tmp_scratch_dir\"] = workshop_temp\n",
    "\n",
    "# This is the name of your database\n",
    "config_dict[\"database\"][\"name\"] = f\"{uid}_workshop_gal\"\n",
    "print(\"Your database is named\", config_dict[\"database\"][\"name\"])\n",
    "\n",
    "# Setup redis queues to pass data to cluster jobs\n",
    "config_dict[\"redis\"][\"basename\"] = f'{uid}_workshop_gal'\n",
    "config_dict[\"redis\"][\"completed_queue\"] = f'{uid}_workshop_gal:done'\n",
    "config_dict[\"redis\"][\"processing_queue\"] = f'{uid}_workshop_gal:proc'\n",
    "config_dict[\"redis\"][\"working_queue\"] = f'{uid}_workshop_gal:working'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The NetworkCandidateGraph\n",
    "The core of AutoCNet is the NetworkCandidateGraph, or NCG, which represents all of the images and their relationships. As the name implies, the NCG is a graph structure where each image is a node and overlapping images have an edge between them. The NCG stores all of the control points and control measures in separate tables and associates them with corresponding images. The last component of the NCG is its spatial abilities. Because it is backed by a PostGIS database, each image, overlap region, and control point has a latitude, longitude geometry associated with it. These three components together allow for a broad range of control processing techniques."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up the NetworkCandidateGraph\n",
    "The first step in creating a NetworkCandidateGraph is to create an object and configure it. This will setup the NCG with all of the information it needs to store and process the data we are going to load into it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ncg = NetworkCandidateGraph()\n",
    "ncg.config_from_dict(config_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading image data\n",
    "Now that our NetworkCandidateGraph is configured, we can load our Galileo images into it.\n",
    "\n",
    "**NOTE: Be very careful re-running this cell as the clear_db=True parameter dumps all of the data out of your database if it already exists**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "galileo_list = '/scratch/jmapel/europa/original_cubes.lis'\n",
    "ncg.add_from_filelist(galileo_list, clear_db=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now look at our NCG and see the relationships between the images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ncg.plot()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise: What do you notice about the graph structure? Why does the graph look that way?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PGAdmin\n",
    "\n",
    "Let's take a look at our NCG's database. You can view all of the data in our NCG on PGAdmin at [autocnet.wr.usgs.gov](https://autocnet.wr.usgs.gov)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading control data\n",
    "\n",
    "If we were generating a network from scratch, we would use some of AutoCNets image matching and spatial tools to pick control points and control measures. We'll see that process in the next workshop, for now we're just going to load in the data from the existing ISIS control network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "galileo_net = '/scratch/jmapel/europa/networks/GLL_FFCombined_thin_SubReg2_Del_2.net'\n",
    "ncg.place_points_from_cnet(galileo_net)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise: Go on PGAdmin and look at what tables are populated in the database and what tables are not populated."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## In the future\n",
    "The steps we went through here are neatly combined into a single function and generally that's what you will use to create a NCG from a list of images and an ISIS control network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "?NetworkCandidateGraph.from_cnet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also reload an existing database into a NCG once it has been configured to point to the database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "?NetworkCandidateGraph.from_database"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting the pairwise image matches\n",
    "All of our control data is now loaded into out NCG and we can start working with it. In this notebook we will be using pairwise compute vision techniques to look for bad control measures.\n",
    "\n",
    "The first process we need to do is collect all of the pairwise matches between images. Our control network currently contains control points and all of the measurements of them. We need to convert these multi-image relationships into all of the common points between each pair of images.\n",
    "\n",
    "To do this, we're going to use Autocnet's apply function to run parallel processing on each edge of the NetworkCandidateGraph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "?NetworkCandidateGraph.apply"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SLURM parameters\n",
    "These parameters will be used when creating SLURM jobs for cluster processing via the apply function. Depending on the complexity of the jobs you are running, you may want to change the walltime and arraychunk parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "walltime=\"00:30:00\"\n",
    "arraychunk=75\n",
    "chunksize=16723\n",
    "\n",
    "# make a new directory for just our matching logs\n",
    "match_log_dir = os.path.join(workshop_logs, \"match\")\n",
    "Path(match_log_dir).mkdir(parents=True, exist_ok=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convert control measures and points to image matches\n",
    "This cell uses apply to run the network_to_matches function on each edge. It is very important that this function only get run once per edge or it will add duplicate matches. So, this cell also contains a check that skips the function if the database already has matches in it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "?NetworkEdge.network_to_matches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "with ncg.session_scope() as session:\n",
    "    num_matches = session.query(Matches).count()\n",
    "if num_matches == 0:\n",
    "    print(\"Loading matches table\")\n",
    "    njobs = ncg.apply('network_to_matches', \n",
    "                      on='edges',\n",
    "                      # SLURM kwargs\n",
    "                      walltime=walltime,\n",
    "                      log_dir=match_log_dir,\n",
    "                      arraychunk=arraychunk,\n",
    "                      chunksize=chunksize)\n",
    "    print(f'Started {njobs} jobs')\n",
    "else:\n",
    "    print(\"Matches table already populated\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Looking at running cluster jobs with SLURM\n",
    "Our cell has completed running in our notebook, but the jobs on the cluster have not finished yet. So, let's take a look at the what SLURM says is running for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!squeue -u $uid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Outlier Detection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Homographies and Reprojective Error\n",
    "The first computer vision technique that we will apply is called a homography. A homography is a transformation between two planes. In the contect of multi-view imagery, a homography is used when two images observe the same planar surface."
   ]
  },
  {
   "attachments": {
    "ea015a00-918a-4166-9f58-30094721de1b.png": {
     "image/png": "[encoded data, 183504 characters]"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![image.png](attachment:ea015a00-918a-4166-9f58-30094721de1b.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we can identify points on a planar surface in two images, then we can compute a homography that transforms how the planar surface appears in one image into how the planar surface appears in the other image. Assuming that our surface is roughly planar, we can use our image to image matches to approximate a homography. Then, we can take a measure in one image and transform it into the matching image. Finally, we compare the transformed measure location to the actual measure location and if they vary sufficiently that match is likely incorrect."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "?NetworkEdge.compute_homography"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Fundamental Matrix\n",
    "The next computer vision technique we will apply is called the fundamental matrix. Similar to a homography, the fundamental matrix describes the relationship between two images that are observing the same thing. Unlike a homography, though, the fundamental matrix does not assume that the two images are observing a planar surface. Instead, the fundamental matrix describes the epipolar geometry between the two images. In short, the epipolar geometry is what lines and points are visible in one image, but occluded in the other."
   ]
  },
  {
   "attachments": {
    "b89ef6c2-49e1-42be-bfdb-3f3fcde30d25.png": {
     "image/png": "[encoded data, 57948 characters]"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![image.png](attachment:b89ef6c2-49e1-42be-bfdb-3f3fcde30d25.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$x = (line, sample, 1)$ is the measure in image A\n",
    "\n",
    "$x^{'} = (line, sample, 1)$ is the measure in image B\n",
    "\n",
    "$F \\in \\mathbb{R}^{3\\times3}$ is the Fundamental matrix for images A and B\n",
    "\n",
    "$x^{T}Fx^{'} = 0$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This geometry is another relationship between matching measures that we can use to find potentially bad matches. Unlike a homography, the fundamental matrix is unique for a pair of images. So, we can compute a fundamental matrix for the image pair and then check that all of the matching measures satisfy the fundamental matrix relationship. Because the fundamental matrix describes the 3D relationship between the images, we need more data to compute it. At a minimum we need 8 matches to compute the fundamental matrix, but for robustness, we want at least 16."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "?NetworkEdge.compute_fundamental_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running the outlier checks\n",
    "Now that we know what we're doing, let's run the outlier checks on our network. Because we have over 100,000 matches on around 5,000 edges to process, we're going to again dispatch this work to the cluster.\n",
    "\n",
    "Just like before we're going to use apply, but this time we're going to pass parameters to the functions we're applying. In both functions we will use ransac try and mitigate the effects of the outliers. For the fundamental matrix we'll pass a tolerance of 3 pixels and for the homography we'll pass a tolerance of 5 pixels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# make a new directory for just our fundamental logs\n",
    "# the other SLURM parameters will stay the same\n",
    "fundamental_log_dir = os.path.join(workshop_logs, \"fundamental\")\n",
    "Path(fundamental_log_dir).mkdir(parents=True, exist_ok=True)\n",
    "\n",
    "njobs = ncg.apply('compute_fundamental_matrix', \n",
    "                  on='edges',\n",
    "                  # homography kwargs\n",
    "                  method='ransac',\n",
    "                  reproj_threshold=3,\n",
    "                  # SLURM kwargs\n",
    "                  walltime=walltime,\n",
    "                  log_dir=fundamental_log_dir,\n",
    "                  arraychunk=arraychunk,\n",
    "                  chunksize=chunksize)\n",
    "\n",
    "print(f'Starting {njobs} fundamental matrix checks')\n",
    "print(f'Logs are located at {fundamental_log_dir}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make a new directory for just our fundamental logs\n",
    "# the other SLURM parameters will stay the same\n",
    "homography_log_dir = os.path.join(workshop_logs, \"homography\")\n",
    "Path(homography_log_dir).mkdir(parents=True, exist_ok=True)\n",
    "\n",
    "njobs = ncg.apply('compute_homography', \n",
    "                  on='edges',\n",
    "                  # fundamental matrix kwargs\n",
    "                  method='ransac',\n",
    "                  reproj_threshold=5,\n",
    "                  # SLURM kwargs\n",
    "                  walltime=walltime,\n",
    "                  log_dir=homography_log_dir,\n",
    "                  arraychunk=arraychunk,\n",
    "                  chunksize=chunksize)\n",
    "\n",
    "print(f'Starting {njobs} homography checks')\n",
    "print(f'Logs are located at {homography_log_dir}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tracking running cluster jobs with REDIS\n",
    "\n",
    "You can also check the queue_length property on the NetworkCandidateGraph object to see how many jobs are either waiting to be processed or in process. This property is how many jobs are waiting in your REDIS job queue."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import some things to make a more active tracker\n",
    "import time\n",
    "from IPython.display import display, clear_output\n",
    "\n",
    "while (ncg.queue_length > 0):\n",
    "    clear_output(wait=True)\n",
    "    display(ncg.queue_length,)\n",
    "    time.sleep(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Looking at the results\n",
    "The outlier checks add a property to each edge called masks. This is a Pandas dataframe that contains a column for each check that has been done on the edge. If the row for a match has a true in it, then that match passed the column's check. Conversely, if the row for a match has a false in it, then that match failed the column's check. We will aggregate this data into a dataframe to get an overview of our results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "match_data = {\n",
    "    \"source\": [],\n",
    "    \"dest\": [],\n",
    "    \"matches\": [],\n",
    "    \"homography\": [],\n",
    "    \"fundamental\": []\n",
    "}\n",
    "for source, dest, edge in ncg.edges(data=\"data\"):\n",
    "    num_matches = len(edge.matches)\n",
    "    # Edges with few matches are more likely to be skewed by outliers so only take edges with 20+ matches\n",
    "    if num_matches < 20:\n",
    "        continue\n",
    "    match_data[\"source\"].append(source)\n",
    "    match_data[\"dest\"].append(dest)\n",
    "    match_data[\"matches\"].append(num_matches)\n",
    "    match_data[\"homography\"].append(sum(edge.masks['homography']))\n",
    "    match_data[\"fundamental\"].append(sum(edge.masks['fundamental']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "match_df = pd.DataFrame(data=match_data)\n",
    "match_df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's somewhat difficult to compare edges when they all have different numbers of matches, so let's compute the pass rates and sort the edges from worst to best."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "match_df[\"homography_percent\"] = match_df[\"homography\"] / match_df[\"matches\"]\n",
    "match_df[\"fundamental_percent\"] = match_df[\"fundamental\"] / match_df[\"matches\"]\n",
    "match_df[[\"homography_percent\",\"fundamental_percent\"]]\n",
    "match_df.sort_values([\"fundamental_percent\", \"homography_percent\"]).head(50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Viewing Nodes\n",
    "In the matches data you'll notice that the \"source\" and \"dest\" columns have just numbers in them instead of image paths or serial numbers. This is because of how AutoCNet stores images in the NetworkCandidateGraph. Each image is stored on a node identified by its image id. We can get things like the path to the image and the image's serial number using the node-view of the NCG."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_1 = ncg.nodes[1]['data'] # Note that we need to access the node data here\n",
    "print(image_1['image_path'])\n",
    "print(image_1.isis_serial)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Viewing Edges\n",
    "Similar to the nodes, edges in our NCG are identified by IDs. Each edge is identified by the IDs of the nodes it connects. You can access the nodes in our NCG using the get_edge_data method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "?NetworkCandidateGraph.get_edge_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "edge_1_2 = ncg.get_edge_data(1, 2)['data'] # Note the data again"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you have an edge, you can do things like compute the overlap between the two images or generate matched. In this case we will be looking at the matches from our control network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# There are a lot of columns in the matches dataframe so we'll make a list of just the ones we want\n",
    "match_columns = [\"point_id\", \"source_measure_id\", \"destin_measure_id\", \"source_x\", \"source_y\", \"destination_x\", \"destination_y\"]\n",
    "edge_1_2.matches[match_columns]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also filter the matches using the edge's mask.\n",
    "\n",
    "The cell below will show us all of the matches that passed the fundamental matrix check"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "edge_1_2.matches.loc[edge_1_2.masks['fundamental'], match_columns]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This next cell will show us all of the matches that failed the fundamental matrix check"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "edge_1_2.matches.loc[~edge_1_2.masks['fundamental'], match_columns] # Note the ~ to negate the fundamental mask"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise: Pick an edge from our matches_df that has some homography failures and find the matches that failed.\n",
    "Use the code fragment in the cell below and replace the `___` with the approriate things"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "failed_edge = ncg.get_edge_data( ___ , ___ )['data']\n",
    "failed_edge.matches.loc[~failed_edge.masks[ ___ ], match_columns]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Viewing Matches\n",
    "Listing the match data is nice, but it's hard to do anything with just that. So, let's look at the actual matches between images.\n",
    "\n",
    "First we're going to pick an edge. In this case we'll be looking at edge 110, 33"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "viewing_edge = ncg.get_edge_data(110, 33)['data']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're going to look at three different sets of matches:\n",
    "\n",
    "1. The matches that failed the homography check\n",
    "1. The matches that failed the fundamental matrix check\n",
    "1. The matches that failed both checks\n",
    "1. The matches that failed either check\n",
    "\n",
    "The code for the first two is exactly like how we viewed the matches on edge 1,2. The code for the last two sets is a little more complicated and will use the all and any functions which allow you to combine multiple masks into a single mask."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "?pd.DataFrame.all"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "?pd.DataFrame.any"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "failed_homography = viewing_edge.matches.loc[(~viewing_edge.masks['homography'])]\n",
    "failed_fundamental = viewing_edge.matches.loc[(~viewing_edge.masks['fundamental'])]\n",
    "failed_both = viewing_edge.matches.loc[(~viewing_edge.masks[['fundamental', 'homography']]).all(axis=1)]\n",
    "failed_either = viewing_edge.matches.loc[(~viewing_edge.masks[['fundamental', 'homography']]).any(axis=1)]\n",
    "print(f'{len(failed_homography)} matches failed the homography check')\n",
    "print(f'{len(failed_fundamental)} matches failed the fundamental matrix check')\n",
    "print(f'{len(failed_both)} matches failed both checks')\n",
    "print(f'{len(failed_either)} matches failed a checks')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It looks like for our particular edge, there are no matches that failed both checks. That's okay it's still worth looking at these failed matches. The next cell will take each match and show us what point it is from and a small chip around the failed measures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "roi_size = 25\n",
    "for idx, match in failed_either.iterrows():\n",
    "    with ncg.session_scope() as session:\n",
    "        point_name = session.query(Points).filter(Points.id == match[\"point_id\"]).first().identifier\n",
    "\n",
    "    source_image = ncg.nodes[match['source']]['data']\n",
    "    dest_image = ncg.nodes[match['destination']]['data']\n",
    "    source_roi = Roi(source_image.geodata, match['source_x'], match['source_y'], size_x=roi_size, size_y=roi_size)\n",
    "    dest_roi = Roi(dest_image.geodata, match['destination_x'], match['destination_y'], size_x=roi_size, size_y=roi_size)\n",
    "\n",
    "    fig, (ax1, ax2) = plt.subplots(1,2)\n",
    "    fig.suptitle(point_name)\n",
    "    ax1.imshow(source_roi.array, cmap='gray')\n",
    "    ax1.plot(source_roi.center[0] + source_roi.axr, source_roi.center[1] + source_roi.ayr, 'ro')\n",
    "    ax1.title.set_text(os.path.split(source_image['image_name'])[-1])\n",
    "    ax2.imshow(dest_roi.array, cmap='gray')\n",
    "    ax2.plot(dest_roi.center[0] + dest_roi.axr, dest_roi.center[1] + dest_roi.ayr, 'bo')\n",
    "    ax2.title.set_text(os.path.split(dest_image['image_name'])[-1])\n",
    "    fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Exercise: Pick a different edge that had failures on it and look at some of the failed match sets\n",
    "You can change viewing_edge and then re-run the last two cells to do this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "viewing_edge = ncg.get_edge_data( ___ , ___ )['data']"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "AutoCNet PLIO (workshop)",
   "language": "python",
   "name": "autocnet_workshop"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
