Merge branch 'env_setup' into 'main'

Added environment setup and maintenance docs

Closes #16

See merge request astrogeology/asc-public-docs!18

docs/how-to-guides/Environment Setup and Maintenance/Anaconda in a Network Environment.md

+# Creating a shared Anaconda installation
+
+## Introduction
+
+Here at the USGS Astrogeology Science Center (ASC) we have a software development team consisting of roughly a dozen full or part time developers, another dozen or so technical users, and a few dozen scientists who all need consistent access to software builds and releases. As we moved to using Anaconda for dependency management and software installation, our IT group quickly flagged an issue where users' Home directories were growing astronomically large from each having their own Anaconda installation. Along with that, we had many users who had never used Anaconda before having to manage and debug their own installations. Fairly quickly, we settled upon having shared Anaconda installations that users could access and use across systems without having to manage their own Anaconda installation or update their environments.
+
+## The systems landscape at the ASC
+
+I'm sure our IT folks could give a much better explanation than me, but here's a quick run-down of our systems at the ASC. The majority of our users have either a Windows desktop, an Ubuntu desktop, or a macbook. We primarily target our software for use on Unix systems so those users who have a windows desktop use PuTTY to connect to Linux Virtual Machines (VMs). Along side individual workstations, we also have a Linux processing cluster and have Network File System (NFS) drives that users can access. Users can mount the NFS drives directly to their Unix work station or they can access them by connecting to the VMs. The primary NFS drive that I'll be talking about is `/nfs/cpkgs/`. This drive is a shared drive that we mount on all of our Unix systems, hence the name cpkgs for common packages. Generally, users manage their own project data either locally, on other NFS drivers, or on high performance storage used by the processing cluster. Files on `/nfs/cpks/` are only updated by software developers or some technical users to setup environments for a project or a set of users. So, most users access software and general purpose data from `/nfs/cpkgs/` but don't store anything there.
+
+## Some background on Anaconda
+
+If you are already familiar with Anaconda, Anaconda channels, Anaconda environments, and Anaconda configuration; then you can go ahead and skip to the next section. If any of those things are new or you have questions about them, this section will give you a brief introduction and some links to further docs.
+
+### Anaconda
+
+First of all, Anaconda is a package manager that allows you to install a whole bunch of different pieces of software with one or two commands. Let's say you want to install opencv so that you can do some computer vision work in Python; all you have to do to get started is run `conda install opencv` and Anaconda will download the latest version of OpenCV (and all of its dependencies) and install the Python bindings into its Python installation. You can download Anaconda and read the docs on their [Individual Edition Home Page](https://www.anaconda.com/products/individual). The company that makes Anaconda has some paid products and support that you can get, but we don't use them so I won't be talking about them.
+
+You can use Anaconda to install packages with `conda install <package-name>`. For example, I can install the latest available version of OpenCV with `conda install opencv`. That will download and install the latest version of OpenCV from https://anaconda.org/anaconda/opencv/files, currently 4.0.1. I can instead install a specific version with `conda install <package-name>=<version-number>.` For example, I can install OpenCV 3.4.1 with `conda install opencv=3.4.1`.
+
+### Environments
+
+So now that you know how to find and install packages, what do you do when you don't want a package available all the time or you want different versions of a package available? By default Anaconda will install packages into the directory you install it in. For example, if I install Anaconda into `~/anaconda` and then I install latest version of OpenCV, it will install the dynamic library into `~/anaconda/lib`, the c++ headers into `~/anaconda/include`, and the Python bindings into `~/anaconda/lib/python3.7/site-packages`. If I then have Anaconda install an older version of OpenCV, let's say 3.2, it will replace everything I just installed with an older version. Uninstalling and reinstalling different versions of a package all the time can be a real pain, so instead let's use [environments](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html) to install them and then we can active the latest OpenCV environment or the OpenCV 3.2 environment depending on what we want. Each environment is basically a separate set of packages that you can swap between whenever you want, and without having to reinstall anything. First, you can create an environment using `conda create my_envrionment`. Then, you can activate that environment by running `conda activate my_environment`. From here, you are ready to install any packages you want and when you're done with it you can deactivate it by running `conda deactivate`.
+
+Using the same example installation as before, I can run `conda create -n my_environment` and I will get a new environment at `~/anaconda/envs/my_environment`. If I then activate that environment with `conda activate my_environment` and install the latest OpenCV with `conda install -c conda-forge opencv` it will install the library into `~/anaconda/envs/my_environment/lib`, the c++ headers into `~/anaconda/envs/my_environment/include`, and the Python bindings into `~/anaconda/envs/my_environment/lib/python3.7/site-packages`. I do some work and then I want to swap to OpenCV 3.2. First, I deactivate my latest OpenCV environment with `conda deactivate`. Next, I make a second environment with `conda create -n opencv_old` which will be located at `~/anaconda/envs/opencv_old` and activate it with `conda activate opencv_old`. Now I have a clean environment and can install OpenCV 3.2 with `conda install -c conda-forge opencv=3.2`. This will install the library into `~/anaconda/envs/opencv_old/lib`, the c++ headers into `~/anaconda/envs/opencv_old/include`, and the Python bindings into `~/anaconda/envs/opencv_old/lib/python3.7/site-packages`.
+
+You can have many different environments in your anaconda installation and swap between them at will. The only stuff shared between different environments is what's in your base environment. Your base environment is the root directory in your Anaconda installation and you can activate it using `conda activate`. In our installation example, the base environment would be located at `~/anaconda`. Notice that by default Anaconda installs packages into your base environment when you don't have another environment active. Because anything installed here will bleed into your other environments, we highly discourage installing anything into your base environment besides what comes with your Anaconda installation.
+
+### Configuration
+
+Constantly specifying channels can be a real chore, so you may want to add a channel to your Anaconda config with `conda config --add channels <channel-name>`. This command will add the channel to your default search and you won't have to specify it every time you install a package anymore. For example you can default to packages from conda-forge with `conda config --add channels conda-forge`. You can also specify a configuration for just an environment by activating the environment and then adding `--env` to your conda config command. For example, I could setup `my_environment` from above to default to searching conda-forge for packages by first activating it with `conda activate my_environment` and then modifying the config with `conda config --env --add channels conda-forge`. This would cause Anaconda to default to conda-forge only when `my_environment` is activate.
+
+All of this works using `.condarc` files at different places. If you run conda config, then it defaults to modifying your personal `.condarc` file at `~/.condarc`. If you use the `--env` option, then it modifies the `.condarc` file in your environment. For example, when I modified `my_environment` to default search conda-forge it added the following to `~/anaconda/envs/my_environment/.condarc`:
+
+```
+channels:
+  - conda-forge
+  - defaults
+```
+
+The [conda config](https://docs.conda.io/projects/conda/en/latest/commands/config.html) can set a whole bunch of other options that are beyond the scope of this document. You can read more about what you can do with `.condarc` files [here](https://conda.io/projects/conda/en/latest/user-guide/configuration/use-condarc.html).
+
+## Our shared Anaconda installation
+
+We wanted to setup a single Anaconda installation in our `/nfs/cpkgs/` NFS drive, but Anaconda requires separate installations for Linux and Mac. So, we instead have two Anaconda installations: `/nfs/cpkgs/anaconda3_linux` and `/nfs/cpkgs/anaconda3_macOS`. As their name suggests, `anaconda3_linux` is for Linux and `anaconda3_macOS` is for Mac. Within each installation we have separate environments for different versions of software. For example, under `/nfs/cpkgs/anaconda3_linux` we have the following environments for different versions of ISIS:
+
+* isis3.6.0
+* isis3.6.2
+* isis3.7.0
+* isis3.7.1
+* isis3.8.0
+* isis3.8.1
+* isis3.9.0
+* isis3.9.1
+* isis4.0.0
+* isis4.0.1
+* isis4.1.0
+* isis4.1.1
+* isis4.2.0
+* isis4.3.0
+
+Whenever there is a new release or release candidate for ISIS, we make a new environment and install the new version into it. This allows users to select any version they want and they can easily upgrade or downgrade for their processing needs. The permissions on the installations are restricted so that users can access all of the files, but cannot modify any of them to prevent accidentally changing an environment.
+
+## Using the shared Anaconda installations
+
+Users can access the shared installation by running the `conda.sh` or `conda.csh` scripts in `/nfs/cpkgs/anaconda3_linux/etc/profile.d/` or `/nfs/cpkgs/anaconda3_macOS/etc/profile.d/`. These will modify their environment to run conda and give them access to the shared environments. Most of our users run bash, so we recommend that users simply add `source "/nfs/cpkgs/anaconda3_linux/etc/profile.d/conda.sh"` or `source "/nfs/cpkgs/anaconda3_macOS/etc/profile.d/conda.sh"` to their `.bashrc` or `.bash_profile`.
+
+Users can even create personal environments with the shared Anaconda utilities. If a user creates an environment with the shared installation, it will create the environment in their home directory instead of inside the installation. For example, if I attempt to create an OpenCV environment with `conda create -n opencv` then it will be located at `~/.conda/envs/opencv`. This is caused by users not having write permissions to the shared installation, so conda defaults back to their home directory. It is okay for users to create one or two environments this way but if users want to have many different environments that they manage themselves and use the shared environments we recommend they setup their own Anaconda installation and follow the steps in the next section to add the shared environments.
+
+## Using the shared Anaconda environments with your own Anaconda installation
+
+Some users want to have their own Anaconda installation and still have access to the shared environments. Some users want to have more fine control over their processing environment while others want to be able to keep their own environments for when they are outside of our network and don't have access to the NFS drives. Thankfully, there is an easy solution to still allow them access to the share environments. Users simply need to run `conda config --add envs_dirs "/nfs/cpkgs/anaconda3_linux/envs"` or `conda config --add envs_dirs "/nfs/cpkgs/anaconda3_macOS/envs"`. This will add the following to their `/.condarc` file:
+
+```
+envs_dirs:
+  - /nfs/cpkgs/anaconda3_linux/envs
+```
+
+Once this is done, they can run `conda activate <shared-env-name>` to activate a shared environment. If the shared environment files are not available because the NFS drive is not mounted, then Anaconda will still continue to operate as before.
+
+## A few warnings
+
+1. NFS drives are not 100% foolproof and you will sometimes have issues, here are a few we have run into. 
+    * An Anaconda package adds some files to the NFS drive and the NFS software modifies them resulting in the Anaconda package thinking it is corrupted.
+    * An Anaconda package uses a SQLite database that does not lock and unlock properly because the NFS drive doesn't handle rapidly locking and unlocking a file well. NFS drives can be very slow for I/O bound processes such as NAIF DSK routines.
+
+1. Anaconda uses fairly aggressive caching. If it finds anything locally that works, it generally doesn't check the cloud for something better. By keeping old environments with old packages around, you can sometimes get solves that use old packages. Even, just adding the environments via the conda config command from the previous section can cause Anaconda to pull old packages from them. Our developers often have to remove the shared environments from their envs_dirs config in order to reproduce user reported installation issues.
+
+1. If you have a networked home directory that you share between Linux and Mac systems, you'll need to add some logic to your `.bashrc` and/or `.bash_profile` files that selects the proper Anaconda installation depending on what OS you are on. If you have a shared home like this, then you likely already have logic like this in your start up files.
\ No newline at end of file

docs/how-to-guides/Environment Setup and Maintenance/Installing ISIS Via Anaconda.md

+# Installation
+
+This installation guide is for ISIS users interested in installing ISIS (3.6.0)+ through conda.
+
+## ISIS Installation With Conda
+
+1. Download either the Anaconda or Miniconda installation script for your OS platform. Anaconda is a much larger distribution of packages supporting scientific python, while Miniconda is a minimal installation and not as large: [Anaconda installer](https://www.anaconda.com/download), [Miniconda installer](https://conda.io/miniconda.html)
+1. If you are running on some variant of Linux, open a terminal window in the directory where you downloaded the script, and run the following commands. In this example, we chose to do a full install of Anaconda, and our OS is Linux-based. Your file name may be different depending on your environment.
+    
+    ```bash
+    chmod +x Anaconda3-5.3.0-Linux-x86_64.sh
+    ./Anaconda3-5.3.0-Linux-x86_64.sh
+    ```
+    This will start the Anaconda installer which will guide you through the installation process.
+
+1. If you are running Mac OS X, a pkg file (which looks similar to Anaconda3-5.3.0-MacOSX-x86\_64.pkg) will be downloaded. Double-click on the file to start the installation process.
+1. After the installation has finished, open up a bash prompt in your terminal window.
+1. If you have an ARM64 Mac (M1/M2) running Catalina (or later), additional prerequisites must be installed for ISIS to run in emulation:
+ - Install [XQuartz](https://www.xquartz.org/). (Tested with XQuartz 2.8.5 on MacOS Catalina)
+ - Install Rosetta2. From the terminal run: `/usr/sbin/softwareupdate --install-rosetta --agree-to-license`
+ - Include the `# MacOS ARM64 Only` lines below
+1. Next setup your Anaconda environment for ISIS. In the bash prompt, run the following commands:
+
+    ```bash
+    
+    #MacOS ARM64 Only - Setup the new environment as an x86_64 environment
+    export CONDA_SUBDIR=osx-64
+    
+    #Create a new conda environment to install ISIS in
+    conda create -n isis
+
+    #Activate the environment
+    conda activate isis
+    
+    #MacOS ARM64 Only - Force installation of x86_64 packages instead of ARM64
+    conda config --env --set subdir osx-64
+
+    #Add the following channels to the environment
+    conda config --env --add channels conda-forge
+    conda config --env --add channels usgs-astrogeology
+
+    #Verify you have the correct channels:
+    conda config --show channels
+
+    #You should see:
+
+    channels:
+        - usgs-astrogeology
+        - conda-forge
+        - defaults
+
+    #The order is important.  If conda-forge is before usgs-astrogeology, you will need to run:
+
+    conda config --env --add channels usgs-astrogeology
+    
+    #Then set channel_priority to flexible in case there is a global channel_priority=strict setting
+    conda config --env --set channel_priority flexible
+    ```
+
+1. The environment is now ready to download ISIS and its dependencies:
+
+    ```bash
+    conda install -c usgs-astrogeology isis
+    ```
+
+1. Finally, setup the environment variables:
+
+    ISIS requires several environment variables to be set in order to run correctly.
+    The variables include: ISISROOT and ISISDATA.
+
+    The following steps are only valid for versions of ISIS after 4.2.0.
+    For older versions of ISIS follow the instructions in [this readme file.](https://github.com/USGS-Astrogeology/ISIS3/blob/adf52de0a04b087411d53f3fe1c9218b06dff92e/README.md)
+
+    There are two methods to configure the environment variables for ISIS:
+
+    A. Using `conda env config vars` *preferred*
+
+       Conda has a built in method for configuring environment variables that are specific to a conda environment since version 4.8.
+       This version number applies only to the conda package, not to the version of miniconda or anaconda that was installed.
+
+       To determine if your version of conda is recent enough run:
+
+           conda --version
+
+       If the version number is less than 4.8, update conda to a newer version by running:
+
+           conda update -n base conda
+
+       The version number should now be greater than 4.8.
+
+       To use the built in environment variable configuration feature, first activate the environment by first running:
+
+           conda activate isis
+
+       After activation, the environment variables can be set using the syntax: `conda config vars set KEY=VALUE`.
+       To set all the environment variables ISIS requires, run the following command, updating the path to `ISISDATA` as needed:
+
+           conda env config vars set ISISROOT=$CONDA_PREFIX ISISDATA=[path to data directory]
+
+       To make these changes take effect, re-activate the isis environment by running:
+
+           conda activate isis
+
+       The environment variables are now set and ISIS is ready for use every time the isis environment is activated.
+
+    !!! Warning "The above method will not enable tab completion for arguments in C-Shell."
+-----
+
+    B. Using the provided isisVarInit.py script:
+
+       To use the default values for: `$ISISROOT` and `$ISISDATA`, run the ISIS variable initialization script with default arguments:
+
+           python $CONDA_PREFIX/scripts/isisVarInit.py
+
+       Executing this script with no arguments will result in $ISISROOT=$CONDA\_PREFIX and $ISISDATA=$CONDA\_PREFIX/data. The user can specify different directories for `$ISISDATA` using the optional value:
+
+           python $CONDA_PREFIX/scripts/isisVarInit.py --data-dir=[path to data directory]
+
+       Now every time the isis environment is activated, $ISISROOT and $ISISDATA will be set to the values passed to isisVarInit.py.
+       This does not happen retroactively, so re-activate the isis environment with one of the following commands:
+
+           for Anaconda 3.4 and up - conda activate isis
+           prior to Anaconda 3.4 - source activate isis
+

docs/how-to-guides/Environment Setup and Maintenance/SPICE Kernel Updates in ISIS.md

+SPICE kernels are required for the use of most ISIS applications. ISIS provides SPICE kernels for each mission
+in the ISISDATA area . For information on how to download the ISISDATA area, see: [Download Mission Data](https://github.com/DOI-USGS/ISIS3#mission-specific-data-downloads)
+
+ISIS provides ongoing automated updates of SPICE kernels for some of its supported missions by checking for new mission kernels, and if available, distributing them via the ISISDATA area available from the rsync server. These supported missions are listed in the table below, alongside their ISIS and Mission update cadences. ‘ISIS Update’ refers to how often ISIS looks for and then distributes new SPICE kernel updates and ‘Mission Update’ refers to how often the mission typically provides updated kernels. If the mission update cadence is unknown, please contact the mission for more information. To receive newly updated kernels, it's necessary to re-run the rsync command described in 
+[Download Mission Data](https://github.com/DOI-USGS/ISIS3#mission-specific-data-downloads).
+
+Missions that are not listed in the table below are provided with a static set of SPICE kernels that are not automatically updated. These are occasionally updated by hand. If you would like to request that the kernels provided with ISIS for one of these missions be updated, please submit a ticket at  [new issue](https://github.com/DOI-USGS/ISIS3/issues/new/choose).
+
+| Mission     | ISIS Update Cadence | Mission Update Cadence |
+| ----------- | ------------------- | ----------------------
+| Cassini     |  daily              | No kernel updates are planned at this time |
+| CaSSIS      |  daily              | 1-3 times a week |
+| Dawn        |  every 12 hours     | Planned update for Vesta SPC shape model and corresponding SPICE kernels in early 2022. |
+| Hayabusa2   |  daily              | No kernel updates are planned at this time |
+| Juno        |  daily              | Unknown |
+| LRO         |  daily              | daily   |
+| MEX         |  once a month       | V2 release date TBD. |
+| MRO         |  daily              | CKs daily/weekly; SPKs semi-weekly; SCLKs, FK and IK updates, etc. - as needed |
+| Odyssey     |  daily              | CKs – daily; Merged CKs – quarterly (month after quarter start); SPKs – weekly on Thursdays; Merged SPKs - quarterly (month after quarter start); SCLKs - monthly |
+
+If you would like to receive notification from NAIF when kernels are publicly released, please sign up with [SPICE Announce](https://naif.jpl.nasa.gov/mailman/listinfo/spice_announce). You will also receive announcements from NAIF regarding new software, bug fixes, training opportunities and similar items. 
\ No newline at end of file

mkdocs.yml

@@ -42,6 +42,10 @@ nav:
       - Generating an ISD, creating a CSM model, and converting coordinates: getting-started/CSM Stack/ImageToGroundTutorial.ipynb
   - How-To Guides: 
     - Home: how-to-guides/index.md
+    - Environment Setup and Maintenance:
+      - Installing ISIS Via Anaconda: how-to-guides/Environment Setup and Maintenance/Installing ISIS Via Anaconda.md
+      - Anaconda in a Network Environment: how-to-guides/Environment Setup and Maintenance/Anaconda in a Network Environment.md
+      - SPICE Kernel Updates in ISIS: how-to-guides/Environment Setup and Maintenance/SPICE Kernel Updates in ISIS.md
     - Image Processing:
       - Map Projecting Images: how-to-guides/Image Processing/Map Projecting Images.md
     - Software Management: