diff --git a/.github/workflows/build-docs.yml b/.github/workflows/build-docs.yml new file mode 100644 index 0000000000000000000000000000000000000000..45a0949b1ba2688e7d6d334e6efab51e3aeaaa53 --- /dev/null +++ b/.github/workflows/build-docs.yml @@ -0,0 +1,34 @@ +name: build-docs + +on: + pull_request: + branches: + - main + push: + branches: + - main + +jobs: + build-docs-job: + runs-on: ubuntu-latest + defaults: + run: + shell: bash -el {0} + steps: + - uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 + - uses: conda-incubator/setup-miniconda@3b0f2504dd76ef23b6d31f291f4913fb60ab5ff3 + with: + miniconda-version: "latest" + activate-environment: knoten-docs + environment-file: docs/environment.yml + auto-activate-base: false + auto-update-conda: true + + - name: Check doc build environment + run: | + conda list + + - name: Build Docs + run: | + cd docs + sphinx-build -b html . public \ No newline at end of file diff --git a/.github/workflows/deploy-docs-aws.yml b/.github/workflows/deploy-docs-aws.yml new file mode 100644 index 0000000000000000000000000000000000000000..9d928068df8010892f79d8b82eb7c52ee9a1bb81 --- /dev/null +++ b/.github/workflows/deploy-docs-aws.yml @@ -0,0 +1,46 @@ +name: deploy-docs-aws + +on: + workflow_run: + workflows: ["build-docs"] + branches: + - main + types: + - completed + +jobs: + deploy-docs-aws-job-test: + if: ${{ github.event.workflow_run.conclusion == 'success' }} + runs-on: ubuntu-latest + defaults: + run: + shell: bash -el {0} + steps: + - uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 + - uses: conda-incubator/setup-miniconda@3b0f2504dd76ef23b6d31f291f4913fb60ab5ff3 + with: + miniconda-version: "latest" + activate-environment: knoten-docs + environment-file: docs/environment.yml + auto-activate-base: false + auto-update-conda: true + + - name: Check doc build environment + run: | + conda list + + - name: Build Docs + run: | + cd docs + sphinx-build -b html . public + + - name: Set AWS credentials for upload + uses: aws-actions/configure-aws-credentials@0e613a0980cbf65ed5b322eb7a1e075d28913a83 + with: + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + aws-region: us-west-2 + + # - name: Upload to S3 + # working-directory: docs + # run: aws s3 sync --delete public s3://asc-public-docs/knoten \ No newline at end of file diff --git a/.gitignore b/.gitignore index 9b71e9e8aeb534451f26db3b32cbc70157ba6edb..01e97e591f0d4ad63503e72196f076aebfc67e38 100644 --- a/.gitignore +++ b/.gitignore @@ -115,3 +115,5 @@ venv.bak/ print.prt *.cub *.CUB + +.DS_Store \ No newline at end of file diff --git a/README.md b/README.md index cd7b981c3edc626a1be1204354b23c30fb99b8b2..8d7e466f668d16af2bb364a8449659f632a541a5 100644 --- a/README.md +++ b/README.md @@ -24,23 +24,39 @@ Please see the **status report** below for the current instruments we have imple ## Installing -You can install the latest build via conda +You can install the latest build via conda. ``` conda install -c usgs-astrogeology -c conda-forge knoten ``` -You can also do a local install using the following steps within a clone of the repository +You can also do a local install using the following steps within a clone of the repository. -1. Install the dependencies -``` -conda env create -f environment.yml -``` +1. Install the dependencies. + Note: creating the environment may take around and hour. If conda is too slow, try using mamba instead. + ``` + conda env create -f environment.yml + ``` -2. Install the package -``` -python setup.py install -``` +1. Install the package + ``` + python setup.py install + ``` + +<hr> + +## Building Docs + +To build the docs: +1. Open the docs directory: `cd docs` +1. Create (if not existing) and activate the environment (or use the main knoten environment if it is already created). + + ```sh + conda env create -f environment.yml #If it does not exist already + conda activate knoten-docs + ``` +1. Run `sphinx-build -b html . public` to build the docs in the `docs/public` directory (or change "public" to a subdirectory name of your choosing). +1. Browse to the `index.html` file in `docs/public` to open the docs. If needed, run a utility like `http-server` in the `docs/public` directory to serve the docs on localhost. <hr> diff --git a/docs/_templates/layout.html b/docs/_templates/layout.html new file mode 100644 index 0000000000000000000000000000000000000000..c4124d18f74c3323627d9a48b326b360f598e1d8 --- /dev/null +++ b/docs/_templates/layout.html @@ -0,0 +1,11 @@ +{% extends '!layout.html' %} + +{% block footer %} + <!-- {{ super() }} --> + {% include "usgsfooter.html" %} +{% endblock %} + +{% block header %} + {% include "usgsheader.html" %} + {{ super() }} +{% endblock %} \ No newline at end of file diff --git a/docs/_templates/usgsfooter.html b/docs/_templates/usgsfooter.html new file mode 100644 index 0000000000000000000000000000000000000000..ab6c4c9948475bc95fef52612bfa7869217c177f --- /dev/null +++ b/docs/_templates/usgsfooter.html @@ -0,0 +1,70 @@ + <footer class="footer"> + <div class="tmp-container"> + + + <div class="footer-doi"> + + <ul class="menu nav"> + <li class="first leaf menu-links menu-level-1"><a href="https://www.doi.gov/privacy">DOI Privacy Policy</a></li> + <li class="leaf menu-links menu-level-1"><a href="https://www.usgs.gov/policies-and-notices">Legal</a></li> + <li class="leaf menu-links menu-level-1"><a href="https://www.usgs.gov/accessibility-and-us-geological-survey">Accessibility</a></li> + <li class="leaf menu-links menu-level-1"><a href="https://www.usgs.gov/sitemap">Site Map</a></li> + <li class="last leaf menu-links menu-level-1"><a href="https://answers.usgs.gov/">Contact USGS</a></li> + </ul> + + </div> + <hr> + + + <div class="footer-doi"> + <ul class="menu nav"> + <li class="first leaf menu-links menu-level-1"><a href="https://www.doi.gov/">U.S. Department of the Interior</a></li> + <li class="leaf menu-links menu-level-1"><a href="https://www.doioig.gov/">DOI Inspector General</a></li> + <li class="leaf menu-links menu-level-1"><a href="https://www.whitehouse.gov/">White House</a></li> + <li class="leaf menu-links menu-level-1"><a href="https://www.whitehouse.gov/omb/management/egov/">E-gov</a></li> + <li class="leaf menu-links menu-level-1"><a href="https://www.doi.gov/pmb/eeo/no-fear-act">No Fear Act</a></li> + <li class="last leaf menu-links menu-level-1"><a href="https://www.usgs.gov/about/organization/science-support/foia">FOIA</a></li> + </ul> + </div> + + + <div class="footer-social-links"> + <ul class="social"> + <li class="follow">Follow</li> + <li class="twitter"> + <a href="https://twitter.com/usgs" target="_blank"> + <i class="fa fa-twitter-square"><span class="only">Twitter</span></i> + </a> + </li> + <li class="facebook"> + <a href="https://facebook.com/usgeologicalsurvey" target="_blank"> + <i class="fa fa-facebook-square"><span class="only">Facebook</span></i> + </a> + </li> + <li class="github"> + <a href="https://github.com/usgs" target="_blank"> + <i class="fa fa-github"><span class="only">GitHub</span></i> + </a> + </li> + <li class="flickr"> + <a href="https://flickr.com/usgeologicalsurvey" target="_blank"> + <i class="fa fa-flickr"><span class="only">Flickr</span></i> + </a> + </li> + <li class="youtube"> + <a href="http://youtube.com/usgs" target="_blank"> + <i class="fa fa-youtube-play"><span class="only">YouTube</span></i> + </a> + </li> + <li class="instagram"> + <a href="https://instagram.com/usgs" target="_blank"> + <i class="fa fa-instagram"><span class="only">Instagram</span></i> + </a> + </li> + </ul> + </div> + + </div> + + </footer> + \ No newline at end of file diff --git a/docs/_templates/usgsheader.html b/docs/_templates/usgsheader.html new file mode 100644 index 0000000000000000000000000000000000000000..04d6d27ffd15354aebbecfab0364bdf59fdeff61 --- /dev/null +++ b/docs/_templates/usgsheader.html @@ -0,0 +1,132 @@ +<!DOCTYPE html> +<html> + <head> + <meta charset="utf-8" /> + <meta http-equiv="X-UA-Compatible" content="IE=edge" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + <title>Knoten</title> + <script async type="text/javascript" src="https://dap.digitalgov.gov/Universal-Federated-Analytics-Min.js?agency=DOI" id="_fed_an_ua_tag"></script> + <script type="text/javascript" src="https://asc-public-docs.s3.us-west-2.amazonaws.com/common/uswds/3.6.0/js/uswds-init.min.js"></script> + <script type="text/javascript" src="https://asc-public-docs.s3.us-west-2.amazonaws.com/common/uswds/3.6.0/js/uswds.min.js"></script> + <link href="https://asc-public-docs.s3.us-west-2.amazonaws.com/common/uswds/3.6.0/css/common.css" rel="stylesheet" type="text/css" media="screen" /> + <link href="https://asc-public-docs.s3.us-west-2.amazonaws.com/common/uswds/3.6.0/css/custom.css" rel="stylesheet" type="text/css" media="screen" /> + <link href="https://asc-public-docs.s3.us-west-2.amazonaws.com/common/uswds/3.6.0/css/uswds.min.css" rel="stylesheet" type="text/css" media="screen" /> + </head> +</html> + + +<section class="usa-banner" aria-label="Official government website"> + <div class="usa-accordion"> + <header class="usa-banner__header"> + <div class="usa-banner__inner"> + <div class="grid-col-auto"> + <img + class="usa-banner__header-flag" + src="https://stac.astrogeology.usgs.gov/docs/img/us_flag_small.png" + alt="U.S. flag" + /> + </div> + <div class="grid-col-fill tablet:grid-col-auto"> + <p class="usa-banner__header-text"> + An official website of the United States government + </p> + <p class="usa-banner__header-action" aria-hidden="true"> + Here’s how you know + </p> + </div> + <button + type="button" + class="usa-accordion__button usa-banner__button" + aria-expanded="false" + aria-controls="gov-banner-default-default" + > + <span class="usa-banner__button-text">Here’s how you know</span> + </button> + </div> + </header> + <div + class="usa-banner__content usa-accordion__content" + id="gov-banner-default-default" + > + <div class="grid-row grid-gap-lg"> + <div class="usa-banner__guidance tablet:grid-col-6"> + <img + class="usa-banner__icon usa-media-block__img" + src="https://stac.astrogeology.usgs.gov/docs/img/icon-dot-gov.svg" + role="img" + alt="" + aria-hidden="true" + /> + <div class="usa-media-block__body"> + <p> + <strong>Official websites use .gov</strong><br />A + <strong>.gov</strong> website belongs to an official government + organization in the United States. + </p> + </div> + </div> + <div class="usa-banner__guidance tablet:grid-col-6"> + <img + class="usa-banner__icon usa-media-block__img" + src="https://asc-public-docs.s3.us-west-2.amazonaws.com/common/uswds/3.6.0/img/icon-https.svg" + role="img" + alt="" + aria-hidden="true" + /> + <div class="usa-media-block__body"> + <p> + <strong>Secure .gov websites use HTTPS</strong><br />A + <strong>lock</strong> ( + <span class="icon-lock" + ><svg + xmlns="http://www.w3.org/2000/svg" + width="52" + height="64" + viewBox="0 0 52 64" + class="usa-banner__lock-image" + role="img" + aria-labelledby="banner-lock-title-default banner-lock-description-default" + focusable="false" + > + <title id="banner-lock-title-default">Lock</title> + <desc id="banner-lock-description-default">A locked padlock</desc> + <path + fill="#000000" + fill-rule="evenodd" + d="M26 0c10.493 0 19 8.507 19 19v9h3a4 4 0 0 1 4 4v28a4 4 0 0 1-4 4H4a4 4 0 0 1-4-4V32a4 4 0 0 1 4-4h3v-9C7 8.507 15.507 0 26 0zm0 8c-5.979 0-10.843 4.77-10.996 10.712L15 19v9h22v-9c0-6.075-4.925-11-11-11z" + /> + </svg> </span + >) or <strong>https://</strong> means you’ve safely connected to + the .gov website. Share sensitive information only on official, + secure websites. + </p> + </div> + </div> + </div> + </div> + </div> + </section> + + + + + <header id="navbar" class="header-nav" role="banner"> + <div class="tmp-container"> + <div class="header-search"> + <a class="logo-header" href="https://www.usgs.gov/" title="Home"> + <img class="img" src="https://stac.astrogeology.usgs.gov/docs/img/logo.png" alt="Home" /> + </a> + <form action="https://www.usgs.gov/science-explorer-results" method="GET" id="search-box"> + <div class="fa-wrapper"><label for="se_search" class="only">Search</label> + <input id="se_search" type="search" name="es" placeholder="Search"> + <button class="fa fa-search" type="submit"> + <span class="only">Search</span> + </button></div> + </form> + </div> + + </div> + + </header> + + diff --git a/docs/conf.py b/docs/conf.py index 765bc896b3dad775637a1102de0955f74b3ed4e4..015b119dab8534bb19def8989a93ba625f295f29 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -44,9 +44,46 @@ exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # -html_theme = 'alabaster' +html_theme = 'sphinx_material' # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ['_static'] \ No newline at end of file +# html_static_path = ['_static'] + +html_sidebars = { + "**": ["logo-text.html", "globaltoc.html", "localtoc.html", "searchbox.html"] +} + +# Material theme options (see theme.conf for more information) +html_theme_options = { + + # Set the name of the project to appear in the navigation. + 'nav_title': 'Knoten', + + # Set you GA account ID to enable tracking + # 'google_analytics_account': 'UA-XXXXX', + + # Specify a base_url used to generate sitemap.xml. If not + # specified, then no sitemap will be built. + # 'base_url': 'https://github.com/DOI-USGS/knoten', + + 'master_doc' : 'False', + 'heroes' : [], + # Set the color and the accent color + 'color_primary': '#000000', + # 'color_accent': 'Red', + 'theme_color' : '#3f51b5', + 'logo_icon': '', + + # Set the repo location to get a badge with stats + 'repo_url': 'https://github.com/DOI-USGS/knoten', + 'repo_name': 'Project', + + # Visible levels of the global TOC; -1 means unlimited + 'globaltoc_depth': 3, + # If False, expand all TOC entries + 'globaltoc_collapse': False, + # If True, show hidden TOC entries + 'globaltoc_includehidden': False, +} \ No newline at end of file diff --git a/docs/environment.yml b/docs/environment.yml index d26691495c2f54252cd3fdab0fb67e2c8c494561..262f0fb8d064501bb49c661995f802f6c7e199b3 100644 --- a/docs/environment.yml +++ b/docs/environment.yml @@ -1,8 +1,15 @@ +name: knoten-docs + channels: - conda-forge + - default + dependencies: - python>=3 - + - pip + - sphinx + - pip: + - sphinx-material diff --git a/docs/index.rst b/docs/index.rst index 667b2b730b2312d5f1a539d0f5c7eddc36e9210f..fabb6d13e88e71995f4759b6e34242e8bf60bba0 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,13 +1,75 @@ -Welcome to knoten's documentation! -================================== +Knoten +====== A library to leverage python wrapped Community Sensor Models (CSMs) for common spatial/sensor operations and testing. -This is a placeholder for knoten's documentation. -For more information, please see: -https://github.com/USGS-Astrogeology/knoten +References +---------- +- CSM (usgscsm): https://github.com/USGS-Astrogeology/usgscsm +- Abstraction Layer for Ephemerides (ALE): https://github.com/USGS-Astrogeology/ale +Overview +-------- +We currently use Knoten to help test our supported CSM implementations against well established `ISIS3 camera models <https://github.com/USGS-Astrogeology/ISIS3>`_. In short, The CSM standard, now at version 3.0.3, is a framework that provides a well-defined application program interface (API) for multiple types of sensors and has been widely adopted by remote sensing software systems (e.g. BAE's Socet GXP, Harris Corp.'s ENVI, Hexagon's ERDAS Imagine, and recently added to the NASA AMES Stereo Pipeline [ASP]). Our support for CSM is explained in this `abstract <https://www.hou.usra.edu/meetings/informatics2018/pdf/6040.pdf>`_ and a recently submitted paper (not yet available). Currently, we support **Framing** and **Pushbroom** (line scanner) types of sensor models in the `usgscsm <https://github.com/USGS-Astrogeology/usgscsm>`_ library. +A secondary requirement for our CSM implementation requires an ALE-generated Image Support Data (ISD). ISDs contain the `SPICE-derived <https://naif.jpl.nasa.gov/naif/toolkit.html>`_ positional (and when needed velocity) description for each image. You can find several generated JSON-formatted examples `here <https://github.com/DOI-USGS/knoten/blob/main/examples/data>`_. + +Please see the **status report** below for the current instruments we have implemented and how well they match our ISIS3 camera models. In the near future, we will continue to address the pixel offsets we currently see. Both the CSM implementations (usgscsm) and ALE are currently in active development and both will be updated as needed to decrease these errors. Thus, none of the instruments have been tested enough for full production use. + + +Installing +---------- + +You can install the latest build via conda:: + + conda install -c usgs-astrogeology -c conda-forge knoten + +You can also do a local install using the following steps within a clone of the repository. + +#. Install the dependencies. + Note: creating the environment may take around and hour. If conda is too slow, try using mamba instead:: + + conda env create -f environment.yml + +#. Install the package:: + + python setup.py install + +Building Docs +------------- + +To build the docs: + +#. Open the docs directory: ``cd docs`` +#. Create (if not existing) and activate the environment (or use the main knoten environment if it is already created):: + + conda env create -f environment.yml #If it does not exist already + conda activate knoten-docs + +#. Run ``sphinx-build -b html . public`` to build the docs in the ``docs/public`` directory (or change "public" to a subdirectory name of your choosing). +#. Browse to the ``index.html`` file in ``docs/public`` to open the docs. If needed, run a utility like ``http-server`` in the ``docs/public`` directory to serve the docs on localhost. + + +Status Report - November 2019 +----------------------------- + +For full testing reports and example usage, please see the linked example Jupyter notebooks in the table below. + + ======================= ============================================================= ====================================== =========================================== ============================================== + Instrument Jupyter Notebooks Production Ready Difference CSM -> ISIS (in pixels) Difference ISIS -> CSM (in pixels) + ======================= ============================================================= ====================================== =========================================== ============================================== + MRO HiRISE `MRO HiRISE <examples/mro_hirise_isis_cmp.ipynb>`_ sub-pixel; in testing for production sample mean=-2.0e-05; line mean=2.5e-08 sample mean=-3.0e-08; line mean=1.2e-04 + MRO CTX `MRO CTX <examples/mro_ctx_isis_cmp.ipynb>`_ nearly sub-pixel; still in research gross error in line sample mean=0.0002; line mean=-0.07 + MEX HRSC `MEX HRSC <examples/mex_hrsc_isis_cmp.ipynb>`_ sub-pixel; in testing for production sample mean=0.000038; line mean=-0.000072 sample mean=-0.000038 ; line mean=-7.512e-05 + LROC NAC `LROC NAC <examples/lrocnac_isis_cmp.ipynb>`_ sub-pixel; in testing for production sample mean=-0.003; line mean=-0.0006 sample mean=0.0005 line mean=0.003 + Kaguya Terrain Camera `Kaguya TC <examples/kaguya_tc_isis_cmp.ipynb>`_ barely sub-pixel; in testing sample mean=0.0001; line mean=0.00003 sample mean=0.009; line mean=-1.242 + Messenger MDIS NAC `MGR MDIS NAC <examples/messenger_mdisnac_isis_cmp.ipynb>`_ sub-pixel; in testing for production sample mean=-0.01; line mean=-0.003 sample mean=0.01; line mean=0.003 + Cassini ISS NAC `CAS ISS NAC <examples/cassini_isis_nac_cmp.ipynb>`_ sub-pixel; in testing for production sample mean=-0.001; line mean=0.01 sample mean=0.001; line mean=-0.01 + Cassini ISS WAC `CAS ISS WAC <examples/cassini_isis_wac_cmp.ipynb>`_ sub-pixel; in testing for production sample mean=0.001; line mean=0.004 sample mean=-0.001; line mean=-0.004 + Dawn Framing Camera `Dawn FC <examples/dawn_fc_isis_cmp.ipynb>`_ sub-pixel; in testing for production sample mean=-0.02; line mean=0.003 sample mean=0.02; line mean=-0.003 + ======================= ============================================================= ====================================== =========================================== ============================================== + +The Difference column (CSM -> ISIS) represents the mean difference in pixels from running usgscsm's *image2ground* and then back to the camera using ISIS3's *campt* (ground2image). The Difference column (ISIS -> CSM) is simply the reverse starting with ISIS3 first with *campt* (image2ground) and then usgscsm's *ground2image*. diff --git a/environment.yml b/environment.yml index 1e935a55dda62d5cd7183c096c450619b54df8c6..e1ff98c69de44e6afd9af1586a3496d6573d9ceb 100644 --- a/environment.yml +++ b/environment.yml @@ -4,7 +4,7 @@ channels: - usgs-astrogeology - plotly dependencies: - - ale>=0.3.2 + - ale>=0.8.8 - coveralls - csmapi>=1.0 - gdal>=3.0.0 @@ -13,6 +13,7 @@ dependencies: - jupyter - matplotlib - numpy + - pip - plio - plotly - plotly-orca @@ -25,4 +26,7 @@ dependencies: - requests - scipy - shapely + - sphinx - usgscsm>=1.3.1 + - pip: + - sphinx-material