Skip to content
Snippets Groups Projects
Commit e79605b8 authored by Rodriguez, Kelvin's avatar Rodriguez, Kelvin
Browse files

Merge branch 'docs_test' into 'main'

Documentation Tests

See merge request astrogeology/asc-public-docs!8
parents 1c216579 98a9ead5
Branches
Tags
No related merge requests found
[codespell]
skip = *.po,*.ts,*.js,*.map,*.css,*.bsp,*.cub,*.tsc,*.tf,*.ti
......@@ -7,13 +7,12 @@
stages:
- build
- test
- deploy
"Build MkDocs":
image: "python:3"
stage: build
only:
- main
before_script:
- pip install -r requirements.txt
script:
......@@ -22,6 +21,37 @@ stages:
paths:
- site/
"Spell Check Docs":
image: "python:3"
stage: test
before_script:
- pip install codespell
script:
- codespell site/
"Link Check Docs":
image: "python:3"
stage: test
before_script:
- pip install linkchecker
script:
- linkchecker site/index.html --ignore-url asc-public-docs/site/docs/software/
"Grammar Check Docs":
image: "node:latest"
stage: test
before_script:
- npm i gramma -g
script:
- git diff --name-only origin/main | grep -e .md -e .MD | sed 's/^/"/;s/$/"/' | xargs -t -L1 gramma check -p || true
allow_failure:
exit_codes:
- 1
"Sync with S3":
image: "python:3"
stage: deploy
......
{
"api_url": "https://api.languagetool.org/v2/check",
"api_key": "",
"dictionary": [
],
"language": "en-US",
"rules": {
"casing": true,
"colloquialisms": true,
"compounding": true,
"confused_words": true,
"false_friends": true,
"gender_neutrality": true,
"grammar": true,
"misc": true,
"punctuation": true,
"redundancy": true,
"regionalisms": true,
"repetitions": true,
"semantics": true,
"style": false,
"typography": false,
"typos": false
}
}
\ No newline at end of file
......@@ -13,7 +13,7 @@ A new ISIS3 class needs to have the following Doxygen tags filled out just above
*/
class ImportMapTemplateWorkOrder : public WorkOrder {....
```
Sometimes, classes are declared inside the header files for other classes. This happens a lot in the $ISISROOT/src/qisis/objs directory where internal XmlHandler classes are defined to handle object serialization.
Sometimes, classes are declared inside the header files for other classes. This happens a lot in the $ISISROOT/src/qisis/objs directory, where internal XmlHandler classes are defined to handle object serialization.
These classes need to be documented as well (as in this example from the ImageList header file):
```C++
......
......@@ -36,13 +36,13 @@ This is a first step towards several places:
For the rest of this document, we will use `appname` as the name of the application that is being worked on. Simple example at https://github.com/DOI-USGS/ISIS3/tree/dev/isis/src/base/apps/crop
1. In the `appname` folder create two new files, `appname.cpp` and `appname.h`. These files are where the application logic will live.
1. In the `appname` folder, create two new files, `appname.cpp` and `appname.h`. These files are where the application logic will live.
1. In `appname.h` and `appname.cpp` create a new function in the `Isis` namespace with the following signature `void appname(UserInterface &ui)`. If the application has a `from` cube, add a second function with the input cube as the first argument `void appname(Cube incube, UserInterface &ui)`.
1. Copy the contents of the existing `IsisMain` function curretly located in `main.cpp` into the new `void appname(Cube incube, UserInterface &ui)` function. (if the app doesn't take any file parameters (i.e., network, text, cube list...) copy the contents into` void appname(UserInterface &ui)`). If there is no input cube, but there are other input files, add them as input parameters similar to how the input cube was done, see spiceinit and cnetcheck tests for examples).
1. Copy the contents of the existing `IsisMain` function currently located in `main.cpp` into the new `void appname(Cube incube, UserInterface &ui)` function. (if the app doesn't take any file parameters (i.e., network, text, cube list...) copy the contents into` void appname(UserInterface &ui)`). If there is no input cube, but there are other input files, add them as input parameters similar to how the input cube was done, see spiceinit and cnetcheck tests for examples.
1. Modify `void appname(UserInterface &ui)` to open the input cube, usually "from", and/or other files and call the second function `void appname(Cube incube, UserInterface &ui)`
1. Copy any helper functions or global variables from `main.cpp` into `appname.cpp`. So as to not pollute the `Isis` namespace and avoid redefining symbols
1. Copy any helper functions or global variables from `main.cpp` into `appname.cpp`. This is to not pollute the `Isis` namespace and avoid redefining symbols
1. Prototype any helper functions at the top of `appname.cpp` and define them at the bottom of `appname.cpp`. Do not define them in `appname.h`.
1. Put all of the required includes in `appname.cpp` and `appname.h`.
1. Put all the required includes in `appname.cpp` and `appname.h`.
1. Remove the call to get the UserInterface from `appname.cpp`; it usually looks like `UserInterface &ui = Application::GetUserInterface();`.
1. In `main.ccp`, add the following
......@@ -61,7 +61,7 @@ void IsisMain() {
```
??? Warning "If your application uses `Application::Log()`"
Due to how the Application singleton works, calling `Application::Log()` outside of an actual ISIS application currently causes a segmentation fault. To avoid this, modify the new `appname` function to return a Pvl that contains all of the PvlGroups that need to be logged instead of calling `Application::Log()`. Then, change your `main.cpp` to
Due to how the Application singleton works, calling `Application::Log()` outside an ISIS application currently causes a segmentation fault. To avoid this, modify the new `appname` function to return a Pvl that contains all the PvlGroups that need to be logged instead of calling `Application::Log()`. Then, change your `main.cpp` to
```C++
#include "Isis.h"
......@@ -97,7 +97,7 @@ The basic interface that we've created so far is simply a mirror of the applicat
### Separating parameter parsing
The first step is to separate the UserInterface parsing from the program logic.
All of the UserInterface parsing should be done in the `appname(UserInterface &ui)` function. Then, all of the actual program logic should be moved into another function also called `appname`. The signature for this function can be quite complex and intricate. Several tips for defining the `appname` function signature are in the next sections, but there is not perfect way to do this for every application.
All the UserInterface parsing should be done in the `appname(UserInterface &ui)` function. Then, all the program logic should be moved into another function also called `appname`. The signature for this function can be quite complex and intricate. Several tips for defining the `appname` function signature are in the next sections, but there is no perfect way to do this for every application.
Once the `appname` function is defined, the `appname(UserInterface &ui)` function simply needs to call it with the appropriate parameters once it has parsed the UserInterface.
......@@ -106,16 +106,16 @@ Once the `appname` function is defined, the `appname(UserInterface &ui)` functio
Most ISIS3 applications were designed to read their inputs from files and then output their results to the command line and/or files. Unfortunately, gtest is very poorly setup to test against files and the command line. To work around this, it is necessary to remove as much file and command line output from the new `appname` functions as possible. Here are some examples of how outputs can be separated from the application logic:
1. Anything that would be logged to the terminal should be simply returned via the log pointer. This way, it can be programmatically validated in gtest. This in fact already needs to be done because of [issues](#if-your-application-uses-applicationlog) with `Application::Log()`.
1. No input filenames should be passed as arguments. All files required by the program should be opened and converted into in-memory objects and then passed to the function. This will help eliminate the need for test data files in many applications. **Make sure that for Cubes the appropriate CubeAttributeInput and CubeAttributeOutput values are set!**
1. No input filenames should be passed as arguments. All files required by the program should be opened and converted into in-memory objects and then passed to the function. This will help eliminate the need for test data files in many applications. **Make sure that for cubes, the appropriate CubeAttributeInput and CubeAttributeOutput values are set!**
### Process, helper functions, and global variables
Many ISIS3 applications make sure of the Process class and its sub-classes. While these classes make file I/O easier, they also tightly couple it to the application logic! Some of them can take objects like cubes instead of filenames, but not all of them. They may need to be refactored to use objects instead of filenames. For now, where-ever possible use objects, but it is acceptable to use filenames if an object cannot be used.
Many ISIS3 applications make sure of the `Process` class and its subclasses. While these classes make file I/O easier, they also tightly couple it to the application logic! Some of them can take objects like cubes instead of filenames, but not all of them. They may need to be refactored to use objects instead of filenames. For now, where-ever possible use objects, but it is acceptable to use filenames if an object cannot be used.
Many of the Process sub-classes use process functions to operate on cubes. These helper functions will need to be pushed into the ISIS3 library. There is a chance that there will be symbol conflicts due to multiple applications using functions with the same name. In this case, the function can simply have its name modified. This is also a natural point at which application logic can be segmented and tested separately.
Many of the Process subclasses use process functions to operate on cubes. These helper functions will need to be pushed into the ISIS3 library. There is a chance that there will be symbol conflicts due to multiple applications using functions with the same name. In this case, the function can simply have its name modified. This is also a natural point at which application logic can be segmented and tested separately.
Because of how the Process sub-classes use process functions, many older ISIS3 applications use global variables to pass additional parameters to helper functions. This can cause serious issues because applications are no longer self-contained executables. To help with this, ProcessByBrick and ProcessByLine have been modified to use a [lambda function with captures](https://en.cppreference.com/w/cpp/language/lambda#Lambda_capture) as the process function. All of the previously global variables can now be defined in the `appname` function and then captured in the lambda function.
Because of how the Process subclasses use process functions, many older ISIS3 applications use global variables to pass additional parameters to helper functions. This can cause serious issues because applications are no longer self-contained executables. To help with this, ProcessByBrick and ProcessByLine have been modified to use a [lambda function with captures](https://en.cppreference.com/w/cpp/language/lambda#Lambda_capture) as the process function. All the previously global variables can now be defined in the `appname` function and then captured in the lambda function.
For example, here's an app called checkerboard which generates an artificial cube with a checkerboard pattern. It requires access to the variable `size` to work but `ProcessByLine.StartProcess` only supports functions `Buffer` objects as input. Therefore, the app uses a global variable in order for the checkerboard function to have `size` in it's scope:
For example, here's an app called checkerboard which generates an artificial cube with a checkerboard pattern. It requires access to the variable `size` to work, but `ProcessByLine.StartProcess` only supports functions `Buffer` objects as input. Therefore, the app uses a global variable in order for the checkerboard function to have `size` in its scope:
```C++
// original foo.cpp
#include "Isis.h"
......@@ -141,7 +141,7 @@ void isismain() {
}
```
can be refactored using a lambdas which captures the size variable:
can be refactored using a lambda which captures the size variable:
```C++
// callable foo.cpp
......@@ -180,8 +180,8 @@ void IsisMain() {
### For an Application
1. Create a new file `FunctionalTestAppname.cpp` in `isis/tests/`. For example, the tests the application `isisimport` should be in `FunctionalTestIsisImport.cpp`.
1. Add you tests to this file
1. Delete the tests and all of the test directories
1. Add your tests to this file
1. Delete the tests and all the test directories
Note: If your tests require the use of ISIS's Null value, you will need to include SpecialPixel.h
......@@ -237,7 +237,7 @@ catch(...) {
Be sure to choose a substring of the exception message that will uniquely identify it, but only use parts of the string identified in the actual code. Anything like `**USER ERROR**` or `in IsisAml.cpp at 1613` that are added based on user preference settings should not be included in the substring. The IException and Preferences classes test that these are properly added onto the error messages.
Normally, if the error message does not contain the substring, gtest will only output the the expression evaluated to false. To make the failure message more helpful, we add `<< e.toString().toStdString()` which outputs the full error message if the test fails.
Normally, if the error message does not contain the substring, gtest will only output the expression evaluated to false. To make the failure message more helpful, add `<< e.toString().toStdString()` which outputs the full error message if the test fails.
### Testing floating point numbers
Comparison between floating point numbers is a common problem in both writing and testing software. Carefully choosing test data can help avoid comparisons that are likely to cause problems. Whenever possible, use test data such that output values can be exactly stored in a double-precision floating point number (c++ double). For example, if a function takes its input and divides it by 116, a good test input would be 232 because 232/116 = 2 which can be stored exactly in a double. On the other hand, 1 would be a poor test input because 1/116 cannot be precisely stored in a double. If a comparison between numbers that cannot be stored precisely is required, gtest provides [Special Assertions](https://github.com/google/googletest/blob/main/docs/reference/assertions.md#floating-point) to make the comparisons more tolerant to different rounding.
......@@ -248,7 +248,7 @@ Because each test case is a completely fresh environment, there is often set-up
In order to conform to our test case naming conventions, all test fixtures need to be named `ClassName_FixtureName`. For example, a fixture that sets up a BundleSettings object in a not-default state for the BundleSettings unit test could be called `BundleSettings_NotDefault`.
### Test parameterization
If the same process needs to be run with several different inputs, it can be easily automated via [value-parameterized tests](https://github.com/google/googletest/blob/main/docs/advanced.md#value-parameterized-tests). In order to maintain test naming conventions, when you call the `INSTANTIATE_TEST_CASE_P` macro make sure the first parameter is. `ClassName`. For example
If the same process needs to be run with several different inputs, it can be easily automated via [value-parameterized tests](https://github.com/google/googletest/blob/main/docs/advanced.md#value-parameterized-tests). In order to maintain test naming conventions, when you call the `INSTANTIATE_TEST_CASE_P` macro, make sure the first parameter is. `ClassName`. For example
```
INSTANTIATE_TEST_CASE_P(
......@@ -257,7 +257,7 @@ INSTANTIATE_TEST_CASE_P(
::testing::Values(BundleSettings::Sigma0, BundleSettings::ParameterCorrections));
```
will ensure that the tests start with `BundleSettings`.
This will ensure that the tests start with `BundleSettings`.
......
......@@ -11,39 +11,39 @@ This document explains the release process for Official Releases, Release Candid
## Step 1: Check current false positive test failures
In this step, we check the currently failing tests. This is currently a judgement call on whether or not the tests are false-positives. This decision should be made by 2+ developers on the development team.
In this step, we check the currently failing tests. This is currently a judgement call on whether the tests are false-positives. This decision should be made by 2+ developers on the development team.
## Step 2: Update the Github documents
## Step 2: Update the GitHub documents
**This step is only required for release candidates and bug fix release. If you are creating a full feature release, skip this step.**
In this step we will update the documents that are stored in the Github repository. Our changes will be going into the dev branch so create a fresh local branch off of the dev branch.
In this step, we will update the documents that are stored in the GitHub repository. Our changes will be going into the dev branch, so create a fresh local branch off of the dev branch.
### Part A: Collecting the Changes in the Release
* Update the Changelog
* For release candidates we need to update the Changelog to label all of the currently unreleased changes as part of this release. Follow the instructions in [CHANGELOG.md](https://raw.githubusercontent.com/DOI-USGS/ISIS3/dev/CHANGELOG.md) for how to do this.
* For bug fix releases we need to update the Changelog to label **only the bug fixes** as part of this release. Follow the instructions in [CHANGELOG.md](https://raw.githubusercontent.com/DOI-USGS/ISIS3/dev/CHANGELOG.md) for how to do this.
* For release candidates we need to update the Changelog to label all the currently unreleased changes as part of this release. Follow the instructions in [CHANGELOG.md](https://raw.githubusercontent.com/DOI-USGS/ISIS3/dev/CHANGELOG.md) for how to do this.
* For bug fix releases, we need to update the Changelog to label **only the bug fixes** as part of this release. Follow the instructions in [CHANGELOG.md](https://raw.githubusercontent.com/DOI-USGS/ISIS3/dev/CHANGELOG.md) for how to do this.
* Update code.json with the new version number and the date last modified.
### Part B: Update the Authors List
* If there are any new contributors to the project since the last release ensure that they are added to the [.zenodo.json](https://github.com/DOI-USGS/ISIS3/blob/dev/.zenodo.json) document, and update the `AUTHORS.rst` file from the .zenodo.json file by running `python $ISISROOT/scripts/zenodo_to_authors.py <path_to_your_clone>/.zenodo.json <path_to_your_clone>/AUTHORS.rst`.
* If there are any new contributors to the project since the last release, ensure that they are added to the [.zenodo.json](https://github.com/DOI-USGS/ISIS3/blob/dev/.zenodo.json) document, and update the `AUTHORS.rst` file from the .zenodo.json file by running `python $ISISROOT/scripts/zenodo_to_authors.py <path_to_your_clone>/.zenodo.json <path_to_your_clone>/AUTHORS.rst`.
### Part C: Submit a Pull Request
* Submit a pull request into the dev branch with your modifications to the Changelog and the Authors list.
* For a bug fix release you will also need to [cherry-pick](https://www.atlassian.com/git/tutorials/cherry-pick) the squashed commit from your pull request into the version branch. If you run into merge conflicts it may be easier to simply redo the above steps with the version branch instead of dev.
* For a bug fix release, you will also need to [cherry-pick](https://www.atlassian.com/git/tutorials/cherry-pick) the squashed commit from your pull request into the version branch. If you run into merge conflicts, it may be easier to simply redo the above steps with the version branch instead of dev.
## Step 3: Set Up the Local and Remote Repositories
In this step, we will prepare the local repository to build from as well as update the remote repository hosted on GitHub. Keep in mind that you will be building from this repo on other systems and plan accordingly by cloning this repo into a directory that you will still have access to as you switch between the machines.
In this step, we will prepare the local repository to build from, as well as update the remote repository hosted on GitHub. Keep in mind that you will be building from this repo on other systems, so plan accordingly by cloning this repo into a directory that you will still have access to as you switch between the machines.
### Part A: Setup Repository
......@@ -102,7 +102,7 @@ In this step, we will prepare the local repository to build from as well as upda
<!---
* To create a new branch, first prepare your local repo by pulling down the merged changes you made earlier (e.g. `git pull upstream dev`)
* Next, create a branch with the appropriate name for the release version in the format `<major version>.<minor verson>`, by running for example: `git branch -b 3.10`. After creation, this branch can be pushed directly to upstream. (`git push upstream 3.10`.)--->
* Next, create a branch with the appropriate name for the release version in the format `<major version>.<minor version>`, by running for example: `git branch -b 3.10`. After creation, this branch can be pushed directly to upstream. (`git push upstream 3.10`.)--->
......@@ -341,7 +341,7 @@ The following operating systems are supported for this release:
(Other Linux/macOS variants may be able to run this release, but are not officially supported.)
If you find a problem with this release, please create an issue on our [github issues page](https://github.com/DOI-USGS/ISIS3/issues/new/choose/)
If you find a problem with this release, please create an issue on our [GitHub issues page](https://github.com/DOI-USGS/ISIS3/issues/new/choose/)
```
......
# How-To Guides
......@@ -38,7 +38,7 @@ These docs use a simple system of defining software documentation in four catego
## Documentation System
We use these four categories to cover the range of potential docs while clarifying to authors and readers for what kind of documentation goes where.
We use these four categories to cover the range of potential docs, while clarifying to authors and readers for what kind of documentation goes where.
| | Getting Started | How-Tos | Concepts | Manuals |
|-----------------|-----------------------|--------------------------------|----------------------------|-------------------------------------------------------|
......@@ -47,7 +47,7 @@ We use these four categories to cover the range of potential docs while clarifyi
| **Has the Goal of** | Enabling Beginners | Showing How To Solve A Problem | Give Detailed Written Explanations | Give Dry Descriptions of Libraries, Services and Apps |
| **Example** | Jupyter notebook with toy data on how to ingest and project images | A breakdown of quirks working with Themis Images | Write-up on how ISIS defines projections | ISIS library Doxygen reference |
This website structure borrows from the [Divio documentation system](https://documentation.divio.com/), except adapted to more specifically adhere to how our software is already structured, renamed the categories to be more specific, and have more focus on the composition (or mode of writing) of the docs to reduce ambiguity. One can't expect any categorical structure for software documentation to be orthogonal to eachother. There will always exist some degree of overlap between categories as the quantity of documentation grows. Documentation in one category can easily have overlapping information with those in others. However, composition of the doc (e.g., jupyter notebook with an explicit starting and end point, short how-to guide with code examples but ambigious starting point and end point, written explanation without code, doxygen API reference) is less ambigious that whether or not your docs can be considered a guide or a tutorial.
This website structure borrows from the [Divio documentation system](https://documentation.divio.com/), except adapted to more specifically adhere to how our software is already structured, renamed the categories to be more specific, and have more focus on the composition (or mode of writing) of the docs to reduce ambiguity. One can't expect any categorical structure for software documentation to be orthogonal to each other. There will always exist some degree of overlap between categories as the quantity of documentation grows. Documentation in one category can easily have overlapping information with those in others. However, composition of the doc (e.g., Jupyter notebook with an explicit starting and end point, short how-to guide with code examples but ambiguous starting point and end point, written explanation without code, doxygen API reference) is less ambiguous than whether your docs can be considered a guide or a tutorial.
## Contributing New Docs and Submitting Issues
......@@ -55,7 +55,7 @@ Before you consider contributing new documentation, ask yourself what category i
Regarding software manuals, issues or contributions should be addressed to the repository for that specific project. The software manuals should have links to their repositories.
You can submit any issues (e.g., addressing grammar errors, factual inacuracies) and find more info on how to contribute new docs through the site's [git repository](https://code.usgs.gov/astrogeology/asc-public-docs).
You can submit any issues (e.g., addressing grammar errors, factual inaccuracies) and find more info on how to contribute new docs through the site's [git repository](https://code.usgs.gov/astrogeology/asc-public-docs).
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment