diff --git a/README.md b/README.md index 3c54e81b81476f169fa402e63bb99c527863b44f..88aeecb51e4a6dce64e8fc5756474e0979182775 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,7 @@ * [Citing ISIS](README.md#Citing-ISIS) * [Start Contributing](https://github.com/USGS-Astrogeology/ISIS3/wiki/How-to-Start-Contributing) * [ISIS Data Area](README.md#The-ISIS-Data-Area) +* [ISIS Test Data](README.md#ISIS-Test-Data) * [Installing Older Versions of ISIS](README.md#Installing-older-versions-of-ISIS) @@ -619,6 +620,33 @@ For versions of ISIS prior to ISIS 4.1.0, please cd into `$ISIS3DATA` instead of rsync -azv --delete --partial isisdist.astrogeology.usgs.gov::isisdata/data/voyager1 . rsync -azv --delete --partial isisdist.astrogeology.usgs.gov::isisdata/data/voyager2 . +### ISIS Test Data +ISIS is comprised of two types of tests, custom Makefile based tests, and GTest based tests. Those that are GTest based, make economical use of data that exists on the ISIS3 repo along with the source, so no special data is required to run those other than the ISIS data area. The Makefile tests depend on a separate source of data that consists of a few gigabytes of input and expected output data used for testing ISIS applications. The Makefile based tests use the ISISTESTDATA environment variable to know where the required data are located. The total size of this test data decreases as we work towards converting Makefile tests to GTests. + +###How to download the ISIS test data with rclone +Test data is hosted using Amazon S3 storage buckets. We recommend using rclone to pull the data into a local directory. You can download rclone using their instructions (see: https://rclone.org/downloads/) or by using an anaconda environment (see: https://docs.anaconda.com/anaconda/install/). If you already have an anaconda environment up, install rclone with: conda install –c conda-forge rclone + +Next, you will want to configure rclone using a default S3 configuration. See: https://rclone.org/s3/ for detailed information on how to configure S3, but for the purposes of downloading the ISIS3 test data, you simply run rclone config which will start an interactive menu. Press enter through it all except for these details: + +1. Set S3 as both your storage type and storage provider +2. Set us-west-2 as both your region to connect to and as the location constraint. +3. Everything else, just leave as the default. + +Example output: https://gist.github.com/Kelvinrr/706bbd54b1c2c30d0ce3d12f7dcaa10a + +Once rclone is configured, simply run: `rclone sync remote:asc-isisdata/isis_testData/ $ISISTESTDATA` +where: + - $ISISTESTDATA is the environment variable defining the location of the ISISTESTDATA + - remote: is the name of the configuration you created earlier. This can be whatever you want to name it, in this case it is named remote. + - asc-isisdata/isis_testData/ is the name of the S3 bucket you’re downloading from + +$ISISTESTDATA should now contain a full clone of the ISIS test data for running Makefile based tests. + +Notes: + - Users can download specific files from the bucket by adding path data or file information to the first argument, that is, to download only the ‘base’ folder from the isis_testData bucket, the user could call: +rclone sync remote:asc-isisdata/isis_testData/base + - It is important that users understand the difference in rclone’s ‘sync’ and ‘copy’ methods. ‘copy’ will overwrite all data in the destination with data from source. ‘sync’ replaces only changed data. + - Syncing / copying in either direction (local -> remote or remote -> local) results in any changed data being overwritten. There is no warning message on overwrite. ## Installing older versions of ISIS ---------------------------------