Merge branch 'test_binary_consistency' into 'master'

Test binary consistency

See merge request giacomo.mulas/np_tmcode!20

.gitlab-ci.yml

@@ -102,7 +102,7 @@ running_stage:
 testing_stage:
    stage: test
    tags: ["np-tmcode"]
-   allow_failure: true
+   allow_failure: false
    needs:
       - job: building_stage
       - job: running_stage
@@ -127,8 +127,15 @@ testing_stage:
       - export FFILE=../../test_data/sphere/OSPH
       - echo "Comparing output of SPHERE"
       - python3 ../../src/scripts/pycompare.py --ffile=$FFILE --cfile=c_OSPH --html
+      - echo "Checking consistency among legacy and HDF5 configuration files"
+      - ../testing/test_TEDF ../../test_data/sphere/DEDFB c_TEDF c_TEDF.hd5
       - cd ../cluster
       - echo "Comparing output of CLUSTER"
       - export FFILE=../../test_data/cluster/OCLU
       - python3 ../../src/scripts/pycompare.py --ffile=$FFILE --cfile=c_OCLU --html
-      
+      - echo "Testing cluster with 24 spheres"
+      - ./np_cluster ../../test_data/cluster/DEDFB_24 ../../test_data/cluster/DCLU_24 .
+      - echo "Checking consistency among legacy and HDF5 configuration files"
+      - ../testing/test_TEDF ../../test_data/cluster/DEDFB_24 c_TEDF c_TEDF.hd5
+      - echo "Checking consistency among legacy and HDF5 TM files"
+      - ../testing/test_TTMS c_TTMS c_TTMS.hd5

build/README.md

@@ -75,3 +75,23 @@ The execution of trapping programs requires at least one of the previous program
    > ./np_trapping ../../test_data/trapping/DFRFME ../../test_data/trapping/DLFFT .
 
 5. Check the consistency between `np_trapping` output files (`c_out66.txt` and `c_out67.txt`) and the legacy *FORTRAN* output for this case (the files named, respectively, `fort.66` and `fort.67` in the `test_data/trapping/` folder). Consider that some of the output values will be affected by numeric noise and take substantially different values. However, this is expected for results whose order of magnitude is clearly below the precision level of the calculation, as they represent results appraching zero that were just approximated with different precision.
+
+### testing
+
+The `testing` folder contains programs that are used to test the consistency of data written in different binary formats. At present, the only binary files that can be read and written both in legacy and _HDF5_ formay are the configuration files (named `TEDF`) and the transition matrix files (named `TTMS`). The _HDF5_ versions of these files (marked with an `hd5` extension) can be inspected by standard _HDF5_ tools, such as `h5dump`. To check the consistency of these two type of files, the `testing` suite provides two programs:
+
+- `test_TEDF`
+
+This program checks for the consistency of the configuration data loaded from a formatted configuration file, a legacy binary file and an _HDF5_ binary file. It must be executed as:
+
+> ./test_TEDF PATH_TO_EDFB PATH_TO_c_TEDF PATH_TO_c_TEDF.hd5
+
+where the command line arguments must be valid paths to, respectively, the formatted configuration file, a binary configuration file produced by one of the _NP_ codes presented above (generally named `c_TEDF`) and a binary configuration file saved in _HDF5_ format (`c_TEDF.hd5`). The program checks for the data in each of the above formats and, after writing a short log to the terminal, returns a result value of 0, in case of success, or a positive error code in case of inconsistent data.
+
+- `test_TTMS`
+
+This program checks for the consistency of transition matrix data files. It is executed similarly to the previous one, but with just two arguments:
+
+> ./test_TTMS PATH_TO_c_TTMS PATH_TO_c_TTMS.hd5
+
+where the arguments must be valid paths to binary transition matrix files saved in the legaci or the _HDF5_ format. Similarly to the previous case, the program writes a short log and returns a result value of 0, for success, or a positive number in case of inconsistency detection.

src/README.md

@@ -4,7 +4,7 @@ This directory collects the source code of the original programs and the develop
 
 ## Instructions
 
-The original code is contained in the folders named `cluster`, `sphere` and `trapping`. Each folder contains a `Makefile` to compile either the whole program set or the single programs. A global `Makefile`, which contains instructions to build all the original source code, is available directly in the `src` folder.
+The original code is contained in the folders named `cluster`, `sphere` and `trapping`. The `testing` folder contains additional programs that are used to perform consistency tests during the development state. Each folder contains a `Makefile` to compile either the whole program set or the single programs. A global `Makefile`, which contains instructions to build all the original source code, is available directly in the `src` folder.
 
 In all cases, build commands executed through `make` will output the object files and the linked binaries in the proper folders under the build directory.
 

src/include/TransitionMatrix.h

@@ -164,9 +164,9 @@ class TransitionMatrix {
   
   /*! \brief Test whether two instances of TransitionMatrix are equal.
    *
-   * Transition matrixes can be the result of a calculation or of a file input operation,
+   * Transition matrices can be the result of a calculation or of a file input operation,
    * reading from a previously computed object. The `==` operator tests for the equivalence
-   * of two transition matrixes, returning `true` if they are equivalnet, `false` otherwise.
+   * of two transition matrices, returning `true` if they are equivalnet, `false` otherwise.
    *
    * \param other: `TransitionMatrix &` Reference to the instance to be compared with.
    * \return result: `bool` True, if the two instances are equal, false otherwise.

src/libnptm/Configuration.cpp

@@ -475,9 +475,10 @@ ScattererConfiguration* ScattererConfiguration::from_dedfb(string dedfb_file_nam
 
   complex<double> ***dc0m = new complex<double>**[configurations];
   complex<double> *dc0 = new complex<double>[configurations]();
+  int dim3 = (_idfc == 0) ? nxi : 1;
   for (int di = 0; di < configurations; di++) {
     dc0m[di] = new complex<double>*[nsph];
-    for (int dj = 0; dj < nsph; dj++) dc0m[di][dj] = new complex<double>[nxi]();
+    for (int dj = 0; dj < nsph; dj++) dc0m[di][dj] = new complex<double>[dim3]();
   } // di loop
   for (int jxi468 = 1; jxi468 <= nxi; jxi468++) {
     if (_idfc != 0 && jxi468 > 1) continue; // jxi468 loop
@@ -574,7 +575,7 @@ ScattererConfiguration* ScattererConfiguration::from_hdf5(string file_name) {
       str_type = "FLOAT64_(" + to_string(nsh) + ")";
       status = hdf_file->read(str_name, str_type, (rcf_vector[i115 - 1]));
     }
-    xi_vec = new double[nxi];
+    xi_vec = new double[nxi]();
     str_name = "XIVEC";
     str_type = "FLOAT64_(" + to_string(nxi) + ")";
     status = hdf_file->read(str_name, str_type, xi_vec);
@@ -586,13 +587,15 @@ ScattererConfiguration* ScattererConfiguration::from_hdf5(string file_name) {
     str_type = "FLOAT64_(" + to_string(element_size) + ")";
     status = hdf_file->read(str_name, str_type, elements);
     dc0m = new complex<double>**[configuration_count];
+    int dc_index = 0;
     for (int di = 0; di < configuration_count; di++) {
       dc0m[di] = new complex<double>*[nsph];
       for (int dj = 0; dj < nsph; dj++) {
 	dc0m[di][dj] = new complex<double>[dim3]();
 	for (int dk = 0; dk < dim3; dk++) {
-	  double rval = elements[(di * nsph) + (dj * dim3) + 2 * dk];
-	  double ival = elements[(di * nsph) + (dj * dim3) + 2 * dk + 1];
+	  double rval = elements[2 * dc_index];
+	  double ival = elements[2 * dc_index + 1];
+	  dc_index++;
 	  dc0m[di][dj][dk] = complex<double>(rval, ival);
 	}
       } // dj loop
@@ -665,9 +668,10 @@ ScattererConfiguration* ScattererConfiguration::from_legacy(string file_name) {
       input.read(reinterpret_cast<char *>(&(_rcf_vec[i115 - 1][nsi])), sizeof(double));
   }
   _dc0m = new complex<double>**[configurations];
+  int dim3 = (_idfc == 0) ? _nxi : 1;
   for (int di = 0; di < configurations; di++) {
     _dc0m[di] = new complex<double>*[_nsph];
-    for (int dj = 0; dj < _nsph; dj++) _dc0m[di][dj] = new complex<double>[_nxi]();
+    for (int dj = 0; dj < _nsph; dj++) _dc0m[di][dj] = new complex<double>[dim3]();
   } // di loop
   for (int jxi468 = 1; jxi468 <= _nxi; jxi468++) {
     if (_idfc != 0 && jxi468 > 1) continue;
@@ -822,7 +826,7 @@ void ScattererConfiguration::write_hdf5(string file_name) {
 
   int dim3 = (idfc == 0) ? number_of_scales : 1;
   int dc0m_size = 2 * dim3 * number_of_spheres * configurations;
-  double *dc0m = new double[dc0m_size];
+  double *dc0m = new double[dc0m_size]();
   int dc0_index = 0;
   for (int jxi468 = 1; jxi468 <= number_of_scales; jxi468++) {
     if (idfc != 0 && jxi468 > 1) continue;
@@ -835,8 +839,9 @@ void ScattererConfiguration::write_hdf5(string file_name) {
 	double dc0_real, dc0_imag;
 	dc0_real = dc0_matrix[i157][i162 - 1][jxi468 - 1].real();
 	dc0_imag = dc0_matrix[i157][i162 - 1][jxi468 - 1].imag();
-	dc0m[dc0_index++] = dc0_real;
-	dc0m[dc0_index++] = dc0_imag;
+	dc0m[2 * dc0_index] = dc0_real;
+	dc0m[2 * dc0_index + 1] = dc0_imag;
+	dc0_index++;
       }
     }
   }
@@ -1108,35 +1113,34 @@ bool ScattererConfiguration::operator ==(ScattererConfiguration &other) {
     if (scale_vec[svi] != other.scale_vec[svi]) {
       return false;
     }
-  int dj_index = 0;
-  for (int dj = 0; dj < number_of_spheres; dj++) {
-    bool check_matrixes = false;
-    if (iog_vec[dj] >= dj + 1) {
-      dj_index = iog_vec[dj] - 1;
-      check_matrixes = true;
-    }
-    if (iog_vec[dj] != other.iog_vec[dj]) {
+  int configurations = 0;
+  int dim3 = (idfc == 0) ? number_of_scales : 1;
+  for (int ci = 1; ci <= number_of_spheres; ci++) {
+    if (iog_vec[ci - 1] != other.iog_vec[ci - 1]) {
       return false;
     }
-    if (radii_of_spheres[dj_index] != other.radii_of_spheres[dj_index]) {
+    if (iog_vec[ci - 1] < ci) continue;
+    configurations++;
+  }
+  for (int ri = 0; ri < configurations; ri++) {
+    if (radii_of_spheres[ri] != other.radii_of_spheres[ri]) {
       return false;
     }
-    int layers = nshl_vec[dj_index];
-    if (layers != other.nshl_vec[dj_index]) {
+    if (nshl_vec[ri] != other.nshl_vec[ri]) {
       return false;
     }
-    if (check_matrixes) {
-      for (int di = 0; di < layers; di++) {
-	if (rcf[dj_index][di] != other.rcf[dj_index][di]) {
+    for (int rj = 0; rj < nshl_vec[ri]; rj++) {
+      if (rcf[ri][rj] != other.rcf[ri][rj]) {
+	return false;
+      }
+    } // rj loop
+    for (int dj = 0; dj < number_of_spheres; dj++) {
+      for (int dk = 0; dk < dim3; dk++) {
+	if (dc0_matrix[ri][dj][dk] != other.dc0_matrix[ri][dj][dk]) {
 	  return false;
 	}
-	for (int dk = 0; dk < number_of_scales; dk++) {
-	  if (dc0_matrix[di][dj_index][dk] != other.dc0_matrix[di][dj_index][dk]) {
-	    return false;
-	  }
-	} // dk loop
-      } // di loop
-    }
-  } // dj loop
+      } // dk loop
+    } // dj loop
+  } // ri loop
   return true;
 }

src/make.inc

@@ -40,7 +40,7 @@ endif
 
 # CXXFLAGS defines the default compilation options for the C++ compiler
 ifndef CXXFLAGS
-#override CXXFLAGS=-O3 -ggdb -pg -coverage -I$(HDF5_INCLUDE)
+#override CXXFLAGS=-O0 -ggdb -pg -coverage -I$(HDF5_INCLUDE)
 override CXXFLAGS=-O3 -I$(HDF5_INCLUDE)
 endif
 

src/testing/test_TEDF.cpp

@@ -59,5 +59,5 @@ int main(int argc, char **argv) {
     printf("Configuration objects c and b are different.\n");
     result += 100;
   }
-  return 0;
+  return result;
 }

src/testing/test_TTMS.cpp

@@ -35,9 +35,9 @@ int main(int argc, char **argv) {
   }
   a = TransitionMatrix::from_binary(legacy_file);
   b = TransitionMatrix::from_binary(hdf5_file, "HDF5");
-  if (*a == *b) printf("Transition matrixes a and b are equal.\n");
+  if (*a == *b) printf("Transition matrices a and b are equal.\n");
   else {
-    printf("Transition matrixes a and b are different.\n");
+    printf("Transition matrices a and b are different.\n");
     result = 1;
   }
   return result;