# SAK documentation

This file contains the overview of the SAK tool for inferring volume densities.

## Repositories

The project remote repository for the code is:
https://www.ict.inaf.it/gitlab/andrea.giannetti/swiss_army_knife_stable/-/tree/main


## Pipeline

The SAK non-LTE, toy model pipeline uses three main layers:

1. **Staging layer:** In the staging layer (`etl/stg`), the [stg_radmc_input_generator.py](../etl/stg/stg_radmc_input_generator.py) file takes care of preparing
   the input files for RADMC, and saves them in the `etl/mdl/radmc_files` folder.

   The [etl/stg/config/config.yml](../etl/stg/config/config.yml) file contains the default values of the parameters used to prepare the RADMC files
   for the postprocessing. All of them are described in detail in the following.

2. **Model layer:** The model layer (`etl/mdl`) takes care of preparing and executing the RADMC command according to the
   configuration in the [etl/mdl/config/config.yml](../etl/mdl/config/config.yml) file. This is done by the [mdl_execute_radmc_command.py](../etl/mdl/mdl_execute_radmc_command.py) script,
   that also creates `radmc3d_postprocessing.sh`.

   The results are then converted to fits cubes, which are saved by default into `prs/fits/cubes`, for later
   processing.

3. **Presentation layer:** In the presentation layer (`etl/prs`) moment-0 maps and line-ratio maps are computed
   executing the [prs_compute_integrated_fluxes_and_ratios.py](../etl/prs/prs_compute_integrated_fluxes_and_ratios.py) script. At the moment, the integration limits cannot be
   specified, and the entire cube is collapsed. *WARNING:* Pay attention to the presence of secondary modeled lines in
   the simulated spectra.

   The script [prs_inspect_results.py](../etl/prs/prs_inspect_results.py) reduces the ratio maps to single points and produces an image of the ratio values
   as a function of gas number density and temperature. Be aware that at the moment the query is hardcoded and works for
   SAK-generated models only.

   There is a final script [prs_density_inference.py](../etl/prs/prs_density_inference.py), that is used for preparing the KDE model and to perform density
   inference, given the measured ratios. It uses the YAML file [etl/config/density_inference_input.yml](../etl/config/density_inference_input.yml) to provide the
   needed input for the script. It produces the `output/run_type/{provided_run_type}/density_pdf*.png` output file,
   where `provided_run_type` is defined in the global configuration file and the wildcard represents the
   source name.
   
   The scripts [prs_ml_training.py](../etl/prs/prs_ml_training.py), [prs_ml_training_ratio.py](../etl/prs/prs_ml_training_ratio.py) and [prs_expand_modelling_with_ml.py](../etl/prs/prs_expand_modelling_with_ml.py) [prs_expand_ratio_modelling_with_ml.py](../etl/prs/prs_expand_ratio_modelling_with_ml.py) can be run before the inference to perform ML-assisted emulation of
   the modelled data, in
   order to expand the grid performed with actual RT computation. These scripts rely on
   the [etl/config/ml_modelling.yml](../etl/config/ml_modelling.yml) to perform training and evaluation of the emulation model, and to actually produce
   emulated data, which are saved to the `etl/inferred_data.csv`. This file is used by the [prs_density_inference.py](../etl/prs/prs_density_inference.py) to
   concatenate these data to those from the formal computation to perform the density inference. In our
   case, XGBoost worked best, and we used this model to perform emulation.

The entire ETL pipeline is executed by the [main.py](../etl/main.py) script, where it is possible to define overrides for the default
values in the specific stage configuration file (so that it's possible to specify an entire grid of models). These
overrides are included into the [etl/config/config.yml](../etl/config/config.yml) configuration file.

4. **Additional files**:
The script [prs_analytical_representations.py](../etl/prs/prs_analytical_representations.py) provides a convenient way of checking the analytical representations of the ratio vs. density curves.
The file [prs_check_biases_poc_sample.py](../etl/prs/prs_check_biases_poc_sample.py) checks for biases in the massive clump sample used in the proof-of-concept.
The scripts [prs_poc_figures.py](../etl/prs/prs_poc_figures.py), [prs_make_comparison_figures.py](../etl/prs/prs_make_comparison_figures.py), and [prs_poc_latex_table.py](../etl/prs/prs_poc_latex_table.py) can be used to reproduce some of the content of the paper, in terms of figures and tables.

### Running the pipeline

The pipeline is now dockerized. To run it clone the repository and in bash run:

```bash 
docker compose up --build
```

from the root project directory. Docker compose will bring up a local database for your runs, with a persistent storage,
so that all the results can be found and inspected. Similarly, a local volume is mounted, so that intermediate files (
radmc files, cubes, moment zero- and ratio images) can be found in the project directory structure, after a run.
Remember to set the environment variables:

* DB_USER;
* DB_PASS;
* DB_HOST;
* DB_NAME;
  so that the database is correctly initialized.

The main routine accepts the following parameters as input:

* --run_id: the run id of the grid to process; if not provided, generates a new run_id;
* --cleanup_scratches: whether to empty the mld/scratches directory;
* --distributed: whether the grid is processed in a distributed environment, in which case it uses a queue from the
  database to process all models in the grid, otherwise, executes with multiprocessing.

To reset the database, use the command:

`docker volume rm swiss_army_knife_db`.

Be aware that at the moment the first run (also after DB reset) fails due to an issue in the DB initialization. Run the
docker compose command once more to start the pipeline.

To obtain the Apptainer image, the script [create_apptainer_image_file.sh](create_apptainer_image_file.sh) is provided. To run the pipeline execute the command from the SAK folder: `srun singularity exec -B .:$HOME swiss_army_knife_latest.sif python main.py`; this preserves the results after the run. The `--distributed True` flag can be used if running on a distributed system. Job dispatching must be taken care of by the user. Slurm provides a very convenient way of doing that, with no adjustment to the code.

The pipeline produces a series of files that are used to produce the final output.

* It compresses and archives in `stg/archive` all the input files for radmc in a zip file, whose name is also used as a
  primary key part in the database. This file can be used to inspect run results and/or to repeat a run manually if
  needed.
* It saves the H2 volume density, dust temperature grids, and the volume density of the species included in the model as
  fits files in `prs/fits/grids`.
* Converts the `image.out` file to a fits cube in `prs/fits/cubes`.
* Computes the moment-zero maps by integrating the full cube, persisting them in `prs/fits/moments`.
* Determines the line ratios pixel-by-pixel and saves the maps in `prs/fits/ratios`.

After the last model is completed, the results are summarized in a series of figures, that are saved in
the `prs/output/run_type/{provided_run_type}/figures`
folder.

The files named `ratio_grid_lines_*.png` show the average ratio across the entire clump, as a function of the
characteristic temperature and volume density.

The pixel-by-pixel ratios as a function of the average line-of-sight H2 volume density is shown
in `ratio_vs_avg_density_los_*.png`. The KDE-smoothed data are computed and save
in `ratio_vs_avg_density_los_kde_*.png`.

To check the opacity of the lines, we include a plot showing the integrated emission px-by-px as a function of the
molecular colum density in that pixel (`coldens_moments_lines_*.png`)).

If density inference is performed, the posterior PDF is saved as `density_inference.png`.

### Configuration files parameters

In this paragraph we describe in more detail the parameters that can be set in the different configuration files, and
their meaning.

#### The staging configuration file ([etl/stg/config/config.yml](../etl/stg/config/config.yml))

The staging config file has three main categories:

* **grid:**
    * grid_type: regular or spherical
    * coordinate_system: cartesian or polar
    * central_density: the central gas number density of the toy model. If you are using a power-law distribution, it
      corresponds also to the maximum possible density in the grid.
    * density_unit: the units in which the number density is expressed, e.g. 'cm^-3'
    * density_powerlaw_idx: the power-law index of the density distribution
    * density_at_reference: the gas number density at a reference value, e.g. at 1 pc
    * distance_reference: the reference radius for scaling the power-law
    * distance_reference_unit: the units of the reference radius, e.g. 'pc', 'AU', 'cm'
    * dust_temperature: the dust temperature value
    * dust_temperature_unit: the dust temperature unit
    * dust_temperature_powerlaw_idx: the dust temperature power-law index
    * microturbulence: the unresolved turbulent velocity of the gas
    * microturbulence_unit: the units of the microturbulence value, e.g. 'km/s'
    * dimN: a dictionary-like parameter, it should include the 'size', 'size_unit', 'shape', and 'refpix' keys. E.g. {"
      size":1, "size_units": "pc", "shape": 3, "refpix": 1}. If only dim1 is provided, the grid is cubic.
        * size: the grid size in physical units
        * size_unit: the units in which the grid size is expressed, e.g. 'pc'
        * shape: the number of cells in the grid for this axis
        * refpix: the reference pixel that corresponds to the grid "centre" from which the distance are computed for
          power-law models, for instance
    * velocity_field: velocity field to apply to the gas, it can only be 'solid' at the moment (the gas is assumed to
      rotate around the y axis). Power-law is in principle also supported, but the wiring is still to be implemented.
    * velocity_gradient: the value of the velocity gradient for solid-body roation.
    * velocity_gradient_unit: the unit of the velocity gradient, e.g. "km/s/pc"

* **stars:**
  Adding the star configuration section makes the program compute the dust temperature distribution given the properties
  of the stars included. _Caveat:_ be careful, test runs show unexpectedtly low dust temperatures using a blackbody.
    * nstars: the number of stars to include
    * rstars: the radius of the star in cm; it is normally ignored, unless the parameter `istar_sphere` is set to 1
    * mstars: the mass in grams; not used in the current version of RADMC
    * star_positions: x, y, z coordinates of the star in cm, expressed as a list of lists
    * star_fluxes: stellar fluxes at each of the computed lambdas in erg cm$^-2$ s$-1$ Hz$-1$; if negative, interpreted
      as the peak temperature of a blackbody
    * nlambdas: number of wavelengths to compute
    * spacing: log or linear
    * lambdas_micron_limits: the limits in wavelength to consider in the run, expressed in micron

* **lines:**
    * species_to_include: the list of molecular species to include in the RADMC postprocessing, e.g. ['e-ch3oh']
    * molecular_abundances: a dict-like parameter, containing the species name and the corresponding fractional
      abundance, e.g. {"e-ch3oh": 1e-8, "p-h2": 1}
    * lines_mode: the line transfer mode. It can be 'lte', 'lvg', 'optically_thin_non_lte', 'user_defined_populations' (
      see the RADMC documentation if in doubt)
    * collision_partners: the list of collision partners to be used; it must appear in the correct order as in the
      molecule_{molname}.inp file of the molecule to be simulated, e.g. ['p-h2']

#### The model configuration file ([etl/mdl/config/config.yml](../etl/mdl/config/config.yml))

The model configuration file has two categories:

* **radmc_postprocessing:**
    * nphotons: the number of photons to use for the postprocessing
    * scattering_mode_max: override the scattering settings in the dust opacity files; 0 excludes scattering, 1 treats
      it in an isotropic way (if defined), 2 includes anisotropic scattering (if defined)
    * iranfreqmode: 1
    * tgas_eq_tdust: whether the gas temperature is equal to the dust temperature (if not, it must be specified or
      computed separately!)

* **radmc_observation:**
    * inclination: the inclination wrt. the line-of-sight in degrees
    * position_angle: the position angle wrt. the observer
    * imolspec: the index of the species to be modeled; defaults to the first
    * iline: the line identifier for the line to be modeled, according to the molecule_{molname}.inp
    * width_kms: the range in km/s to be modeled around the line
    * nchannels: the number of channels to be considered
    * npix: the number of pixels in the final image; WARNING: it must be a multiple of the grid shape
    * threads: number of threads to be used by radmc
    * image_size_pc: the size of the image to produce; it is useful to get a good alignment

#### The global configuration file ([etl/config/config.yml](../etl/config/config.yml))

The global configuration file, in addition to the run_type name, has two categories, "computation" and "overrides":

* run_type: name of the folder where the results are grouped and saved in the prs step
* **computation:**
    * threads: number of threads to include in the multiprocessing pool

* **overrides:**
    * dust_temperature_grid_type: the spacing in the dust temperature grid, can be: 'linear', 'log'
    * dust_temperature_limits: the limits in the dust temperature grid, e.g. [10, 30]; the last point is excluded
    * dust_temperature_step: the step size in the grid; if the spacing is logarithmic, it represents the increase
      factor,
      e.g. 10 means steps of one order of magnitude
    * gas_density_grid_type: the spacing in the gas number density grid, can be: 'linear', 'log'
    * gas_density_limits: the limits in the gas number density grid, e.g. [1e4, 1e8]; the last point is excluded
    * gas_density_step: the step size in the grid; if the spacing is logarithmic, it represents the increase factor,
      e.g. 10
      means steps of one order of magnitude
    * gas_density_unit: the units in which the gas number density is expressed, e.g. 'cm^-3'
    * lines_to_process: the list of line identifiers to process, according to the molecule_{molname}.inp file,
      e.g. [['87', '86'], ['88', '87']]. The parameter is gives as a list of lists, so that the program knows which
      ratios
      to compute.

#### The density inference input file ([etl/config/density_inference_input.yml](../etl/config/density_inference_input.yml))

This file contains the measured ratios, their uncertainties, and a few other parameters to perform the inference.

* use_model_for_inference: the name of the folder where the model results to be used are stored.
* integrated_intensities_uncertainties: a (potentially nested) dictionary of measured integrated intensities. The keys
  are the line id and the source name that will be concatenated to the output plot names. A subset can be given,
  depending on what is available from observations. Each key can contain both a float representing the integrated
  intensity or a dictionary of sources, each with their own data.
* measured_integrated_intensities: the rms uncertainty associated to the integrated intensity measurement. It has the
  same structure as the integrated intensity dictionary.
* ratios_to_include: a list of strings representing the ratios to consider in the density determination, in the form
  of 'transition_id_1-transition_id_2'.
* simulated_ratio_realizations: the number of simulated ratios to generate
* recompute_kde: True - retrain the model, False - unpickles it
* probability_threshold: the probability threshold to be used to compute the highest probability density interval (this
  is the probability in the wings!).
* limit_rows [optional]: limits the number of rows to extract from the data to reduce computational time for tests.
* points_per_axis [optional]: the number of points on the density and ratio axes, to compute the probability
  distributions.
* nthreads [optional]: the number of threads to be used for computation.

#### The ML-emulation input file ([etl/config/ml_modelling.yml](../etl/config/ml_modelling.yml))

This file determines what tasks are performed as part of the `prs_ml_training.py` `prs_expand_modelling_with_ml.py`
scripts.

* retrain: boolean to specify whether the model should be retrained from scratch (`prs_ml_training.py`) or reloaded to
  perform inference (`prs_expand_modelling_with_ml.py`).
* overrides: same as in the global configuration file.
* model_type: the kind of model to use to perform emulation (relevant only if retrain is true); admitted values are '
  XGBoost', 'XGBoost_gridsearch' (XGBoost with gridsearch on the hyperparameters and CV), 'RandomForest', 'auto_skl' (
  AutoSklearnRegressor).
* model_parameters: depending on model_type, a dictionary of the parameters to pass to the model. For example, XGBoost
  supports `n_estimators`, `max_depth`, `learning_rate`, etc; compare the documentation of the xgboost, sklearn, and
  auto-sklearn packages for a comprehensive list. For the XGBoost with grid search, it is a nested dictionary with two
  keys:
    * param_grid: a dictionary of the hyperparameters to explore, each with a corresponding list of [start, stop, step]
      values to pass to numpy.arange.
    * param_gridsearch: a dictionary of parameters for the GridSearchCV object; verbosity is put to 3 by default.

## Database

The database is an instance of PostgreSQL, created via docker compose or with the `apptainer.def` singularity definition
file.

The database ER diagram is shown in `publications/6373bb408e4040043398e495/figures/sak_database.png`. In the following
we document the tables and their relationships.

### Description of the tables

The tables and their column are described in detail in this section.

#### grid_parameters

The `grid_parameters` table contains the metadata of the models grid computed in the SAK run. The primary key is
composed of the `run_id` and `zipped_grid_name` columns.

| Column name                     | Type         | PK    | FK    | FK Table | Description                                                                                                                                                               |
|---------------------------------|--------------|-------|-------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| zipped_grid_name                | varchar(150) | True  |       |          | The name of the compressed grid file containing all the input used to generate the model                                                                                  |
| grid_type                       | text         | False |       |          | The type of the grid to be constructed (to date regular or spherical)                                                                                                     |
| coordinate_system               | text         | False |       |          | The coordinate system to use in the grid (to date cartesian or polar)                                                                                                     |
| central_density                 | Float        | False |       |          | The maximum density above which the grid is cut and filled with; if 1e8 cm^-3, every point above this threshold is set to 1e8 cm^-3 instead; expressed in cm^-3           |
| density_powerlaw_index          | Float        | False |       |          | The power-law index used to construct the volume density grid                                                                                                             |
| density_at_reference            | Float        | False |       |          | The value of the volume density at the reference radius; expressed in cm^-3                                                                                               |
| dust_temperature                | Float        | False |       |          | The maximum value of the dust temperature above which the grid is cut and filled with; if 2000K, every point above this threshold is set to 2000K instead; expressed in K |
| dust_temperature_powerlaw_index | Float        | False |       |          | The power-law index used to construct the dust temperature grid                                                                                                           |
| dust_temperature_at_reference   | Float        | False |       |          | The dust temperature value at the reference radius; expressed in K                                                                                                        |
| microturbulence                 | Float        | False |       |          | The microturbulence value (standard deviation) to add to the thermal broadening of the lines; expressed in km/s                                                           |
| velocity_field                  | text         | False |       |          | The type of velocity field to use to construct the velocity grid (to date 'solid' for solid-body rotation                                                                 |
| velocity_gradient               | text         | False |       |          | The value of the velocity gradient to use for the velocity field construction; expressed in km/s/pc for solid-body rotation                                               |
| velocity_powerlaw_index         | Float        | False |       |          | The index of the power-law used to construct the velocity grid                                                                                                            |
| velocity_at_reference           | Float        | False |       |          | The value of the velocity at the reference radius; expressed in km/s                                                                                                      |
| distance_reference              | Float        | False |       |          | The value of the reference radius; expressed in pc                                                                                                                        |
| maximum_radius                  | Float        | False |       |          | Grid cutoff radius, the external radius of the fragment                                                                                                                   |
| grid_size_1                     | Float        | False |       |          | The dimension of the grid along the x axis; expressed in pc                                                                                                               |
| grid_shape_1                    | Float        | False |       |          | The number of pixels along the x axis                                                                                                                                     |
| grid_refpix_1                   | Float        | False |       |          | The reference pixel for the x axis                                                                                                                                        |
| grid_size_2                     | Float        | False |       |          | The dimension of the grid along the y axis; expressed in pc                                                                                                               |
| grid_shape_2                    | Float        | False |       |          | The number of pixels along the y axis                                                                                                                                     |
| grid_refpix_2                   | Float        | False |       |          | The reference pixel for the y axis                                                                                                                                        |
| grid_size_3                     | Float        | False |       |          | The dimension of the grid along the z axis; expressed in pc                                                                                                               |
| grid_shape_3                    | Float        | False |       |          | The reference pixel for the z axis                                                                                                                                        |
| grid_refpix_3                   | Float        | False |       |          | The reference pixel for the z axis                                                                                                                                        |
| created_on                      | DateTime     | False | False |          | The timestamp of the record creation                                                                                                                                      |
| run_id                          | text         | True  |       |          | The id of the run                                                                                                                                                         |

#### grid_files

This table contains the name of the fits files where the physical grids have been saved.

| Column name      | Type         | PK    | FK    | FK Table        | Description                                                                              |
|------------------|--------------|-------|-------|-----------------|------------------------------------------------------------------------------------------|
| zipped_grid_name | varchar(150) | True  | True  | grid_parameters | The name of the compressed grid file containing all the input used to generate the model |
| quantity         | varchar(30)  | True  | False |                 | The quantity saved in the fits file e.g. temperature, volume density, etc.               |
| fits_grid_name   | text         | False | False |                 | The name of the fits file with the physical grid                                         |
| created_on       | DateTime     | False | False |                 | The timestamp of the record creation                                                     |
| run_id           | text         | True  | True  | grid_parameters | The id of the run                                                                        |

#### stars_parameters

In this table there are the parameters used to describe the (potential) distribution of stars in the grid.

| Column name           | Type         | PK    | FK    | FK Table        | Description                                                                              |
|-----------------------|--------------|-------|-------|-----------------|------------------------------------------------------------------------------------------|
| zipped_grid_name      | varchar(150) | True  | True  | grid_parameters | The name of the compressed grid file containing all the input used to generate the model |
| nstars                | integer      | False | False |                 | The number of stars to include in the grid                                               |
| rstars                | Array(float) | False | False |                 | The radiuses of the stars in the grid (as described in the RADMC3D docs)                 |
| mstars                | Array(float) | False | False |                 | The masses of the stars in the grid (as described in the RADMC3D docs)                   |
| star_positions        | Array(float) | False | False |                 | The array of stellar positions                                                           |
| star_fluxes           | Array(float) | False | False |                 | The arrays of the stellar fluxes, for each star                                          |
| nlamdbas              | integer      | False | False |                 | The number of wavelengths to consider for the stellar spectrum                           |
| spacing               | text         | False | False |                 |                                                                                          |
| lambdas_micron_limits | Array(float) | False | False |                 | The spectrum limits array, expressed in microns                                          |
| created_on            | DateTime     | False | False |                 | The timestamp of the record creation                                                     |
| run_id                | text         | True  | True  | grid_parameters | The id of the run                                                                        |

#### lines_parameters

The `lines_parameters` table contains the mode used by RADMC to perform the radiative transfer.

| Column name      | Type         | PK    | FK    | FK Table        | Description                                                                              |
|------------------|--------------|-------|-------|-----------------|------------------------------------------------------------------------------------------|
| zipped_grid_name | varchar(150) | True  | True  | grid_parameters | The name of the compressed grid file containing all the input used to generate the model |
| lines_mode       | varchar(20)  | False | False |                 | The mode used by RADMC to perform the radiative transfer                                 |
| created_on       | DateTime     | False | False |                 | The timestamp of the record creation                                                     |
| run_id           | text         | True  | True  | grid_parameters | The id of the run                                                                        |

#### species_and_partners

This table contains the species and their partners that are used in the radiative transfer computation.

| Column name                           | Type         | PK    | FK    | FK Table        | Description                                                                                    |
|---------------------------------------|--------------|-------|-------|-----------------|------------------------------------------------------------------------------------------------|
| zipped_grid_name                      | varchar(150) | True  | True  | grid_parameters | The name of the compressed grid file containing all the input used to generate the model       |
| species_to_include                    | varchar(100) | True  | False |                 | A molecular species for which RT was performed                                                 |
| molecular_abundance                   | float        | False | False |                 | The base abundance of the molecule                                                             |
| threshold                             | float        | False | False |                 | The temperature threshold above which the temperature is assumed to evaporate from dust grains |
| abundance_jump                        | float        | False | False |                 | The factor by which the base abundance is scaled after evaporation                             |
| collision_partner                     | varchar(100) | True  | False |                 | A collision partner used for excitation                                                        |
| molecular_abundance_collision_partner | float        | False | False |                 | The molecular abundance of the collision partner                                               |
| created_on                            | DateTime     | False | False |                 | The timestamp of the record creation                                                           |
| run_id                                | text         | True  | True  | grid_parameters | The id of the run                                                                              |

#### model_pars

This table includes the configuration parameters used for RADMC.

| Column name         | Type         | PK    | FK    | FK Table        | Description                                                                              |
|---------------------|--------------|-------|-------|-----------------|------------------------------------------------------------------------------------------|
| zipped_grid_name    | varchar(150) | False | True  | grid_parameters | The name of the compressed grid file containing all the input used to generate the model |
| fits_cube_name      | varchar(100) | True  | False |                 | A molecular species for which RT was performed                                           |
| nphotons            | float        | False | False |                 | The number of photons used in the RT computation                                         |
| scattering_mode_max | integer      | False | False |                 | The type of scattering considered in RADMC                                               |
| iranfreqmode        | integer      | False | False |                 |                                                                                          |
| tgas_eq_tdust       | integer      | False | False |                 | Whether or not the gas and dust temperatures are assumed equal                           |
| inclination         | float        | False | False |                 | The model inclination                                                                    |
| position_angle      | float        | False | False |                 | The model position angle                                                                 |
| imolspec            | float        | False | False |                 | The species identifier                                                                   |
| iline               | float        | False | False |                 | The line identifier, according to the molecular transitions descriptor file              |
| width_kms           | float        | False | False |                 | The width of the spectrum to be computed                                                 |
| nchannels           | float        | False | False |                 | The number of channel into which the spectrum will be divided                            |
| npix                | float        | False | False |                 | The number of pixels used for the output image                                           |
| created_on          | DateTime     | False | False |                 | The timestamp of the record creation                                                     |
| run_id              | text         | True  | True  | grid_parameters | The id of the run                                                                        |

#### moment_zero_maps

This table contains the information on the moment-zero images and their computation.

| Column name            | Type         | PK    | FK    | FK Table         | Description                                                                                       |
|------------------------|--------------|-------|-------|------------------|---------------------------------------------------------------------------------------------------|
| mom_zero_name          | varchar(150) | True  | False |                  | The name of the moment-zero fits file                                                             | 
| fits_cube_name         | varchar(150) | False | True  | model_parameters | The name of the fits cube from which the moment zero was computed                                 |                               
| integration_limit_low  | float        | False | False |                  | The lower integration limit for the computation of the moment zero; if null includes all channels |
| integration_limit_high | float        | False | False |                  | The upper integration limit for the computation of the moment zero; if null includes all channels |
| aggregated_moment_zero | float        | False | False |                  | The moment zero computed for the entire cube, according to aggregation_function                   |
| aggregation_function   | varchar(20)  | False | False |                  | The function used to aggregate the pixel-values of the moment-zero maps                           |
| created_on             | DateTime     | False | False |                  | The timestamp of the record creation                                                              |
| run_id                 | text         | True  | True  | moment_zero_maps | The id of the run                                                                                 |

#### ratio_maps

This table includes the information about the ratio computation.

| Column name          | Type         | PK    | FK    | FK Table         | Description                                                               |
|----------------------|--------------|-------|-------|------------------|---------------------------------------------------------------------------|
| ratio_map_name       | varchar(150) | True  | False |                  | The name of the fits file that contains the ratio maps                    |
| mom_zero_name_1      | varchar(150) | False | True  | moment_zero_maps | The name of the moment-zero map constituting the numerator of the ratio   |
| mom_zero_name_2      | varchar(150) | False | True  | moment_zero_maps | The name of the moment-zero map constituting the denominator of the ratio |
| aggregated_ratio     | float        | False | False |                  | The value of the ratio computed over the full simulated cubes             |
| aggregation_function | varchar(20)  | False | False |                  | The function used to compute the aggregated ratio                         |
| created_on           | DateTime     | False | False |                  | The timestamp of the record creation                                      |
| run_id               | text         | True  | True  | moment_zero_maps | The id of the run                                                         |

#### tmp_execution_queue

An auxiliary table used to efficiently parallelize computation in a distributed environment.

| Column name              | Type    | PK    | FK    | FK Table | Description                                                                                          |
|--------------------------|---------|-------|-------|----------|------------------------------------------------------------------------------------------------------|
| row_id                   | integer | False | False |          | A row counter                                                                                        |
| run_id                   | text    | True  | False |          | The id of the run                                                                                    |
| dust_temperature         | float   | True  | False |          | The characteristic dust temperature of the model                                                     |
| density                  | float   | True  | False |          | The characteristic number density temperature of the model                                           |
| line                     | integer | True  | False |          | The line ID                                                                                          |
| density_keyword          | text    | True  | False |          | The column used in the query to select the input (central_density or density_at_reference)           |
| dust_temperature_keyword | text    | True  | False |          | The column used in the query to select the input (dust_temperature or dust_temperature_at_reference) |
| fits_cube_name           | text    | False | False |          | The name of the associated fits cube                                                                 |
| done                     | bool    | False | False |          | Whether the image was processed already                                                              |