diff --git a/JDL.md b/JDL.md new file mode 100644 index 0000000000000000000000000000000000000000..6726b8c1c4a37fe866bb4e784402f966e0cf773c --- /dev/null +++ b/JDL.md @@ -0,0 +1,8 @@ + +## Multi-cutout Job Description Language {#jdl} + +The JDL is an JSON-array of standard SODA parameters. Result status of a cut is appended to each array element and such JSON is included into the resulting compressed tar.gz file. + +TBD... + + diff --git a/README.md b/README.md index 5706084174beac7ba4f290e1e444d6c05fea1e77..8f7af4eeef42ce63a15d43a4ffa7d6040819ebff 100644 --- a/README.md +++ b/README.md @@ -1,108 +1,60 @@ + ## vlkb-soda is a web-application to access astronomical data stored in FITS-files. -It implements SODA IVOA recommendation v1.0 (https://ivoa.net/documents/SODA/20170517/index.html). - -*It can be used with vlkb-siav2 (https://ict.inaf.it/gitlab/ViaLactea/vlkb-siav2) to provide complete data discovery and access service.* - - -The SODA docker image is available in this repository and may be launched by an **compose.yaml** as in this example: - -```yaml -version: '3' - -services: - - soda: - container_name: soda-vlkb - image: git.ia2.inaf.it:5050/vialactea/vlkb-soda/soda:1.7 - ports: - - 18019:8080 - environment: - - ACCESS_CONTEXT_ROOT=ska#datasets - volumes: - - /srv/ska/surveys:/srv/surveys:ro - restart: always - -``` - -The port number, ACCESS_CONTEXT_ROOT and directory to FITS-files must be adjusted for the given instance. - -The above configuration example assumes: - -* 'SKA' project holds FITS-files in sub-directories under: `/srv/ska/surveys` -* will be accessed remotely by URL: `http://<server>/ska/datasets/soda` -* identified by SODA-param ID which is an **IVOID** of the form: +It implements [IVOA SODA v1.0](https://ivoa.net/documents/SODA/20170517/index.html) and it supports security by OpenIDConnect/OAuth2.0 and protects connections by TLS. -`ID=ivo://<authority>/<resource-key>?<relative-pathname>#extnum` +When used with [vlkb-siav2](https://ict.inaf.it/gitlab/ViaLactea/vlkb-siav2) they provide complete data-discovery and access solution. -The service decodes IVOID part after the question mark '?'. +## Launching the service -The pathname in the ID is relative to the path in compose.yaml volume-mapping (e.g. `/srv/ska/surveys`). +The vlkb-soda service is available as a docker-image and may be launched as examplified in this [**compose.yaml**](docker/example-compose-soda.yaml). The port number, ACCESS_CONTEXT_ROOT and directory to FITS-files need to be adjusted for the given instance. To see that the instance is running, access the availability endpoint: -The FITS extension number `#extnum` is optional. Default is the primary HDU. - -Note that the service appends '**soda**' to ACCESS_CONTEXT_ROOT. -Then, a data access example: ```bash -curl -s -k --get - --data-urlencode "ID=ivo://some-auth/some-key?some/path/somefile.fits" - --data-urlencode "CIRCLE=202.256 47.493 0.2" - -v -o soda.fits - http://localhost:18019/ska/datasets/soda +curl --get http://localhost:8004/vlkb/datasets/availability ``` -cuts the file `/srv/ska/surveys/some/path/somefile.fits`. - -The root URL `http://localhost:18019/ska/datasets/soda` returns the service descriptor. - - - -## SODA for VLKB - -Additionally to SODAv1 it also provides VLKB-specific functions: -- the filtering paramters (POS, BAND) can be given in GALACTIC and VELO-LSRK-km/s coordinates systems -- supports non-WCS cuts by pixels grid -- inserts additional metadata (RESTFREQ, CUNIT) if not present in FITS-headers -- can create and hold a cut-file server-side for later download, and -- provides count of undefined values in the cut-file +## Enpoints -If the cuts are hold server-side also the following needs ot be configured: +- **availability** is a [VOSI](https://ivoa.net/documents/VOSI/20170524/REC-VOSI-1.1.html) end-point and returns information whether the service is active +- **capabilities** is a [VOSI](https://ivoa.net/documents/VOSI/20170524/REC-VOSI-1.1.html) end-point which describes service' functions and paramters +- **soda** synchronous end-point provides [SODA](https://ivoa.net/documents/SODA/20170517/index.html) service +- **uws_mcutout** a non-standard asynchronous [UWS](https://ivoa.net/documents/UWS/20161024/REC-UWS-1.1-20161024.html#ApplicationsOfUWS) end-point which allows several cuts to be specified in one request by own [Job Description Language](JDL.md). All cuts are returned compressed in one tar.gz file -* RESPONSE_FORMAT : application/x-vlkb-xml (default is application/fits) -* URL_CUTOUTS : server-side cuts can be downloaded form this URL +## Security -## Installation +The data store may hold public and/or private collections. -The service consists of +The vlkb-soda supports OIDC/OAuth2.0 protocol and will validate access token in the request. +Non authenticated requests may be allowed by configuration, and will access only public data. -- **vlkb-soda** web-application and associated **vlkbd** daemon to access FITS file contents based on VO SODA -- **vlkb** command line utility to perform some of the functionalities of the web-applications on command line +If request passes token validation, group-based authorization check is performed. A user may access the data +if at least on of the user's groups is allowed the access. Access rights for data collections are held in the authorization table. -There are rpm, deb and war packages avaialable for Debian, CentOS and Fedora. +## Docker image -### Install from packages (rpm/deb and war) +is available from the gitlab-repository of this project. -There is a war-package for the cutout web-application +Configuration parameters are set by these docker environment variables: -- vlkb-soda-X.Y.Z.war -And two packages for linux executables (deb or rpm) +| parameter | description | +|---------|--------------| +| **ACCESS_CONTEXT_ROOT** | root of the vlkb-soda end-points | +| **OIDC_INTROSPECT** | URL to OAuth2.0 introspect end-point for token validation | +| **OIDC_RAP_URL** | root URL of an Open ID Connect comaptible identity service | +| **OIDC_GMS_URL** | root URL of Group Management Service | +| **OIDC_CLIENT** | client-id of a Relying party | +| **OIDC_SECRET** | secret of of the client | +| **AUTHZ_DB_URL** | DB where a table with authorization info ('goups' column) can be found | +| **AUTHZ_DB_SCHEMA** | schema of the authorization table | +| **AUTHZ_DB_USERNAME** | user in the DB with authorization table | +| **AUTHZ_DB_PASSWORD** | password for the DB-user | +| **URL_CUTOUTS** | server-side cuts can be downloaded form this URL | -- vlkbd-X.Y.Z.deb implements the cutout engine and should be installed together with vlkb-cutout-\*.war -- vlkb-X.Y.Z.deb is a optional utility - -Additionally optional vlkb-obscore utility for the vlkb-search web-app: -- vlkb-obscore-X.Y.Z.deb is a optional tool to create ObsCore table for vlkb-search-\*.war - -To download version X.Y, add one of the above package names to - -```bash -FIXME curl -O --header "PRIVATE-TOKEN: <security-token>" "https://ict.inaf.it/gitlab/api/v4/projects/79/packages/generic/vlkb-datasets/X.Y/<package-name>" - -``` +Context-root parameter uses 'tomcat syntax' in which the forward slash is replaced with hash: vlkb#datasets -> /vlkb/datasets