Just an other commit...

docs/source/aspis.rst

@@ -21,7 +21,7 @@ integrated approach. This encompasses the whole chain of phenomena from the Sun
 to other planets. In this endeavour, it will be crucial to exploit a variety of products (datasets, codes, models),
 which ultimately populate the ASPIS system. ASPIS unifies multiple SWE resources in a flexible
 architecture to allow users to perform advanced scientific studies, both through a web interface, and a dedicated
-SW package called ASPIS.py. A keyword, metadata-based system enhances search capabilities, especially as
+SW package called ASPISpy. A keyword, metadata-based system enhances search capabilities, especially as
 data volume grows with recorded events. The ASPIS system is open-source, allowing researchers to
 study problems that span across disciplinary boundaries. It creates an environment where scientists can
 discover, understand, and model the connection between solar phenomena, interplanetary disturbances, and

docs/source/aspis_py.rst

-ASPIS.py package
+ASPISpy package
 ----------------
 
 Intro
 ^^^^^
 
-ASPIS.py is a package of code that allows access to the data contained in ASPIS database and other
+ASPISpy is a package of code that allows access to the data contained in ASPIS database and other
 various community databases, to perform client-side specific analyses of this data and to
 visualize it. The data available are of two types: data in the internal database produced by the CAESAR
 community and information on external data in other databases. Along with access to the data, the
-ASPIS.py package contains code packages that can be imported as package objects, with which to perform
+ASPISpy package contains code packages that can be imported as package objects, with which to perform
 analysis and/or visualization of the data.
 
-The criterion of ASPIS.py is to act as a collector for the data and analysis software of the Space Weather (SWx) Italian
+The criterion of ASPISpy is to act as a collector for the data and analysis software of the Space Weather (SWx) Italian
 community and to foster cooperation within the community. In fact, one can use the data and analysis
 methods developed by the community and, likewise, share one’s own data and methods. Indeed, we
-have defined a standard template in ASPIS.py for generic data analysis code, and provide a how-to for
+have defined a standard template in ASPISpy for generic data analysis code, and provide a how-to for
 developers in the SWx community to convert their own code into the standard template, making it
-easily usable for the entire ASPIS.py community in a standard way. In this way ASPIS.py’s primary goal
+easily usable for the entire ASPISpy community in a standard way. In this way ASPISpy’s primary goal
 is to promote collaboration and cooperation within the SWx community. It facilitates the sharing
 of data and analysis methods among researchers, enabling them to benefit from each other’s work. In
-summary, ASPIS.py is a tool designed to streamline the access, analysis, and sharing of 
+summary, ASPISpy is a tool designed to streamline the access, analysis, and sharing of 
 data and analysis methods within the SWx community. It aims to enhance collaboration and
 standardization, making it easier for researchers to work with and build upon each other’s work in the
 field of SWX and astrophysics in general.
@@ -27,35 +27,35 @@ field of SWX and astrophysics in general.
 Functions
 ^^^^^^^^^
 
-The functions of the ASPIS.py module are conceived to implement the following key components of the
+The functions of the ASPISpy module are conceived to implement the following key components of the
 ASPIS service:
 
-1. Access for Advanced Users: ASPIS.py is designed to cater to advanced users in the SWx
+1. Access for Advanced Users: ASPISpy is designed to cater to advanced users in the SWx
 community. It offers a set of functions that can be executed programmatically through a Python
 interpreter. This capability allows users to perform more complex computations and analysis tasks,
 surpassing the limitations of a web application.
 
-2. Basic Visualization Tools: While ASPIS.py provides basic visualization tools/functions, it also en-
+2. Basic Visualization Tools: While ASPISpy provides basic visualization tools/functions, it also en-
 courages users to leverage other freely available modules like Sunpy and Astropy for more ad-
 vanced visualization needs. This approach ensures compatibility and flexibility.
 
-3. Ready-to-Use Python Snippets: ASPIS.py intends to provide ready-to-use Python code snippets.
+3. Ready-to-Use Python Snippets: ASPISpy intends to provide ready-to-use Python code snippets.
 These snippets are designed to assist entry-level users, making it easier for them to get started
 with the module. Advanced users can further customize these code snippets to suit their specific
 research requirements.
 
-4. Integration with User Programs: The expectation is that advanced users of ASPIS.py will seam-
+4. Integration with User Programs: The expectation is that advanced users of ASPISpy will seam-
 lessly integrate the functions provided by the module into their own analysis and visualization pro-
 grams. This promotes flexibility and allows users to tailor their solutions to their research needs.
 
-5. Structured Information in the Database: ASPIS.py relies on structured information within the database.
+5. Structured Information in the Database: ASPISpy relies on structured information within the database.
 This information likely includes details about various SWx entities such as phenomena,
 data descriptions, and chains, which are essential for conducting meaningful analyses.
 
-6. Data Outside the Database: ASPIS.py is designed to work with data that exists outside the database
+6. Data Outside the Database: ASPISpy is designed to work with data that exists outside the database
 but corresponds to the descriptions available within the database. This flexibility allows researchers
 to combine data from various sources to enhance their analysis.
-In summary, ASPIS.py is a versatile tool that caters to both entry-level and advanced users in the
+In summary, ASPISpy is a versatile tool that caters to both entry-level and advanced users in the
 SWx community. It provides functions for querying, visualizing, analyzing, and modeling astro-
 physical data. The inclusion of code snippets and use case examples simplifies the usage for a broad
 range of researchers, and advanced users can extend and customize the functionality to meet their spe-
@@ -64,10 +64,13 @@ cific research objectives.
 Installation
 ^^^^^^^^^^^^
 
-To install ASPIS.py, a user typically needs to follow these general steps. Please note that the specific
-installation process may vary depending on the availability and distribution method of ASPIS.py. The user
-has to ensure that Python is installed on the system before proceeding. The user can install ASPIS.py
-using pip, which is the Python package manager:
+**ASPISpy needs python 3.9 and higher to run.**
+
+To install ASPISpy, a user typically needs to follow these general steps. Please note that the specific
+installation process may vary depending on the availability and distribution method of ASPISpy. The user
+has to ensure that Python is installed on the system before proceeding. 
+
+The user can install ASPISpy using pip, which is the Python package manager:
 
 ``mkdir aspispy``
 
@@ -94,7 +97,7 @@ the provided Wheel file. Note that version number may have changed.
 Use case example
 ^^^^^^^^^^^^^^^^
 
-Here we show an example of the way a user can use a specific product of the ASPIS.py package.
+Here we show an example of the way a user can use a specific product of the ASPISpy package.
 from aspis import products as aps
 from datetime import datetime
 
@@ -141,13 +144,13 @@ their usage and any additional requirements for proper execution.
 Code snippets and standardization
 .................................
 
-The idea of ASPIS.py is that code developed by different members of the Space Weather community in contiguous 
+The idea of ASPISpy is that code developed by different members of the Space Weather community in contiguous 
 fields can be made available to each other and shared in exactly the same way as is usually done
 with data. In the area of data, standardization of formats and thus the use of databases in which to store
 information in a consistent and shared manner has been adopted for years. As for software, the situation
 is more complex because codes have very heterogeneous characteristics among themselves and code
 groupings still have strong limitations. Inspired by some recent experiences aimed at standardizing the
-use of codes and comparing the performance of different methods on the same data, ASPIS.py aims to
+use of codes and comparing the performance of different methods on the same data, ASPISpy aims to
 provide a structural scheme with the ability to make the use of code developed by different members of
 the Space Weather community for even different purposes available to the entire community.
 

docs/source/index.rst

 Welcome to ASPIS documentation!
 ===================================
 
-This is the **ASPIS** service live documentation. Here you can learn how to use **ASPIS** webapp, aspis.py library and have a look about the informations of the data products hosted in **ASPIS** systems. 
+This is the **ASPIS** service live documentation. Here you can learn how to use **ASPIS** webapp, ASPISpy package and have a look about the informations of the data products hosted in **ASPIS** systems. 
 
 .. image:: assets/images/logo_ASPIS_prtscrn.png
   :height: 320

docs/source/products/data_products.rst

@@ -7,8 +7,8 @@ Atmosphere
 
    filerst_ATMO/*
 
-Earth image/map
----------------
+Earth
+-----
 
 .. toctree::
    :titlesonly:
@@ -61,8 +61,8 @@ Data Acquired at Lagrange 1 position
 
    filerst_L1/*
 
-Mercury image/map
------------------
+Mercury
+-------
 
 .. toctree::
    :titlesonly:
@@ -70,7 +70,7 @@ Mercury image/map
 
    filerst_Mercury/*
 
-Sun image/map
+Sun
 -------------
 
 .. toctree::
@@ -79,7 +79,7 @@ Sun image/map
 
    filerst_SUN/*
 
-Venus image/map
+Venus
 ---------------
 
 .. toctree::

docs/source/products/filerst_IP/unitov_icme_tableset.rst

@@ -55,7 +55,7 @@ Dataset summary
   * - **observable:**
     - 
   * - **latest update:**
-    - 2024-03-28 05:08:17
+    - 2024-05-13 07:34:22
 
 Columns specification
 ~~~~~~~~~~~~~~~~~~~~~
@@ -230,3 +230,136 @@ Columns specification
      - char
      - char
      - string (CME_0111_param_inv.txt ) - name of the file containing PDBM information from the statistical inversion procedure
+
+Code example
+~~~~~~~~~~~~
+
+``"""``
+
+``This is an example script for accessing client-side ASPIS datasets.``
+
+``The client is provided by the ASPISpy package and in particular the connection is resolved by the CaesarAPI class.``
+
+``This example code allows the user to fetch a product from the ASPIS DB and``
+
+``save it in a ready-to-use pandas.DataFrame structure.``
+
+``To use this code to fetch another product, simply change``
+
+``the product name and possibly the time range and columns of the dataset``
+
+``(for the filtered_search method).``
+
+``Notes:``
+
+``- The CaesarAPI class provides a complete interface to the ASPIS DB through several methods,``
+
+``each of which corresponds to a different API endpoint.``
+
+``- Once CaesarAPI has been instantiated through authentication, the user can call the different methods``
+
+``(e.g., CaesarAPI.download_data, CaesarAPI.filtered_search, etc. etc.).``
+
+``- The import and initialization are the same for each code fragment.``
+
+``"""``
+
+``#### The following lines of code should be executed only once``
+
+``#### during session initialization. If they are re-executed,``
+
+``#### the session in ASPIS DB is recreated from scratch.``
+
+``#### Once the session is initialized, the user can use of all``
+
+``#### the methods provided by the CaesarAPI class.``
+
+``from aspis.CaesarAPIwrapper.request_caesar import CaesarAPI``
+
+``import pandas as pd``
+
+``# The following lines to disable SSL certificate warning.``
+
+``# Note: In this prototype version, the certificate is not available``
+
+``import requests``
+
+``from requests.packages.urllib3.exceptions import InsecureRequestWarning``
+
+``requests.packages.urllib3.disable_warnings(InsecureRequestWarning)``
+
+``# Initialize the CaesarAPI connector``
+
+``api = CaesarAPI(credentials={'email': '<your_email>','password': '<your_password>'})``
+
+``# [INFO] user <your_email> authenticated``
+
+``# Define the product to be fetched``
+
+``product_name = "unitov_icme_tableset"``
+
+``# FETCH THE FULL DATASET``
+
+``r, df = api.get_product_detail(product_name)``
+
+``# r is the full response of the request, df is the pandas.DataFrame``
+
+``# which contains the dataset.``
+
+``# > print(df.head())``
+
+``#     id                 time     HPC_Tx     HPC_Ty  HPC_distance  CME_num  ...``
+
+``# 0   0  1997-01-06T15:10:42  25.210143  -3.221796      0.983319        1   ...``
+
+``# 1   1  1997-02-07T00:30:05  53.055793  -7.993757      0.986300        2   ...``
+
+``# 2   2  1997-04-07T14:27:44  14.917489 -25.972924      1.001209        3   ...``
+
+``# 3   3  1997-05-12T05:30:05   6.568649  21.393300      1.010308        4   ...``
+
+``# 4   4  1997-05-21T21:00:53 -63.993601 -35.996400      1.012279        5   ...``
+
+``# FETCH A SUBSET of <product_name>``
+
+``df = api.filtered_search(prod_id=product_name,filters=['time < 2005-01-01','time > 2000-01-01'],cols=['time','bz','dst','v_r'],return_dataframe=True)``
+
+``# [INFO] Executing the following query:``
+
+``#     query MyQuery {``
+
+``#       filtered_search(filters: [{field: "time", operator: "lt", value: "2005-01-01"}{field: "time", operator: "gt", value: "2000-01-01"}]) {``
+
+``#         ... on UnitovIcmeTablesetType {``
+
+``#           time``
+
+``#           bz``
+
+``#           dst``
+
+``#           v_r``
+
+``#         }``
+
+``#       }``
+
+``#     }``
+
+``# > print(df.head())``
+
+``#                   time  bz  dst      v_r``
+
+``# 0  2000-01-18T17:54:05  16  -97  1132.60``
+
+``# 1  2000-02-08T09:30:05   7  -25  1364.00``
+
+``# 2  2000-02-10T02:30:05  13 -133  1097.20``
+
+``# 3  2000-02-12T04:31:20   5  -67  1286.90``
+
+``# 4  2000-02-17T19:31:23  15  -26   566.07``
+
+``
+``
+

docs/source/webapp.rst

@@ -9,12 +9,12 @@ ASPIS webapp aims to provide an easy, user friendly, access to the data containe
 ASPIS webapp will be suited with tools able to easy access and visualize data. Technically, ASPIS
 web-app will access to database using a set of dedicated API (GET) who will act as middleware to the
 Database queries. A key feature will be the function to visualize several data-set in the same page. The
-feature, strongly related to the casual-chain [ref. 1] concept is designed to take into account the standard
+feature, strongly related to the casual-chain concept is designed to take into account the standard
 work-flow of the Space Weather community while they are inspecting correlation and liked phenomena
 inside different products. A small and protected amount of server-side and/or client side models should
 be run over visualized data for simple and light computation, notwithstanding the intention of the web-app
 is to facilitate the data exploration and cross-visualization. Users will be encouraged to implement heavy
-computations and advanced data handling using ASPIS.py [ref. 9] libraries.
+computations and advanced data handling using ASPISpy package.
 
 Functionalities
 ^^^^^^^^^^^^^^^
@@ -38,7 +38,7 @@ Advanced functionalities will be protected and will be accessible only to user i
 • Run internal, server-side, ASPIS models over an identified data-set
 • (optional) Light computation of model and data comparison
 
-The Access Control List (ACL) will be provided by the ASPIS machines (Fig. 5)
+The Access Control List (ACL) will be provided by the ASPIS machines.
 
 Time-based multi-plot visualization
 ...................................
@@ -54,6 +54,7 @@ Data-set are generated by instruments spatially referenced by acquisition site (
 physically mounted on a Spacecraft or on Earth ), by target position (ex. The instrument target is the
 Sun) and by line of sight (ex. the instrument acquire 20 square deg in a direction of the sky). This info
 can be used to facilitate the user to visualize:
+
 • Instruments positions in a specified time span
 • Targets positions in a specified time span
 • Targets coverage in instruments line of sights.
@@ -62,17 +63,27 @@ can be used to facilitate the user to visualize:
 Event-chain visualization
 .........................
 
-Causal-chains (ref. 1) will get a special section in the webapp. Causal-chains will take advantage from the
+Causal-chains will get a special section in the webapp. Causal-chains will take advantage from the
 other basic visualization plots and can be associated here as a pre-defined organization of data subsets
 plots. Web-app dedicated section will be able to list and search all causal-chains. Web-app will be able
 to advise the user if, in a nominal visualization plot, the data subset is part of a know causal-chain and
 propose a suitable visualization of such a causal-chain.
 
-Data proximity suggestions
-..........................
+Data Discovery
+..............
+
+Aspis Data Discovery section is organized in four thematic sub-panels:
+
+• Phenomena
+• Targets
+• Scientific Topics
+• Smart Discovery
 
-User would be prompt by the system in case other data-sets, respect to ones already visualized, will
-have a "proximity" (in terms of observed target, data coherence, data meaning, spatial nearness ) and
-are available in the ASPIS Database system for the observed time-span or in a relevant subsequent time
-(to consider propagation of the phenomena to other instruments).
+The last selection method is strongly related to the causal chain concept. 
+The concept is designed to consider several possible workflows of the Space Weather community while searching for correlations or linked phenomena inside different products.
+This criterion is called "smart discovery" and takes inspiration from the casual chain concept.
+This discovery panel tries to represent a semantic generalization of the "casual chains" contained in the DB, presenting to the user a selection of data products that are linked for their 'conceptual proximity', i.e. they either are often associated by causality or correlation in Space Weather studies or are alternative data sources.
+A few smart discovery associations are offered to the user (e.g., interplanetary disturbances studies, flare AR studies, geoeffective disturbances studies), possibly updating the existing associations and adding others, following the researchers' experience and interests.
+Of course, jumping from one smart discovery representation to another is possible since some data products are present in more than one representation and can act as pivot points.
 
+On the right size of the Data Discovery panel the list of the user selected products will be shown.