Now, it is recommended to throw a DBCancelledException from any DBConnection
long processing. It is already done for the upload of a table, the execution
of an ADQL query and the setting of a whole TAP_SCHEMA. The flag
JDBCConnection#cancelled has now a bit different meaning: it is set even if
the Statement.cancel() fails so that any JDBCConnection function can see that
a cancellation has been requested.

Two comments:

- JDBCConnection.getDBMSName(String url) is now deprecated ; it may disappear
  in a next version of the ADQLLibrary.
- In case the DBMS name can not be retrieved (generally because of an
  incomplete JDBC driver implementation), NULL will be returned.

Few useless casts have also been removed.

It is also now recommended to make DBConnection.executeQuery(ADQLQuery)
return NULL if the query has been aborted (indeed, the DBConnection is
the only one that can reliably know that fact). JDBCConnection has been
adapted consequently.

Two embedded DBMS are used: H2 and a little SQLite. JDBC drivers are provided
in the `lib` directory. The databases are created and deleted automatically
by the JUnit tests (see `test/tap/db_testtools/DBTools for more details`).

The ANT and Gradle build scripts have been updated to reflect all these test
modifications.

This issue has been raised with an H2 database.

This is very helpful if TAP_SCHEMA (or some of its tables and their columns)
has a different name in the database.
The mapping can be specified to JDBCConnection or directly in the configuration
file.

Schema were still ordered by schema_name instead of schema_index when this
latter exists.

Commit following 6fc7f8fd.

These two columns let recommend an order, respectively, for tables in their
schema, and columns in their table.
In addition of these new columns, "arraysize" has been also added.
All these new columns are already supported when creating a TAP_SCHEMA from
an XML file.

column in TAP_SCHEMA. Such names are normalized (without any prefix and without
double quotes) when getADQLName() is called, but are returned just trimmed
by getRawName(). This latter is just used by TAPMetadata when writing the XML
description of all TAP tables.

See the commit bd621842,
for the first part of the fix (which actually did not really
fixed the problem: connections "idle in transaction" were
still in the database ; the connection being inside an opened transaction,
it generates lock issues in the database in addition of probably taking
some memory resources).

The transaction and Statement were closed too early before.
  - Fetching the row was not possible once the first bunch of fetched
rows was over.
  - The problem of "statement is aborted" preventing the re-use of
a same DB connection was apparently still there, but occurred less often.

  Now, any transaction potentially started in a DB connection is always
closed after one of the public functions of JDBCConnection is called ;
except executeQuery(ADQLQuery) whose the call MUST be wrapped inside a
try...catch block in which DBConnection.cancel(true) MUST be called
in case of error (in order to effectively end any started transaction).

log message (i.e. event = "TABLE_CREATED").

opened by JDBCConnection.executeQuery(...) because a fetch size is set, this
transaction was never closed.

1) [TAP & UWS] ]MAJOR BUG FIX: The abortion of an SQL query is now correctly
implemented. Before this fix, 2 mistakes prevented this clean abortion:
  a/ The thread was not cancelled because the SQL query execution was
blocking the thread. Then the thread could not treat the interruption though
it was flagged as interrupted.
  b/ The function UWSJob.isStopped() considered the job as stopped because
the interrupted flag was set, even though the thread was still processing
(and the database too). Because of that it returned true and the job phase
was ABORTED though the thread was still running.
  NOW:
  a/ TAPJob calls the function Statement.cancel() (if supported) in order
to cancel the SQL query execution properly inside the database.
  b/ The function UWSJob.isStopped() does not test any more the interrupted flag
and returns true only if the thread is really stopped.
  IN BRIEF: It is now sure that a job in the phase ABORTED is really stopped
(that's to say: thread stopped AND DB query execution stopped).

2) [TAP] BUG FIX: When the writing of a result is abnormaly interrupted for any
reason, the file which was being written is deleted.

(merge with branch 'unknownFctType')

Such value is replaced by NULL while ingesting it into a Postgres
database.

[TAP] Small fix: the translated SQL query was logged only after the query execution ; so, in case of error,
the SQL query is not logged and can not be checked by service provider.

[UWS,TAP] Errors and log management improvements. Particularly, now TAP and UWS are able to manage correctly HTTP request abortions (i.e. when the user stop the request before the response has been fully sent, or when there is a connection problem or a time-out). Such abortions are considered by UWS and TAP merely as job abortion/cancel. No error is logged any more. In addition of this correction, log entries concerning the execution of a TAP sync/async job have been modified so that having more coherents messages. And stack traces of exception that occurred when executing a job (sync or async, tap or uws) are displayed just once: at the JOB END log entry, and not by the HTTP RESPONSE_SENT entry. And finally, output flush and interruption detection are made more often when writing a query result (the flush is particularly important when combining with fetch-size > 0 in synchronous mode....the sync response is then a streaming output).

[TAP] Add the possibility to set the FetchSize on a DB Statement for the execution of ADQL queries. It is also possible to set a fetchSize different for synchronous and asynchronous queries.

[TAP] Default TAPSchema DB name set to the ADQL name ; so if the column TAP_SCHEMA.schemas.dbname is NULL or empty, the DB name must be set to the ADQL name. For the moment no empty schema is allowed.

[TAP] Qualify by default the standard TAP schema tables in TAP_SCHEMA.tables & Correct the primary key in TAP_SCHEMA.tables (it must be only 'table_name' and not 'schema_name, table_name').

[TAP] Add a new database access method in the configuration file: get a Datasource from JNDI. & Better support of connection pooling (TAPFactory.countFreeConnections() has been removed ; when the creation of a database connection fails with an SQLException, it is considered that no connection are momentarily available...async jobs will be queued and all sync requests will be rejected).

[TAP] Add an XML TableSet parser. The main modification done in JDBCConnection is about the schema prefix of table when the DBMS does not support schemas: now, only standard tables are expected with the prefix 'TAP_SCHEMA_' and the upload tables also with 'TAP_UPLOAD_'.

[TAP] Deal with the DB type 'BOOLEAN' while uploading boolean values in the database => convert the boolean in a short value (1 or 0).

[ADQL,TAP] Add geometry format in output and correct upload of STC-S regions. Geometrical type conversion from and into a DB type is now required in all JDBCTranslator. This allows formatting of geometrical column value coming from the database, but also the translation of STC-S expressions provided in uploaded table into geometrical column values.

[ADQL,TAP] Add STC-S and UDFs support in the ADQL parser. Now, it is possible to provide a list of allowed UDFs, regions and coordinate systems. The ServiceConnection of TAP is now able to provide these lists and to propagate them to the ADQLExecutor. UDFs and allowed regions are now listed automatically in the /capabilities resource of TAP. The type 'geometry' is now fully supported in ADQL. That's why the new function 'isGeometry()' has been added to all ADQLOperand extensions. Now the DBChecker is also able to check roughly types of columns and UDFs (unknown when parsing syntactically a query). The syntax of STC-S regions (expressed in the REGION function) are now checked by DBChecker. However, for the moment, geometries are not serialized in STC-S in the output....but it should be possible in some way in the next commit(s).

[UWS,TAP] Fix timestamp format. Now, each returned date will expressed in UTC and using the ISO8601 format. Any given date is also expected in UTC (or with a time zone offset) and in ISO8601Format. These rules apply also for uploaded tables and for timestamp columns in a query result.

[TAP,UWS] Manage thrown Error properly in synchronous and asynchronous jobs execution + Modify some log messages + Add log messages in JDBCConnection (for add and drop uploaded tables).

[UWS,TAP] Review the exceptions and logs management + Add log file rotation (with a configurable rotation frequency)