Skip to content
  1. Mar 20, 2019
  2. Mar 13, 2019
    • Grégory Mantelet's avatar
      [ADQL] Add to the parser a function attempting to quickly fix an ADQL query. · 15cd5944
      Grégory Mantelet authored
      This new function - ADQLParser.tryQuickFix(...) - fixes the most common issues
      with ADQL queries:
      
      - replace Unicode confusable characters by their ASCII/UTF-8 version,
      - double-quote SQL reserved words/terms (e.g. `public`, `year`, `date`),
      - double-quote ADQL function names used a column name/alias (e.g. `distance`,
        `min`, `avg`),
      - double-quote invalid regular identifiers (e.g. `_RAJ2000`, `2mass`).
      
      The last point is far from being perfect but should work at least for
      identifiers starting with a digit or an underscore, or an identifier including
      one of the following character: `?`, `!`, `$`, `@`, `#`, `{`, `}`, `[`, `]`,
      `~`, `^` and '`'.
      
      It should also been noted that double-quoting a column/table name will make it
      case-sensitive. Then, it is possible that the query does not pass even after the
      double-quote operation ; the case would have to be checked by the user.
      
      Finally, there is no attempt to fix column and table names (i.e. case
      sensitivity and/or typos) using tables/columns list/metadata. That could be a
      possible evolution of this function or an additional feature to implement in the
      parser.
      15cd5944
  3. Mar 08, 2019
    • Grégory Mantelet's avatar
      [UWS,TAP] Externalize `org.json`. · 93701deb
      Grégory Mantelet authored
      `org.json` was included in the sources of UWS-/TAP-Lib because of a manual
      small correction. This is no longer needed. So this library is now a dependency
      (included in the generated JAR). This has also been upgraded to the version of
      August 2018.
      
      _This commit aims to reproduce the original content of the Pull Request #38
       by @vforchi ._
      93701deb
  4. Mar 06, 2019
  5. Mar 05, 2019
  6. Feb 19, 2019
  7. Jan 28, 2019
  8. Nov 13, 2018
  9. Nov 08, 2018
  10. Nov 07, 2018
    • Grégory Mantelet's avatar
      [TAP] Fix memory issues for the 'text/plain' format. · 3e65794d
      Grégory Mantelet authored
      The previous text formatting process was storing the entire table in memory....
      hence OutOfMemoryError when dealing with large table.
      
      Now, this process is done entirely in memory only for a table having less than
      1000 lines. For a larger table, its content is stored in a temporary file. This
      file is deleted after usage or in case of error.
      
      This formatting process has been tested under JVM monitoring (both JConsole and
      VisualVM) and tables larger than 3,000,000 rows, with success.
      
      This commit fixes the GitHub issue #40
      3e65794d
  11. Nov 02, 2018
    • Grégory Mantelet's avatar
      [TAP] Fix cancellation of a text formatting process. · f912c20a
      Grégory Mantelet authored
      Before this fix, cancelling a TAP job (async or not) which was formatting the
      result in ASCII may failed, especially for large results. This was due to a
      non interruptible alignment process. This process is now checking whether a
      cancellation has been requested before formatting a new result line/row ; if so,
      the process is immediately stopped and the job can be cleaning declared as
      aborted.
      f912c20a
  12. Oct 26, 2018
    • Grégory Mantelet's avatar
      fb4cbcfd
    • Grégory Mantelet's avatar
      [TAP] Fix the TAP_SCHEMA mapping. · a5406513
      Grégory Mantelet authored
      When defining in the configuration file a different name for TAP_SCHEMA content,
      the service implementor was also forced to define the same mapping in the
      database with the column `dbName`.
      
      This is no longer necessary. From now on, the `dbName` column will be ignored
      for all standard TAP_SCHEMA content. Instead, the name specified in the
      configuration file (if any) will be used instead. This way, the mapping for
      standard TAP_SCHEMA content is only specified once and at only one place:
      the configuration file.
      
      _This commit resolves the GitHub issue #98_
      a5406513
  13. Oct 22, 2018
  14. Sep 21, 2018
  15. Sep 19, 2018
  16. Sep 14, 2018
  17. Sep 05, 2018
    • Grégory Mantelet's avatar
      [UWS,TAP] Fix the configuration file for the UPLOAD feature. · 7bd91a1c
      Grégory Mantelet authored
      The property `upload_default_db_limit` has been deprecated. Indeed, in the
      current state of the TAP protocol, this makes no sense: the user can not change
      the limit size (in bytes or rows) for uploaded tables.
      
      The property `upload_max_file_size` has been deprecated. It is actually
      duplicated: `upload_max_db_limit`, if expressed in bytes already lets put a
      limit on the maximum size of an uploaded table/file.
      
      The property `upload_max_request_size` has been added. It lets set a maximum
      size for a whole HTTP Multipart Request. By default it is set to 250MB.
      
      The default value of `upload_max_db_size` is now 1 million rows.
      
      The UPLOAD feature is still disabled by default (i.e. `upload_enabled=false`).
      7bd91a1c
    • Grégory Mantelet's avatar
      [UWS,TAP] Fix destruction of a job: error files were never deleted. · 02a4a7f5
      Grégory Mantelet authored
      This bug occurred "just" due to an un-desired inverted test, since UWS-1.1 is
      implemented (UWSLib-4.3 and TAPLib-2.2).
      02a4a7f5
  18. Aug 21, 2018
  19. Aug 20, 2018
  20. Aug 10, 2018
    • Grégory Mantelet's avatar
      [TAP] Return a clear error message in case of duplicated items in UPLOADs. · e447a487
      Grégory Mantelet authored
      Before correction, if two uploaded tables have been submitted by the user with
      the same name, or if one uploaded table contained duplicated column names, an
      obscure error message coming from the database was returned to the user.
      
      Now, duplicated items (tables and columns) are searched before ingestion in the
      database. When one is detected, an error is immediately returned to the user and
      the query is aborted.
      e447a487
    • Grégory Mantelet's avatar
      [TAP] Fix disk space consumption with UPLOAD for synchronous jobs. · 25f373f6
      Grégory Mantelet authored
      Files uploaded by the user when creating/executing a synchronous job were never
      deleted after the job execution.
      
      The same problem applied for the tables already uploaded in the database (in
      `TAP_UPLOAD`) when an error occurred before the end of the UPLOAD process.
      
      Now, in case of error when uploading one or more files, or in case of success of
      the job, all uploaded files and their corresponding database tables are deleted
      after the end of the job.
      25f373f6
    • Grégory Mantelet's avatar
  21. Aug 07, 2018
  22. Jul 30, 2018
    • Grégory Mantelet's avatar
      [UWS] Fix recurrent ConcurrentModificationException during service overload. · ded9cead
      Grégory Mantelet authored
      This error occurred generally during the backup process while trying to
      backup the job list of a specific user. If several of his jobs were running
      and changing state during the backup process, this
      ConcurrentModificationException was thrown. This generally happens when the same
      user submits a lot of shorts jobs in the same time.
      
      This exception was due to a non thread-safe usage of
      UWSParameters.additionalParams. To fix this issue, instead of creating it as a
      normal HashMap, it is now created as a ConcurrentHashMap.
      
      The same modification has also been applied to UWSParameters.params. In addition
      of the replacement of HashMap into ConcurrentHashMap, all `synchronized` blocks
      have been removed....there should not be needed any more.
      ded9cead
  23. Jul 27, 2018
    • Grégory Mantelet's avatar
      [UWS,TAP] Fix the weekly log file rotation. · 1f4bc6b1
      Grégory Mantelet authored
      When enabled, it was generating a file each minute on the day before the
      specified day of week.
      
      For instance: if the log rotation frequency was `W 1 0 0` (so, weekly on Sunday
      at 00:00). The rotation was performed on Saturday midnight. But, because of a
      bad index correction, the rotation kept going on the whole day of Saturday.
      Since the rotated file is suffixed by the timestamp with hours and minutes
      (no seconds), it actually generated a new log file for each minute of the
      saturday. Of course, each time the file contained only one line (or 2 with some
      luck)...which is pretty useless.
      1f4bc6b1
  24. Jul 04, 2018