- * Checks the existence of tables and columns, but also adds database metadata - * on {@link ADQLTable} and {@link ADQLColumn} instances when they are resolved. + * In addition to check the existence of tables and columns referenced in the query, + * this checked will also attach database metadata on these references ({@link ADQLTable} + * and {@link ADQLColumn} instances when they are resolved. *
* *These information are:
@@ -59,35 +91,242 @@ import adql.search.SimpleSearchHandler; * * *Note: - * Knowing DB metadata of {@link ADQLTable} and {@link ADQLColumn} is particularly useful for the translation of the ADQL query to SQL, because the ADQL name of columns and tables - * can be replaced in SQL by their DB name, if different. This mapping is done automatically by {@link adql.translator.PostgreSQLTranslator}. + * Knowing DB metadata of {@link ADQLTable} and {@link ADQLColumn} is particularly useful for the translation of the ADQL query to SQL, + * because the ADQL name of columns and tables can be replaced in SQL by their DB name, if different. This mapping is done automatically + * by {@link adql.translator.JDBCTranslator}. *
* * @author Grégory Mantelet (CDS;ARI) - * @version 1.2 (04/2014) + * @version 1.3 (05/2015) */ public class DBChecker implements QueryChecker { /** List of all available tables ({@link DBTable}). */ protected SearchTableList lstTables; + /**List of all allowed geometrical functions (i.e. CONTAINS, REGION, POINT, COORD2, ...).
+ *+ * If this list is NULL, all geometrical functions are allowed. + * However, if not, all items of this list must be the only allowed geometrical functions. + * So, if the list is empty, no such function is allowed. + *
+ * @since 1.3 */ + protected String[] allowedGeo = null; + + /**List of all allowed coordinate systems.
+ *+ * Each item of this list must be of the form: "{frame} {refpos} {flavor}". + * Each of these 3 items can be either of value, a list of values expressed with the syntax "({value1}|{value2}|...)" + * or a '*' to mean all possible values. + *
+ *Note: since a default value (corresponding to the empty string - '') should always be possible for each part of a coordinate system, + * the checker will always add the default value (UNKNOWNFRAME, UNKNOWNREFPOS or SPHERICAL2) into the given list of possible values for each coord. sys. part.
+ *+ * If this list is NULL, all coordinates systems are allowed. + * However, if not, all items of this list must be the only allowed coordinate systems. + * So, if the list is empty, none is allowed. + *
+ * @since 1.3 */ + protected String[] allowedCoordSys = null; + + /**A regular expression built using the list of allowed coordinate systems. + * With this regex, it is possible to known whether a coordinate system expression is allowed or not.
+ *If NULL, all coordinate systems are allowed.
+ * @since 1.3 */ + protected String coordSysRegExp = null; + + /**List of all allowed User Defined Functions (UDFs).
+ *+ * If this list is NULL, any encountered UDF will be allowed. + * However, if not, all items of this list must be the only allowed UDFs. + * So, if the list is empty, no UDF is allowed. + *
+ * @since 1.3 */ + protected FunctionDef[] allowedUdfs = null; + /* ************ */ /* CONSTRUCTORS */ /* ************ */ /** - * Builds a {@link DBChecker} with an empty list of tables. + *Builds a {@link DBChecker} with an empty list of tables.
+ * + *Verifications done by this object after creation:
+ *Builds a {@link DBChecker} with the given list of known tables.
+ * + *Verifications done by this object after creation:
+ *Builds a {@link DBChecker} with the given list of known tables and with a restricted list of user defined functions.
+ * + *Verifications done by this object after creation:
+ *Builds a {@link DBChecker} with the given list of known tables and with a restricted list of user defined functions.
+ * + *Verifications done by this object after creation:
+ *Builds a {@link DBChecker}.
+ * + *Verifications done by this object after creation:
+ *Note: * Only if the given collection is NOT an instance of {@link SearchTableList}, - * the collection will be copied inside a new {@link SearchTableList}. + * the collection will be copied inside a new {@link SearchTableList}, otherwise it is used as provided. *
* * @param tables List of {@link DBTable}s. */ - public final void setTables(final Collection
- * Map<DBTable,ADQLTable> mapTables;
- *
- * For each ADQLTable t
- * if (t.isSubQuery())
- * dbTable = generateDBTable(t.getSubQuery, t.getAlias());
- * else
- * dbTable = resolveTable(t);
- * t.setDBLink(dbTable);
- * dbTables.put(t, dbTable);
- * End
- *
- * For each SelectAllColumns c
- * table = c.getAdqlTable();
- * if (table != null){
- * dbTable = resolveTable(table);
- * if (dbTable == null)
- * dbTable = query.getFrom().getTablesByAlias(table.getTableName(), table.isCaseSensitive(IdentifierField.TABLE));
- * if (dbTable == null)
- * throw new UnresolvedTableException(table);
- * table.setDBLink(dbTable);
- * }
- * End
- *
- * SearchColumnList list = query.getFrom().getDBColumns();
- *
- * For each ADQLColumn c
- * dbColumn = resolveColumn(c, list);
- * c.setDBLink(dbColumn);
- * c.setAdqlTable(mapTables.get(dbColumn.getTable()));
- * End
- *
- * For each ColumnReference colRef
- * checkColumnReference(colRef, query.getSelect(), list);
- * End
- *
+ * Process several (semantic) verifications in the given ADQL query.
+ * + *Main verifications done in this function:
+ *Check DB items (tables and columns) used in the given ADQL query.
+ * + *Operations done in this function:
+ *Search all table references inside the given query, resolve them against the available tables, and if there is only one match, + * attach the matching metadata to them.
+ * + * Management of sub-query tables + *+ * If a table is not a DB table reference but a sub-query, this latter is first checked (using {@link #check(ADQLQuery, Stack)} ; + * but the father list must not contain tables of the given query, because on the same level) and then corresponding table metadata + * are generated (using {@link #generateDBTable(ADQLQuery, String)}) and attached to it. + *
+ * + * Management of "{table}.*" in the SELECT clause + *+ * For each of this SELECT item, this function tries to resolve the table name. If only one match is found, the corresponding ADQL table object + * is got from the list of resolved tables and attached to this SELECT item (thus, the joker item will also have the good metadata, + * particularly if the referenced table is a sub-query). + *
+ * + * @param query Query in which the existence of tables must be checked. + * @param fathersList List of all columns available in the father queries and that should be accessed in sub-queries. + * Each item of this stack is a list of columns available in each father-level query. + * Note: this parameter is NULL if this function is called with the root/father query as parameter. + * @param errors List of errors to complete in this function each time an unknown table or column is encountered. + * + * @return An associative map of all the resolved tables. + */ + protected MapSearch all column references inside the given query, resolve them thanks to the given tables' metadata, + * and if there is only one match, attach the matching metadata to them.
+ * + * Management of selected columns' references + *+ * A column reference is not only a direct reference to a table column using a column name. + * It can also be a reference to an item of the SELECT clause (which will then call a "selected column"). + * That kind of reference can be either an index (an unsigned integer starting from 1 to N, where N is the + * number selected columns), or the name/alias of the column. + *
+ *+ * These references are also checked, in a second step, in this function. Thus, column metadata are + * also attached to them, as common columns. + *
+ * + * @param query Query in which the existence of tables must be checked. + * @param fathersList List of all columns available in the father queries and that should be accessed in sub-queries. + * Each item of this stack is a list of columns available in each father-level query. + * Note: this parameter is NULL if this function is called with the root/father query as parameter. + * @param mapTables List of all resolved tables. + * @param list List of column metadata to complete in this function each time a column reference is resolved. + * @param errors List of errors to complete in this function each time an unknown table or column is encountered. + */ + protected void resolveColumns(final ADQLQuery query, final StackResolve the given column, that's to say search for the corresponding {@link DBColumn}.
* - * @return The corresponding {@link DBTable} if found, null otherwise. - * - * @throws ParseException An {@link UnresolvedTableException} if the given table can't be resolved. - */ - protected DBTable resolveTable(final ADQLTable table) throws ParseException{ - ArrayListResolves the given column, that's to say searches for the corresponding {@link DBColumn}.
- *The third parameter is used only if this function is called inside a subquery. In this case, - * column is tried to be resolved with the first list (dbColumns). If no match is found, - * the resolution is tried with the father columns list (fatherColumns).
+ *+ * The third parameter is used only if this function is called inside a sub-query. In this case, + * the column is tried to be resolved with the first list (dbColumns). If no match is found, + * the resolution is tried with the father columns list (fathersList). + *
* * @param column The column to resolve. * @param dbColumns List of all available {@link DBColumn}s. - * @param fathersList List of all columns available in the father query ; a list for each father-level. + * @param fathersList List of all columns available in the father queries and that should be accessed in sub-queries. + * Each item of this stack is a list of columns available in each father-level query. + * Note: this parameter is NULL if this function is called with the root/father query as parameter. * * @return The corresponding {@link DBColumn} if found. Otherwise an exception is thrown. * @@ -386,14 +696,14 @@ public class DBChecker implements QueryChecker { } /** - * Checks whether the given column reference corresponds to a selected item (column or an expression with an alias) + * Check whether the given column reference corresponds to a selected item (column or an expression with an alias) * or to an existing column. * - * @param colRef The column reference which must be checked. - * @param select The SELECT clause of the ADQL query. - * @param dbColumns The list of all available {@link DBColumn}s. + * @param colRef The column reference which must be checked. + * @param select The SELECT clause of the ADQL query. + * @param dbColumns The list of all available columns. * - * @return The corresponding {@link DBColumn} if this reference is actually the name of a column, null otherwise. + * @return The corresponding {@link DBColumn} if this reference is actually the name of a column, null otherwise. * * @throws ParseException An {@link UnresolvedColumnException} if the given column can't be resolved * or an {@link UnresolvedTableException} if its table reference can't be resolved. @@ -431,19 +741,16 @@ public class DBChecker implements QueryChecker { } } - /* ************************************* */ - /* DBTABLE & DBCOLUMN GENERATION METHODS */ - /* ************************************* */ /** - * Generates a {@link DBTable} corresponding to the given sub-query with the given table name. - * This {@link DBTable} which contains all {@link DBColumn} returned by {@link ADQLQuery#getResultingColumns()}. + * Generate a {@link DBTable} corresponding to the given sub-query with the given table name. + * This {@link DBTable} will contain all {@link DBColumn} returned by {@link ADQLQuery#getResultingColumns()}. * * @param subQuery Sub-query in which the specified table must be searched. * @param tableName Name of the table to search. * - * @return The corresponding {@link DBTable} if the table has been found in the given sub-query, null otherwise. + * @return The corresponding {@link DBTable} if the table has been found in the given sub-query, null otherwise. * - * @throws ParseException Can be used to explain why the table has not been found. + * @throws ParseException Can be used to explain why the table has not been found. Note: not used by default. */ public static DBTable generateDBTable(final ADQLQuery subQuery, final String tableName) throws ParseException{ DefaultDBTable dbTable = new DefaultDBTable(tableName); @@ -455,6 +762,473 @@ public class DBChecker implements QueryChecker { return dbTable; } + /* ************************* */ + /* CHECKING METHODS FOR UDFs */ + /* ************************* */ + + /** + *Search all UDFs (User Defined Functions) inside the given query, and then + * check their signature against the list of allowed UDFs.
+ * + *Note: + * When more than one allowed function match, the function is considered as correct + * and no error is added. + * However, in case of multiple matches, the return type of matching functions could + * be different and in this case, there would be an error while checking later + * the types. In such case, throwing an error could make sense, but the user would + * then need to cast some parameters to help the parser identifying the right function. + * But the type-casting ability is not yet possible in ADQL. + *
+ * + * @param query Query in which UDFs must be checked. + * @param errors List of errors to complete in this function each time a UDF does not match to any of the allowed UDFs. + * + * @since 1.3 + */ + protected void checkUDFs(final ADQLQuery query, final UnresolvedIdentifiersException errors){ + // 1. Search all UDFs: + ISearchHandler sHandler = new SearchUDFHandler(); + sHandler.search(query); + + // If no UDF are allowed, throw immediately an error: + if (allowedUdfs.length == 0){ + for(ADQLObject result : sHandler) + errors.addException(new UnresolvedFunctionException((UserDefinedFunction)result)); + } + // 2. Try to resolve all of them: + else{ + ArrayListTell whether the type of all parameters of the given ADQL function + * is resolved.
+ * + *A parameter type may not be resolved for 2 main reasons:
+ *Check all geometries.
+ * + *Operations done in this function:
+ *Check whether the specified geometrical function is allowed by this implementation.
+ * + *Note: + * If the list of allowed geometrical functions is empty, this function will always add an errors to the given list. + * Indeed, it means that no geometrical function is allowed and so that the specified function is automatically not supported. + *
+ * + * @param fctName Name of the geometrical function to test. + * @param fct The function instance being or containing the geometrical function to check. Note: this function can be the function to test or a function embedding the function under test (i.e. RegionFunction). + * @param binSearch The object to use in order to search a function name inside the list of allowed functions. + * It is able to perform a binary search inside a sorted array of String objects. The interest of + * this object is its compare function which must be overridden and tells how to compare the item + * to search and the items of the array (basically, a non-case-sensitive comparison between 2 strings). + * @param errors List of errors to complete in this function each time a geometrical function is not supported. + * + * @since 1.3 + */ + protected void checkGeometryFunction(final String fctName, final ADQLFunction fct, final BinarySearchSearch all explicit coordinate system declarations, check their syntax and whether they are allowed by this implementation.
+ * + *Note: + * "explicit" means here that all {@link StringConstant} instances. Only coordinate systems expressed as string can + * be parsed and so checked. So if a coordinate system is specified by a column, no check can be done at this stage... + * it will be possible to perform such test only at the execution. + *
+ * + * @param query Query in which coordinate systems must be checked. + * @param errors List of errors to complete in this function each time a coordinate system has a wrong syntax or is not supported. + * + * @see #checkCoordinateSystem(StringConstant, UnresolvedIdentifiersException) + * + * @since 1.3 + */ + protected void resolveCoordinateSystems(final ADQLQuery query, final UnresolvedIdentifiersException errors){ + ISearchHandler sHandler = new SearchCoordSysHandler(); + sHandler.search(query); + for(ADQLObject result : sHandler) + checkCoordinateSystem((StringConstant)result, errors); + } + + /** + * Parse and then check the coordinate system contained in the given {@link StringConstant} instance. + * + * @param adqlCoordSys The {@link StringConstant} object containing the coordinate system to check. + * @param errors List of errors to complete in this function each time a coordinate system has a wrong syntax or is not supported. + * + * @see STCS#parseCoordSys(String) + * @see #checkCoordinateSystem(adql.db.STCS.CoordSys, ADQLOperand, UnresolvedIdentifiersException) + * + * @since 1.3 + */ + protected void checkCoordinateSystem(final StringConstant adqlCoordSys, final UnresolvedIdentifiersException errors){ + String coordSysStr = adqlCoordSys.getValue(); + try{ + checkCoordinateSystem(STCS.parseCoordSys(coordSysStr), adqlCoordSys, errors); + }catch(ParseException pe){ + errors.addException(new ParseException(pe.getMessage())); // TODO Missing object position! + } + } + + /** + * Check whether the given coordinate system is allowed by this implementation. + * + * @param coordSys Coordinate system to test. + * @param operand The operand representing or containing the coordinate system under test. + * @param errors List of errors to complete in this function each time a coordinate system is not supported. + * + * @since 1.3 + */ + protected void checkCoordinateSystem(final CoordSys coordSys, final ADQLOperand operand, final UnresolvedIdentifiersException errors){ + if (coordSysRegExp != null && coordSys != null && !coordSys.toFullSTCS().matches(coordSysRegExp)) + errors.addException(new ParseException("Coordinate system \"" + ((operand instanceof StringConstant) ? ((StringConstant)operand).getValue() : coordSys.toString()) + "\" (= \"" + coordSys.toFullSTCS() + "\") not allowed in this implementation.")); // TODO Missing object position! + List of accepted coordinate systems + } + + /** + *Search all STC-S expressions inside the given query, parse them (and so check their syntax) and then determine + * whether the declared coordinate system and the expressed region are allowed in this implementation.
+ * + *Note: + * In the current ADQL language definition, STC-S expressions can be found only as only parameter of the REGION function. + *
+ * + * @param query Query in which STC-S expressions must be checked. + * @param binSearch The object to use in order to search a region name inside the list of allowed functions/regions. + * It is able to perform a binary search inside a sorted array of String objects. The interest of + * this object is its compare function which must be overridden and tells how to compare the item + * to search and the items of the array (basically, a non-case-sensitive comparison between 2 strings). + * @param errors List of errors to complete in this function each time the STC-S syntax is wrong or each time the declared coordinate system or region is not supported. + * + * @see STCS#parseRegion(String) + * @see #checkRegion(adql.db.STCS.Region, RegionFunction, BinarySearch, UnresolvedIdentifiersException) + * + * @since 1.3 + */ + protected void resolveSTCSExpressions(final ADQLQuery query, final BinarySearchCheck the given region.
+ * + *The following points are checked in this function:
+ *Search all operands whose the type is not yet known and try to resolve it now + * and to check whether it matches the type expected by the syntactic parser.
+ * + *+ * Only two operands may have an unresolved type: columns and user defined functions. + * Indeed, their type can be resolved only if the list of available columns and UDFs is known, + * and if columns and UDFs used in the query are resolved successfully. + *
+ * + *+ * When an operand type is still unknown, they will own the three kinds of type and + * so this function won't raise an error: it is thus automatically on the expected type. + * This behavior is perfectly correct because if the type is not resolved + * that means the item/operand has not been resolved in the previous steps and so that + * an error about this item has already been raised. + *
+ * + *Important note: + * This function does not check the types exactly, but just roughly by considering only three categories: + * string, numeric and geometry. + *
+ * + * @param query Query in which unknown types must be resolved and checked. + * @param errors List of errors to complete in this function each time a types does not match to the expected one. + * + * @see UnknownType + * + * @since 1.3 + */ + protected void checkTypes(final ADQLQuery query, final UnresolvedIdentifiersException errors){ + // Search all unknown types: + ISearchHandler sHandler = new SearchUnknownTypeHandler(); + sHandler.search(query); + + // Check whether their type matches the expected one: + UnknownType unknown; + for(ADQLObject result : sHandler){ + unknown = (UnknownType)result; + switch(unknown.getExpectedType()){ + case 'G': + case 'g': + if (!unknown.isGeometry()) + errors.addException(new ParseException("Type mismatch! A geometry was expected instead of \"" + unknown.toADQL() + "\".")); // TODO Add the ADQLOperand position! + break; + case 'N': + case 'n': + if (!unknown.isNumeric()) + errors.addException(new ParseException("Type mismatch! A numeric value was expected instead of \"" + unknown.toADQL() + "\".")); // TODO Add the ADQLOperand position! + break; + case 'S': + case 's': + if (!unknown.isString()) + errors.addException(new ParseException("Type mismatch! A string value was expected instead of \"" + unknown.toADQL() + "\".")); // TODO Add the ADQLOperand position! + break; + } + } + } + + /* ******************************** */ + /* METHODS CHECKING THE SUB-QUERIES */ + /* ******************************** */ + + /** + *Search all sub-queries found in the given query but not in the clause FROM. + * These sub-queries are then checked using {@link #check(ADQLQuery, Stack)}.
+ * + * Fathers stack + *+ * Each time a sub-query must be checked with {@link #check(ADQLQuery, Stack)}, + * the list of all columns available in each of its father queries must be provided. + * This function is composing itself this stack by adding the given list of available + * columns (= all columns resolved in the given query) at the end of the given stack. + * If this stack is given empty, then a new stack is created. + *
+ *+ * This modification of the given stack is just the execution time of this function. + * Before returning, this function removes the last item of the stack. + *
+ * + * + * @param query Query in which sub-queries must be checked. + * @param fathersList List of all columns available in the father queries and that should be accessed in sub-queries. + * Each item of this stack is a list of columns available in each father-level query. + * Note: this parameter is NULL if this function is called with the root/father query as parameter. + * @param availableColumns List of all columns resolved in the given query. + * @param errors List of errors to complete in this function each time a semantic error is encountered. + * + * @since 1.3 + */ + protected void checkSubQueries(final ADQLQuery query, StackLet replacing every {@link DefaultUDF}s whose a {@link FunctionDef} is set by their corresponding {@link UserDefinedFunction} class.
+ * + *Important note: + * If the replacer can not be created using the class returned by {@link FunctionDef#getUDFClass()}, no replacement is performed. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (02/2015) + * @since 1.3 + */ + private static class ReplaceDefaultUDFHandler extends SimpleReplaceHandler { + private final UnresolvedIdentifiersException errors; + + public ReplaceDefaultUDFHandler(final UnresolvedIdentifiersException errorsContainer){ + errors = errorsContainer; + } + + @Override + protected boolean match(ADQLObject obj){ + return (obj.getClass().getName().equals(DefaultUDF.class.getName())) && (((DefaultUDF)obj).getDefinition() != null) && (((DefaultUDF)obj).getDefinition().getUDFClass() != null); + /* Note: detection of DefaultUDF is done on the exact class name rather than using "instanceof" in order to have only direct instances of DefaultUDF, + * and not extensions of it. Indeed, DefaultUDFs are generally created automatically by the ADQLQueryFactory ; so, extensions of it can only be custom + * UserDefinedFunctions. */ + } + + @Override + protected ADQLObject getReplacer(ADQLObject objToReplace) throws UnsupportedOperationException{ + try{ + // get the associated UDF class: + Class udfClass = ((DefaultUDF)objToReplace).getDefinition().getUDFClass(); + // get the constructor with a single parameter of type ADQLOperand[]: + Constructor constructor = udfClass.getConstructor(ADQLOperand[].class); + // create a new instance of this UDF class with the operands stored in the object to replace: + return constructor.newInstance((Object)(((DefaultUDF)objToReplace).getParameters())); /* note: without this class, each item of the given array will be considered as a single parameter. */ + }catch(Exception ex){ + // IF NO INSTANCE CAN BE CREATED... + // ...keep the error for further report: + errors.addException(new UnresolvedFunctionException("Impossible to represent the function \"" + ((DefaultUDF)objToReplace).getName() + "\": the following error occured while creating this representation: \"" + ((ex instanceof InvocationTargetException) ? "[" + ex.getCause().getClass().getSimpleName() + "] " + ex.getCause().getMessage() : ex.getMessage()) + "\"", (DefaultUDF)objToReplace)); + // ...keep the same object (i.e. no replacement): + return objToReplace; + } + } + } + + /** + * Let searching geometrical functions. + * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + private static class SearchGeometryHandler extends SimpleSearchHandler { + @Override + protected boolean match(ADQLObject obj){ + return (obj instanceof GeometryFunction); + } + } + + /** + *Let searching all ADQL objects whose the type was not known while checking the syntax of the ADQL query. + * These objects are {@link ADQLColumn}s and {@link UserDefinedFunction}s.
+ * + *Important note: + * Only {@link UnknownType} instances having an expected type equals to 'S' (or 's' ; for string) or 'N' (or 'n' ; for numeric) + * are kept by this handler. Others are ignored. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + private static class SearchUnknownTypeHandler extends SimpleSearchHandler { + @Override + protected boolean match(ADQLObject obj){ + if (obj instanceof UnknownType){ + char expected = ((UnknownType)obj).getExpectedType(); + return (expected == 'G' || expected == 'g' || expected == 'S' || expected == 's' || expected == 'N' || expected == 'n'); + }else + return false; + } + } + + /** + * Let searching all explicit declaration of coordinate systems. + * So, only {@link StringConstant} objects will be returned. + * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + private static class SearchCoordSysHandler extends SimpleSearchHandler { + @Override + protected boolean match(ADQLObject obj){ + if (obj instanceof PointFunction || obj instanceof BoxFunction || obj instanceof CircleFunction || obj instanceof PolygonFunction) + return (((GeometryFunction)obj).getCoordinateSystem() instanceof StringConstant); + else + return false; + } + + @Override + protected void addMatch(ADQLObject matchObj, ADQLIterator it){ + results.add(((GeometryFunction)matchObj).getCoordinateSystem()); + } + + } + + /** + * Let searching all {@link RegionFunction}s. + * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + private static class SearchRegionHandler extends SimpleSearchHandler { + @Override + protected boolean match(ADQLObject obj){ + if (obj instanceof RegionFunction) + return (((RegionFunction)obj).getParameter(0) instanceof StringConstant); + else + return false; + } + + } + + /** + *Implement the binary search algorithm over a sorted array.
+ * + *+ * The only difference with the standard implementation of Java is + * that this object lets perform research with a different type + * of object than the types of array items. + *
+ * + *+ * For that reason, the "compare" function must always be implemented. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * + * @paramSearch the given item in the given array.
+ * + *+ * In case the given object matches to several items of the array, + * this function will return the smallest index, pointing thus to the first + * of all matches. + *
+ * + * @param searchItem Object for which a corresponding array item must be searched. + * @param array Array in which the given object must be searched. + * + * @return The array index of the first item of all matches. + */ + public int search(final S searchItem, final T[] array){ + s = 0; + e = array.length - 1; + while(s < e){ + // middle of the sorted array: + m = s + ((e - s) / 2); + // compare the fct with the middle item of the array: + comp = compare(searchItem, array[m]); + // if the fct is after, trigger the inspection of the right part of the array: + if (comp > 0) + s = m + 1; + // otherwise, the left part: + else + e = m; + } + if (s != e || compare(searchItem, array[s]) != 0) + return -1; + else + return s; + } + + /** + * Compare the search item and the array item. + * + * @param searchItem Item whose a corresponding value must be found in the array. + * @param arrayItem An item of the array. + * + * @return Negative value if searchItem is less than arrayItem, 0 if they are equals, or a positive value if searchItem is greater. + */ + protected abstract int compare(final S searchItem, final T arrayItem); + } + } diff --git a/src/adql/db/DBColumn.java b/src/adql/db/DBColumn.java index a803717e449bb7d1bf8594a3aee51a1b632eb2f9..c987e062cc161f5b20488c8e4652accc00455463 100644 --- a/src/adql/db/DBColumn.java +++ b/src/adql/db/DBColumn.java @@ -16,7 +16,8 @@ package adql.db; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeGet the type of this column (as closed as possible from the "database" type).
+ * + *Note: + * The returned type should be as closed as possible from a type listed by the IVOA in the TAP protocol description into the section UPLOAD. + *
+ * + * @return Its type. + * + * @since 1.3 + */ + public DBType getDatatype(); + /** * Gets the table which contains this {@link DBColumn}. * diff --git a/src/adql/db/DBCommonColumn.java b/src/adql/db/DBCommonColumn.java index fbbc73deaa2bf277b3a55ad77e25208f6dbb4274..44c6642ad34a01ddf654ec1d99c018136650d3d5 100644 --- a/src/adql/db/DBCommonColumn.java +++ b/src/adql/db/DBCommonColumn.java @@ -16,12 +16,13 @@ package adql.db; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeMakes a copy of this instance of {@link DBTable}, with the possibility to change the DB and ADQL names.
+ * + *IMPORTANT:
+ * The given DB and ADQL name may be NULL. If NULL, the copy will contain exactly the same full name (DB and/or ADQL).
+ * And they may be qualified (that's to say: prefixed by the schema name or by the catalog and schema name). It means that it is possible to
+ * change the catalog, schema and table name in the copy.
+ * For instance:
+ *
+ * Describe a full column type as it is described in the IVOA document of TAP.
+ * Thus, this object contains 2 attributes: type (or datatype) and length (or size).
+ *
The length/size may be not defined ; in this case, its value is set to {@link #NO_LENGTH} or is negative or null.
+ * + *All datatypes declared in the IVOA recommendation document of TAP are listed in an enumeration type: {@link DBDatatype}. + * It is used to set the attribute type/datatype of this class.
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ +public class DBType { + + /** + * List of all datatypes declared in the IVOA recommendation of TAP (in the section UPLOAD). + * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + public static enum DBDatatype{ + SMALLINT, INTEGER, BIGINT, REAL, DOUBLE, BINARY, VARBINARY, CHAR, VARCHAR, BLOB, CLOB, TIMESTAMP, POINT, REGION; + } + + /** Special value in case no length/size is specified. */ + public static final int NO_LENGTH = -1; + + /** Datatype of a column. */ + public final DBDatatype type; + + /** The length parameter (only few datatypes need this parameter: char, varchar, binary and varbinary). */ + public final int length; + + /** + * Build a TAP column type by specifying a datatype. + * + * @param datatype Column datatype. + */ + public DBType(final DBDatatype datatype){ + this(datatype, NO_LENGTH); + } + + /** + * Build a TAP column type by specifying a datatype and a length (needed only for datatypes like char, varchar, binary and varbinary). + * + * @param datatype Column datatype. + * @param length Length of the column value (needed only for datatypes like char, varchar, binary and varbinary). + */ + public DBType(final DBDatatype datatype, final int length){ + if (datatype == null) + throw new NullPointerException("Missing TAP column datatype !"); + this.type = datatype; + this.length = length; + } + + public boolean isNumeric(){ + switch(type){ + case SMALLINT: + case INTEGER: + case BIGINT: + case REAL: + case DOUBLE: + /* Note: binaries are also included here because they can also be considered as Numeric, + * but not for JOINs. */ + case BINARY: + case VARBINARY: + case BLOB: + return true; + default: + return false; + } + } + + public boolean isBinary(){ + switch(type){ + case BINARY: + case VARBINARY: + case BLOB: + return true; + default: + return false; + } + } + + public boolean isString(){ + switch(type){ + case CHAR: + case VARCHAR: + case CLOB: + case TIMESTAMP: + return true; + default: + return false; + } + } + + public boolean isGeometry(){ + return (type == DBDatatype.POINT || type == DBDatatype.REGION); + } + + public boolean isCompatible(final DBType t){ + if (t == null) + return false; + else if (isBinary() == t.isBinary()) + return (type == DBDatatype.BLOB && t.type == DBDatatype.BLOB) || (type != DBDatatype.BLOB && t.type != DBDatatype.BLOB); + else if (isNumeric() == t.isNumeric()) + return true; + else if (isGeometry() == t.isGeometry()) + return (type == t.type); + else if (isString()) + return (type == DBDatatype.CLOB && t.type == DBDatatype.CLOB) || (type != DBDatatype.CLOB && t.type != DBDatatype.CLOB); + else + return (type == t.type); + } + + @Override + public String toString(){ + if (length > 0) + return type + "(" + length + ")"; + else + return type.toString(); + } + +} diff --git a/src/adql/db/DefaultDBColumn.java b/src/adql/db/DefaultDBColumn.java index 8496501aa3ee7ecc3fb15619c8bddecafb98e3ef..a4ed9e3bc557bc8cba391bc68c35e269dbf27ea9 100644 --- a/src/adql/db/DefaultDBColumn.java +++ b/src/adql/db/DefaultDBColumn.java @@ -16,20 +16,27 @@ package adql.db; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeSet the type of this column.
+ * + *Note 1: + * The given type should be as closed as possible from a type listed by the IVOA in the TAP protocol description into the section UPLOAD. + *
+ * + *Note 2: + * there is no default value. Consequently if this parameter is NULL, + * the type should be considered as unknown. It means that any comparison with + * any type will always return 'true'. + *
+ * + * @param type New type of this column. + * + * @since 1.3 + */ + public final void setDatatype(final DBType type){ + this.type = type; + } + + @Override public final String getDBName(){ return dbName; } + @Override public final DBTable getTable(){ return table; } @@ -84,8 +164,9 @@ public class DefaultDBColumn implements DBColumn { this.table = table; } + @Override public DBColumn copy(final String dbName, final String adqlName, final DBTable dbTable){ - return new DefaultDBColumn(dbName, adqlName, dbTable); + return new DefaultDBColumn(dbName, adqlName, type, dbTable); } } diff --git a/src/adql/db/DefaultDBTable.java b/src/adql/db/DefaultDBTable.java index baf71400273ce0d32a193328d7b20833cf6aded6..ccc3752df7a56eb33715da0eda46ca0f5fc38d81 100644 --- a/src/adql/db/DefaultDBTable.java +++ b/src/adql/db/DefaultDBTable.java @@ -17,18 +17,19 @@ package adql.db; * along with ADQLLibrary. If not, seeBuilds a default {@link DBTable} with the given DB name.
@@ -247,8 +248,52 @@ public class DefaultDBTable implements DBTable { return splitRes; } + /** + *Join the last 3 items of the given string array with a dot ('.'). + * These three parts should be: [0]=catalog name, [1]=schema name, [2]=table name.
+ * + *+ * If the array contains less than 3 items, all the given items will be though joined. + * However, if it contains more than 3 items, only the three last items will be. + *
+ * + *A null item will be written as an empty string (string of length 0 ; "").
+ * + *+ * In the case the first and the third items are not null, but the second is null, the final string will contain in the middle two dots. + * Example: if the array is {"cat", NULL, "table"}, then the joined string will be: "cat..table". + *
+ * + * @param nameParts String items to join. + * + * @return A string joining the 3 last string items of the given array, + * or an empty string if the given array is NULL. + * + * @since 1.3 + */ + public static final String joinTableName(final String[] nameParts){ + if (nameParts == null) + return ""; + + StringBuffer str = new StringBuffer(); + boolean empty = true; + for(int i = (nameParts.length <= 3) ? 0 : (nameParts.length - 3); i < nameParts.length; i++){ + if (!empty) + str.append('.'); + + String part = (nameParts[i] == null) ? null : nameParts[i].trim(); + if (part != null && part.length() > 0){ + str.append(part); + empty = false; + } + } + return str.toString(); + } + @Override - public DBTable copy(final String dbName, final String adqlName){ + public DBTable copy(String dbName, String adqlName){ + dbName = (dbName == null) ? joinTableName(new String[]{dbCatalogName,dbSchemaName,this.dbName}) : dbName; + adqlName = (adqlName == null) ? joinTableName(new String[]{adqlCatalogName,adqlSchemaName,this.adqlName}) : adqlName; DefaultDBTable copy = new DefaultDBTable(dbName, adqlName); for(DBColumn col : this){ if (col instanceof DBCommonColumn) @@ -258,5 +303,4 @@ public class DefaultDBTable implements DBTable { } return copy; } - } diff --git a/src/adql/db/FunctionDef.java b/src/adql/db/FunctionDef.java new file mode 100644 index 0000000000000000000000000000000000000000..82107d0a232a33d2e1d113c795304261f1bfda66 --- /dev/null +++ b/src/adql/db/FunctionDef.java @@ -0,0 +1,541 @@ +package adql.db; + +/* + * This file is part of ADQLLibrary. + * + * ADQLLibrary is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * ADQLLibrary is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with ADQLLibrary. If not, seeDefinition of any function that could be used in ADQL queries.
+ * + *+ * A such definition can be built manually thanks to the different constructors of this class, + * or by parsing a string function definition form using the static function {@link #parse(String)}. + *
+ * + *+ * The syntax of the expression expected by {@link #parse(String)} is the same as the one used to build + * the string returned by {@link #toString()}. Here is this syntax: + *
+ *{fctName}([{param1Name} {param1Type}, ...])[ -> {returnType}]
+ *
+ * + * A description of this function may be set thanks to the public class attribute {@link #description}. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (02/2015) + * + * @since 1.3 + */ +public class FunctionDef implements ComparableString representation of this function.
+ *The syntax of this representation is the following (items between brackets are optional):
+ *{fctName}([{param1Name} {param1Type}, ...])[ -> {returnType}] */
+ private final String serializedForm;
+
+ /** String representation of this function dedicated to comparison with any function signature.
+ *This form is different from the serialized form on the following points:
+ *So the syntax of this form is the following (items between brackets are optional ; xxx is a string of 3 characters, each being either 0 or 1):
+ *{fctName}([xxx, ...]) */
+ private final String compareForm;
+
+ /**
+ * Class of the {@link UserDefinedFunction} which must represent the UDF defined by this {@link FunctionDef} in the ADQL tree.
+ *This class MUST have a constructor with a single parameter of type {@link ADQLOperand}[].
+ *If this {@link FunctionDef} is defining an ordinary ADQL function, this attribute must be NULL. It is used only for user defined functions.
+ */ + private Class udfClass = null; + + /** + *Definition of a function parameter.
+ * + *This definition is composed of two items: the name and the type of the parameter.
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + public static final class FunctionParam { + /** Parameter name. Ensured not null */ + public final String name; + /** Parameter type. Ensured not null */ + public final DBType type; + + /** + * Create a function parameter. + * + * @param paramName Name of the parameter to create. MUST NOT be NULL + * @param paramType Type of the parameter to create. MUST NOT be NULL + */ + public FunctionParam(final String paramName, final DBType paramType){ + if (paramName == null) + throw new NullPointerException("Missing name! The function parameter can not be created."); + if (paramType == null) + throw new NullPointerException("Missing type! The function parameter can not be created."); + this.name = paramName; + this.type = paramType; + } + } + + /** + *Create a function definition.
+ * + *The created function will have no return type and no parameter.
+ * + * @param fctName Name of the function. + */ + public FunctionDef(final String fctName){ + this(fctName, null, null); + } + + /** + *Create a function definition.
+ * + *The created function will have a return type (if the provided one is not null) and no parameter.
+ * + * @param fctName Name of the function. + * @param returnType Return type of the function. If NULL, this function will have no return type + */ + public FunctionDef(final String fctName, final DBType returnType){ + this(fctName, returnType, null); + } + + /** + *Create a function definition.
+ * + *The created function will have no return type and some parameters (except if the given array is NULL or empty).
+ * + * @param fctName Name of the function. + * @param params Parameters of this function. If NULL or empty, this function will have no parameter. + */ + public FunctionDef(final String fctName, final FunctionParam[] params){ + this(fctName, null, params); + } + + public FunctionDef(final String fctName, final DBType returnType, final FunctionParam[] params){ + // Set the name: + if (fctName == null) + throw new NullPointerException("Missing name! Can not create this function definition."); + this.name = fctName; + + // Set the parameters: + this.params = (params == null || params.length == 0) ? null : params; + this.nbParams = (params == null) ? 0 : params.length; + + // Set the return type; + this.returnType = returnType; + if (returnType != null){ + isNumeric = returnType.isNumeric(); + isString = returnType.isString(); + isGeometry = returnType.isGeometry(); + }else + isNumeric = isString = isGeometry = false; + + // Serialize in Strings (serializedForm and compareForm) this function definition: + StringBuffer bufSer = new StringBuffer(name), bufCmp = new StringBuffer(name.toLowerCase()); + bufSer.append('('); + for(int i = 0; i < nbParams; i++){ + bufSer.append(params[i].name).append(' ').append(params[i].type); + bufCmp.append(params[i].type.isNumeric() ? '1' : '0').append(params[i].type.isString() ? '1' : '0').append(params[i].type.isGeometry() ? '1' : '0'); + if (i + 1 < nbParams) + bufSer.append(", "); + } + bufSer.append(')'); + if (returnType != null) + bufSer.append(" -> ").append(returnType); + serializedForm = bufSer.toString(); + compareForm = bufCmp.toString(); + } + + /** + * Tell whether this function returns a numeric. + * + * @return true if this function returns a numeric, false otherwise. + */ + public final boolean isNumeric(){ + return isNumeric; + } + + /** + * Tell whether this function returns a string. + * + * @return true if this function returns a string, false otherwise. + */ + public final boolean isString(){ + return isString; + } + + /** + * Tell whether this function returns a geometry. + * + * @return true if this function returns a geometry, false otherwise. + */ + public final boolean isGeometry(){ + return isGeometry; + } + + /** + * Get the number of parameters required by this function. + * + * @return Number of required parameters. + */ + public final int getNbParams(){ + return nbParams; + } + + /** + * Get the definition of the indParam-th parameter of this function. + * + * @param indParam Index of the parameter whose the definition must be returned. + * + * @return Definition of the specified parameter. + * + * @throws ArrayIndexOutOfBoundsException If the given index is negative or bigger than the number of parameters. + */ + public final FunctionParam getParam(final int indParam) throws ArrayIndexOutOfBoundsException{ + if (indParam < 0 || indParam >= nbParams) + throw new ArrayIndexOutOfBoundsException(indParam); + else + return params[indParam]; + } + + /** + *Get the class of the {@link UserDefinedFunction} able to represent the function defined here in an ADQL tree.
+ * + *Note:
+ * This getter should return always NULL if the function defined here is not a user defined function.
+ *
+ * However, if this {@link FunctionDef} is defining a user defined function and this function returns NULL,
+ * the library will create on the fly a {@link DefaultUDF} corresponding to this definition when needed.
+ * Indeed this UDF class is useful only if the translation from ADQL (to SQL for instance) of the defined
+ * function has a different signature (e.g. a different name) in the target language (e.g. SQL).
+ *
Set the class of the {@link UserDefinedFunction} able to represent the function defined here in an ADQL tree.
+ * + *Note:
+ * If this {@link FunctionDef} defines an ordinary ADQL function - and not a user defined function - no class should be set here.
+ *
+ * However, if it defines a user defined function, there is no obligation to set a UDF class. It is useful only if the translation
+ * from ADQL (to SQL for instance) of the function has a different signature (e.g. a different name) in the target language (e.g. SQL).
+ * If the signature is the same, there is no need to set a UDF class ; a {@link DefaultUDF} will be created on the fly by the library
+ * when needed if it turns out that no UDF class is set.
+ *
Let parsing the serialized form of a function definition.
+ * + *The expected syntax is (items between brackets are optional):
+ *{fctName}([{param1Name} {param1Type}, ...])[ -> {returnType}]
+ *
+ * + * Allowed parameter types and return types should be one the types listed by the UPLOAD section of the TAP recommendation document. + * These types are listed in the enumeration object {@link DBType}. + * However, other types should be accepted like the common database types...but it should be better to not rely on that + * since the conversion of those types to TAP types should not be exactly what is expected. + *
+ * + * @param strDefinition Serialized function definition to parse. + * + * @return The object representation of the given string definition. + * + * @throws ParseException If the given string has a wrong syntax or uses unknown types. + */ + public static FunctionDef parse(final String strDefinition) throws ParseException{ + if (strDefinition == null) + throw new NullPointerException("Missing string definition to build a FunctionDef!"); + + // Check the global syntax of the function definition: + Matcher m = fctPattern.matcher(strDefinition); + if (m.matches()){ + + // Get the function name: + String fctName = m.group(1); + + // Parse and get the return type: + DBType returnType = null; + if (m.group(3) != null){ + returnType = parseType(m.group(5), (m.group(7) == null) ? DBType.NO_LENGTH : Integer.parseInt(m.group(7))); + if (returnType == null) + throw new ParseException("Unknown return type: \"" + m.group(4).trim() + "\"!"); + } + + // Get the parameters, if any: + String paramsList = m.group(2); + FunctionParam[] params = null; + if (paramsList != null && paramsList.trim().length() > 0){ + + // Check the syntax of the parameters' list: + if (!paramsList.matches(fctParamsRegExp)) + throw new ParseException("Wrong parameters syntax! Expected syntax: \"(Compare this function definition with the given ADQL function item.
+ * + *+ * The comparison is done only on the function name and on rough type of the parameters. + * "Rough type" means here that just the kind of type is tested: numeric, string or geometry. + * Anyway, the return type is never tested by this function, since such information is usually + * not part of a function signature. + *
+ * + *The notion of "greater" and "less" are defined here according to the three following test steps:
+ *This class helps dealing with the subset of STC-S expressions described by the section "6 Use of STC-S in TAP (informative)" + * of the TAP Recommendation 1.0 (27th March 2010). This subset is limited to the most common coordinate systems and regions.
+ * + *Note: + * No instance of this class can be created. Its usage is only limited to its static functions and classes. + *
+ * + *+ * The function {@link #parseCoordSys(String)} is able to parse a string containing only the STC-S expression of a coordinate system + * (or an empty string or null which would be interpreted as the default coordinate system - UNKNOWNFRAME UNKNOWNREFPOS SPHERICAL2). + * When successful, this parsing returns an object representation of the coordinate system: {@link CoordSys}. + *
+ *+ * To serialize into STC-S a coordinate system, you have to create a {@link CoordSys} instance with the desired values + * and to call the function {@link CoordSys#toSTCS()}. The static function {@link #toSTCS(CoordSys)} is just calling the + * {@link CoordSys#toSTCS()} on the given coordinate system. + *
+ * + *+ * As for the coordinate system, there is a static function to parse the STC-S representation of a geometrical region: {@link #parseRegion(String)}. + * Here again, when the parsing is successful an object representation is returned: {@link Region}. + *
+ *+ * This class lets also serializing into STC-S a region. The procedure is the same as with a coordinate system: create a {@link Region} and then + * call {@link Region#toString()}. + *
+ *+ * The class {@link Region} lets also dealing with the {@link ADQLFunction} implementing a region. It is then possible to create a {@link Region} + * object from a such {@link ADQLFunction} and to get the corresponding STC-S representation. The static function {@link #toSTCS(GeometryFunction)} + * is a helpful function which do these both actions in once. + *
+ *Note: + * The conversion from {@link ADQLFunction} to {@link Region} or STC-S is possible only if the {@link ADQLFunction} contains constants as parameter. + * Thus, a such function using a column, a concatenation, a math operation or using another function can not be converted into STC-S using this class. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (12/2014) + * @since 1.3 + */ +public final class STCS { + + /** + * Empty private constructor ; in order to prevent any instance creation. + */ + private STCS(){} + + /* ***************** */ + /* COORDINATE SYSTEM */ + /* ***************** */ + + /** Regular expression for a STC-S representation of a coordinate system. It takes into account the fact that each part of + * a coordinate system is optional and so that a full coordinate system expression can be reduced to an empty string. */ + private final static String coordSysRegExp = Frame.regexp + "?\\s*" + RefPos.regexp + "?\\s*" + Flavor.regexp + "?"; + /** Regular expression of an expression exclusively limited to a coordinate system. */ + private final static String onlyCoordSysRegExp = "^\\s*" + coordSysRegExp + "\\s*$"; + /** Regular expression of a default coordinate system: either an empty string or a string containing only default values. */ + private final static String defaultCoordSysRegExp = "^\\s*" + Frame.DEFAULT + "?\\s*" + RefPos.DEFAULT + "?\\s*" + Flavor.DEFAULT + "?\\s*$"; + /** Regular expression of a pattern describing a set of allowed coordinate systems. See {@link #buildAllowedRegExp(String)} for more details. */ + /* With this regular expression, we get the following matching groups: + * 0: All the expression + * 1+(6*N): The N-th part of the coordinate system (N is an unsigned integer between 0 and 2 (included) ; it is reduced to '*' if the two following groups are NULL + * 2+(6*N): A single value for the N-th part + * 3+(6*N): A list of values for the N-th part + * 4+(6*N): First value of the list for the N-th part + * 5+(6*N): All the other values (starting with a |) of the list for the N-th part + * 6+(6*N): Last value of the list for the N-th part. + */ + private final static String allowedCoordSysRegExp = "^\\s*" + buildAllowedRegExp(Frame.regexp) + "\\s+" + buildAllowedRegExp(RefPos.regexp) + "\\s+" + buildAllowedRegExp(Flavor.regexp) + "\\s*$"; + + /** Pattern of an allowed coordinate system pattern. This object has been compiled with {@link #allowedCoordSysRegExp}. */ + private final static Pattern allowedCoordSysPattern = Pattern.compile(allowedCoordSysRegExp); + + /** Human description of the syntax of a full coordinate system expression. */ + private final static String COORD_SYS_SYNTAX = "\"[" + Frame.regexp + "] [" + RefPos.regexp + "] [" + Flavor.regexp + "]\" ; an empty string is also allowed and will be interpreted as the coordinate system locally used"; + + /** + * Build the regular expression of a string defining the allowed values for one part of the whole coordinate system. + * + * @param rootRegExp All allowed part values. + * + * @return The corresponding regular expression. + */ + private static String buildAllowedRegExp(final String rootRegExp){ + return "(" + rootRegExp + "|\\*|(\\(\\s*" + rootRegExp + "\\s*(\\|\\s*" + rootRegExp + "\\s*)*\\)))"; + } + + /** + *List of all possible frames in an STC expression.
+ * + *+ * When no value is specified, the default one is {@link #UNKNOWNFRAME}. + * The default value is also accessible through the attribute {@link #DEFAULT} + * and it is possible to test whether a frame is the default with the function {@link #isDefault()}. + *
+ * + *Note: + * The possible values listed in this enumeration are limited to the subset of STC-S described by the section "6 Use of STC-S in TAP (informative)" + * of the TAP Recommendation 1.0 (27th March 2010). + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + public static enum Frame{ + ECLIPTIC, FK4, FK5, GALACTIC, ICRS, UNKNOWNFRAME; + + /** Default value for a frame: {@link #UNKNOWNFRAME}. */ + public static final Frame DEFAULT = UNKNOWNFRAME; + + /** Regular expression to test whether a string is a valid frame or not. This regular expression does not take into account + * the case of an empty string (which means "default frame"). */ + public static final String regexp = buildRegexp(Frame.class); + + /** + * Tell whether this frame is the default one. + * + * @return true if this is the default frame, false + */ + public final boolean isDefault(){ + return this == DEFAULT; + } + } + + /** + *List of all possible reference positions in an STC expression.
+ * + *+ * When no value is specified, the default one is {@link #UNKNOWNREFPOS}. + * The default value is also accessible through the attribute {@link #DEFAULT} + * and it is possible to test whether a reference position is the default with the function {@link #isDefault()}. + *
+ * + *Note: + * The possible values listed in this enumeration are limited to the subset of STC-S described by the section "6 Use of STC-S in TAP (informative)" + * of the TAP Recommendation 1.0 (27th March 2010). + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + public static enum RefPos{ + BARYCENTER, GEOCENTER, HELIOCENTER, LSR, TOPOCENTER, RELOCATABLE, UNKNOWNREFPOS; + + /** Default value for a reference position: {@link #UNKNOWNREFPOS}. */ + public static final RefPos DEFAULT = UNKNOWNREFPOS; + + /** Regular expression to test whether a string is a valid reference position or not. This regular expression does not take into account + * the case of an empty string (which means "default reference position"). */ + public static final String regexp = buildRegexp(RefPos.class); + + /** + * Tell whether this reference position is the default one. + * + * @return true if this is the default reference position, false + */ + public final boolean isDefault(){ + return this == DEFAULT; + } + } + + /** + *List of all possible flavors in an STC expression.
+ * + *+ * When no value is specified, the default one is {@link #SPHERICAL2}. + * The default value is also accessible through the attribute {@link #DEFAULT} + * and it is possible to test whether a flavor is the default with the function {@link #isDefault()}. + *
+ * + *Note: + * The possible values listed in this enumeration are limited to the subset of STC-S described by the section "6 Use of STC-S in TAP (informative)" + * of the TAP Recommendation 1.0 (27th March 2010). + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + public static enum Flavor{ + CARTESIAN2, CARTESIAN3, SPHERICAL2; + + /** Default value for a flavor: {@link #SPHERICAL2}. */ + public static final Flavor DEFAULT = SPHERICAL2; + + /** Regular expression to test whether a string is a valid flavor or not. This regular expression does not take into account + * the case of an empty string (which means "default flavor"). */ + public static final String regexp = buildRegexp(Flavor.class); + + /** + * Tell whether this flavor is the default one. + * + * @return true if this is the default flavor, false + */ + public final boolean isDefault(){ + return this == DEFAULT; + } + } + + /** + * Build a regular expression covering all possible values of the given enumeration. + * + * @param enumType Class of an enumeration type. + * + * @return The build regular expression or "\s*" if the given enumeration contains no constants/values. + * + * @throws IllegalArgumentException If the given class is not an enumeration type. + */ + private static String buildRegexp(final Class enumType) throws IllegalArgumentException{ + // The given class must be an enumeration type: + if (!enumType.isEnum()) + throw new IllegalArgumentException("An enum class was expected, but a " + enumType.getName() + " has been given!"); + + // Get the enumeration constants/values: + Object[] constants = enumType.getEnumConstants(); + if (constants == null || constants.length == 0) + return "\\s*"; + + // Concatenate all constants with pipe to build a choice regular expression: + StringBuffer buf = new StringBuffer("("); + for(int i = 0; i < constants.length; i++){ + buf.append(constants[i]); + if ((i + 1) < constants.length) + buf.append('|'); + } + return buf.append(')').toString(); + } + + /** + *Object representation of an STC coordinate system.
+ * + *+ * A coordinate system is composed of three parts: a frame ({@link #frame}), + * a reference position ({@link #refpos}) and a flavor ({@link #flavor}). + *
+ * + *+ * The default value - also corresponding to an empty string - should be: + * {@link Frame#UNKNOWNFRAME} {@link RefPos#UNKNOWNREFPOS} {@link Flavor#SPHERICAL2}. + * Once built, it is possible to know whether the coordinate system is the default one + * or not thanks to function {@link #isDefault()}. + *
+ * + *+ * An instance of this class can be easily serialized into STC-S using {@link #toSTCS()}, {@link #toFullSTCS()} + * or {@link #toString()}. {@link #toFullSTCS()} will display default values explicitly + * on the contrary to {@link #toSTCS()} which will replace them by empty strings. + *
+ * + *Important note: + * The flavors CARTESIAN2 and CARTESIAN3 can not be used with other frame and reference position than + * UNKNOWNFRAME and UNKNOWNREFPOS. In the contrary case an {@link IllegalArgumentException} is throw. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + public static class CoordSys { + /** First item of a coordinate system expression: the frame. */ + public final Frame frame; + + /** Second item of a coordinate system expression: the reference position. */ + public final RefPos refpos; + + /** Third and last item of a coordinate system expression: the flavor. */ + public final Flavor flavor; + + /** Indicate whether all parts of the coordinate system are set to their default value. */ + private final boolean isDefault; + + /** STC-S representation of this coordinate system. Default items are not written (that's to say, they are replaced by an empty string). */ + private final String stcs; + + /** STC-S representation of this coordinate system. Default items are explicitly written. */ + private final String fullStcs; + + /** + * Build a default coordinate system (UNKNOWNFRAME UNKNOWNREFPOS SPHERICAL2). + */ + public CoordSys(){ + this(null, null, null); + } + + /** + * Build a coordinate system with the given parts. + * + * @param fr Frame part. + * @param rp Reference position part. + * @param fl Flavor part. + * + * @throws IllegalArgumentException If a cartesian flavor is used with a frame and reference position other than UNKNOWNFRAME and UNKNOWNREFPOS. + */ + public CoordSys(final Frame fr, final RefPos rp, final Flavor fl) throws IllegalArgumentException{ + frame = (fr == null) ? Frame.DEFAULT : fr; + refpos = (rp == null) ? RefPos.DEFAULT : rp; + flavor = (fl == null) ? Flavor.DEFAULT : fl; + + if (flavor != Flavor.SPHERICAL2 && (frame != Frame.UNKNOWNFRAME || refpos != RefPos.UNKNOWNREFPOS)) + throw new IllegalArgumentException("a coordinate system expressed with a cartesian flavor MUST have an UNKNOWNFRAME and UNKNOWNREFPOS!"); + + isDefault = frame.isDefault() && refpos.isDefault() && flavor.isDefault(); + + stcs = ((!frame.isDefault() ? frame + " " : "") + (!refpos.isDefault() ? refpos + " " : "") + (!flavor.isDefault() ? flavor : "")).trim(); + fullStcs = frame + " " + refpos + " " + flavor; + } + + /** + * Build a coordinate system by parsing the given STC-S expression. + * + * @param coordsys STC-S expression representing a coordinate system. Empty string and NULL are allowed values ; they correspond to a default coordinate system. + * + * @throws ParseException If the syntax of the given STC-S expression is wrong or if it is not a coordinate system only. + */ + public CoordSys(final String coordsys) throws ParseException{ + CoordSys tmp = new STCSParser().parseCoordSys(coordsys); + frame = tmp.frame; + refpos = tmp.refpos; + flavor = tmp.flavor; + isDefault = tmp.isDefault; + stcs = tmp.stcs; + fullStcs = tmp.fullStcs; + } + + /** + * Tell whether this is the default coordinate system (UNKNOWNFRAME UNKNOWNREFPOS SPHERICAL2). + * + * @return true if it is the default coordinate system, false otherwise. + */ + public final boolean isDefault(){ + return isDefault; + } + + /** + * Get the STC-S expression of this coordinate system, + * in which default values are not written (they are replaced by empty strings). + * + * @return STC-S representation of this coordinate system. + */ + public String toSTCS(){ + return stcs; + } + + /** + * Get the STC-S expression of this coordinate system, + * in which default values are explicitly written. + * + * @return STC-S representation of this coordinate system. + */ + public String toFullSTCS(){ + return fullStcs; + } + + /** + * Convert this coordinate system into a STC-S expression. + * + * @see java.lang.Object#toString() + * @see #toSTCS() + */ + @Override + public String toString(){ + return stcs; + } + } + + /** + * Parse the given STC-S representation of a coordinate system. + * + * @param stcs STC-S expression of a coordinate system. Note: a NULL or empty string will be interpreted as a default coordinate system. + * + * @return The object representation of the specified coordinate system. + * + * @throws ParseException If the given expression has a wrong STC-S syntax. + */ + public static CoordSys parseCoordSys(final String stcs) throws ParseException{ + return (new STCSParser().parseCoordSys(stcs)); + } + + /** + *Convert an object representation of a coordinate system into an STC-S expression.
+ * + *Note: + * A NULL object will be interpreted as the default coordinate system and so an empty string will be returned. + * Otherwise, this function is equivalent to {@link CoordSys#toSTCS()} (in which default values for each + * coordinate system part is not displayed). + *
+ * + * @param coordSys The object representation of the coordinate system to convert into STC-S. + * + * @return The corresponding STC-S expression. + * + * @see CoordSys#toSTCS() + * @see CoordSys#toFullSTCS() + */ + public static String toSTCS(final CoordSys coordSys){ + if (coordSys == null) + return ""; + else + return coordSys.toSTCS(); + } + + /** + *Build a big regular expression gathering all of the given coordinate system syntaxes.
+ * + *
+ * Each item of the given list must respect a strict syntax. Each part of the coordinate system
+ * may be a single value, a list of values or a '*' (meaning all values are allowed).
+ * A list of values must have the following syntax: ({value1}|{value2}|...).
+ * An empty string is NOT here accepted.
+ *
Example:
+ * (ICRS|FK4|FK5) * SPHERICAL2 is OK,
+ * but (ICRS|FK4|FK5) * is not valid because the flavor value is not defined.
+ *
+ * Since the default value of each part of a coordinate system should always be possible, + * this function ensure these default values are always possible in the returned regular expression. + * Thus, if some values except the default one are specified, the default value is automatically appended. + *
+ * + *Note: + * If the given array is NULL, all coordinate systems are allowed. + * But if the given array is empty, none except an empty string or the default value will be allowed. + *
+ * + * @param allowedCoordSys List of all coordinate systems that are allowed. + * + * @return The corresponding regular expression. + * + * @throws ParseException If the syntax of one of the given allowed coordinate system is wrong. + */ + public static String buildCoordSysRegExp(final String[] allowedCoordSys) throws ParseException{ + // NULL array => all coordinate systems are allowed: + if (allowedCoordSys == null) + return onlyCoordSysRegExp; + // Empty array => no coordinate system (except the default one) is allowed: + else if (allowedCoordSys.length == 0) + return defaultCoordSysRegExp; + + // The final regular expression must be reduced to a coordinate system and nothing else before: + StringBuffer finalRegExp = new StringBuffer("^\\s*("); + + // For each allowed coordinate system: + Matcher m; + int nbCoordSys = 0; + for(int i = 0; i < allowedCoordSys.length; i++){ + + // NULL item => skipped! + if (allowedCoordSys[i] == null) + continue; + else{ + if (nbCoordSys > 0) + finalRegExp.append('|'); + nbCoordSys++; + } + + // Check its syntax and identify all of its parts: + m = allowedCoordSysPattern.matcher(allowedCoordSys[i].toUpperCase()); + if (m.matches()){ + finalRegExp.append('('); + for(int g = 0; g < 3; g++){ // See the comment after the Javadoc of #allowedCoordSysRegExp for a complete list of available groups returned by the pattern. + + // SINGLE VALUE: + if (m.group(2 + (6 * g)) != null) + finalRegExp.append('(').append(defaultChoice(g, m.group(2 + (6 * g)))).append(m.group(2 + (6 * g))).append(')'); + + // LIST OF VALUES: + else if (m.group(3 + (6 * g)) != null) + finalRegExp.append('(').append(defaultChoice(g, m.group(3 + (6 * g)))).append(m.group(3 + (6 * g)).replaceAll("\\s", "").substring(1)); + + // JOKER (*): + else{ + switch(g){ + case 0: + finalRegExp.append(Frame.regexp); + break; + case 1: + finalRegExp.append(RefPos.regexp); + break; + case 2: + finalRegExp.append(Flavor.regexp); + break; + } + finalRegExp.append('?'); + } + finalRegExp.append("\\s*"); + } + finalRegExp.append(')'); + }else + throw new ParseException("Wrong allowed coordinate system syntax for the " + (i + 1) + "-th item: \"" + allowedCoordSys[i] + "\"! Expected: \"frameRegExp refposRegExp flavorRegExp\" ; where each xxxRegExp = (xxx | '*' | '('xxx ('|' xxx)*')'), frame=\"" + Frame.regexp + "\", refpos=\"" + RefPos.regexp + "\" and flavor=\"" + Flavor.regexp + "\" ; an empty string is also allowed and will be interpreted as '*' (so all possible values)."); + } + + // The final regular expression must be reduced to a coordinate system and nothing else after: + finalRegExp.append(")\\s*$"); + + return (nbCoordSys > 0) ? finalRegExp.toString() : defaultCoordSysRegExp; + } + + /** + * Get the default value appended by a '|' character, ONLY IF the given value does not already contain the default value. + * + * @param g Index of the coordinate system part (0: Frame, 1: RefPos, 2: Flavor, another value will return an empty string). + * @param value Value in which the default value must prefix. + * + * @return A prefix for the given value (the default value and a '|' if the default value is not already in the given value, "" otherwise). + */ + private static String defaultChoice(final int g, final String value){ + switch(g){ + case 0: + return value.contains(Frame.DEFAULT.toString()) ? "" : Frame.DEFAULT + "|"; + case 1: + return value.contains(RefPos.DEFAULT.toString()) ? "" : RefPos.DEFAULT + "|"; + case 2: + return value.contains(Flavor.DEFAULT.toString()) ? "" : Flavor.DEFAULT + "|"; + default: + return ""; + } + } + + /* ****** */ + /* REGION */ + /* ****** */ + + /** + *List all possible region types allowed in an STC-S expression.
+ * + *Note: + * The possible values listed in this enumeration are limited to the subset of STC-S described by the section "6 Use of STC-S in TAP (informative)" + * of the TAP Recommendation 1.0 (27th March 2010). + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + public static enum RegionType{ + POSITION, CIRCLE, BOX, POLYGON, UNION, INTERSECTION, NOT; + } + + /** + *Object representation of an STC region.
+ * + *+ * This class contains a field for each possible parameter of a region. Depending of the region type + * some are not used. In such case, these unused fields are set to NULL. + *
+ * + *+ * An instance of this class can be easily serialized into STC-S using {@link #toSTCS()}, {@link #toFullSTCS()} + * or {@link #toString()}. {@link #toFullSTCS()} will display default value explicit + * on the contrary to {@link #toSTCS()} which will replace them by empty strings. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + public static class Region { + /** Type of the region. */ + public final RegionType type; + + /** Coordinate system used by this region. + * Note: only the NOT region does not declare a coordinate system ; so only for this region this field is NULL. */ + public final CoordSys coordSys; + + /** List of coordinates' pairs. The second dimension of this array represents a pair of coordinates ; it is then an array of two elements. + * Note: this field is used by POINT, BOX, CIRCLE and POLYGON. */ + public final double[][] coordinates; + + /** Width of the BOX region. */ + public final double width; + + /** Height of the BOX region. */ + public final double height; + + /** Radius of the CIRCLE region. */ + public final double radius; + + /** List of regions unified (UNION), intersected (INTERSECTION) or avoided (NOT). */ + public final Region[] regions; + + /** STC-S representation of this region, in which default values of the coordinate system (if any) are not written (they are replaced by empty strings). + * Note: This attribute is NULL until the first call of the function {@link #toSTCS()} where it is built. */ + private String stcs = null; + + /** STC-S representation of this region, in which default values of the coordinate system (if any) are explicitly written. + * Note: This attribute is NULL until the first call of the function {@link #toFullSTCS()} where it is built. */ + private String fullStcs = null; + + /** The ADQL function object representing this region. + * Note: this attribute is NULL until the first call of the function {@link #toGeometry()} or {@link #toGeometry(ADQLQueryFactory)}. */ + private GeometryFunction geometry = null; + + /** + *Constructor for a POINT/POSITION region.
+ * + *Important note: + * The array of coordinates is used like that. No copy is done. + *
+ * + * @param coordSys Coordinate system. note: It MAY BE null ; if so, the default coordinate system will be chosen + * @param coordinates A pair of coordinates ; coordinates[0] and coordinates[1]. + */ + public Region(final CoordSys coordSys, final double[] coordinates){ + this(coordSys, new double[][]{coordinates}); + } + + /** + *Constructor for a POINT/POSITION or a POLYGON region.
+ * + *Whether it is a polygon or a point depends on the number of given coordinates:
+ *Important note: + * The array of coordinates is used like that. No copy is done. + *
+ * + * @param coordSys Coordinate system. note: It MAY BE null ; if so, the default coordinate system will be chosen + * @param coordinates List of coordinates' pairs ; coordinates[n] = 1 pair = 2 items (coordinates[n][0] and coordinates[n][1]) ; if 1 pair, it is a POINT/POSITION, but if more, it is a POLYGON. + */ + public Region(final CoordSys coordSys, final double[][] coordinates){ + // Check roughly the coordinates: + if (coordinates == null || coordinates.length == 0) + throw new NullPointerException("Missing coordinates!"); + else if (coordinates[0].length != 2) + throw new IllegalArgumentException("Wrong number of coordinates! Expected at least 2 pairs of coordinates (so coordinates[0], coordinates[1] and coordinates[n].length = 2)."); + + // Decide of the region type in function of the number of coordinates' pairs: + type = (coordinates.length > 1) ? RegionType.POLYGON : RegionType.POSITION; + + // Set the coordinate system (if NULL, choose the default one): + this.coordSys = (coordSys == null ? new CoordSys() : coordSys); + + // Set the coordinates: + this.coordinates = coordinates; + + // Set the other fields as not used: + width = Double.NaN; + height = Double.NaN; + radius = Double.NaN; + regions = null; + } + + /** + *Constructor for a CIRCLE region.
+ * + *Important note: + * The array of coordinates is used like that. No copy is done. + *
+ * + * @param coordSys Coordinate system. note: It MAY BE null ; if so, the default coordinate system will be chosen + * @param coordinates A pair of coordinates ; coordinates[0] and coordinates[1]. + * @param radius The circle radius. + */ + public Region(final CoordSys coordSys, final double[] coordinates, final double radius){ + // Check roughly the coordinates: + if (coordinates == null || coordinates.length == 0) + throw new NullPointerException("Missing coordinates!"); + else if (coordinates.length != 2) + throw new IllegalArgumentException("Wrong number of coordinates! Expected exactly 2 values."); + + // Set the region type: + type = RegionType.CIRCLE; + + // Set the coordinate system (if NULL, choose the default one): + this.coordSys = (coordSys == null ? new CoordSys() : coordSys); + + // Set the coordinates: + this.coordinates = new double[][]{coordinates}; + + // Set the radius: + this.radius = radius; + + // Set the other fields as not used: + width = Double.NaN; + height = Double.NaN; + regions = null; + } + + /** + *Constructor for a BOX region.
+ * + *Important note: + * The array of coordinates is used like that. No copy is done. + *
+ * + * @param coordSys Coordinate system. note: It MAY BE null ; if so, the default coordinate system will be chosen + * @param coordinates A pair of coordinates ; coordinates[0] and coordinates[1]. + * @param width Width of the box. + * @param height Height of the box. + */ + public Region(final CoordSys coordSys, final double[] coordinates, final double width, final double height){ + // Check roughly the coordinates: + if (coordinates == null || coordinates.length == 0) + throw new NullPointerException("Missing coordinates!"); + else if (coordinates.length != 2) + throw new IllegalArgumentException("Wrong number of coordinates! Expected exactly 2 values."); + + // Set the region type: + type = RegionType.BOX; + + // Set the coordinate system (if NULL, choose the default one): + this.coordSys = (coordSys == null ? new CoordSys() : coordSys); + + // Set the coordinates: + this.coordinates = new double[][]{coordinates}; + + // Set the size of the box: + this.width = width; + this.height = height; + + // Set the other fields as not used: + radius = Double.NaN; + regions = null; + } + + /** + *Constructor for a UNION or INTERSECTION region.
+ * + *Important note: + * The array of regions is used like that. No copy is done. + *
+ * + * @param unionOrIntersection Type of the region to create. Note: It can be ONLY a UNION or INTERSECTION. Another value will throw an IllegalArgumentException). + * @param coordSys Coordinate system. note: It MAY BE null ; if so, the default coordinate system will be chosen + * @param regions Regions to unite or to intersect. Note: At least two regions must be provided. + */ + public Region(final RegionType unionOrIntersection, final CoordSys coordSys, final Region[] regions){ + // Check the type: + if (unionOrIntersection == null) + throw new NullPointerException("Missing type of region (UNION or INTERSECTION here)!"); + else if (unionOrIntersection != RegionType.UNION && unionOrIntersection != RegionType.INTERSECTION) + throw new IllegalArgumentException("Wrong region type: \"" + unionOrIntersection + "\"! This constructor lets create only an UNION or INTERSECTION region."); + + // Check the list of regions: + if (regions == null || regions.length == 0) + throw new NullPointerException("Missing regions to " + (unionOrIntersection == RegionType.UNION ? "unite" : "intersect") + "!"); + else if (regions.length < 2) + throw new IllegalArgumentException("Wrong number of regions! Expected at least 2 regions."); + + // Set the region type: + type = unionOrIntersection; + + // Set the coordinate system (if NULL, choose the default one): + this.coordSys = (coordSys == null ? new CoordSys() : coordSys); + + // Set the regions: + this.regions = regions; + + // Set the other fields as not used: + coordinates = null; + radius = Double.NaN; + width = Double.NaN; + height = Double.NaN; + } + + /** + * Constructor for a NOT region. + * + * @param region Any region to not select. + */ + public Region(final Region region){ + // Check the region parameter: + if (region == null) + throw new NullPointerException("Missing region to NOT select!"); + + // Set the region type: + type = RegionType.NOT; + + // Set the regions: + this.regions = new Region[]{region}; + + // Set the other fields as not used: + coordSys = null; + coordinates = null; + radius = Double.NaN; + width = Double.NaN; + height = Double.NaN; + } + + /** + *Build a Region from the given ADQL representation.
+ * + *Note: + * Only {@link PointFunction}, {@link CircleFunction}, {@link BoxFunction}, {@link PolygonFunction} and {@link RegionFunction} + * are accepted here. Other extensions of {@link GeometryFunction} will throw an {@link IllegalArgumentException}. + *
+ * + * @param geometry The ADQL representation of the region to create here. + * + * @throws IllegalArgumentException If the given geometry is neither of {@link PointFunction}, {@link BoxFunction}, {@link PolygonFunction} and {@link RegionFunction}. + * @throws ParseException If the declared coordinate system, the coordinates or the STC-S definition has a wrong syntax. + */ + public Region(final GeometryFunction geometry) throws IllegalArgumentException, ParseException{ + if (geometry == null) + throw new NullPointerException("Missing geometry to convert into STCS.Region!"); + + if (geometry instanceof PointFunction){ + type = RegionType.POSITION; + coordSys = STCS.parseCoordSys(extractString(geometry.getCoordinateSystem())); + coordinates = new double[][]{{extractNumeric(((PointFunction)geometry).getCoord1()),extractNumeric(((PointFunction)geometry).getCoord2())}}; + width = Double.NaN; + height = Double.NaN; + radius = Double.NaN; + regions = null; + }else if (geometry instanceof CircleFunction){ + type = RegionType.CIRCLE; + coordSys = STCS.parseCoordSys(extractString(geometry.getCoordinateSystem())); + coordinates = new double[][]{{extractNumeric(((CircleFunction)geometry).getCoord1()),extractNumeric(((CircleFunction)geometry).getCoord2())}}; + radius = extractNumeric(((CircleFunction)geometry).getRadius()); + width = Double.NaN; + height = Double.NaN; + regions = null; + }else if (geometry instanceof BoxFunction){ + type = RegionType.BOX; + coordSys = STCS.parseCoordSys(extractString(geometry.getCoordinateSystem())); + coordinates = new double[][]{{extractNumeric(((BoxFunction)geometry).getCoord1()),extractNumeric(((BoxFunction)geometry).getCoord2())}}; + width = extractNumeric(((BoxFunction)geometry).getWidth()); + height = extractNumeric(((BoxFunction)geometry).getHeight()); + radius = Double.NaN; + regions = null; + }else if (geometry instanceof PolygonFunction){ + PolygonFunction poly = (PolygonFunction)geometry; + type = RegionType.POLYGON; + coordSys = STCS.parseCoordSys(extractString(poly.getCoordinateSystem())); + coordinates = new double[(poly.getNbParameters() - 1) / 2][2]; + for(int i = 0; i < coordinates.length; i++) + coordinates[i] = new double[]{extractNumeric(poly.getParameter(1 + i * 2)),extractNumeric(poly.getParameter(2 + i * 2))}; + width = Double.NaN; + height = Double.NaN; + radius = Double.NaN; + regions = null; + }else if (geometry instanceof RegionFunction){ + Region r = STCS.parseRegion(extractString(((RegionFunction)geometry).getParameter(0))); + type = r.type; + coordSys = r.coordSys; + coordinates = r.coordinates; + width = r.width; + height = r.height; + radius = r.radius; + regions = r.regions; + }else + throw new IllegalArgumentException("Unknown region type! Only geometrical function PointFunction, CircleFunction, BoxFunction, PolygonFunction and RegionFunction are allowed."); + } + + /** + * Extract a string value from the given {@link ADQLOperand} + * which is expected to be a {@link StringConstant} instance. + * + * @param op A string operand. + * + * @return The string value embedded in the given operand. + * + * @throws ParseException If the given operand is not an instance of {@link StringConstant}. + */ + private static String extractString(final ADQLOperand op) throws ParseException{ + if (op == null) + throw new NullPointerException("Missing operand!"); + else if (op instanceof StringConstant) + return ((StringConstant)op).getValue(); + else + throw new ParseException("Can not convert into STC-S a non string argument (including ADQLColumn and Concatenation)!"); + } + + /** + * Extract a numeric value from the given {@link ADQLOperand} + * which is expected to be a {@link NumericConstant} instance + * or a {@link NegativeOperand} embedding a {@link NumericConstant}. + * + * @param op A numeric operand. + * + * @return The numeric value embedded in the given operand. + * + * @throws ParseException If the given operand is not an instance of {@link NumericConstant} or a {@link NegativeOperand}. + */ + private static double extractNumeric(final ADQLOperand op) throws ParseException{ + if (op == null) + throw new NullPointerException("Missing operand!"); + else if (op instanceof NumericConstant) + return Double.parseDouble(((NumericConstant)op).getValue()); + else if (op instanceof NegativeOperand) + return extractNumeric(((NegativeOperand)op).getOperand()) * -1; + else + throw new ParseException("Can not convert into STC-S a non numeric argument (including ADQLColumn and Operation)!"); + } + + /** + *Get the STC-S representation of this region (in which default values + * of the coordinate system are not written ; they are replaced by empty strings).
+ * + *Note: + * This function build the STC-S just once and store it in a class attribute. + * The value of this attribute is then returned at next calls of this function. + *
+ * + * @return Its STC-S representation. + */ + public String toSTCS(){ + if (stcs != null) + return stcs; + else{ + // Write the region type: + StringBuffer buf = new StringBuffer(type.toString()); + + // Write the coordinate system (except for NOT): + if (type != RegionType.NOT){ + String coordSysStr = coordSys.toSTCS(); + if (coordSysStr != null && coordSysStr.length() > 0) + buf.append(' ').append(coordSysStr); + buf.append(' '); + } + + // Write the other parameters (coordinates, regions, ...): + switch(type){ + case POSITION: + case POLYGON: + appendCoordinates(buf, coordinates); + break; + case CIRCLE: + appendCoordinates(buf, coordinates); + buf.append(' ').append(radius); + break; + case BOX: + appendCoordinates(buf, coordinates); + buf.append(' ').append(width).append(' ').append(height); + break; + case UNION: + case INTERSECTION: + case NOT: + buf.append('('); + appendRegions(buf, regions, false); + buf.append(')'); + break; + } + + // Return the built STC-S: + return (stcs = buf.toString()); + } + } + + /** + *Get the STC-S representation of this region (in which default values + * of the coordinate system are explicitly written).
+ * + *Note: + * This function build the STC-S just once and store it in a class attribute. + * The value of this attribute is then returned at next calls of this function. + *
+ * + * @return Its STC-S representation. + */ + public String toFullSTCS(){ + if (fullStcs != null) + return fullStcs; + else{ + // Write the region type: + StringBuffer buf = new StringBuffer(type.toString()); + + // Write the coordinate system (except for NOT): + if (type != RegionType.NOT){ + String coordSysStr = coordSys.toFullSTCS(); + if (coordSysStr != null && coordSysStr.length() > 0) + buf.append(' ').append(coordSysStr); + buf.append(' '); + } + + // Write the other parameters (coordinates, regions, ...): + switch(type){ + case POSITION: + case POLYGON: + appendCoordinates(buf, coordinates); + break; + case CIRCLE: + appendCoordinates(buf, coordinates); + buf.append(' ').append(radius); + break; + case BOX: + appendCoordinates(buf, coordinates); + buf.append(' ').append(width).append(' ').append(height); + break; + case UNION: + case INTERSECTION: + case NOT: + buf.append('('); + appendRegions(buf, regions, true); + buf.append(')'); + break; + } + + // Return the built STC-S: + return (fullStcs = buf.toString()); + } + } + + /** + * Append all the given coordinates to the given buffer. + * + * @param buf Buffer in which coordinates must be appended. + * @param coords Coordinates to append. + */ + private static void appendCoordinates(final StringBuffer buf, final double[][] coords){ + for(int i = 0; i < coords.length; i++){ + if (i > 0) + buf.append(' '); + buf.append(coords[i][0]).append(' ').append(coords[i][1]); + } + } + + /** + * Append all the given regions in the given buffer. + * + * @param buf Buffer in which regions must be appended. + * @param regions Regions to append. + * @param fullCoordSys Indicate whether the coordinate system of the regions must explicitly display the default values. + */ + private static void appendRegions(final StringBuffer buf, final Region[] regions, final boolean fullCoordSys){ + for(int i = 0; i < regions.length; i++){ + if (i > 0) + buf.append(' '); + if (fullCoordSys) + buf.append(regions[i].toFullSTCS()); + else + buf.append(regions[i].toSTCS()); + } + } + + @Override + public String toString(){ + return toSTCS(); + } + + /** + *Convert this region into its corresponding ADQL representation.
+ * + *Note: + * This function is using the default ADQL factory, built using {@link ADQLQueryFactory#ADQLQueryFactory()}. + *
+ * + * @return The corresponding ADQL representation. + * + * @see #toGeometry(ADQLQueryFactory) + */ + public GeometryFunction toGeometry(){ + return toGeometry(null); + } + + /** + *Convert this region into its corresponding ADQL representation.
+ * + *Note: + * This function build the ADQL representation just once and store it in a class attribute. + * The value of this attribute is then returned at next calls of this function. + *
+ * + * @param factory The factory of ADQL objects to use. + * + * @return The corresponding ADQL representation. + */ + public GeometryFunction toGeometry(ADQLQueryFactory factory){ + if (factory == null) + factory = new ADQLQueryFactory(); + + try{ + if (geometry != null) + return geometry; + else{ + StringConstant coordSysObj = factory.createStringConstant(coordSys == null ? "" : coordSys.toString()); + switch(type){ + case POSITION: + return (geometry = factory.createPoint(coordSysObj, toNumericObj(coordinates[0][0], factory), toNumericObj(coordinates[0][1], factory))); + case CIRCLE: + return (geometry = factory.createCircle(coordSysObj, toNumericObj(coordinates[0][0], factory), toNumericObj(coordinates[0][1], factory), toNumericObj(radius, factory))); + case BOX: + return (geometry = factory.createBox(coordSysObj, toNumericObj(coordinates[0][0], factory), toNumericObj(coordinates[0][1], factory), toNumericObj(width, factory), toNumericObj(height, factory))); + case POLYGON: + ArrayListConvert a numeric value into an ADQL representation:
+ * + *Convert into STC-S the given ADQL representation of a geometrical function.
+ * + *Important note: + * Only {@link PointFunction}, {@link CircleFunction}, {@link BoxFunction}, {@link PolygonFunction} + * and {@link RegionFunction} are accepted here. Other extensions of {@link GeometryFunction} will + * throw an {@link IllegalArgumentException}. + *
+ * + * @param region ADQL representation of the region to convert into STC-S. + * + * @return The corresponding STC-S expression. + * + * @throws ParseException If the given object is NULL or not of the good type. + */ + public static String toSTCS(final GeometryFunction region) throws ParseException{ + if (region == null) + throw new NullPointerException("Missing region to serialize into STC-S!"); + return (new Region(region)).toSTCS(); + } + + /* *************************** */ + /* PARSER OF STC-S EXPRESSIONS */ + /* *************************** */ + + /** + * Let parse any STC-S expression. + * + * @author Grégory Mantelet (ARI) + * @version 1.3 (11/2014) + * @since 1.3 + */ + private static class STCSParser { + /** Regular expression of a numerical value. */ + private final static String numericRegExp = "(\\+|-)?(\\d+(\\.\\d*)?|\\.\\d+)([Ee](\\+|-)?\\d+)?"; + + /** Position of the next characters to read in the STC-S expression to parse. */ + private int pos; + /** Full STC-S expression to parse. */ + private String stcs; + /** Last read token (can be a numeric, a string, a region type, ...). */ + private String token; + /** Buffer used to read tokens. */ + private StringBuffer buffer; + + /** + * Exception sent when the end of the expression + * (EOE = End Of Expression) is reached. + * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ + private static class EOEException extends ParseException { + private static final long serialVersionUID = 1L; + + /** Build a simple EOEException. */ + public EOEException(){ + super("Unexpected End Of Expression!"); + } + } + + /** + * Build the STC-S parser. + */ + public STCSParser(){} + + /** + * Parse the given STC-S expression, expected as a coordinate system. + * + * @param stcs The STC-S expression to parse. + * + * @return The corresponding object representation of the specified coordinate system. + * + * @throws ParseException If the syntax of the given STC-S expression is wrong or if it is not a coordinate system. + */ + public CoordSys parseCoordSys(final String stcs) throws ParseException{ + init(stcs); + CoordSys coordsys = null; + try{ + coordsys = coordSys(); + end(COORD_SYS_SYNTAX); + return coordsys; + }catch(EOEException ex){ + ex.printStackTrace(); + return new CoordSys(); + } + } + + /** + * Parse the given STC-S expression, expected as a geometrical region. + * + * @param stcs The STC-S expression to parse. + * + * @return The corresponding object representation of the specified geometrical region. + * + * @throws ParseException If the syntax of the given STC-S expression is wrong or if it is not a geometrical region. + */ + public Region parseRegion(final String stcs) throws ParseException{ + init(stcs); + Region region = region(); + end("\"POSITIONGet the next meaningful word. This word can be a numeric, any string constant or a region type.
+ * + *+ * In case the end of the expression is reached before getting any meaningful character, an {@link EOEException} is thrown. + *
+ * + * @return The full read word/token. + * + * @throws EOEException If the end of the STC-S expression is reached before getting any meaningful character. + */ + private String nextToken() throws EOEException{ + // Skip all spaces: + skipSpaces(); + + // Fetch all characters until word separator (a space or a open/close parenthesis): + while(pos < stcs.length() && !Character.isWhitespace(stcs.charAt(pos)) && stcs.charAt(pos) != '(' && stcs.charAt(pos) != ')') + buffer.append(stcs.charAt(pos++)); + + // If no character has been fetched while at least one was expected, throw an exception: + if (buffer.length() == 0) + throw new EOEException(); + + // Save the read token and reset the buffer: + token = buffer.toString(); + buffer.delete(0, token.length()); + + return token; + } + + /** + * Read the next token as a numeric. + * If not a numeric, a {@link ParseException} is thrown. + * + * @return The read numerical value. + * + * @throws ParseException If the next token is not a numerical expression. + */ + private double numeric() throws ParseException{ + if (nextToken().matches(numericRegExp)) + return Double.parseDouble(token); + else + throw new ParseException("a numeric was expected!", new TextPosition(1, pos - token.length(), 1, pos)); // TODO Check the begin and end! + } + + /** + * Read the next 2 tokens as a coordinate pairs (so as 2 numerical values). + * If not 2 numeric, a {@link ParseException} is thrown. + * + * @return The read coordinate pairs. + * + * @throws ParseException If the next 2 tokens are not 2 numerical expressions. + */ + private double[] coordPair() throws ParseException{ + skipSpaces(); + int startPos = pos; + try{ + return new double[]{numeric(),numeric()}; + }catch(ParseException pe){ + if (pe instanceof EOEException) + throw pe; + else + throw new ParseException("a coordinates pair (2 numerics separated by one or more spaces) was expected!", new TextPosition(1, startPos, 1, pos)); // TODO Check the begin and end! + } + } + + /** + * Read and parse the next tokens as a coordinate system expression. + * If they do not match, a {@link ParseException} is thrown. + * + * @return The object representation of the read coordinate system. + * + * @throws ParseException If the next tokens are not representing a valid coordinate system. + */ + private CoordSys coordSys() throws ParseException{ + // Skip all spaces: + skipSpaces(); + + // Backup the current position: + /* (because every parts of a coordinate system are optional ; + * like this, it will be possible to go back in the expression + * to parse if optional parts are not written) */ + String oldToken = token; + int startPos = pos; + + Frame fr = null; + RefPos rp = null; + Flavor fl = null; + + try{ + // Read the token: + nextToken(); + // Try to parse it as a frame: + if ((fr = frame()) != null){ + // if success, go the next token: + startPos = pos; + oldToken = token; + nextToken(); + } + // Try to parse the last read token as a reference position: + if ((rp = refpos()) != null){ + // if success, go the next token: + startPos = pos; + oldToken = token; + nextToken(); + } + // Try to parse the last read token as a flavor: + if ((fl = flavor()) == null){ + // if NOT a success, go back "in time" (go back to the position before reading the token): + pos = startPos; + token = oldToken; + } + }catch(EOEException ex){ + /* End Of Expression may happen here since all parts of a coordinate system are optional. + * So, there is no need to treat the error. */ + } + + // Build the object representation of the read coordinate system: + /* Note: if nothing has been read for one or all parts of the coordinate system, + * the NULL value will be replaced automatically in the constructor + * by the default value of the corresponding part(s). */ + try{ + return new CoordSys(fr, rp, fl); + }catch(IllegalArgumentException iae){ + throw new ParseException(iae.getMessage(), new TextPosition(1, startPos, 1, pos)); + } + } + + /** + * Parse the last read token as FRAME. + * + * @return The corresponding enumeration item, or NULL if the last token is not a valid FRAME item. + */ + private Frame frame(){ + try{ + return Frame.valueOf(token.toUpperCase()); + }catch(IllegalArgumentException iae){ + return null; + } + } + + /** + * Parse the last read token as REFERENCE POSITION. + * + * @return The corresponding enumeration item, or NULL if the last token is not a valid REFERENCE POSITION item. + */ + private RefPos refpos(){ + try{ + return RefPos.valueOf(token.toUpperCase()); + }catch(IllegalArgumentException iae){ + return null; + } + } + + /** + * Parse the last read token as FLAVOR. + * + * @return The corresponding enumeration item, or NULL if the last token is not a valid FLAVOR item. + */ + private Flavor flavor(){ + try{ + return Flavor.valueOf(token.toUpperCase()); + }catch(IllegalArgumentException iae){ + return null; + } + } + + /** + * Read and parse the next tokens as a geometrical region. + * If they do not match, a {@link ParseException} is thrown. + * + * @return The object representation of the read geometrical region. + * + * @throws ParseException If the next tokens are not representing a valid geometrical region. + */ + private Region region() throws ParseException{ + // Skip all spaces: + skipSpaces(); + + // Read the next token (it should be the region type): + int startPos = pos; + token = nextToken().toUpperCase(); + + /* Identify the region type, next the expected parameters and finally build the corresponding object representation */ + // POSITION case: + if (token.equals("POSITION")){ + try{ + CoordSys coordSys = coordSys(); + double[] coords = coordPair(); + return new Region(coordSys, coords); + }catch(Exception e){ + throw buildException(e, "\"POSITIONGet the signature of the function given in parameter.
+ * + *+ * In this signature, just the name and the type of all the parameters are written. + * The return type is never part of a function signature. + *
+ * + *Note 1: + * A parameter type can be either "NUMERIC", "STRING" or "GEOMETRY". In order to be the most generic has possible, + * no more precision about a type is returned here. If the parameter is none of these type kinds, "???" is returned. + *
+ * + *Note 2: + * If the given object is NULL, an empty string is returned. + *
+ * + * @param fct Function whose the signature must be returned. + * + * @return The corresponding signature. + */ + public static String getFctSignature(final ADQLFunction fct){ + if (fct == null) + return ""; + + StringBuffer buf = new StringBuffer(fct.getName().toLowerCase()); + buf.append('('); + for(int i = 0; i < fct.getNbParameters(); i++){ + if (fct.getParameter(i).isNumeric()) + buf.append("NUMERIC"); + else if (fct.getParameter(i).isString()) + buf.append("STRING"); + else if (fct.getParameter(i).isGeometry()) + buf.append("GEOMETRY"); + else + buf.append("???"); + + if ((i + 1) < fct.getNbParameters()) + buf.append(", "); + } + buf.append(')'); + return buf.toString(); + } + +} diff --git a/src/adql/db/exception/UnresolvedIdentifiersException.java b/src/adql/db/exception/UnresolvedIdentifiersException.java index b141befb55700ffa2a51f0d39fcba06d58d61d0f..446ae7933b57555b89a9e97500d6744c103a2c69 100644 --- a/src/adql/db/exception/UnresolvedIdentifiersException.java +++ b/src/adql/db/exception/UnresolvedIdentifiersException.java @@ -16,7 +16,8 @@ package adql.db.exception; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeTo customize the object representation you merely have to extends the appropriate functions of this class.
* * @author Grégory Mantelet (CDS;ARI) - * @version 1.3 (05/2014) + * @version 1.4 (06/2015) * * @see ADQLParser */ public class ADQLQueryFactory { - protected boolean allowUnknownFunctions = false; - + /** + * Type of table JOIN. + * + * @author Grégory Mantelet (CDS) + * @version 1.0 (08/2011) + */ public static enum JoinType{ CROSS, INNER, OUTER_LEFT, OUTER_RIGHT, OUTER_FULL; } + /** + * Create a query factory. + */ public ADQLQueryFactory(){ ; } - public ADQLQueryFactory(boolean allowUnknownFunctions){ - this.allowUnknownFunctions = allowUnknownFunctions; - } - public ADQLQuery createQuery() throws Exception{ return new ADQLQuery(); } @@ -268,21 +271,29 @@ public class ADQLQueryFactory { /** *Creates the user defined functions called as the given name and with the given parameters.
- *IMPORTANT: This function must be overridden if some user defined functions are available.
+ * + *+ * By default, this function returns a {@link DefaultUDF} instance. It is generic enough to cover every kind of functions. + * But you can of course override this function in order to return your own instance of {@link UserDefinedFunction}. + * In this case, you may not forget to call the super function (super.createUserDefinedFunction(name, params)) so that + * all other unknown functions are still returned as {@link DefaultUDF} instances. + *
+ * + *IMPORTANT: + * The tests done to check whether a user defined function is allowed/managed in this implementation, is done later by the parser. + * Only declared UDF will pass the test of the parser. For that, you should give it a list of allowed UDFs (each UDF will be then + * represented by a {@link FunctionDef} object). + *
* * @param name Name of the user defined function to create. * @param params Parameters of the user defined function to create. * - * @return The corresponding user defined function. + * @return The corresponding user defined function (by default an instance of {@link DefaultUDF}). * - * @throws Exception An {@link UnsupportedOperationException} by default, otherwise any other type of error may be - * thrown if there is a problem while creating the function. + * @throws Exception If there is a problem while creating the function. */ public UserDefinedFunction createUserDefinedFunction(String name, ADQLOperand[] params) throws Exception{ - if (allowUnknownFunctions) - return new DefaultUDF(name, params); - else - throw new UnsupportedOperationException("No ADQL function called \"" + name + "\" !"); + return new DefaultUDF(name, params); } public DistanceFunction createDistance(PointFunction point1, PointFunction point2) throws Exception{ @@ -317,7 +328,7 @@ public class ADQLQueryFactory { return new RegionFunction(param); } - public PolygonFunction createPolygon(ADQLOperand coordSys, VectorSince it is a list, it is possible to add, remove, modify and iterate on a such object.
* - * @author Grégory Mantelet (CDS) - * @version 06/2011 + * @author Grégory Mantelet (CDS;ARI) + * @version 1.4 (06/2015) * * @see ClauseADQL * @see ClauseConstraints @@ -44,7 +45,7 @@ public abstract class ADQLList< T extends ADQLObject > implements ADQLObject, It private final VectorThe resulting object of the {@link ADQLParser} is an object of this class.
* * @author Grégory Mantelet (CDS;ARI) - * @version 1.3 (05/2014) + * @version 1.4 (06/2015) */ public class ADQLQuery implements ADQLObject { @@ -61,7 +61,7 @@ public class ADQLQuery implements ADQLObject { private ClauseADQLIt merely encapsulates an operand and allows to associate to it an alias (according to the following syntax: "SELECT operand AS alias").
* * @author Grégory Mantelet (CDS;ARI) - * @version 1.3 (05/2014) + * @version 1.4 (06/2015) * * @see ClauseSelect */ @@ -46,7 +46,7 @@ public class SelectItem implements ADQLObject { private boolean caseSensitive = false; /** Position of this Select item in the ADQL query string. - * @since 1.3 */ + * @since 1.4 */ private TextPosition position = null; /** @@ -172,7 +172,7 @@ public class SelectItem implements ADQLObject { * Set the position of this {@link SelectItem} in the given ADQL query string. * * @param position New position of this {@link SelectItem}. - * @since 1.3 + * @since 1.4 */ public final void setPosition(final TextPosition position){ this.position = position; diff --git a/src/adql/query/TextPosition.java b/src/adql/query/TextPosition.java index 5bda46a1d2326edf8bf0c3f5eff91de8c023f98c..5324ebab077160eed2e873d290f21779ad2161f8 100644 --- a/src/adql/query/TextPosition.java +++ b/src/adql/query/TextPosition.java @@ -16,8 +16,8 @@ package adql.query; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeThis function returns true if the sub-query given in parameter returns at least one result, else it returns false.
* * @author Grégory Mantelet (CDS;ARI) - * @version 1.3 (05/2014) + * @version 1.4 (06/2015) */ public class Exists implements ADQLConstraint { @@ -41,7 +41,7 @@ public class Exists implements ADQLConstraint { private ADQLQuery subQuery; /** Position of this {@link Exists} in the given ADQL query string. - * @since 1.3 */ + * @since 1.4 */ private TextPosition position = null; /** @@ -97,7 +97,7 @@ public class Exists implements ADQLConstraint { * Set the position of this {@link Exists} in the given ADQL query string. * * @param position New position of this {@link Exists}. - * @since 1.3 + * @since 1.4 */ public final void setPosition(final TextPosition position){ this.position = position; diff --git a/src/adql/query/constraint/In.java b/src/adql/query/constraint/In.java index a5079849a12b210a40004d0dddd848766259a538..67f37e4d6228f3a5ce28ee8c00d0ed65f5a60715 100644 --- a/src/adql/query/constraint/In.java +++ b/src/adql/query/constraint/In.java @@ -16,8 +16,8 @@ package adql.query.constraint; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeNote: In the most cases, this list is generated on the fly !
* * @return All the available {@link DBColumn}s. - * @throws UnresolvedJoin If a join is not possible. + * @throws UnresolvedJoinException If a join is not possible. */ - public SearchColumnList getDBColumns() throws UnresolvedJoin; + public SearchColumnList getDBColumns() throws UnresolvedJoinException; /** * Gets all {@link ADQLTable} instances contained in this FROM part (itself included, if it is an {@link ADQLTable}). @@ -72,7 +72,7 @@ public interface FromContent extends ADQLObject { * Set the position of this {@link FromContent} in the given ADQL query string. * * @param position New position of this {@link FromContent}. - * @since 1.3 + * @since 1.4 */ public void setPosition(final TextPosition position); diff --git a/src/adql/query/operand/ADQLColumn.java b/src/adql/query/operand/ADQLColumn.java index 590f8b5048e1b2e4fc187093d7526767cb1eeb8a..eebce23b893d6c0f15336343678928ca3e5671a9 100644 --- a/src/adql/query/operand/ADQLColumn.java +++ b/src/adql/query/operand/ADQLColumn.java @@ -16,7 +16,8 @@ package adql.query.operand; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeAny ADQL operand (an operation, a constant, a column name, a function, ...) must implement this interface - * and indicates whether it corresponds to a numeric or a string value.
+ * and indicates whether it corresponds to a numeric, a string or a geometrical region value. * - * @author Grégory Mantelet (CDS) - * @version 11/2010 + * @author Grégory Mantelet (CDS;ARI) + * @version 1.3 (10/2014) */ public interface ADQLOperand extends ADQLObject { + /** + * Tell whether this operand is numeric or not. + * + * @return true if this operand is numeric, false otherwise. + */ public boolean isNumeric(); + /** + * Tell whether this operand is a string or not. + * + * @return true if this operand is a string, false otherwise. + */ public boolean isString(); + /** + * Tell whether this operand is a geometrical region or not. + * + * @return true if this operand is a geometry, false otherwise. + * + * @since 1.3 + */ + public boolean isGeometry(); + } diff --git a/src/adql/query/operand/Concatenation.java b/src/adql/query/operand/Concatenation.java index a09b7905adc6e885cb0af492d19ef8a05e990782..24536c72cfafd754bc359a462ca7afe757b62dfc 100644 --- a/src/adql/query/operand/Concatenation.java +++ b/src/adql/query/operand/Concatenation.java @@ -16,7 +16,8 @@ package adql.query.operand; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeOperand whose the type can not be known at the parsing time. + * A post-parsing step with column metadata is needed to resolved their types.
+ * + *Note: + * For the moment, only two operands are concerned: columns ({@link ADQLColumn}) and user defined functions ({@link UserDefinedFunction}). + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (10/2014) + * @since 1.3 + */ +public interface UnknownType extends ADQLOperand { + + /** + * Get the type expected by the syntactic parser according to the context. + * + * @return Expected type: 'n' or 'N' for numeric, 's' or 'S' for string, 'g' or 'G' for geometry. + */ + public char getExpectedType(); + + /** + * Set the type expected for this operand. + * + * @param c Expected type: 'n' or 'N' for numeric, 's' or 'S' for string, 'g' or 'G' for geometry. + */ + public void setExpectedType(final char c); + +} diff --git a/src/adql/query/operand/WrappedOperand.java b/src/adql/query/operand/WrappedOperand.java index 92ab7bfaae0ca5b6c2d354c2f2a6e7cf74710657..ca36d7a03793796702ddd36c38aaa9e843751c47 100644 --- a/src/adql/query/operand/WrappedOperand.java +++ b/src/adql/query/operand/WrappedOperand.java @@ -16,8 +16,8 @@ package adql.query.operand; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeLet set the signature/definition/description of this user defined function.
+ * + *IMPORTANT: + * No particular checks are done here except on the function name which MUST + * be the same (case insensitive) as the name of the given definition. + * Advanced checks must have been done before calling this setter. + *
+ * + * @param def The definition applying to this parsed UDF, or NULL if none has been found. + * + * @throws IllegalArgumentException If the name in the given definition does not match the name of this parsed function. + * + * @since 1.3 + */ + public final void setDefinition(final FunctionDef def) throws IllegalArgumentException{ + if (def != null && (def.name == null || !functionName.equalsIgnoreCase(def.name))) + throw new IllegalArgumentException("The parsed function name (" + functionName + ") does not match to the name of the given UDF definition (" + def.name + ")."); + + this.definition = def; + } + @Override public final boolean isNumeric(){ - return true; + return (definition == null || definition.isNumeric()); } @Override public final boolean isString(){ - return true; + return (definition == null || definition.isString()); + } + + @Override + public final boolean isGeometry(){ + return (definition == null || definition.isGeometry()); } @Override public ADQLObject getCopy() throws Exception{ - return new DefaultUDF(this); + DefaultUDF copy = new DefaultUDF(this); + copy.setDefinition(definition); + return copy; } @Override @@ -115,4 +163,17 @@ public final class DefaultUDF extends UserDefinedFunction { return oldParam; } + @Override + public String translate(final ADQLTranslator caller) throws TranslationException{ + StringBuffer sql = new StringBuffer(functionName); + sql.append('('); + for(int i = 0; i < parameters.size(); i++){ + if (i > 0) + sql.append(',').append(' '); + sql.append(caller.translate(parameters.get(i))); + } + sql.append(')'); + return sql.toString(); + } + } diff --git a/src/adql/query/operand/function/MathFunction.java b/src/adql/query/operand/function/MathFunction.java index 88e06e06d171d19ef2ff0bea28cc38f50ef6c484..49e1a4a675e436658378ac736e4cf919057784f1 100644 --- a/src/adql/query/operand/function/MathFunction.java +++ b/src/adql/query/operand/function/MathFunction.java @@ -16,8 +16,8 @@ package adql.query.operand.function; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeTranslate this User Defined Function into the language supported by the given translator.
+ * + *VERY IMPORTANT: This function MUST NOT use {@link ADQLTranslator#translate(UserDefinedFunction)} to translate itself. + * The given {@link ADQLTranslator} must be used ONLY to translate UDF's operands.
+ * + *Implementation example (extract of {@link DefaultUDF#translate(ADQLTranslator)}):
+ *
+ * public String translate(final ADQLTranslator caller) throws TranslationException{
+ * StringBuffer sql = new StringBuffer(functionName);
+ * sql.append('(');
+ * for(int i = 0; i < parameters.size(); i++){
+ * if (i > 0)
+ * sql.append(',').append(' ');
+ * sql.append(caller.translate(parameters.get(i)));
+ * }
+ * sql.append(')');
+ * return sql.toString();
+ * }
+ *
+ *
+ *
+ * @param caller Translator to use in order to translate ONLY function parameters.
+ *
+ * @return The translation of this UDF into the language supported by the given translator.
+ *
+ * @throws TranslationException If one of the parameters can not be translated.
+ *
+ * @since 1.3
+ */
+ public abstract String translate(final ADQLTranslator caller) throws TranslationException;
}
diff --git a/src/adql/query/operand/function/geometry/AreaFunction.java b/src/adql/query/operand/function/geometry/AreaFunction.java
index aa59e83a0d08dcf4fe51f30bf7e1776726822c42..bb9003d5484091d54efde436410ce6c866f7c9b6 100644
--- a/src/adql/query/operand/function/geometry/AreaFunction.java
+++ b/src/adql/query/operand/function/geometry/AreaFunction.java
@@ -16,8 +16,8 @@ package adql.query.operand.function.geometry;
* You should have received a copy of the GNU Lesser General Public License
* along with ADQLLibrary. If not, see Inappropriate geometries for this construct (e.g. POINT) SHOULD either return zero or throw an error message. This choice must be done in an extended class of {@link AreaFunction}.
* * @author Grégory Mantelet (CDS;ARI) - * @version 1.3 (05/2014) + * @version 1.4 (06/2015) */ public class AreaFunction extends GeometryFunction { @@ -108,6 +108,11 @@ public class AreaFunction extends GeometryFunction { return false; } + @Override + public boolean isGeometry(){ + return false; + } + @Override public ADQLOperand[] getParameters(){ return new ADQLOperand[]{parameter.getValue()}; @@ -147,4 +152,4 @@ public class AreaFunction extends GeometryFunction { throw new ArrayIndexOutOfBoundsException("No " + index + "-th parameter for the function \"" + getName() + "\" !"); } -} \ No newline at end of file +} diff --git a/src/adql/query/operand/function/geometry/BoxFunction.java b/src/adql/query/operand/function/geometry/BoxFunction.java index 485c0982b8f483ceaf31a56574eb7261eb1e591d..7585fd67b13412b1548c24e41e9e288e8d2054f5 100644 --- a/src/adql/query/operand/function/geometry/BoxFunction.java +++ b/src/adql/query/operand/function/geometry/BoxFunction.java @@ -16,8 +16,8 @@ package adql.query.operand.function.geometry; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeIt represents any geometric function of ADQL.
* * @author Grégory Mantelet (CDS;ARI) - * @version 1.3 (05/2014) + * @version 1.4 (06/2015) */ public abstract class GeometryFunction extends ADQLFunction { @@ -83,13 +85,13 @@ public abstract class GeometryFunction extends ADQLFunction { * @param coordSys Its new coordinate system. * @throws UnsupportedOperationException If this function is not associated with a coordinate system. * @throws NullPointerException If the given operand is null. - * @throws Exception If the given operand is not a string. + * @throws ParseException If the given operand is not a string. */ - public void setCoordinateSystem(ADQLOperand coordSys) throws UnsupportedOperationException, NullPointerException, Exception{ + public void setCoordinateSystem(ADQLOperand coordSys) throws UnsupportedOperationException, NullPointerException, ParseException{ if (coordSys == null) - throw new NullPointerException(""); + this.coordSys = new StringConstant(""); else if (!coordSys.isString()) - throw new Exception("A coordinate system must be a string literal: \"" + coordSys.toADQL() + "\" is not a string operand !"); + throw new ParseException("A coordinate system must be a string literal: \"" + coordSys.toADQL() + "\" is not a string operand!"); else{ this.coordSys = coordSys; setPosition(null); @@ -101,13 +103,13 @@ public abstract class GeometryFunction extends ADQLFunction { * which, in general, is either a GeometryFunction or a Column. * * @author Grégory Mantelet (CDS;ARI) - * @version 05/2014 + * @version 1.4 (06/2015) */ public static final class GeometryValue< F extends GeometryFunction > implements ADQLOperand { private ADQLColumn column; private F geomFunct; /** Position of this {@link GeometryValue} in the ADQL query string. - * @since 1.3 */ + * @since 1.4 */ private TextPosition position = null; public GeometryValue(ADQLColumn col) throws NullPointerException{ @@ -172,6 +174,11 @@ public abstract class GeometryFunction extends ADQLFunction { return position; } + @Override + public boolean isGeometry(){ + return getValue().isGeometry(); + } + @Override public ADQLObject getCopy() throws Exception{ return new GeometryValueImplementation of {@link ADQLTranslator} which translates ADQL queries in SQL queries.
+ * + *+ * It is already able to translate all SQL standard features, but lets abstract the translation of all + * geometrical functions. So, this translator must be extended as {@link PostgreSQLTranslator} and + * {@link PgSphereTranslator} are doing. + *
+ * + *Note: + * Its default implementation of the SQL syntax has been inspired by the PostgreSQL one. + * However, it should work also with SQLite and MySQL, but some translations might be needed + * (as it is has been done for PostgreSQL about the mathematical functions). + *
+ * + *+ * {@link PgSphereTranslator} extends {@link PostgreSQLTranslator} and is able to translate geometrical + * functions according to the syntax given by PgSphere. But it can also convert geometrical types + * (from and toward the database), translate PgSphere regions into STC expression and vice-versa. + *
+ * + *+ * {@link PostgreSQLTranslator} overwrites the translation of mathematical functions whose some have + * a different name or signature. Besides, it is also implementing the translation of the geometrical + * functions. However, it does not really translate them. It is just returning the ADQL expression + * (by calling {@link #getDefaultADQLFunction(ADQLFunction)}). + * And so, of course, the execution of a SQL query containing geometrical functions and translated + * using this translator will not work. It is just a default implementation in case there is no interest + * of these geometrical functions. + *
+ * + *+ * In ADQL and in SQL, it is possible to tell the parser to respect the exact case or not of an identifier (schema, table or column name) + * by surrounding it with double quotes. However ADQL identifiers and SQL ones may be different. In that way, the case sensitivity specified + * in ADQL on the different identifiers can not be kept in SQL. That's why this translator lets specify a general rule on which types of + * SQL identifier must be double quoted. This can be done by implementing the abstract function {@link #isCaseSensitive(IdentifierField)}. + * The functions translating column and table names will call this function in order to surround the identifiers by double quotes or not. + * So, be careful if you want to override the functions translating columns and tables! + *
+ * + *+ * The default behavior of this translator is to translate the ADQL "TOP" into the SQL "LIMIT" at the end of the query. + * This is ok for some DBMS, but not all. So, if your DBMS does not know the "LIMIT" keyword, you should override the function + * translating the whole query: {@link #translate(ADQLQuery)}. Here is its current implementation: + *
+ *
+ * StringBuffer sql = new StringBuffer(translate(query.getSelect()));
+ * sql.append("\nFROM ").append(translate(query.getFrom()));
+ * if (!query.getWhere().isEmpty())
+ * sql.append('\n').append(translate(query.getWhere()));
+ * if (!query.getGroupBy().isEmpty())
+ * sql.append('\n').append(translate(query.getGroupBy()));
+ * if (!query.getHaving().isEmpty())
+ * sql.append('\n').append(translate(query.getHaving()));
+ * if (!query.getOrderBy().isEmpty())
+ * sql.append('\n').append(translate(query.getOrderBy()));
+ * if (query.getSelect().hasLimit())
+ * sql.append("\nLimit ").append(query.getSelect().getLimit());
+ * return sql.toString();
+ *
+ *
+ * + * All ADQL functions are by default not translated. Consequently, the SQL translation is + * actually the ADQL expression. Generally the ADQL expression is generic enough. However some mathematical functions may need + * to be translated differently. For instance {@link PostgreSQLTranslator} is translating differently: LOG, LOG10, RAND and TRUNC. + *
+ * + *Note: + * Geometrical regions and types have not been managed here. They stay abstract because it is obviously impossible to have a generic + * translation and conversion ; it totally depends from the database system. + *
+ * + *+ * The FROM clause is translated into SQL as written in ADQL. There is no differences except the identifiers that are replaced. + * The tables' aliases and their case sensitivity are kept like in ADQL. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (05/2015) + * @since 1.3 + * + * @see PostgreSQLTranslator + * @see PgSphereTranslator + */ +public abstract class JDBCTranslator implements ADQLTranslator { + + /** + *Tell whether the specified identifier MUST be translated so that being interpreted case sensitively or not. + * By default, an identifier that must be translated with case sensitivity will be surrounded by double quotes. + * But, if this function returns FALSE, the SQL name will be written just as given in the metadata, without double quotes.
+ * + *WARNING: + * An {@link IdentifierField} object can be a SCHEMA, TABLE, COLUMN and ALIAS. However, in this translator, + * aliases are translated like in ADQL (so, with the same case sensitivity specification as in ADQL). + * So, this function will never be used to know the case sensitivity to apply to an alias. It is then + * useless to write a special behavior for the ALIAS value. + *
+ * + * @param field The identifier whose the case sensitive to apply is asked. + * + * @return true if the specified identifier must be translated case sensitivity, false otherwise (included if ALIAS or NULL). + */ + public abstract boolean isCaseSensitive(final IdentifierField field); + + /** + *Get the qualified DB name of the schema containing the given table.
+ * + *Note: + * This function will, by default, add double quotes if the schema name must be case sensitive in the SQL query. + * This information is provided by {@link #isCaseSensitive(IdentifierField)}. + *
+ * + * @param table A table of the schema whose the qualified DB name is asked. + * + * @return The qualified (with DB catalog name prefix if any, and with double quotes if needed) DB schema name, + * or an empty string if there is no schema or no DB name. + */ + public String getQualifiedSchemaName(final DBTable table){ + if (table == null || table.getDBSchemaName() == null) + return ""; + + StringBuffer buf = new StringBuffer(); + + if (table.getDBCatalogName() != null) + appendIdentifier(buf, table.getDBCatalogName(), IdentifierField.CATALOG).append('.'); + + appendIdentifier(buf, table.getDBSchemaName(), IdentifierField.SCHEMA); + + return buf.toString(); + } + + /** + *Get the qualified DB name of the given table.
+ * + *Note: + * This function will, by default, add double quotes if the table name must be case sensitive in the SQL query. + * This information is provided by {@link #isCaseSensitive(IdentifierField)}. + *
+ * + * @param table The table whose the qualified DB name is asked. + * + * @return The qualified (with DB catalog and schema prefix if any, and with double quotes if needed) DB table name, + * or an empty string if the given table is NULL or if there is no DB name. + * + * @see #getTableName(DBTable, boolean) + */ + public String getQualifiedTableName(final DBTable table){ + return getTableName(table, true); + } + + /** + *Get the DB name of the given table. + * The second parameter lets specify whether the table name must be prefixed by the qualified schema name or not.
+ * + *Note: + * This function will, by default, add double quotes if the table name must be case sensitive in the SQL query. + * This information is provided by {@link #isCaseSensitive(IdentifierField)}. + *
+ * + * @param table The table whose the DB name is asked. + * @param withSchema true if the qualified schema name must prefix the table name, false otherwise. + * + * @return The DB table name (prefixed by the qualified schema name if asked, and with double quotes if needed), + * or an empty string if the given table is NULL or if there is no DB name. + * + * @since 2.0 + */ + public String getTableName(final DBTable table, final boolean withSchema){ + if (table == null) + return ""; + + StringBuffer buf = new StringBuffer(); + if (withSchema){ + buf.append(getQualifiedSchemaName(table)); + if (buf.length() > 0) + buf.append('.'); + } + appendIdentifier(buf, table.getDBName(), IdentifierField.TABLE); + + return buf.toString(); + } + + /** + *Get the DB name of the given column
+ * + *Note: + * This function will, by default, add double quotes if the column name must be case sensitive in the SQL query. + * This information is provided by {@link #isCaseSensitive(IdentifierField)}. + *
+ * + *Caution: + * The given column may be NULL and in this case an empty string will be returned. + * But if the given column is not NULL, its DB name MUST NOT BE NULL! + *
+ * + * @param column The column whose the DB name is asked. + * + * @return The DB column name (with double quotes if needed), + * or an empty string if the given column is NULL. + */ + public String getColumnName(final DBColumn column){ + return (column == null) ? "" : appendIdentifier(new StringBuffer(), column.getDBName(), IdentifierField.COLUMN).toString(); + } + + /** + * Appends the given identifier in the given StringBuffer. + * + * @param str The string buffer. + * @param id The identifier to append. + * @param field The type of identifier (column, table, schema, catalog or alias ?). + * + * @return The string buffer + identifier. + */ + public final StringBuffer appendIdentifier(final StringBuffer str, final String id, final IdentifierField field){ + return appendIdentifier(str, id, isCaseSensitive(field)); + } + + /** + * Appends the given identifier to the given StringBuffer. + * + * @param str The string buffer. + * @param id The identifier to append. + * @param caseSensitive true to format the identifier so that preserving the case sensitivity, false otherwise. + * + * @return The string buffer + identifier. + */ + public static final StringBuffer appendIdentifier(final StringBuffer str, final String id, final boolean caseSensitive){ + if (caseSensitive) + return str.append('"').append(id).append('"'); + else + return str.append(id); + } + + @Override + @SuppressWarnings({"unchecked","rawtypes"}) + public String translate(ADQLObject obj) throws TranslationException{ + if (obj instanceof ADQLQuery) + return translate((ADQLQuery)obj); + else if (obj instanceof ADQLList) + return translate((ADQLList)obj); + else if (obj instanceof SelectItem) + return translate((SelectItem)obj); + else if (obj instanceof ColumnReference) + return translate((ColumnReference)obj); + else if (obj instanceof ADQLTable) + return translate((ADQLTable)obj); + else if (obj instanceof ADQLJoin) + return translate((ADQLJoin)obj); + else if (obj instanceof ADQLOperand) + return translate((ADQLOperand)obj); + else if (obj instanceof ADQLConstraint) + return translate((ADQLConstraint)obj); + else + return obj.toADQL(); + } + + @Override + public String translate(ADQLQuery query) throws TranslationException{ + StringBuffer sql = new StringBuffer(translate(query.getSelect())); + + sql.append("\nFROM ").append(translate(query.getFrom())); + + if (!query.getWhere().isEmpty()) + sql.append('\n').append(translate(query.getWhere())); + + if (!query.getGroupBy().isEmpty()) + sql.append('\n').append(translate(query.getGroupBy())); + + if (!query.getHaving().isEmpty()) + sql.append('\n').append(translate(query.getHaving())); + + if (!query.getOrderBy().isEmpty()) + sql.append('\n').append(translate(query.getOrderBy())); + + if (query.getSelect().hasLimit()) + sql.append("\nLimit ").append(query.getSelect().getLimit()); + + return sql.toString(); + } + + /* *************************** */ + /* ****** LIST & CLAUSE ****** */ + /* *************************** */ + @Override + public String translate(ADQLList list) throws TranslationException{ + if (list instanceof ClauseSelect) + return translate((ClauseSelect)list); + else if (list instanceof ClauseConstraints) + return translate((ClauseConstraints)list); + else + return getDefaultADQLList(list); + } + + /** + * Gets the default SQL output for a list of ADQL objects. + * + * @param list List to format into SQL. + * + * @return The corresponding SQL. + * + * @throws TranslationException If there is an error during the translation. + */ + protected String getDefaultADQLList(ADQLList list) throws TranslationException{ + String sql = (list.getName() == null) ? "" : (list.getName() + " "); + + for(int i = 0; i < list.size(); i++) + sql += ((i == 0) ? "" : (" " + list.getSeparator(i) + " ")) + translate(list.get(i)); + + return sql; + } + + @Override + public String translate(ClauseSelect clause) throws TranslationException{ + String sql = null; + + for(int i = 0; i < clause.size(); i++){ + if (i == 0){ + sql = clause.getName() + (clause.distinctColumns() ? " DISTINCT" : ""); + }else + sql += " " + clause.getSeparator(i); + + sql += " " + translate(clause.get(i)); + } + + return sql; + } + + @Override + public String translate(ClauseConstraints clause) throws TranslationException{ + if (clause instanceof ConstraintsGroup) + return "(" + getDefaultADQLList(clause) + ")"; + else + return getDefaultADQLList(clause); + } + + @Override + public String translate(SelectItem item) throws TranslationException{ + if (item instanceof SelectAllColumns) + return translate((SelectAllColumns)item); + + StringBuffer translation = new StringBuffer(translate(item.getOperand())); + if (item.hasAlias()){ + translation.append(" AS "); + appendIdentifier(translation, item.getAlias(), item.isCaseSensitive()); + }else{ + translation.append(" AS "); + appendIdentifier(translation, item.getName(), true); + } + + return translation.toString(); + } + + @Override + public String translate(SelectAllColumns item) throws TranslationException{ + HashMapConvert any type provided by the ADQL/TAP library into a type understandable by a JDBC driver.
+ * + *Note: + * The returned DBMS type may contain some parameters between brackets. + *
+ * + * @param type The ADQL/TAP library's type to convert. + * + * @return The corresponding DBMS type or NULL if the specified type is unknown. + */ + public abstract String convertTypeToDB(final DBType type); + + /** + *Parse the given JDBC column value as a geometry object and convert it into a {@link Region}.
+ * + *Note: + * Generally the returned object will be used to get its STC-S expression. + *
+ * + *Note: + * If the given column value is NULL, NULL will be returned. + *
+ * + *Important note: + * This function is called ONLY for value of columns flagged as geometries by + * {@link #convertTypeFromDB(int, String, String, String[])}. So the value should always + * be of the expected type and format. However, if it turns out that the type is wrong + * and that the conversion is finally impossible, this function SHOULD throw a + * {@link DataReadException}. + *
+ * + * @param jdbcColValue A JDBC column value (returned by ResultSet.getObject(int)). + * + * @return The corresponding {@link Region} if the given value is a geometry. + * + * @throws ParseException If the given object is not a geometrical object + * or can not be transformed into a {@link Region} object. + */ + public abstract Region translateGeometryFromDB(final Object jdbcColValue) throws ParseException; + + /** + *Convert the given STC region into a DB column value.
+ * + *Note: + * This function is used only by the UPLOAD feature, to import geometries provided as STC-S expression in + * a VOTable document inside a DB column. + *
+ * + *Note: + * If the given region is NULL, NULL will be returned. + *
+ * + * @param region The region to store in the DB. + * + * @return The corresponding DB column object. + * + * @throws ParseException If the given STC Region can not be converted into a DB object. + */ + public abstract Object translateGeometryToDB(final Region region) throws ParseException; + +} diff --git a/src/adql/translator/PgSphereTranslator.java b/src/adql/translator/PgSphereTranslator.java index efe38074322b8e850227a9f5169d2346b23958de..96c509c52744219b63c7560a82988cd06a72d720 100644 --- a/src/adql/translator/PgSphereTranslator.java +++ b/src/adql/translator/PgSphereTranslator.java @@ -16,12 +16,22 @@ package adql.translator; * You should have received a copy of the GNU Lesser General Public License * along with ADQLLibrary. If not, seeTranslates all ADQL objects into the SQL adaptation of Postgres+PgSphere. * Actually only the geometrical functions are translated in this class. * The other functions are managed by {@link PostgreSQLTranslator}.
* - * @author Grégory Mantelet (CDS) - * @version 01/2012 - * - * @see PostgreSQLTranslator + * @author Grégory Mantelet (CDS;ARI) + * @version 1.3 (11/2014) */ public class PgSphereTranslator extends PostgreSQLTranslator { + /** Angle between two points generated while transforming a circle into a polygon. + * This angle is computed by default to get at the end a polygon of 32 points. + * @see #circleToPolygon(double[], double) + * @since 1.3 */ + protected static double ANGLE_CIRCLE_TO_POLYGON = 2 * Math.PI / 32; + /** - * Builds a PgSphereTranslator which takes into account the case sensitivity on column names. - * It means that column names which have been written between double quotes, will be also translated between double quotes. + * Builds a PgSphereTranslator which always translates in SQL all identifiers (schema, table and column) in a case sensitive manner ; + * in other words, schema, table and column names will be surrounded by double quotes in the SQL translation. * * @see PostgreSQLTranslator#PostgreSQLTranslator() */ @@ -58,23 +69,24 @@ public class PgSphereTranslator extends PostgreSQLTranslator { } /** - * Builds a PgSphereTranslator. + * Builds a PgSphereTranslator which always translates in SQL all identifiers (schema, table and column) in the specified case sensitivity ; + * in other words, schema, table and column names will all be surrounded or not by double quotes in the SQL translation. * - * @param column true to take into account the case sensitivity of column names, false otherwise. + * @param allCaseSensitive true to translate all identifiers in a case sensitive manner (surrounded by double quotes), false for case insensitivity. * * @see PostgreSQLTranslator#PostgreSQLTranslator(boolean) */ - public PgSphereTranslator(boolean column){ - super(column); + public PgSphereTranslator(boolean allCaseSensitive){ + super(allCaseSensitive); } /** - * Builds a PgSphereTranslator. + * Builds a PgSphereTranslator which will always translate in SQL identifiers with the defined case sensitivity. * - * @param catalog true to take into account the case sensitivity of catalog names, false otherwise. - * @param schema true to take into account the case sensitivity of schema names, false otherwise. - * @param table true to take into account the case sensitivity of table names, false otherwise. - * @param column true to take into account the case sensitivity of column names, false otherwise. + * @param catalog true to translate catalog names with double quotes (case sensitive in the DBMS), false otherwise. + * @param schema true to translate schema names with double quotes (case sensitive in the DBMS), false otherwise. + * @param table true to translate table names with double quotes (case sensitive in the DBMS), false otherwise. + * @param column true to translate column names with double quotes (case sensitive in the DBMS), false otherwise. * * @see PostgreSQLTranslator#PostgreSQLTranslator(boolean, boolean, boolean, boolean) */ @@ -103,11 +115,12 @@ public class PgSphereTranslator extends PostgreSQLTranslator { public String translate(BoxFunction box) throws TranslationException{ StringBuffer str = new StringBuffer("sbox("); + str.append("spoint(").append("radians(").append(translate(box.getCoord1())).append("-(").append(translate(box.getWidth())).append("/2.0)),"); + str.append("radians(").append(translate(box.getCoord2())).append("-(").append(translate(box.getHeight())).append("/2.0))),"); + str.append("spoint(").append("radians(").append(translate(box.getCoord1())).append("+(").append(translate(box.getWidth())).append("/2.0)),"); - str.append("radians(").append(translate(box.getCoord2())).append("+(").append(translate(box.getHeight())).append("/2.0))),"); + str.append("radians(").append(translate(box.getCoord2())).append("+(").append(translate(box.getHeight())).append("/2.0))))"); - str.append("spoint(").append("radians(").append(translate(box.getCoord1())).append("-(").append(translate(box.getWidth())).append("/2.0)),"); - str.append("radians(").append(translate(box.getCoord2())).append("-(").append(translate(box.getHeight())).append("/2.0))))"); return str.toString(); } @@ -156,8 +169,8 @@ public class PgSphereTranslator extends PostgreSQLTranslator { @Override public String translate(AreaFunction areaFunction) throws TranslationException{ - StringBuffer str = new StringBuffer("degrees(area("); - str.append(translate(areaFunction.getParameter())).append("))"); + StringBuffer str = new StringBuffer("degrees(degrees(area("); + str.append(translate(areaFunction.getParameter())).append(")))"); return str.toString(); } @@ -185,4 +198,534 @@ public class PgSphereTranslator extends PostgreSQLTranslator { return super.translate(comp); } + @Override + public DBType convertTypeFromDB(final int dbmsType, final String rawDbmsTypeName, String dbmsTypeName, final String[] params){ + // If no type is provided return VARCHAR: + if (dbmsTypeName == null || dbmsTypeName.trim().length() == 0) + return new DBType(DBDatatype.VARCHAR, DBType.NO_LENGTH); + + // Put the dbmsTypeName in lower case for the following comparisons: + dbmsTypeName = dbmsTypeName.toLowerCase(); + + if (dbmsTypeName.equals("spoint")) + return new DBType(DBDatatype.POINT); + else if (dbmsTypeName.equals("scircle") || dbmsTypeName.equals("sbox") || dbmsTypeName.equals("spoly")) + return new DBType(DBDatatype.REGION); + else + return super.convertTypeFromDB(dbmsType, rawDbmsTypeName, dbmsTypeName, params); + } + + @Override + public String convertTypeToDB(final DBType type){ + if (type != null){ + if (type.type == DBDatatype.POINT) + return "spoint"; + else if (type.type == DBDatatype.REGION) + return "spoly"; + } + return super.convertTypeToDB(type); + } + + @Override + public Region translateGeometryFromDB(final Object jdbcColValue) throws ParseException{ + // A NULL value stays NULL: + if (jdbcColValue == null) + return null; + // Only a special object is expected: + else if (!(jdbcColValue instanceof PGobject)) + throw new ParseException("Incompatible type! The column value \"" + jdbcColValue.toString() + "\" was supposed to be a geometrical object."); + + PGobject pgo = (PGobject)jdbcColValue; + + // In case one or both of the fields of the given object are NULL: + if (pgo == null || pgo.getType() == null || pgo.getValue() == null || pgo.getValue().length() == 0) + return null; + + // Extract the object type and its value: + String objType = pgo.getType().toLowerCase(); + String geomStr = pgo.getValue(); + + /* Only spoint, scircle, sbox and spoly are supported ; + * these geometries are parsed and transformed in Region instances:*/ + if (objType.equals("spoint")) + return (new PgSphereGeometryParser()).parsePoint(geomStr); + else if (objType.equals("scircle")) + return (new PgSphereGeometryParser()).parseCircle(geomStr); + else if (objType.equals("sbox")) + return (new PgSphereGeometryParser()).parseBox(geomStr); + else if (objType.equals("spoly")) + return (new PgSphereGeometryParser()).parsePolygon(geomStr); + else + throw new ParseException("Unsupported PgSphere type: \"" + objType + "\"! Impossible to convert the column value \"" + geomStr + "\" into a Region."); + } + + @Override + public Object translateGeometryToDB(final Region region) throws ParseException{ + // A NULL value stays NULL: + if (region == null) + return null; + + try{ + PGobject dbRegion = new PGobject(); + StringBuffer buf; + + // Build the PgSphere expression from the given geometry in function of its type: + switch(region.type){ + + case POSITION: + dbRegion.setType("spoint"); + dbRegion.setValue("(" + region.coordinates[0][0] + "d," + region.coordinates[0][1] + "d)"); + break; + + case POLYGON: + dbRegion.setType("spoly"); + buf = new StringBuffer("{"); + for(int i = 0; i < region.coordinates.length; i++){ + if (i > 0) + buf.append(','); + buf.append('(').append(region.coordinates[i][0]).append("d,").append(region.coordinates[i][1]).append("d)"); + } + buf.append('}'); + dbRegion.setValue(buf.toString()); + break; + + case BOX: + dbRegion.setType("spoly"); + buf = new StringBuffer("{"); + // south west + buf.append('(').append(region.coordinates[0][0] - region.width / 2).append("d,").append(region.coordinates[0][1] - region.height / 2).append("d),"); + // north west + buf.append('(').append(region.coordinates[0][0] - region.width / 2).append("d,").append(region.coordinates[0][1] + region.height / 2).append("d),"); + // north east + buf.append('(').append(region.coordinates[0][0] + region.width / 2).append("d,").append(region.coordinates[0][1] + region.height / 2).append("d),"); + // south east + buf.append('(').append(region.coordinates[0][0] + region.width / 2).append("d,").append(region.coordinates[0][1] - region.height / 2).append("d)"); + buf.append('}'); + dbRegion.setValue(buf.toString()); + break; + + case CIRCLE: + dbRegion.setType("spoly"); + dbRegion.setValue(circleToPolygon(region.coordinates[0], region.radius)); + break; + + default: + throw new ParseException("Unsupported geometrical region: \"" + region.type + "\"!"); + } + return dbRegion; + }catch(SQLException e){ + /* This error could never happen! */ + return null; + } + } + + /** + *Convert the specified circle into a polygon. + * The generated polygon is formatted using the PgSphere syntax.
+ * + *Note: + * The center coordinates and the radius are expected in degrees. + *
+ * + * @param center Center of the circle ([0]=ra and [1]=dec). + * @param radius Radius of the circle. + * + * @return The PgSphere serialization of the corresponding polygon. + * + * @since 1.3 + */ + protected String circleToPolygon(final double[] center, final double radius){ + double angle = 0, x, y; + StringBuffer buf = new StringBuffer(); + while(angle < 2 * Math.PI){ + x = center[0] + radius * Math.cos(angle); + y = center[1] + radius * Math.sin(angle); + if (buf.length() > 0) + buf.append(','); + buf.append('(').append(x).append("d,").append(y).append("d)"); + angle += ANGLE_CIRCLE_TO_POLYGON; + } + return "{" + buf + "}"; + } + + /** + *Let parse a geometry serialized with the PgSphere syntax.
+ * + *+ * There is one function parseXxx(String) for each supported geometry. + * These functions always return a {@link Region} object, + * which is the object representation of an STC region. + *
+ * + *Only the following geometries are supported:
+ *+ * This parser supports all the known PgSphere representations of an angle. + * However, it always returns angle (coordinates, radius, width and height) in degrees. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 1.3 (11/2014) + * @since 1.3 + */ + protected static class PgSphereGeometryParser { + /** Position of the next characters to read in the PgSphere expression to parse. */ + private int pos; + /** Full PgSphere expression to parse. */ + private String expr; + /** Last read token (either a string/numeric or a separator). */ + private String token; + /** Buffer used to read tokens. */ + private StringBuffer buffer; + + private static final char OPEN_PAR = '('; + private static final char CLOSE_PAR = ')'; + private static final char COMMA = ','; + private static final char LESS_THAN = '<'; + private static final char GREATER_THAN = '>'; + private static final char OPEN_BRACE = '{'; + private static final char CLOSE_BRACE = '}'; + private static final char DEGREE = 'd'; + private static final char HOUR = 'h'; + private static final char MINUTE = 'm'; + private static final char SECOND = 's'; + + /** + * Exception sent when the end of the expression + * (EOE = End Of Expression) is reached. + * + * @author Grégory Mantelet (ARI) + * @version 1.3 (11/2014) + * @since 1.3 + */ + private static class EOEException extends ParseException { + private static final long serialVersionUID = 1L; + + /** Build a simple EOEException. */ + public EOEException(){ + super("Unexpected End Of PgSphere Expression!"); + } + } + + /** + * Build the PgSphere parser. + */ + public PgSphereGeometryParser(){} + + /** + * Prepare the parser in order to read the given PgSphere expression. + * + * @param newStcs New PgSphere expression to parse from now. + */ + private void init(final String newExpr){ + expr = (newExpr == null) ? "" : newExpr; + token = null; + buffer = new StringBuffer(); + pos = 0; + } + + /** + * Finalize the parsing. + * No more characters (except eventually some space characters) should remain in the PgSphere expression to parse. + * + * @throws ParseException If other non-space characters remains. + */ + private void end() throws ParseException{ + // Skip all spaces: + skipSpaces(); + + // If there is still some characters, they are not expected, and so throw an exception: + if (expr.length() > 0 && pos < expr.length()) + throw new ParseException("Unexpected end of PgSphere region expression: \"" + expr.substring(pos) + "\" was unexpected!", new TextPosition(1, pos, 1, expr.length())); + + // Reset the buffer, token and the PgSphere expression to parse: + buffer = null; + expr = null; + token = null; + } + + /** + * Tool function which skips all next space characters until the next meaningful characters. + */ + private void skipSpaces(){ + while(pos < expr.length() && Character.isWhitespace(expr.charAt(pos))) + pos++; + } + + /** + *Get the next meaningful word. This word can be a numeric, any string constant or a separator. + * This function returns this token but also stores it in the class attribute {@link #token}.
+ * + *+ * In case the end of the expression is reached before getting any meaningful character, + * an {@link EOEException} is thrown. + *
+ * + * @return The full read word/token, or NULL if the end has been reached. + */ + private String nextToken() throws EOEException{ + // Skip all spaces: + skipSpaces(); + + if (pos >= expr.length()) + throw new EOEException(); + + // Fetch all characters until word separator (a space or a open/close parenthesis): + buffer.append(expr.charAt(pos++)); + if (!isSyntaxSeparator(buffer.charAt(0))){ + while(pos < expr.length() && !isSyntaxSeparator(expr.charAt(pos))){ + // skip eventual white-spaces: + if (!Character.isWhitespace(expr.charAt(pos))) + buffer.append(expr.charAt(pos)); + pos++; + } + } + + // Save the read token and reset the buffer: + token = buffer.toString(); + buffer.delete(0, token.length()); + + return token; + } + + /** + *Tell whether the given character is a separator defined in the syntax.
+ * + *Here, the following characters are considered as separators/specials: + * ',', 'd', 'h', 'm', 's', '(', ')', '<', '>', '{' and '}'.
+ * + * @param c Character to test. + * + * @return true if the given character must be considered as a separator, false otherwise. + */ + private static boolean isSyntaxSeparator(final char c){ + return (c == COMMA || c == DEGREE || c == HOUR || c == MINUTE || c == SECOND || c == OPEN_PAR || c == CLOSE_PAR || c == LESS_THAN || c == GREATER_THAN || c == OPEN_BRACE || c == CLOSE_BRACE); + } + + /** + * Get the next character and ensure it is the same as the character given in parameter. + * If the read character is not matching the expected one, a {@link ParseException} is thrown. + * + * @param expected Expected character. + * + * @throws ParseException If the next character is not matching the given one. + */ + private void nextToken(final char expected) throws ParseException{ + // Skip all spaces: + skipSpaces(); + + // Test whether the end is reached: + if (pos >= expr.length()) + throw new EOEException(); + + // Fetch the next character: + char t = expr.charAt(pos++); + token = new String(new char[]{t}); + + /* Test the the fetched character with the expected one + * and throw an error if they don't match: */ + if (t != expected) + throw new ParseException("Incorrect syntax for \"" + expr + "\"! \"" + expected + "\" was expected instead of \"" + t + "\".", new TextPosition(1, pos - 1, 1, pos)); + } + + /** + * Parse the given PgSphere geometry as a point. + * + * @param pgsphereExpr The PgSphere expression to parse as a point. + * + * @return A {@link Region} implementing a STC Position region. + * + * @throws ParseException If the PgSphere syntax of the given expression is wrong or does not correspond to a point. + */ + public Region parsePoint(final String pgsphereExpr) throws ParseException{ + // Init the parser: + init(pgsphereExpr); + // Parse the expression: + double[] coord = parsePoint(); + // No more character should remain after that: + end(); + // Build the STC Position region: + return new Region(null, coord); + } + + /** + * Internal spoint parsing function. It parses the PgSphere expression stored in this parser as a point. + * + * @return The ra and dec coordinates (in degrees) of the parsed point. + * + * @throws ParseException If the PgSphere syntax of the given expression is wrong or does not correspond to a point. + * + * @see #parseAngle() + * @see #parsePoint(String) + */ + private double[] parsePoint() throws ParseException{ + nextToken(OPEN_PAR); + double x = parseAngle(); + nextToken(COMMA); + double y = parseAngle(); + nextToken(CLOSE_PAR); + return new double[]{x,y}; + } + + /** + * Parse the given PgSphere geometry as a circle. + * + * @param pgsphereExpr The PgSphere expression to parse as a circle. + * + * @return A {@link Region} implementing a STC Circle region. + * + * @throws ParseException If the PgSphere syntax of the given expression is wrong or does not correspond to a circle. + */ + public Region parseCircle(final String pgsphereExpr) throws ParseException{ + // Init the parser: + init(pgsphereExpr); + + // Parse the expression: + nextToken(LESS_THAN); + double[] center = parsePoint(); + nextToken(COMMA); + double radius = parseAngle(); + nextToken(GREATER_THAN); + + // No more character should remain after that: + end(); + + // Build the STC Circle region: + return new Region(null, center, radius); + } + + /** + * Parse the given PgSphere geometry as a box. + * + * @param pgsphereExpr The PgSphere expression to parse as a box. + * + * @return A {@link Region} implementing a STC Box region. + * + * @throws ParseException If the PgSphere syntax of the given expression is wrong or does not correspond to a box. + */ + public Region parseBox(final String pgsphereExpr) throws ParseException{ + // Init the parser: + init(pgsphereExpr); + + // Parse the expression: + nextToken(OPEN_PAR); + double[] southwest = parsePoint(); + nextToken(COMMA); + double[] northeast = parsePoint(); + nextToken(CLOSE_PAR); + + // No more character should remain after that: + end(); + + // Build the STC Box region: + double width = Math.abs(northeast[0] - southwest[0]), height = Math.abs(northeast[1] - southwest[1]); + double[] center = new double[]{northeast[0] - width / 2,northeast[1] - height / 2}; + return new Region(null, center, width, height); + } + + /** + * Parse the given PgSphere geometry as a point. + * + * @param pgsphereExpr The PgSphere expression to parse as a point. + * + * @return A {@link Region} implementing a STC Position region. + * + * @throws ParseException If the PgSphere syntax of the given expression is wrong or does not correspond to a point. + */ + public Region parsePolygon(final String pgsphereExpr) throws ParseException{ + // Init the parser: + init(pgsphereExpr); + + // Parse the expression: + nextToken(OPEN_BRACE); + ArrayListRead the next tokens as an angle expression and returns the corresponding angle in degrees.
+ * + *This function supports the 4 following syntaxes:
+ *Translates all ADQL objects into the SQL adaptation of Postgres.
+ *Translates all ADQL objects into an SQL interrogation query designed for PostgreSQL.
* - *IMPORTANT: The geometrical functions are translated exactly as in ADQL. - * You will probably need to extend this translator to correctly manage the geometrical functions. - * An extension is already available for PgSphere: {@link PgSphereTranslator}.
+ *Important: + * The geometrical functions are translated exactly as in ADQL. + * You will probably need to extend this translator to correctly manage the geometrical functions. + * An extension is already available for PgSphere: {@link PgSphereTranslator}. + *
* * @author Grégory Mantelet (CDS;ARI) - * @version 1.2 (03/2014) + * @version 1.3 (11/2014) * * @see PgSphereTranslator */ -public class PostgreSQLTranslator implements ADQLTranslator { +public class PostgreSQLTranslator extends JDBCTranslator { - protected boolean inSelect = false; + /**Indicate the case sensitivity to apply to each SQL identifier (only SCHEMA, TABLE and COLUMN).
+ * + *Note: + * In this implementation, this field is set by the constructor and never modified elsewhere. + * It would be better to never modify it after the construction in order to keep a certain consistency. + *
+ */ protected byte caseSensitivity = 0x00; /** - * Builds a PostgreSQLTranslator which takes into account the case sensitivity on column names. - * It means that column names which have been written between double quotes, will be also translated between double quotes. + * Builds a PostgreSQLTranslator which always translates in SQL all identifiers (schema, table and column) in a case sensitive manner ; + * in other words, schema, table and column names will be surrounded by double quotes in the SQL translation. */ public PostgreSQLTranslator(){ - this(true); + caseSensitivity = 0x0F; } /** - * Builds a PostgreSQLTranslator. + * Builds a PostgreSQLTranslator which always translates in SQL all identifiers (schema, table and column) in the specified case sensitivity ; + * in other words, schema, table and column names will all be surrounded or not by double quotes in the SQL translation. * - * @param column true to take into account the case sensitivity of column names, false otherwise. + * @param allCaseSensitive true to translate all identifiers in a case sensitive manner (surrounded by double quotes), false for case insensitivity. */ - public PostgreSQLTranslator(final boolean column){ - caseSensitivity = IdentifierField.COLUMN.setCaseSensitive(caseSensitivity, column); + public PostgreSQLTranslator(final boolean allCaseSensitive){ + caseSensitivity = allCaseSensitive ? (byte)0x0F : (byte)0x00; } /** - * Builds a PostgreSQLTranslator. + * Builds a PostgreSQLTranslator which will always translate in SQL identifiers with the defined case sensitivity. * - * @param catalog true to take into account the case sensitivity of catalog names, false otherwise. - * @param schema true to take into account the case sensitivity of schema names, false otherwise. - * @param table true to take into account the case sensitivity of table names, false otherwise. - * @param column true to take into account the case sensitivity of column names, false otherwise. + * @param catalog true to translate catalog names with double quotes (case sensitive in the DBMS), false otherwise. + * @param schema true to translate schema names with double quotes (case sensitive in the DBMS), false otherwise. + * @param table true to translate table names with double quotes (case sensitive in the DBMS), false otherwise. + * @param column true to translate column names with double quotes (case sensitive in the DBMS), false otherwise. */ public PostgreSQLTranslator(final boolean catalog, final boolean schema, final boolean table, final boolean column){ caseSensitivity = IdentifierField.CATALOG.setCaseSensitive(caseSensitivity, catalog); @@ -125,521 +98,22 @@ public class PostgreSQLTranslator implements ADQLTranslator { caseSensitivity = IdentifierField.COLUMN.setCaseSensitive(caseSensitivity, column); } - /** - * Appends the full name of the given table to the given StringBuffer. - * - * @param str The string buffer. - * @param dbTable The table whose the full name must be appended. - * - * @return The string buffer + full table name. - */ - public final StringBuffer appendFullDBName(final StringBuffer str, final DBTable dbTable){ - if (dbTable != null){ - if (dbTable.getDBCatalogName() != null) - appendIdentifier(str, dbTable.getDBCatalogName(), IdentifierField.CATALOG).append('.'); - - if (dbTable.getDBSchemaName() != null) - appendIdentifier(str, dbTable.getDBSchemaName(), IdentifierField.SCHEMA).append('.'); - - appendIdentifier(str, dbTable.getDBName(), IdentifierField.TABLE); - } - return str; - } - - /** - * Appends the given identifier in the given StringBuffer. - * - * @param str The string buffer. - * @param id The identifier to append. - * @param field The type of identifier (column, table, schema, catalog or alias ?). - * - * @return The string buffer + identifier. - */ - public final StringBuffer appendIdentifier(final StringBuffer str, final String id, final IdentifierField field){ - return appendIdentifier(str, id, field.isCaseSensitive(caseSensitivity)); - } - - /** - * Appends the given identifier to the given StringBuffer. - * - * @param str The string buffer. - * @param id The identifier to append. - * @param caseSensitive true to format the identifier so that preserving the case sensitivity, false otherwise. - * - * @return The string buffer + identifier. - */ - public static final StringBuffer appendIdentifier(final StringBuffer str, final String id, final boolean caseSensitive){ - if (caseSensitive) - return str.append('\"').append(id).append('\"'); - else - return str.append(id); - } - - @Override - @SuppressWarnings("unchecked") - public String translate(ADQLObject obj) throws TranslationException{ - if (obj instanceof ADQLQuery) - return translate((ADQLQuery)obj); - else if (obj instanceof ADQLList) - return translate((ADQLList)obj); - else if (obj instanceof SelectItem) - return translate((SelectItem)obj); - else if (obj instanceof ColumnReference) - return translate((ColumnReference)obj); - else if (obj instanceof ADQLTable) - return translate((ADQLTable)obj); - else if (obj instanceof ADQLJoin) - return translate((ADQLJoin)obj); - else if (obj instanceof ADQLOperand) - return translate((ADQLOperand)obj); - else if (obj instanceof ADQLConstraint) - return translate((ADQLConstraint)obj); - else - return obj.toADQL(); - } - - @Override - public String translate(ADQLQuery query) throws TranslationException{ - StringBuffer sql = new StringBuffer(translate(query.getSelect())); - - sql.append("\nFROM ").append(translate(query.getFrom())); - - if (!query.getWhere().isEmpty()) - sql.append('\n').append(translate(query.getWhere())); - - if (!query.getGroupBy().isEmpty()) - sql.append('\n').append(translate(query.getGroupBy())); - - if (!query.getHaving().isEmpty()) - sql.append('\n').append(translate(query.getHaving())); - - if (!query.getOrderBy().isEmpty()) - sql.append('\n').append(translate(query.getOrderBy())); - - if (query.getSelect().hasLimit()) - sql.append("\nLimit ").append(query.getSelect().getLimit()); - - return sql.toString(); - } - - /* *************************** */ - /* ****** LIST & CLAUSE ****** */ - /* *************************** */ - @Override - public String translate(ADQLList list) throws TranslationException{ - if (list instanceof ClauseSelect) - return translate((ClauseSelect)list); - else if (list instanceof ClauseConstraints) - return translate((ClauseConstraints)list); - else - return getDefaultADQLList(list); - } - - /** - * Gets the default SQL output for a list of ADQL objects. - * - * @param list List to format into SQL. - * - * @return The corresponding SQL. - * - * @throws TranslationException If there is an error during the translation. - */ - protected String getDefaultADQLList(ADQLList list) throws TranslationException{ - String sql = (list.getName() == null) ? "" : (list.getName() + " "); - - boolean oldInSelect = inSelect; - inSelect = (list.getName() != null) && list.getName().equalsIgnoreCase("select"); - - try{ - for(int i = 0; i < list.size(); i++) - sql += ((i == 0) ? "" : (" " + list.getSeparator(i) + " ")) + translate(list.get(i)); - }finally{ - inSelect = oldInSelect; - } - - return sql; - } - @Override - public String translate(ClauseSelect clause) throws TranslationException{ - String sql = null; - - for(int i = 0; i < clause.size(); i++){ - if (i == 0){ - sql = clause.getName() + (clause.distinctColumns() ? " DISTINCT" : ""); - }else - sql += " " + clause.getSeparator(i); - - sql += " " + translate(clause.get(i)); - } - - return sql; - } - - @Override - public String translate(ClauseConstraints clause) throws TranslationException{ - if (clause instanceof ConstraintsGroup) - return "(" + getDefaultADQLList(clause) + ")"; - else - return getDefaultADQLList(clause); - } - - @Override - public String translate(SelectItem item) throws TranslationException{ - if (item instanceof SelectAllColumns) - return translate((SelectAllColumns)item); - - StringBuffer translation = new StringBuffer(translate(item.getOperand())); - if (item.hasAlias()){ - translation.append(" AS "); - appendIdentifier(translation, item.getAlias(), item.isCaseSensitive()); - }else - translation.append(" AS ").append(item.getName()); - - return translation.toString(); - } - - @Override - public String translate(SelectAllColumns item) throws TranslationException{ - HashMapGets the default SQL output for the given geometrical function.
- * - *Note: By default, only the ADQL serialization is returned.
- * - * @param fct The geometrical function to translate. - * - * @return The corresponding SQL. - * - * @throws TranslationException If there is an error during the translation. - */ - protected String getDefaultGeometryFunction(GeometryFunction fct) throws TranslationException{ - if (inSelect) - return "'" + fct.toADQL().replaceAll("'", "''") + "'"; - else - return getDefaultADQLFunction(fct); - } - - @Override - public String translate(GeometryValue geomValue) throws TranslationException{ - return translate(geomValue.getValue()); - } - @Override public String translate(ExtractCoord extractCoord) throws TranslationException{ - return getDefaultGeometryFunction(extractCoord); + return getDefaultADQLFunction(extractCoord); } @Override public String translate(ExtractCoordSys extractCoordSys) throws TranslationException{ - return getDefaultGeometryFunction(extractCoordSys); + return getDefaultADQLFunction(extractCoordSys); } @Override public String translate(AreaFunction areaFunction) throws TranslationException{ - return getDefaultGeometryFunction(areaFunction); + return getDefaultADQLFunction(areaFunction); } @Override public String translate(CentroidFunction centroidFunction) throws TranslationException{ - return getDefaultGeometryFunction(centroidFunction); + return getDefaultADQLFunction(centroidFunction); } @Override public String translate(DistanceFunction fct) throws TranslationException{ - return getDefaultGeometryFunction(fct); + return getDefaultADQLFunction(fct); } @Override public String translate(ContainsFunction fct) throws TranslationException{ - return getDefaultGeometryFunction(fct); + return getDefaultADQLFunction(fct); } @Override public String translate(IntersectsFunction fct) throws TranslationException{ - return getDefaultGeometryFunction(fct); + return getDefaultADQLFunction(fct); } @Override public String translate(BoxFunction box) throws TranslationException{ - return getDefaultGeometryFunction(box); + return getDefaultADQLFunction(box); } @Override public String translate(CircleFunction circle) throws TranslationException{ - return getDefaultGeometryFunction(circle); + return getDefaultADQLFunction(circle); } @Override public String translate(PointFunction point) throws TranslationException{ - return getDefaultGeometryFunction(point); + return getDefaultADQLFunction(point); } @Override public String translate(PolygonFunction polygon) throws TranslationException{ - return getDefaultGeometryFunction(polygon); + return getDefaultADQLFunction(polygon); } @Override public String translate(RegionFunction region) throws TranslationException{ - return getDefaultGeometryFunction(region); + return getDefaultADQLFunction(region); + } + + @Override + public DBType convertTypeFromDB(final int dbmsType, final String rawDbmsTypeName, String dbmsTypeName, final String[] params){ + // If no type is provided return VARCHAR: + if (dbmsTypeName == null || dbmsTypeName.trim().length() == 0) + return new DBType(DBDatatype.VARCHAR, DBType.NO_LENGTH); + + // Put the dbmsTypeName in lower case for the following comparisons: + dbmsTypeName = dbmsTypeName.toLowerCase(); + + // Extract the length parameter (always the first one): + int lengthParam = DBType.NO_LENGTH; + if (params != null && params.length > 0){ + try{ + lengthParam = Integer.parseInt(params[0]); + }catch(NumberFormatException nfe){} + } + + // SMALLINT + if (dbmsTypeName.equals("smallint") || dbmsTypeName.equals("int2") || dbmsTypeName.equals("smallserial") || dbmsTypeName.equals("serial2") || dbmsTypeName.equals("boolean") || dbmsTypeName.equals("bool")) + return new DBType(DBDatatype.SMALLINT); + // INTEGER + else if (dbmsTypeName.equals("integer") || dbmsTypeName.equals("int") || dbmsTypeName.equals("int4") || dbmsTypeName.equals("serial") || dbmsTypeName.equals("serial4")) + return new DBType(DBDatatype.INTEGER); + // BIGINT + else if (dbmsTypeName.equals("bigint") || dbmsTypeName.equals("int8") || dbmsTypeName.equals("bigserial") || dbmsTypeName.equals("bigserial8")) + return new DBType(DBDatatype.BIGINT); + // REAL + else if (dbmsTypeName.equals("real") || dbmsTypeName.equals("float4")) + return new DBType(DBDatatype.REAL); + // DOUBLE + else if (dbmsTypeName.equals("double precision") || dbmsTypeName.equals("float8")) + return new DBType(DBDatatype.DOUBLE); + // BINARY + else if (dbmsTypeName.equals("bit")) + return new DBType(DBDatatype.BINARY, lengthParam); + // VARBINARY + else if (dbmsTypeName.equals("bit varying") || dbmsTypeName.equals("varbit")) + return new DBType(DBDatatype.VARBINARY, lengthParam); + // CHAR + else if (dbmsTypeName.equals("char") || dbmsTypeName.equals("character")) + return new DBType(DBDatatype.CHAR, lengthParam); + // VARCHAR + else if (dbmsTypeName.equals("varchar") || dbmsTypeName.equals("character varying")) + return new DBType(DBDatatype.VARCHAR, lengthParam); + // BLOB + else if (dbmsTypeName.equals("bytea")) + return new DBType(DBDatatype.BLOB); + // CLOB + else if (dbmsTypeName.equals("text")) + return new DBType(DBDatatype.CLOB); + // TIMESTAMP + else if (dbmsTypeName.equals("timestamp") || dbmsTypeName.equals("timestamptz") || dbmsTypeName.equals("time") || dbmsTypeName.equals("timetz") || dbmsTypeName.equals("date")) + return new DBType(DBDatatype.TIMESTAMP); + // Default: + else + return new DBType(DBDatatype.VARCHAR, DBType.NO_LENGTH); + } + + @Override + public String convertTypeToDB(final DBType type){ + if (type == null) + return "VARCHAR"; + + switch(type.type){ + + case SMALLINT: + case INTEGER: + case REAL: + case BIGINT: + case CHAR: + case VARCHAR: + case TIMESTAMP: + return type.type.toString(); + + case DOUBLE: + return "DOUBLE PRECISION"; + + case BINARY: + case VARBINARY: + return "bytea"; + + case BLOB: + return "bytea"; + + case CLOB: + return "TEXT"; + + case POINT: + case REGION: + default: + return "VARCHAR"; + } + } + + @Override + public Region translateGeometryFromDB(final Object jdbcColValue) throws ParseException{ + throw new ParseException("Unsupported geometrical value! The value \"" + jdbcColValue + "\" can not be parsed as a region."); + } + + @Override + public Object translateGeometryToDB(final Region region) throws ParseException{ + throw new ParseException("Geometries can not be uploaded in the database in this implementation!"); } } diff --git a/src/cds/utils/TextualSearchList.java b/src/cds/utils/TextualSearchList.java index dfbff861dff639e2e287660ad31c36893d57ac73..3dd9888b0f0f866f36f2a590c9cf09dcc2d78b4c 100644 --- a/src/cds/utils/TextualSearchList.java +++ b/src/cds/utils/TextualSearchList.java @@ -17,7 +17,7 @@ package cds.utils; * along with ADQLLibrary. If not, seeLet process completely an ADQL query.
* + *Thus, this class aims to apply the following actions (in the given order):
+ *+ * This executor is able to process queries coming from a synchronous job (the result must be written directly in the HTTP response) + * and from an asynchronous job (the result must be written, generally, in a file). Two start(...) functions let deal with + * the differences between the two job execution modes: {@link #start(AsyncThread)} for asynchronous jobs + * and {@link #start(Thread, String, TAPParameters, HttpServletResponse)} for synchronous jobs. + *
+ * + *Uploaded tables must be provided in VOTable format.
+ * + *+ * Query results must be formatted in the format specified by the user in the job parameters. A corresponding formatter ({@link OutputFormat}) + * is asked to the description of the TAP service ({@link ServiceConnection}). If none can be found, VOTable will be chosen by default. + *
+ * + *It is totally possible to customize some parts of the ADQL query processing. However, the main algorithm must remain the same and is implemented + * by {@link #start()}. This function is final, like {@link #start(AsyncThread)} and {@link #start(Thread, String, TAPParameters, HttpServletResponse)}, + * which are just preparing the execution for {@link #start()} in function of the job execution mode (asynchronous or synchronous). + *
+ * + *Note: + * {@link #start()} is using the Template Method Design Pattern: it defines the skeleton/algorithm of the processing, and defers some steps + * to other functions. + *
+ * + *+ * So, you are able to customize almost all individual steps of the ADQL query processing: {@link #parseADQL()}, {@link #executeADQL(ADQLQuery)} and + * {@link #writeResult(TableIterator, OutputFormat, OutputStream)}. + *
+ * + *Note: + * Note that the formatting of the result is done by an OutputFormat and that the executor is just calling the appropriate function of the formatter. + *
+ * + *+ * There is no way in this executor to customize the upload. However, it does not mean it can not be customized. + * Indeed you can do it easily by extending {@link Uploader} and by providing the new class inside your {@link TAPFactory} implementation + * (see {@link TAPFactory#createUploader(DBConnection)}). + *
+ * + * @author Grégory Mantelet (CDS;ARI) + * @version 2.0 (04/2015) */ -public class ADQLExecutor< R > { +public class ADQLExecutor { - protected final ServiceConnectionGet the report of the query execution. It helps indicating the execution progression and the duration of each step.
+ * + *Note: + * Before starting the execution (= before the call of a "start(...)" function), this function will return NULL. + * It is set when the query processing starts and remains not NULL after that (even when the execution is finished). + *
+ * + * @return The execution report. + */ public final TAPExecutionReport getExecReport(){ return report; } - public boolean hasUploadedTables(){ - return (uploadSchema != null) && (uploadSchema.getNbTables() > 0); - } - - protected final DBConnectionGet the object to use in order to write the query result in the appropriate format + * (either the asked one, or else VOTable).
+ * + * @return The appropriate result formatter to use. Can not be NULL! + * + * @throws TAPException If no format corresponds to the asked one and if no default format (for VOTable) can be found. + * + * @see ServiceConnection#getOutputFormat(String) + */ + protected OutputFormat getFormatter() throws TAPException{ + // Search for the corresponding formatter: + String format = tapParams.getFormat(); + OutputFormat formatter = service.getOutputFormat((format == null) ? "votable" : format); + if (format != null && formatter == null) + formatter = service.getOutputFormat("votable"); - tapParams.set(TAPJob.PARAM_PROGRESSION, ExecutionProgression.EXECUTING_SQL); - start = System.currentTimeMillis(); - R result = executeQuery(sqlQuery, adql); - report.setDuration(ExecutionProgression.EXECUTING_SQL, System.currentTimeMillis() - start); + // Format the result: + if (formatter == null) + throw new TAPException("Impossible to format the query result: no formatter has been found for the given MIME type \"" + format + "\" and for the default MIME type \"votable\" (short form) !"); - return result; + return formatter; } - public final TAPExecutionReport start(final AsyncThreadStart the asynchronous processing of the ADQL query.
+ * + *+ * This function initialize the execution report, get the execution parameters (including the query to process) + * and then call {@link #start()}. + *
+ * + * @param thread The asynchronous thread which asks the query processing. + * + * @return The resulting execution report. + * + * @throws UWSException If any error occurs while executing the ADQL query. + * @throws InterruptedException If the job has been interrupted (by the user or a time-out). + * + * @see #start() + */ + public final TAPExecutionReport start(final AsyncThread thread) throws UWSException, InterruptedException{ if (this.thread != null || this.report != null) - throw new UWSException(UWSException.INTERNAL_SERVER_ERROR, "This ADQLExecutor has already been executed !"); + throw new UWSException(UWSException.INTERNAL_SERVER_ERROR, "This ADQLExecutor has already been executed!"); this.thread = thread; @@ -159,164 +221,480 @@ public class ADQLExecutor< R > { this.report = new TAPExecutionReport(tapJob.getJobId(), false, tapParams); this.response = null; - return start(); + try{ + return start(); + }catch(IOException ioe){ + throw new UWSException(ioe); + }catch(TAPException te){ + throw new UWSException(te.getHttpErrorCode(), te); + } + } + + /** + *Create the database connection required for the ADQL execution.
+ * + *Note: This function has no effect if the DB connection already exists.
+ * + * @param jobID ID of the job which will be executed by this {@link ADQLExecutor}. + * This ID will be the database connection ID. + * + * @throws TAPException If the DB connection creation fails. + * + * @see TAPFactory#getConnection(String) + * + * @since 2.0 + */ + public final void initDBConnection(final String jobID) throws TAPException{ + if (dbConn == null) + dbConn = service.getFactory().getConnection(jobID); } - public final TAPExecutionReport start(final Thread thread, final String jobId, final TAPParameters params, final HttpServletResponse response) throws TAPException, UWSException, InterruptedException, ParseException, TranslationException, SQLException{ + /** + *Start the synchronous processing of the ADQL query.
+ * + *This function initialize the execution report and then call {@link #start()}.
+ * + * @param thread The synchronous thread which asks the query processing. + * @param jobId ID of the corresponding job. + * @param params All execution parameters (including the query to process). + * @param response Object in which the result or the error must be written. + * + * @return The resulting execution report. + * + * @throws TAPException If any error occurs while executing the ADQL query. + * @throws IOException If any error occurs while writing the result in the given {@link HttpServletResponse}. + * @throws InterruptedException If the job has been interrupted (by the user or a time-out). + * + * @see #start() + */ + public final TAPExecutionReport start(final Thread thread, final String jobId, final TAPParameters params, final HttpServletResponse response) throws TAPException, IOException, InterruptedException{ if (this.thread != null || this.report != null) - throw new TAPException("This ADQLExecutor has already been executed !"); + throw new TAPException("This ADQLExecutor has already been executed!"); this.thread = thread; this.tapParams = params; this.report = new TAPExecutionReport(jobId, true, tapParams); this.response = response; - return start(); + try{ + return start(); + }catch(UWSException ue){ + throw new TAPException(ue, ue.getHttpErrorCode()); + } } - protected final TAPExecutionReport start() throws TAPException, UWSException, InterruptedException, ParseException, TranslationException, SQLException{ + /** + *Process the ADQL query.
+ * + *This function calls the following function (in the same order):
+ *+ * The execution report is updated gradually. Besides a job parameter - progression - is set at each step of the process in order to + * notify the user of the progression of the query execution. This parameter is removed at the end of the execution if it is successful. + *
+ * + *The "interrupted" flag of the associated thread is often tested so that stopping the execution as soon as possible.
+ * + * @return The updated execution report. + * + * @throws TAPException If any error occurs while executing the ADQL query. + * @throws UWSException If any error occurs while executing the ADQL query. + * @throws IOException If an error happens while writing the result in the specified {@link HttpServletResponse}. + * That kind of error can be thrown only in synchronous mode. + * In asynchronous, the error is stored as job error report and is never propagated. + * @throws InterruptedException If the job has been interrupted (by the user or a time-out). + */ + protected final TAPExecutionReport start() throws TAPException, UWSException, IOException, InterruptedException{ + logger.logTAP(LogLevel.INFO, report, "START_EXEC", (report.synchronous ? "Synchronous" : "Asynchronous") + " execution of an ADQL query STARTED.", null); + + // Save the start time (for reporting usage): long start = System.currentTimeMillis(); + + TableIterator queryResult = null; + try{ - // Upload tables if needed: - if (tapParams != null && tapParams.getTableLoaders() != null && tapParams.getTableLoaders().length > 0){ - tapParams.set(TAPJob.PARAM_PROGRESSION, ExecutionProgression.UPLOADING); + // Get a "database" connection: + initDBConnection(report.jobID); + + // 1. UPLOAD TABLES, if there is any: + if (tapParams.getUploadedTables() != null && tapParams.getUploadedTables().length > 0){ + startStep(ExecutionProgression.UPLOADING); uploadTables(); + endStep(); } if (thread.isInterrupted()) throw new InterruptedException(); - // Parse, translate in SQL and execute the ADQL query: - R queryResult = executeADQL(); - if (queryResult == null || thread.isInterrupted()) + // 2. PARSE THE ADQL QUERY: + startStep(ExecutionProgression.PARSING); + // Parse the query: + ADQLQuery adqlQuery = null; + try{ + adqlQuery = parseADQL(); + }catch(ParseException pe){ + if (report.synchronous) + throw new TAPException("Incorrect ADQL query: " + pe.getMessage(), pe, UWSException.BAD_REQUEST, tapParams.getQuery(), progression); + else + throw new UWSException(UWSException.BAD_REQUEST, pe, "Incorrect ADQL query: " + pe.getMessage()); + } + // List all resulting columns (it will be useful later to format the result): + report.resultingColumns = adqlQuery.getResultingColumns(); + endStep(); + + if (thread.isInterrupted()) throw new InterruptedException(); - // Write the result: - tapParams.set(TAPJob.PARAM_PROGRESSION, ExecutionProgression.WRITING_RESULT); - writeResult(queryResult); + // 3. EXECUTE THE ADQL QUERY: + startStep(ExecutionProgression.EXECUTING_ADQL); + queryResult = executeADQL(adqlQuery); + endStep(); - logger.info("JOB " + report.jobID + " COMPLETED"); - tapParams.set(TAPJob.PARAM_PROGRESSION, ExecutionProgression.FINISHED); + if (thread.isInterrupted()) + throw new InterruptedException(); + // 4. WRITE RESULT: + startStep(ExecutionProgression.WRITING_RESULT); + writeResult(queryResult); + endStep(); + + // Report the COMPLETED status: + tapParams.remove(TAPJob.PARAM_PROGRESSION); report.success = true; + // Set the total duration in the report: + report.setTotalDuration(System.currentTimeMillis() - start); + + // Log and report the end of this execution: + logger.logTAP(LogLevel.INFO, report, "END_EXEC", "ADQL query execution finished.", null); + return report; - }catch(NullPointerException npe){ - npe.printStackTrace(); - throw npe; }finally{ + // Close the result if any: + if (queryResult != null){ + try{ + queryResult.close(); + }catch(DataReadException dre){ + logger.logTAP(LogLevel.WARNING, report, "END_EXEC", "Can not close the database query result!", dre); + } + } + + // Drop all the uploaded tables (they are not supposed to exist after the query execution): try{ dropUploadedTables(); }catch(TAPException e){ - logger.error("JOB " + report.jobID + "\tCan not drop uploaded tables !", e); + logger.logTAP(LogLevel.WARNING, report, "END_EXEC", "Can not drop the uploaded tables from the database!", e); } - try{ - closeDBConnection(); - }catch(TAPException e){ - logger.error("JOB " + report.jobID + "\tCan not close the DB connection !", e); + + // Free the connection (so that giving it back to a pool, if any, otherwise, just free resources): + if (dbConn != null){ + service.getFactory().freeConnection(dbConn); + dbConn = null; } - report.setTotalDuration(System.currentTimeMillis() - start); - logger.queryFinished(report); } } - protected ADQLQuery parseADQL() throws ParseException, InterruptedException, TAPException{ - ADQLQueryFactory queryFactory = service.getFactory().createQueryFactory(); - QueryChecker queryChecker = service.getFactory().createQueryChecker(uploadSchema); - ADQLParser parser; - if (queryFactory == null) - parser = new ADQLParser(queryChecker); - else - parser = new ADQLParser(queryChecker, queryFactory); - parser.setCoordinateSystems(service.getCoordinateSystems()); - parser.setDebug(false); - //logger.info("Job "+report.jobID+" - 1/5 Parsing ADQL...."); - return parser.parseQuery(tapParams.getQuery()); + /** + *Memorize the time at which the step starts, the step ID and update the job parameter "progression" + * (to notify the user about the progression of the query processing).
+ * + *Note: + * If for some reason the job parameter "progression" can not be updated, no error will be thrown. A WARNING message + * will be just written in the log. + *
+ * + *Note: + * This function is designed to work with {@link #endStep()}, which must be called after it, when the step is finished (successfully or not). + *
+ * + * @param progression ID of the starting step. + * + * @see #endStep() + */ + private void startStep(final ExecutionProgression progression){ + // Save the start time (for report usage): + startStep = System.currentTimeMillis(); + // Memorize the current step: + this.progression = progression; + // Update the job parameter "progression", to notify the user about the progression of the query processing: + try{ + tapParams.set(TAPJob.PARAM_PROGRESSION, this.progression); + }catch(UWSException ue){ + // should not happen, but just in case... + logger.logTAP(LogLevel.WARNING, report, "START_STEP", "Can not set/update the informative job parameter \"" + TAPJob.PARAM_PROGRESSION + "\" (this parameter would be just for notification purpose about the execution progression)!", ue); + } } - protected String translateADQL(ADQLQuery query) throws TranslationException, InterruptedException, TAPException{ - ADQLTranslator translator = service.getFactory().createADQLTranslator(); - //logger.info("Job "+report.jobID+" - 2/5 Translating ADQL..."); - return translator.translate(query); + /** + *Set the duration of the current step in the execution report.
+ * + *Note: + * The start time and the ID of the step are then forgotten. + *
+ * + *Note: + * This function is designed to work with {@link #startStep(ExecutionProgression)}, which must be called before it, when the step is starting. + * It marks the end of a step. + *
+ * + * @see #startStep(ExecutionProgression) + */ + private void endStep(){ + if (progression != null){ + // Set the duration of this step in the execution report: + report.setDuration(progression, System.currentTimeMillis() - startStep); + // No start time: + startStep = -1; + // No step for the moment: + progression = null; + } } - protected R executeQuery(String sql, ADQLQuery adql) throws SQLException, InterruptedException, TAPException{ - //logger.info("Job "+report.jobID+" - 3/5 Creating DBConnection...."); - DBConnectionCreate in the "database" all tables uploaded by the user (only for this specific query execution).
+ * + *Note: + * Obviously, nothing is done if no table has been uploaded. + *
+ * + * @throws TAPException If any error occurs while reading the uploaded table + * or while importing them in the database. + */ + private final void uploadTables() throws TAPException{ + // Fetch the tables to upload: + DALIUpload[] tables = tapParams.getUploadedTables(); + + // Upload them, if needed: + if (tables.length > 0){ + logger.logTAP(LogLevel.INFO, report, "UPLOADING", "Loading uploaded tables (" + tables.length + ")", null); + uploadSchema = service.getFactory().createUploader(dbConn).upload(tables); + } } - protected OutputFormatParse the ADQL query provided in the parameters by the user.
+ * + *The query factory and the query checker are got from the TAP factory.
+ * + *+ * The configuration of this TAP service list all allowed coordinate systems. These are got here and provided to the query checker + * in order to ensure the coordinate systems used in the query are in this list. + *
+ * + *+ * The row limit specified in the ADQL query (with TOP) is checked and adjusted (if needed). Indeed, this limit + * can not exceed MAXREC given in parameter and the maximum value specified in the configuration of this TAP service. + * In the case no row limit is specified in the query or the given value is greater than MAXREC, (MAXREC+1) is used by default. + * The "+1" aims to detect overflows. + *
+ * + * @return The object representation of the ADQL query. + * + * @throws ParseException If the given ADQL query can not be parsed or if the construction of the object representation has failed. + * @throws InterruptedException If the thread has been interrupted. + * @throws TAPException If the TAP factory is unable to create the ADQL factory or the query checker. + */ + protected ADQLQuery parseADQL() throws ParseException, InterruptedException, TAPException{ + // Log the start of the parsing: + logger.logTAP(LogLevel.INFO, report, "PARSING", "Parsing ADQL: " + tapParams.getQuery().replaceAll("(\t|\r?\n)+", " "), null); + + // Create the ADQL parser: + ADQLParser parser = service.getFactory().createADQLParser(); + if (parser == null){ + logger.logTAP(LogLevel.WARNING, null, "PARSING", "No ADQL parser returned by the TAPFactory! The default implementation is used instead.", null); + parser = new ADQLParser(); + } - // Format the result: - if (formatter == null) - throw new TAPException("Impossible to format the query result: no formatter has been found for the given MIME type \"" + format + "\" and for the default MIME type \"votable\" (short form) !"); + // Set the ADQL factory: + if (parser.getQueryFactory() == null || parser.getQueryFactory().getClass() == ADQLQueryFactory.class) + parser.setQueryFactory(service.getFactory().createQueryFactory()); - return formatter; + // Set the query checker: + if (parser.getQueryChecker() == null) + parser.setQueryChecker(service.getFactory().createQueryChecker(uploadSchema)); + + // Parse the ADQL query: + ADQLQuery query = parser.parseQuery(tapParams.getQuery()); + + // Set or check the row limit: + final int limit = query.getSelect().getLimit(); + final Integer maxRec = tapParams.getMaxRec(); + if (maxRec != null && maxRec > -1){ + if (limit <= -1 || limit > maxRec) + query.getSelect().setLimit(maxRec + 1); + } + + return query; } - protected final void writeResult(R queryResult) throws InterruptedException, TAPException, UWSException{ - OutputFormatExecute in "database" the given object representation of an ADQL query.
+ * + *By default, this function is just calling {@link DBConnection#executeQuery(ADQLQuery)} and then it returns the value returned by this call.
+ * + *Note: + * An INFO message is logged at the end of the query execution in order to report the result status (success or error) + * and the execution duration. + *
+ * + * @param adql The object representation of the ADQL query to execute. + * + * @return The result of the query, + * or NULL if the query execution has failed. + * + * @throws InterruptedException If the thread has been interrupted. + * @throws TAPException If the {@link DBConnection} has failed to deal with the given ADQL query. + * + * @see DBConnection#executeQuery(ADQLQuery) + */ + protected TableIterator executeADQL(final ADQLQuery adql) throws InterruptedException, TAPException{ + // Log the start of execution: + logger.logTAP(LogLevel.INFO, report, "START_DB_EXECUTION", "ADQL query: " + adql.toADQL().replaceAll("(\t|\r?\n)+", " "), null); + + // Set the fetch size, if any: + if (service.getFetchSize() != null && service.getFetchSize().length >= 1){ + if (report.synchronous && service.getFetchSize().length >= 2) + dbConn.setFetchSize(service.getFetchSize()[1]); + else + dbConn.setFetchSize(service.getFetchSize()[0]); + } + + // Execute the ADQL query: + TableIterator result = dbConn.executeQuery(adql); - // Synchronous case: + // Log the success or failure: + if (result == null) + logger.logTAP(LogLevel.INFO, report, "END_DB_EXECUTION", "Query execution aborted after " + (System.currentTimeMillis() - startStep) + "ms!", null); + else + logger.logTAP(LogLevel.INFO, report, "END_DB_EXECUTION", "Query successfully executed in " + (System.currentTimeMillis() - startStep) + "ms!", null); + + return result; + } + + /** + *Write the given query result into the appropriate format in the appropriate output + * (HTTP response for a synchronous execution, otherwise a file or any output provided by UWS).
+ * + *This function prepare the output in function of the execution type (synchronous or asynchronous). + * Once prepared, the result, the output and the formatter to use are given to {@link #writeResult(TableIterator, OutputFormat, OutputStream)} + * which will really process the result formatting and writing. + *
+ * + * @param queryResult The result of the query execution in database. + * + * @throws InterruptedException If the thread has been interrupted. + * @throws IOException If an error happens while writing the result in the {@link HttpServletResponse}. + * That kind of error can be thrown only in synchronous mode. + * In asynchronous, the error is stored as job error report and is never propagated. + * @throws TAPException If an error occurs while getting the appropriate formatter or while formatting or writing (synchronous execution) the result. + * @throws UWSException If an error occurs while getting the output stream or while writing (asynchronous execution) the result. + * + * @see #writeResult(TableIterator, OutputFormat, OutputStream) + */ + protected final void writeResult(final TableIterator queryResult) throws InterruptedException, IOException, TAPException, UWSException{ + // Log the start of the writing: + logger.logTAP(LogLevel.INFO, report, "WRITING_RESULT", "Writing the query result", null); + + // Get the appropriate result formatter: + OutputFormat formatter = getFormatter(); + + // CASE SYNCHRONOUS: if (response != null){ - long start = System.currentTimeMillis(); - try{ - response.setContentType(formatter.getMimeType()); - writeResult(queryResult, formatter, response.getOutputStream()); - }catch(IOException ioe){ - throw new TAPException("Impossible to get the output stream of the HTTP request to write the result of the job " + report.jobID + " !", ioe); - }finally{ - report.setDuration(ExecutionProgression.WRITING_RESULT, System.currentTimeMillis() - start); - } + long start = -1; + + // Set the HTTP content type to the MIME type of the result format: + response.setContentType(formatter.getMimeType()); + + // Set the character encoding: + response.setCharacterEncoding(UWSToolBox.DEFAULT_CHAR_ENCODING); + + // Write the formatted result in the HTTP response output: + start = System.currentTimeMillis(); + writeResult(queryResult, formatter, response.getOutputStream()); - }// Asynchronous case: + logger.logTAP(LogLevel.INFO, report, "RESULT_WRITTEN", "Result formatted (in " + formatter.getMimeType() + " ; " + (report.nbRows < 0 ? "?" : report.nbRows) + " rows ; " + ((report.resultingColumns == null) ? "?" : report.resultingColumns.length) + " columns) in " + ((start <= 0) ? "?" : (System.currentTimeMillis() - start)) + "ms!", null); + } + // CASE ASYNCHRONOUS: else{ - long start = System.currentTimeMillis(); + long start = -1, end = -1; try{ + // Create a UWS Result object to store the result + // (the result will be stored in a file and this object is the association between the job and the result file): JobThread jobThread = (JobThread)thread; Result result = jobThread.createResult(); + + // Set the MIME type of the result format in the result description: result.setMimeType(formatter.getMimeType()); + + // Write the formatted result in the file output: + start = System.currentTimeMillis(); writeResult(queryResult, formatter, jobThread.getResultOutput(result)); + end = System.currentTimeMillis(); + + // Set the size (in bytes) of the result in the result description: result.setSize(jobThread.getResultSize(result)); + + // Add the result description and link in the job description: jobThread.publishResult(result); + + logger.logTAP(LogLevel.INFO, report, "RESULT_WRITTEN", "Result formatted (in " + formatter.getMimeType() + " ; " + (report.nbRows < 0 ? "?" : report.nbRows) + " rows ; " + ((report.resultingColumns == null) ? "?" : report.resultingColumns.length) + " columns) in " + ((start <= 0 || end <= 0) ? "?" : (end - start)) + "ms!", null); + }catch(IOException ioe){ - throw new UWSException(UWSException.INTERNAL_SERVER_ERROR, ioe, "Impossible to get the output stream of the result file to write the result of the job " + report.jobID + " !"); - }finally{ - report.setDuration(ExecutionProgression.WRITING_RESULT, System.currentTimeMillis() - start); + throw new UWSException(UWSException.INTERNAL_SERVER_ERROR, ioe, "Impossible to write in the file into the result of the job " + report.jobID + " must be written!"); } } } - protected void writeResult(R queryResult, OutputFormatFormat and write the given result in the given output with the given formatter.
+ * + *By default, this function is just calling {@link OutputFormat#writeResult(TableIterator, OutputStream, TAPExecutionReport, Thread)}.
+ * + *Note: + * {@link OutputFormat#writeResult(TableIterator, OutputStream, TAPExecutionReport, Thread)} is often testing the "interrupted" flag of the + * thread in order to stop as fast as possible if the user has cancelled the job or if the thread has been interrupted for another reason. + *
+ * + * @param queryResult Query result to format and to output. + * @param formatter The object able to write the result in the appropriate format. + * @param output The stream in which the result must be written. + * + * @throws InterruptedException If the thread has been interrupted. + * @throws IOException If there is an error while writing the result in the given stream. + * @throws TAPException If there is an error while formatting the result. + */ + protected void writeResult(TableIterator queryResult, OutputFormat formatter, OutputStream output) throws InterruptedException, IOException, TAPException{ formatter.writeResult(queryResult, output, report, thread); } + /** + *Drop all tables uploaded by the user from the database.
+ * + *Note: + * By default, if an error occurs while dropping a table from the database, the error will just be logged ; it won't be thrown/propagated. + *
+ * + * @throws TAPException If a grave error occurs. By default, no exception is thrown ; they are just logged. + */ protected void dropUploadedTables() throws TAPException{ if (uploadSchema != null){ // Drop all uploaded tables: - DBConnectionBuild a basic TAPFactory. + * Nothing is done except setting the service connection and the given error writer.
+ * + *Then the error writer will be used when creating a UWS service and a job thread.
+ * + * @param service Configuration of the TAP service. MUST NOT be NULL + * @param errorWriter Object to use to format and write the errors for the user. + * + * @throws NullPointerException If the given {@link ServiceConnection} is NULL. + * + * @see TAPFactory#TAPFactory(ServiceConnection) + */ + protected AbstractTAPFactory(final ServiceConnection service, final ServiceErrorWriter errorWriter) throws NullPointerException{ + super(service); + this.errorWriter = errorWriter; } @Override - public UWSService createUWS() throws TAPException, UWSException{ - return new UWSService(this.service.getFactory(), this.service.getFileManager(), this.service.getLogger()); + public final ServiceErrorWriter getErrorWriter(){ + return errorWriter; } + /* *************** */ + /* ADQL MANAGEMENT */ + /* *************** */ + + /** + *Note: + * Unless the standard implementation - {@link ADQLExecutor} - does not fit exactly your needs, + * it should not be necessary to extend this class and to extend this function (implemented here by default). + *
+ */ @Override - public UWSBackupManager createUWSBackupManager(final UWSService uws) throws TAPException, UWSException{ - return null; + public ADQLExecutor createADQLExecutor() throws TAPException{ + return new ADQLExecutor(service); } + /** + *Note: + * This function should be extended if you want to customize the ADQL grammar. + *
+ */ @Override - public UWSJob createJob(HttpServletRequest request, JobOwner owner) throws UWSException{ - if (!service.isAvailable()) - throw new UWSException(HttpServletResponse.SC_SERVICE_UNAVAILABLE, service.getAvailability()); + public ADQLParser createADQLParser() throws TAPException{ + return new ADQLParser(); + } - try{ - TAPParameters tapParams = (TAPParameters)createUWSParameters(request); - return new TAPJob(owner, tapParams); - }catch(TAPException te){ - throw new UWSException(UWSException.INTERNAL_SERVER_ERROR, te, "Can not create a TAP asynchronous job !"); - } + /** + *Note: + * This function should be extended if you have customized the creation of any + * {@link ADQLQuery} part ; it could be the addition of one or several user defined function + * or the modification of any ADQL function or clause specific to your implementation. + *
+ */ + @Override + public ADQLQueryFactory createQueryFactory() throws TAPException{ + return new ADQLQueryFactory(); } + /** + *This implementation gathers all tables published in this TAP service and those uploaded + * by the user. Then it calls {@link #createQueryChecker(Collection)} with this list in order + * to create a query checked. + *
+ * + *Note: + * This function can not be overridded, but {@link #createQueryChecker(Collection)} can be. + *
+ */ @Override - public UWSJob createJob(String jobId, JobOwner owner, final UWSParameters params, long quote, long startTime, long endTime, ListCreate an object able to check the consistency between the ADQL query and the database. + * That's to say, it checks whether the tables and columns used in the query really exist + * in the database.
+ * + *Note: + * This implementation just create a {@link DBChecker} instance with the list given in parameter. + *
+ * + * @param tables List of all available tables (and indirectly, columns). + * + * @return A new ADQL query checker. + * + * @throws TAPException If any error occurs while creating the query checker. + */ + protected QueryChecker createQueryChecker(final CollectionThis implementation just create an {@link Uploader} instance with the given database connection.
+ * + *Note: + * This function should be overrided if you need to change the DB name of the TAP_UPLOAD schema. + * Indeed, by overriding this function you can specify a given TAPSchema to use as TAP_UPLOAD schema + * in the constructor of {@link Uploader}. But do not forget that this {@link TAPSchema} instance MUST have + * an ADQL name equals to "TAP_UPLOAD", otherwise, a TAPException will be thrown. + *
+ */ @Override - public final JobThread createJobThread(final UWSJob job) throws UWSException{ + public Uploader createUploader(final DBConnection dbConn) throws TAPException{ + return new Uploader(service, dbConn); + } + + /* ************** */ + /* UWS MANAGEMENT */ + /* ************** */ + + /** + *This implementation just create a {@link UWSService} instance.
+ * + *Note: + * This implementation is largely enough for a TAP service. It is not recommended to override + * this function. + *
+ */ + @Override + public UWSService createUWS() throws TAPException{ try{ - return new AsyncThreadThis implementation does not provided a backup manager. + * It means that no asynchronous job will be restored and backuped.
+ * + *You must override this function if you want enable the backup feature.
+ */ + @Override + public UWSBackupManager createUWSBackupManager(final UWSService uws) throws TAPException{ + return null; } /** - * Extracts the parameters from the given request (multipart or not). - * This function is used only to set UWS parameters, not to create a TAP query (for that, see {@link TAPParameters}). + *This implementation provides a basic {@link TAPJob} instance.
* - * @see uws.service.AbstractUWSFactory#extractParameters(javax.servlet.http.HttpServletRequest, uws.service.UWS) + *+ * If you need to add or modify the behavior of some functions of a {@link TAPJob}, + * you must override this function and return your own extension of {@link TAPJob}. + *
*/ @Override - public UWSParameters createUWSParameters(HttpServletRequest request) throws UWSException{ + protected TAPJob createTAPJob(final HttpServletRequest request, final JobOwner owner) throws UWSException{ try{ - return new TAPParameters(request, service, getExpectedAdditionalParameters(), getInputParamControllers()); + TAPParameters tapParams = createTAPParameters(request); + return new TAPJob(owner, tapParams); }catch(TAPException te){ - throw new UWSException(UWSException.INTERNAL_SERVER_ERROR, te); + if (te.getCause() != null && te.getCause() instanceof UWSException) + throw (UWSException)te.getCause(); + else + throw new UWSException(UWSException.INTERNAL_SERVER_ERROR, te, "Can not create a TAP asynchronous job!"); } } + /** + *This implementation provides a basic {@link TAPJob} instance.
+ * + *+ * If you need to add or modify the behavior of some functions of a {@link TAPJob}, + * you must override this function and return your own extension of {@link TAPJob}. + *
+ */ @Override - public UWSParameters createUWSParameters(MapThis implementation extracts standard TAP parameters from the given request.
+ * + *+ * Non-standard TAP parameters are added in a map inside the returned {@link TAPParameters} object + * and are accessible with {@link TAPParameters#get(String)} and {@link TAPParameters#getAdditionalParameters()}. + * However, if you want to manage them in another way, you must extend {@link TAPParameters} and override + * this function in order to return an instance of your extension. + *
+ */ @Override - public ADQLQueryFactory createQueryFactory() throws TAPException{ - return new ADQLQueryFactory(); + public TAPParameters createTAPParameters(final HttpServletRequest request) throws TAPException{ + return new TAPParameters(request, service); } + /** + *This implementation extracts standard TAP parameters from the given request.
+ * + *+ * Non-standard TAP parameters are added in a map inside the returned {@link TAPParameters} object + * and are accessible with {@link TAPParameters#get(String)} and {@link TAPParameters#getAdditionalParameters()}. + * However, if you want to manage them in another way, you must extend {@link TAPParameters} and override + * this function in order to return an instance of your extension. + *
+ */ @Override - public QueryChecker createQueryChecker(TAPSchema uploadSchema) throws TAPException{ - TAPMetadata meta = service.getTAPMetadata(); - ArrayListCheck whether this thread is able to start right now.
+ * + *+ * Basically, this function asks to the {@link ADQLExecutor} to get a database connection. If no DB connection is available, + * then this thread can not start and this function return FALSE. In all the other cases, TRUE is returned. + *
+ * + *Warning: This function will indirectly open and keep a database connection, so that the job can be started just after its call. + * If it turns out that the execution won't start just after this call, the DB connection should be closed in some way in order to save database resources.
+ * + * @return true if this thread can start right now, false otherwise. + * + * @since 2.0 + */ + public final boolean isReadyForExecution(){ + try{ + executor.initDBConnection(job.getJobId()); + return true; + }catch(TAPException te){ + return false; } - super.interrupt(); } @Override @@ -55,12 +83,6 @@ public class AsyncThread< R > extends JobThread { throw ie; }catch(UWSException ue){ throw ue; - }catch(TAPException te){ - throw new UWSException(te.getHttpErrorCode(), te, te.getMessage()); - }catch(ParseException pe){ - throw new UWSException(UWSException.BAD_REQUEST, pe, pe.getMessage()); - }catch(TranslationException te){ - throw new UWSException(UWSException.INTERNAL_SERVER_ERROR, te, te.getMessage()); }catch(Exception ex){ throw new UWSException(UWSException.INTERNAL_SERVER_ERROR, ex, "Error while processing the ADQL query of the job " + job.getJobId() + " !"); }finally{ @@ -68,6 +90,11 @@ public class AsyncThread< R > extends JobThread { } } + /** + * Get the description of the job that this thread is executing. + * + * @return The executed job. + */ public final TAPJob getTAPJob(){ return (TAPJob)job; } diff --git a/src/tap/ExecutionProgression.java b/src/tap/ExecutionProgression.java index 618d2104813c7d0584be6f136e4b4390b3ef95a6..4086ccdec638ef9b3139e7dae75949c38e291884 100644 --- a/src/tap/ExecutionProgression.java +++ b/src/tap/ExecutionProgression.java @@ -16,9 +16,16 @@ package tap; * You should have received a copy of the GNU Lesser General Public License * along with TAPLibrary. If not, seeDescription and parameters list of a TAP service.
+ * + *+ * Through this object, it is possible to configure the different limits and formats, + * but also to list all available tables and columns, to declare geometry features as all allowed user defined functions + * and to say where log and other kinds of files must be stored. + *
+ * + * @author Grégory Mantelet (CDS;ARI) + * @version 2.0 (03/2015) + */ +public interface ServiceConnection { + /** + * List of possible limit units. + * + * @author Grégory Mantelet (CDS;ARI) + * @version 2.0 (01/2015) + */ public static enum LimitUnit{ - rows, bytes; + rows("row"), bytes("byte"), kilobytes("kilobyte"), megabytes("megabyte"), gigabytes("gigabyte"); + + private final String str; + + private LimitUnit(final String str){ + this.str = str; + } + + /** + * Tells whether the given unit has the same type (bytes or rows). + * + * @param anotherUnit A unit. + * + * @return true if the given unit has the same type, false otherwise. + * + * @since 1.1 + */ + public boolean isCompatibleWith(final LimitUnit anotherUnit){ + if (this == rows) + return anotherUnit == rows; + else + return anotherUnit != rows; + } + + /** + * Gets the factor to convert into bytes the value expressed in this unit. + * Note: if this unit is not a factor of bytes, 1 is returned (so that the factor does not affect the value). + * + * @return The factor need to convert a value expressed in this unit into bytes, or 1 if not a bytes derived unit. + * + * @since 1.1 + */ + public long bytesFactor(){ + switch(this){ + case bytes: + return 1; + case kilobytes: + return 1000; + case megabytes: + return 1000000; + case gigabytes: + return 1000000000l; + default: + return 1; + } + } + + /** + * Compares the 2 given values (each one expressed in the given unit). + * Conversions are done internally in order to make a correct comparison between the 2 limits. + * + * @param leftLimit Value/Limit of the comparison left part. + * @param leftUnit Unit of the comparison left part value. + * @param rightLimit Value/Limit of the comparison right part. + * @param rightUnit Unit of the comparison right part value. + * + * @return the value 0 if x == y; a value less than 0 if x < y; and a value greater than 0 if x > y + * + * @throws TAPException If the two given units are not compatible. + * + * @see tap.ServiceConnection.LimitUnit#isCompatibleWith(tap.ServiceConnection.LimitUnit) + * @see #bytesFactor() + * @see Integer#compare(int, int) + * @see Long#compare(long, long) + * + * @since 1.1 + */ + public static int compare(final int leftLimit, final LimitUnit leftUnit, final int rightLimit, final LimitUnit rightUnit) throws TAPException{ + if (!leftUnit.isCompatibleWith(rightUnit)) + throw new TAPException("Limit units (" + leftUnit + " and " + rightUnit + ") are not compatible!"); + + if (leftUnit == rows || leftUnit == rightUnit) + return compare(leftLimit, rightLimit); + else + return compare(leftLimit * leftUnit.bytesFactor(), rightLimit * rightUnit.bytesFactor()); + } + + /** + *(Strict copy of Integer.compare(int,int) of Java 1.7)
+ *+ * Compares two {@code int} values numerically. + * The value returned is identical to what would be returned by: + *
+ *+ * Integer.valueOf(x).compareTo(Integer.valueOf(y)) + *+ * + * @param x the first {@code int} to compare + * @param y the second {@code int} to compare + * @return the value {@code 0} if {@code x == y}; + * a value less than {@code 0} if {@code x < y}; and + * a value greater than {@code 0} if {@code x > y} + * + * @since 1.1 + */ + private static int compare(int x, int y){ + return (x < y) ? -1 : ((x == y) ? 0 : 1); + } + + /** + *
(Strict copy of Integer.compare(long,long) of Java 1.7)
+ *+ * Compares two {@code long} values numerically. + * The value returned is identical to what would be returned by: + *
+ *+ * Long.valueOf(x).compareTo(Long.valueOf(y)) + *+ * + * @param x the first {@code long} to compare + * @param y the second {@code long} to compare + * @return the value {@code 0} if {@code x == y}; + * a value less than {@code 0} if {@code x < y}; and + * a value greater than {@code 0} if {@code x > y} + * + * @since 1.1 + */ + public static int compare(long x, long y){ + return (x < y) ? -1 : ((x == y) ? 0 : 1); + } + + @Override + public String toString(){ + return str; + } } + /** + * [OPTIONAL] + *
Name of the service provider ; it can be an organization as an individual person.
+ * + *There is no restriction on the syntax or on the label to use ; this information is totally free
+ * + *It will be used as additional information (INFO tag) in any VOTable and HTML output.
+ * + * @return The TAP service provider or NULL to leave this field blank. + */ public String getProviderName(); + /** + * [OPTIONAL] + *Description of the service provider.
+ * + *It will be used as additional information (INFO tag) in any VOTable output.
+ * + * @return The TAP service description or NULL to leave this field blank. + */ public String getProviderDescription(); + /** + * [MANDATORY] + *This function tells whether the TAP service is available + * (that's to say, "able to execute requests" ; resources like /availability, /capabilities and /tables may still work).
+ * + *+ * A message explaining the current state of the TAP service could be provided thanks to {@link #getAvailability()}. + *
+ * + * @return true to enable all TAP resources, false to disable all of them (except /availability). + */ public boolean isAvailable(); + /** + * [OPTIONAL] + *Get an explanation about the current TAP service state (working or not). + * This message aims to provide more details to the users about the availability of this service, + * or more particularly about its unavailability.
+ * + * @return Explanation about the TAP service state. + */ public String getAvailability(); + /** + * [MANDATORY] + *This function sets the state of the whole TAP service. + * If true, all TAP resources will be able to execute resources. + * If false, /sync and /async won't answer any more to requests and a HTTP-503 (Service unavailable) + * error will be returned. + *
+ * + * @param isAvailable true to enable all resources, false to forbid /sync and /async (all other resources will still be available). + * @param message A message describing the current state of the service. If NULL, a default message may be set by the library. + * + * @since 2.0 + */ + public void setAvailable(final boolean isAvailable, final String message); + + /** + * [OPTIONAL] + *Get the limit of the retention period (in seconds).
+ * + *+ * It is the maximum period while an asynchronous job can leave in the jobs list + * and so can stay on the server. + *
+ * + *Important notes:
+ *Get the limit of the job execution duration (in milliseconds).
+ * + *+ * It is the duration of a running job (including the query execution). + * This duration is used for synchronous AND asynchronous jobs. + *
+ * + *Important notes:
+ *Get the limit of the job execution result.
+ * + *+ * This value will limit the size of the query results, either in rows or in bytes. + * The type of limit is defined by the function {@link #getOutputLimitType()}. + *
+ * + *Important notes:
+ *Important note:
+ * Currently, the default implementations of the library is only able to deal with output limits in ROWS.
+ * Anyway, in order to save performances, it is strongly recommended to use ROWS limit rather than in bytes. Indeed, the rows limit can be taken
+ * into account at the effective execution of the query (so before getting the result), on the contrary of the bytes limit which
+ * will be applied on the query result.
+ *
Get the type of each output limit set by this service connection (and accessible with {@link #getOutputLimit()}).
+ * + *Important notes:
+ *Important note:
+ * Currently, the default implementations of the library is only able to deal with output limits in ROWS.
+ * Anyway, in order to save performances, it is strongly recommended to use ROWS limit rather than in bytes. Indeed, the rows limit can be taken
+ * into account at the effective execution of the query (so before getting the result), on the contrary of the bytes limit which
+ * will be applied on the query result.
+ *
Get the object to use in order to identify users at the origin of requests.
+ * + * @return NULL if no user identification should be done, a {@link UserIdentifier} instance otherwise. + */ public UserIdentifier getUserIdentifier(); + /** + * [MANDATORY] + *This function let enable or disable the upload capability of this TAP service.
+ * + *Note: + * If the upload is disabled, the request is aborted and an HTTP-400 error is thrown each time some tables are uploaded. + *
+ * + * @return true to enable the upload capability, false to disable it. + */ public boolean uploadEnabled(); + /** + * [OPTIONAL] + *Get the maximum size of EACH uploaded table.
+ * + *+ * This value is expressed either in rows or in bytes. + * The unit limit is defined by the function {@link #getUploadLimitType()}. + *
+ * + *Important notes:
+ *Important note: + * To save performances, it is recommended to use BYTES limit rather than in rows. Indeed, the bytes limit can be taken + * into account at directly when reading the bytes of the request, on the contrary of the rows limit which + * requires to parse the uploaded tables. + *
+ * + * @return NULL if no limit must be set, or a two-items array ([0]: default value, [1]: maximum value). + * + * @see #getUploadLimitType() + */ public int[] getUploadLimit(); + /** + * [OPTIONAL] + *Get the type of each upload limit set by this service connection (and accessible with {@link #getUploadLimit()}).
+ * + *Important notes:
+ *Important note: + * To save performances, it is recommended to use BYTES limit rather than in rows. Indeed, the bytes limit can be taken + * into account at directly when reading the bytes of the request, on the contrary of the rows limit which + * requires to parse the uploaded tables. + *
+ * + * @return NULL if limits should be expressed in ROWS, or a two-items array ([0]: type of getUploadLimit()[0], [1]: type of getUploadLimit()[1]). + * + * @see #getUploadLimit() + */ public LimitUnit[] getUploadLimitType(); + /** + * [OPTIONAL] + *Get the maximum size of the whole set of all tables uploaded in one request. + * This size is expressed in bytes.
+ * + *IMPORTANT 1: + * This value is always used when the upload capability is enabled. + *
+ * + *IMPORTANT 2: + * The value returned by this function MUST always be positive. + * A zero or negative value will throw an exception later while + * reading parameters in a request with some uploaded tables. + *
+ * + * @return A positive (>0) value corresponding to the maximum number of bytes of all uploaded tables sent in one request. + */ public int getMaxUploadSize(); + /** + * [MANDATORY] + *Get the list of all available tables and columns.
+ * + *+ * This object is really important since it lets the library check ADQL queries properly and set the good type + * and formatting in the query results. + *
+ * + * @return A TAPMetadata object. NULL is not allowed and will throw a grave error at the service initialization. + */ public TAPMetadata getTAPMetadata(); + /** + * [OPTIONAL] + *Get the list of all allowed coordinate systems.
+ * + * Special values + * + *Two special values can be returned by this function:
+ *+ * Each item of this list is a pattern and not a simple coordinate system. + * Thus each item MUST respect the following syntax: + *
+ *{framePattern} {refposPattern} {flavorPattern}
+ * + * Contrary to a coordinate system expression, all these 3 information are required. + * Each may take 3 kinds of value: + *
+ *({value1}|{value2}|...) (i.e. "(ICRS|FK4)"),
+ * For instance: (ICRS|FK4) HELIOCENTER * is a good syntax,
+ * but not ICRS or ICRS HELIOCENTER.
+ *
Note: + * Even if not explicitly part of the possible values, the default value of each part (i.e. UNKNOWNFRAME for frame) is always taken into account by the library. + * Particularly, the empty string will always be allowed even if not explicitly listed in the list returned by this function. + *
+ * + * @return NULL to allow ALL coordinate systems, an empty list to allow NO coordinate system, + * or a list of coordinate system patterns otherwise. + */ public CollectionGet the list of all allowed geometrical functions.
+ * + * Special values + * + *Two special values can be returned by this function:
+ *+ * Each item of the returned list MUST be a function name (i.e. "CONTAINS", "POINT"). + * It can also be a type of STC region to forbid (i.e. "POSITION", "UNION"). + *
+ * + *The given names are not case sensitive.
+ * + * @return NULL to allow ALL geometrical functions, an empty list to allow NO geometrical function, + * or a list of geometrical function names otherwise. + * + * @since 2.0 + */ + public CollectionGet the list of all allowed User Defined Functions (UDFs).
+ * + * Special values + * + *Two special values can be returned by this function:
+ *+ * Each item of the returned list MUST be an instance of {@link FunctionDef}. + *
+ * + * @return NULL to allow ALL unknown functions, an empty list to allow NO unknown function, + * or a list of user defined functions otherwise. + * + * @since 2.0 + */ + public CollectionGet the maximum number of asynchronous jobs that can run in the same time.
+ * + *A null or negative value means no limit on the number of running asynchronous jobs.
+ * + * @return Maximum number of running jobs (≤0 => no limit). + * + * @since 2.0 + */ + public int getNbMaxAsyncJobs(); + + /** + * [MANDATORY] + *Get the logger to use in the whole service when any error, warning or info happens.
+ * + *IMPORTANT: + * If NULL is returned by this function, grave errors will occur while executing a query or managing an error. + * It is strongly recommended to provide a logger, even a basic implementation. + *
+ * + *Piece of advice: + * A default implementation like {@link DefaultTAPLog} would be most of time largely enough. + *
+ * + * @return An instance of {@link TAPLog}. + */ public TAPLog getLogger(); - public TAPFactoryGet the object able to build other objects essentials to configure the TAP service or to run every queries.
+ * + *IMPORTANT: + * If NULL is returned by this function, grave errors will occur while initializing the service. + *
+ * + *Piece of advice: + * The {@link TAPFactory} is an interface which contains a lot of functions to implement. + * It is rather recommended to extend {@link AbstractTAPFactory}: just 2 functions + * ({@link AbstractTAPFactory#freeConnection(DBConnection)} and {@link AbstractTAPFactory#getConnection(String)}) + * will have to be implemented. + *
+ * + * @return An instance of {@link TAPFactory}. + * + * @see AbstractTAPFactory + */ + public TAPFactory getFactory(); + + /** + * [MANDATORY] + *Get the object in charge of the files management. + * This object manages log, error, result and backup files of the whole service.
+ * + *IMPORTANT: + * If NULL is returned by this function, grave errors will occur while initializing the service. + *
+ * + *Piece of advice: + * The library provides a default implementation of the interface {@link UWSFileManager}: + * {@link LocalUWSFileManager}, which stores all files on the local file-system. + *
+ * + * @return An instance of {@link UWSFileManager}. + */ + public UWSFileManager getFileManager(); - public TAPFileManager getFileManager(); + /** + * [MANDATORY] + *Get the list of all available output formats.
+ * + *IMPORTANT:
+ *Get the output format having the given MIME type (or short MIME type ~ alias).
+ * + *IMPORTANT: + * This function MUST always return an {@link OutputFormat} instance when the MIME type "votable" is given in parameter. + *
+ * + * @param mimeOrAlias MIME type or short MIME type of the format to get. + * + * @return The corresponding {@link OutputFormat} or NULL if not found. + */ + public OutputFormat getOutputFormat(final String mimeOrAlias); - public OutputFormatGet the size of result blocks to fetch from the database.
+ * + *+ * Rather than fetching a query result in a whole, it may be possible to specify to the database + * that results may be retrieved by blocks whose the size can be specified by this function. + * If supported by the DBMS and the JDBC driver, this feature may help sparing memory and avoid + * too much waiting time from the TAP /sync users (and thus, avoiding some HTTP client timeouts). + *
+ * + *Note: + * Generally, this feature is well supported by DBMS. But for that, the used JDBC driver must use + * the V3 protocol. If anyway, this feature is supported neither by the DBMS, the JDBC driver nor your + * {@link DBConnection}, no error will be thrown if a value is returned by this function: it will be silently + * ignored by the library. + *
+ * + * @return null or an array of 1 or 2 integers. + * If null (or empty array), no attempt to set fetch size will be done and so, ONLY the default + * value of the {@link DBConnection} will be used. + * [0]=fetchSize for async queries, [1]=fetchSize for sync queries. + * If [1] is omitted, it will be considered as equals to [0]. + * If a fetchSize is negative or null, the default value of your JDBC driver will be used. + * + * @since 2.0 + */ + public int[] getFetchSize(); } diff --git a/src/tap/TAPException.java b/src/tap/TAPException.java index 89a4feeef6723558f5d32f9203862329310ec361..edfe745e98908b2e2669acc03bc10d740278d10d 100644 --- a/src/tap/TAPException.java +++ b/src/tap/TAPException.java @@ -16,129 +16,373 @@ package tap; * You should have received a copy of the GNU Lesser General Public License * along with TAPLibrary. If not, seeAny exception that occurred while a TAP service activity.
+ * + *Most of the time this exception wraps another exception (e.g. {@link UWSException}).
+ * + *It contains an HTTP status code, set by default to HTTP-500 (Internal Server Error).
+ * + * @author Grégory Mantelet (CDS;ARI) + * @version 2.0 (04/2015) + */ public class TAPException extends Exception { private static final long serialVersionUID = 1L; + /** An ADQL query which were executed when the error occurred. */ private String adqlQuery = null; + + /** The ADQL query execution status (e.g. uploading, parsing, executing) just when the error occurred. */ private ExecutionProgression executionStatus = null; + /** The HTTP status code to set in the HTTP servlet response if the exception reaches the servlet. */ private int httpErrorCode = UWSException.INTERNAL_SERVER_ERROR; + /** + * Standard TAP exception: no ADQL query or execution status specified. + * The corresponding HTTP status code will be HTTP-500 (Internal Server Error). + * + * @param message Message explaining the error. + */ public TAPException(String message){ super(message); } + /** + * Standard TAP exception: no ADQL query or execution status specified. + * The corresponding HTTP status code is set by the second parameter. + * + * @param message Message explaining the error. + * @param httpErrorCode HTTP response status code. (if ≤ 0, 500 will be set by default) + */ public TAPException(String message, int httpErrorCode){ super(message); this.httpErrorCode = httpErrorCode; } + /** + * TAP exception with the ADQL query which were executed when the error occurred. + * No execution status specified. + * The corresponding HTTP status code will be HTTP-500 (Internal Server Error). + * + * @param message Message explaining the error. + * @param query The ADQL query which were executed when the error occurred. + */ public TAPException(String message, String query){ super(message); adqlQuery = query; } + /** + * TAP exception with the ADQL query which were executed when the error occurred. + * No execution status specified. + * The corresponding HTTP status code is set by the second parameter. + * + * @param message Message explaining the error. + * @param httpErrorCode HTTP response status code. (if ≤ 0, 500 will be set by default) + * @param query The ADQL query which were executed when the error occurred. + */ public TAPException(String message, int httpErrorCode, String query){ this(message, httpErrorCode); adqlQuery = query; } + /** + * TAP exception with the ADQL query which were executed when the error occurred, + * AND with its execution status (e.g. uploading, parsing, executing, ...). + * The corresponding HTTP status code will be HTTP-500 (Internal Server Error). + * + * @param message Message explaining the error. + * @param query The ADQL query which were executed when the error occurred. + * @param status Execution status/phase of the given ADQL query when the error occurred. + */ public TAPException(String message, String query, ExecutionProgression status){ this(message, query); executionStatus = status; } + /** + * TAP exception with the ADQL query which were executed when the error occurred, + * AND with its execution status (e.g. uploading, parsing, executing, ...). + * The corresponding HTTP status code is set by the second parameter. + * + * @param message Message explaining the error. + * @param httpErrorCode HTTP response status code. (if ≤ 0, 500 will be set by default) + * @param query The ADQL query which were executed when the error occurred. + * @param status Execution status/phase of the given ADQL query when the error occurred. + */ public TAPException(String message, int httpErrorCode, String query, ExecutionProgression status){ this(message, httpErrorCode, query); executionStatus = status; } + /** + *TAP exception wrapping the given {@link UWSException}.
+ * + *The message of this TAP exception will be exactly the same as the one of the given exception.
+ * + *+ * Besides, the cause of this TAP exception will be the cause of the given exception ONLY if it has one ; + * otherwise it will the given exception. + *
+ * + *The HTTP status code will be the same as the one of the given {@link UWSException}.
+ * + * @param ue The exception to wrap. + */ public TAPException(UWSException ue){ - this(ue.getMessage(), ue.getCause(), ue.getHttpErrorCode()); + this(ue.getMessage(), (ue.getCause() == null ? ue : ue.getCause()), ue.getHttpErrorCode()); } + /** + *TAP exception wrapping the given {@link UWSException}.
+ * + *The message of this TAP exception will be exactly the same as the one of the given exception.
+ * + *+ * Besides, the cause of this TAP exception will be the cause of the given exception ONLY if it has one ; + * otherwise it will the given exception. + *
+ * + *The HTTP status code will be the one given in second parameter.
+ * + * @param cause The exception to wrap. + * @param httpErrorCode HTTP response status code. (if ≤ 0, 500 will be set by default) + */ public TAPException(UWSException cause, int httpErrorCode){ this(cause); this.httpErrorCode = httpErrorCode; } + /** + *TAP exception wrapping the given {@link UWSException} and storing the current ADQL query execution status.
+ * + *The message of this TAP exception will be exactly the same as the one of the given exception.
+ * + *+ * Besides, the cause of this TAP exception will be the cause of the given exception ONLY if it has one ; + * otherwise it will the given exception. + *
+ * + *The HTTP status code will be the one given in second parameter.
+ * + * @param cause The exception to wrap. + * @param httpErrorCode HTTP response status code. (if ≤ 0, 500 will be set by default) + * @param status Execution status/phase of the given ADQL query when the error occurred. + */ public TAPException(UWSException cause, int httpErrorCode, ExecutionProgression status){ this(cause, httpErrorCode); this.executionStatus = status; } + /** + * Build a {@link TAPException} with the given cause. The built exception will have NO MESSAGE. + * No execution status specified. + * The corresponding HTTP status code will be HTTP-500 (Internal Server Error). + * + * @param cause The cause of this exception. + */ public TAPException(Throwable cause){ super(cause); } + /** + * Build a {@link TAPException} with the given cause. The built exception will have NO MESSAGE. + * No execution status specified. + * The corresponding HTTP status code is set by the second parameter. + * + * @param cause The cause of this exception. + * @param httpErrorCode HTTP response status code. (if ≤ 0, 500 will be set by default) + */ public TAPException(Throwable cause, int httpErrorCode){ super(cause); this.httpErrorCode = httpErrorCode; } + /** + * Build a {@link TAPException} with the given cause AND with the ADQL query which were executed when the error occurred. + * The built exception will have NO MESSAGE. + * No execution status specified. + * The corresponding HTTP status code will be HTTP-500 (Internal Server Error). + * + * @param cause The cause of this exception. + * @param query The ADQL query which were executed when the error occurred. + */ public TAPException(Throwable cause, String query){ super(cause); adqlQuery = query; } + /** + * Build a {@link TAPException} with the given cause AND with the ADQL query which were executed when the error occurred. + * The built exception will have NO MESSAGE. + * No execution status specified. + * The corresponding HTTP status code is set by the second parameter. + * + * @param cause The cause of this exception. + * @param httpErrorCode HTTP response status code. (if ≤ 0, 500 will be set by default) + * @param query The ADQL query which were executed when the error occurred. + */ public TAPException(Throwable cause, int httpErrorCode, String query){ this(cause, httpErrorCode); adqlQuery = query; } + /** + * Build a {@link TAPException} with the given cause AND with the ADQL query which were executed when the error occurred + * AND with its execution status (e.g. uploading, parsing, executing, ...). + * The built exception will have NO MESSAGE. + * The corresponding HTTP status code will be HTTP-500 (Internal Server Error). + * + * @param cause The cause of this exception. + * @param query The ADQL query which were executed when the error occurred. + * @param status Execution status/phase of the given ADQL query when the error occurred. + */ public TAPException(Throwable cause, String query, ExecutionProgression status){ this(cause, query); executionStatus = status; } + /** + * Build a {@link TAPException} with the given cause AND with the ADQL query which were executed when the error occurred + * AND with its execution status (e.g. uploading, parsing, executing, ...). + * The built exception will have NO MESSAGE. + * The corresponding HTTP status code is set by the second parameter. + * + * @param cause The cause of this exception. + * @param httpErrorCode HTTP response status code. (if ≤ 0, 500 will be set by default) + * @param query The ADQL query which were executed when the error occurred. + * @param status Execution status/phase of the given ADQL query when the error occurred. + */ public TAPException(Throwable cause, int httpErrorCode, String query, ExecutionProgression status){ this(cause, httpErrorCode, query); executionStatus = status; } + /** + * Build a {@link TAPException} with the given message and cause. + * No execution status specified. + * The corresponding HTTP status code will be HTTP-500 (Internal Server Error). + * + * @param message Message of this exception. + * @param cause The cause of this exception. + */ public TAPException(String message, Throwable cause){ super(message, cause); } + /** + * Build a {@link TAPException} with the given message and cause. + * No execution status specified. + * The corresponding HTTP status code is set by the third parameter. + * + * @param message Message of this exception. + * @param cause The cause of this exception. + * @param httpErrorCode HTTP response status code. (if ≤ 0, 500 will be set by default) + */ public TAPException(String message, Throwable cause, int httpErrorCode){ super(message, cause); this.httpErrorCode = httpErrorCode; } + /** + * Build a {@link TAPException} with the given message and cause, + * AND with the ADQL query which were executed when the error occurred. + * No execution status specified. + * The corresponding HTTP status code will be HTTP-500 (Internal Server Error). + * + * @param message Message of this exception. + * @param cause The cause of this exception. + * @param query The ADQL query which were executed when the error occurred. + */ public TAPException(String message, Throwable cause, String query){ super(message, cause); adqlQuery = query; } + /** + * Build a {@link TAPException} with the given message and cause, + * AND with the ADQL query which were executed when the error occurred. + * No execution status specified. + * The corresponding HTTP status code is set by the third parameter. + * + * @param message Message of this exception. + * @param cause The cause of this exception. + * @param httpErrorCode HTTP response status code. (if ≤ 0, 500 will be set by default) + * @param query The ADQL query which were executed when the error occurred. + */ public TAPException(String message, Throwable cause, int httpErrorCode, String query){ this(message, cause, httpErrorCode); adqlQuery = query; } + /** + * Build a {@link TAPException} with the given message and cause, + * AND with the ADQL query which were executed when the error occurred + * AND with its execution status (e.g. uploading, parsing, executing, ...). + * No execution status specified. + * The corresponding HTTP status code will be HTTP-500 (Internal Server Error). + * + * @param message Message of this exception. + * @param cause The cause of this exception. + * @param query The ADQL query which were executed when the error occurred. + * @param status Execution status/phase of the given ADQL query when the error occurred. + */ public TAPException(String message, Throwable cause, String query, ExecutionProgression status){ this(message, cause, query); executionStatus = status; } + /** + * Build a {@link TAPException} with the given message and cause, + * AND with the ADQL query which were executed when the error occurred + * AND with its execution status (e.g. uploading, parsing, executing, ...). + * No execution status specified. + * The corresponding HTTP status code is set by the third parameter. + * + * @param message Message of this exception. + * @param cause The cause of this exception. + * @param httpErrorCode HTTP response status code. (if ≤ 0, 500 will be set by default) + * @param query The ADQL query which were executed when the error occurred. + * @param status Execution status/phase of the given ADQL query when the error occurred. + */ public TAPException(String message, Throwable cause, int httpErrorCode, String query, ExecutionProgression status){ this(message, cause, httpErrorCode, query); executionStatus = status; } + /** + *Get the HTTP status code to set in the HTTP response.
+ * + *If the set value is ≤ 0, 500 will be returned instead.
+ * + * @return The HTTP response status code. + */ public int getHttpErrorCode(){ - return httpErrorCode; + return (httpErrorCode <= 0) ? UWSException.INTERNAL_SERVER_ERROR : httpErrorCode; } + /** + * Get the ADQL query which were executed when the error occurred. + * + * @return Executed ADQL query. + */ public String getQuery(){ return adqlQuery; } + /** + * Get the execution status/phase of an ADQL query when the error occurred. + * + * @return ADQL query execution status. + */ public ExecutionProgression getExecutionStatus(){ return executionStatus; } diff --git a/src/tap/TAPExecutionReport.java b/src/tap/TAPExecutionReport.java index bf95af1a449dd836cc4b16f1c65960e90d7db4b4..fe2733f07ac4fe4fc6e2bbb3be59de678e503fb5 100644 --- a/src/tap/TAPExecutionReport.java +++ b/src/tap/TAPExecutionReport.java @@ -16,90 +16,167 @@ package tap; * You should have received a copy of the GNU Lesser General Public License * along with TAPLibrary. If not, seeReport the execution (including the parsing and the output writing) of an ADQL query. + * It gives information on the job parameters, the job ID, whether it is a synchronous task or not, times of each execution step (uploading, parsing, executing and writing), + * the resulting columns and the success or not of the execution.
+ * + *This report is completely filled by {@link ADQLExecutor}, and aims to be used/read only at the end of the job or when it is definitely finished.
+ * + * @author Grégory Mantelet (CDS;ARI) + * @version 2.0 (04/2015) + */ public class TAPExecutionReport { + /** ID of the job whose the execution is reported here. */ public final String jobID; + + /** Indicate whether this execution is done in a synchronous or asynchronous job. */ public final boolean synchronous; + + /** List of all parameters provided in the user request. */ public final TAPParameters parameters; - public String sqlTranslation = null; + /** List of all resulting columns. Empty array, if not yet known. */ public DBColumn[] resultingColumns = new DBColumn[0]; - protected final long[] durations = new long[]{-1,-1,-1,-1,-1}; + /** Total number of written rows. + * @since 2.0 */ + public long nbRows = -1; + + /** Duration of all execution steps. For the moment only 4 steps (in the order): uploading, parsing, executing and writing. */ + protected final long[] durations = new long[]{-1,-1,-1,-1}; + + /** Total duration of the job execution. */ protected long totalDuration = -1; + /** Indicate whether this job has ended successfully or not. At the beginning or while executing, this field is always FALSE. */ public boolean success = false; + /** + * Build an empty execution report. + * + * @param jobID ID of the job whose the execution must be described here. + * @param synchronous true if the job is synchronous, false otherwise. + * @param params List of all parameters provided by the user for the execution. + */ public TAPExecutionReport(final String jobID, final boolean synchronous, final TAPParameters params){ this.jobID = jobID; this.synchronous = synchronous; parameters = params; } + /** + *Map the execution progression with an index inside the {@link #durations} array.
+ * + *Warning: for the moment, only {@link ExecutionProgression#UPLOADING}, {@link ExecutionProgression#PARSING}, + * {@link ExecutionProgression#EXECUTING_ADQL} and {@link ExecutionProgression#WRITING_RESULT} are managed.
+ * + * @param tapProgression Execution progression. + * + * @return Index in the array {@link #durations}, or -1 if the given execution progression is not managed. + */ protected int getIndexDuration(final ExecutionProgression tapProgression){ switch(tapProgression){ case UPLOADING: return 0; case PARSING: return 1; - case TRANSLATING: + case EXECUTING_ADQL: return 2; - case EXECUTING_SQL: - return 3; case WRITING_RESULT: - return 4; + return 3; default: return -1; } } - public final long getDuration(final ExecutionProgression tapProgression){ - int indDuration = getIndexDuration(tapProgression); + /** + * Get the duration corresponding to the given job execution step. + * + * @param tapStep Job execution step. + * + * @return The corresponding duration (in ms), or -1 if this step has not been (yet) processed. + * + * @see #getIndexDuration(ExecutionProgression) + */ + public final long getDuration(final ExecutionProgression tapStep){ + int indDuration = getIndexDuration(tapStep); if (indDuration < 0 || indDuration >= durations.length) return -1; else return durations[indDuration]; } - public final void setDuration(final ExecutionProgression tapProgression, final long duration){ - int indDuration = getIndexDuration(tapProgression); + /** + * Set the duration corresponding to the given execution step. + * + * @param tapStep Job execution step. + * @param duration Duration (in ms) of the given execution step. + */ + public final void setDuration(final ExecutionProgression tapStep, final long duration){ + int indDuration = getIndexDuration(tapStep); if (indDuration < 0 || indDuration >= durations.length) return; else durations[indDuration] = duration; } + /** + * Get the execution of the UPLOAD step. + * @return Duration (in ms). + * @see #getDuration(ExecutionProgression) + */ public final long getUploadDuration(){ return getDuration(ExecutionProgression.UPLOADING); } + /** + * Get the execution of the PARSE step. + * @return Duration (in ms). + * @see #getDuration(ExecutionProgression) + */ public final long getParsingDuration(){ return getDuration(ExecutionProgression.PARSING); } - public final long getTranslationDuration(){ - return getDuration(ExecutionProgression.TRANSLATING); - } - + /** + * Get the execution of the EXECUTION step. + * @return Duration (in ms). + * @see #getDuration(ExecutionProgression) + */ public final long getExecutionDuration(){ - return getDuration(ExecutionProgression.EXECUTING_SQL); + return getDuration(ExecutionProgression.EXECUTING_ADQL); } + /** + * Get the execution of the FORMAT step. + * @return Duration (in ms). + * @see #getDuration(ExecutionProgression) + */ public final long getFormattingDuration(){ return getDuration(ExecutionProgression.WRITING_RESULT); } + /** + * Get the total duration of the job execution. + * @return Duration (in ms). + */ public final long getTotalDuration(){ return totalDuration; } + /** + * Set the total duration of the job execution. + * @param duration Duration (in ms) to set. + */ public final void setTotalDuration(final long duration){ totalDuration = duration; } diff --git a/src/tap/TAPFactory.java b/src/tap/TAPFactory.java index 785be4795659e53d11bc2b0ffecf32477784f5dd..f4f6b7a46a43753e8d3093437b02ef6490d7f351 100644 --- a/src/tap/TAPFactory.java +++ b/src/tap/TAPFactory.java @@ -16,43 +16,469 @@ package tap; * You should have received a copy of the GNU Lesser General Public License * along with TAPLibrary. If not, seeLet build essential objects of the TAP service.
+ * + *Basically, it means answering to the following questions:
+ *Get the object to use when an error must be formatted and written to the user.
+ * + *This formatted error will be either written in an HTTP response or in a job error summary.
+ * + * @return The error writer to use. + * + * @since 2.0 + */ + public abstract ServiceErrorWriter getErrorWriter(); + + /* ******************* */ + /* DATABASE CONNECTION */ + /* ******************* */ + + /** + *Get a free database connection.
+ * + *+ * Free means this connection is not currently in use and will be exclusively dedicated to the function/process/thread + * which has asked for it by calling this function. + *
+ * + *Note: + * This function can create on the fly a new connection OR get a free one from a connection pool. Considering the + * creation time of a database connection, the second way is recommended. + *
+ * + *IMPORTANT: + * The returned connection MUST be freed after having used it. + *
+ * + *WARNING: + * Some implementation may free the connection automatically when not used for a specific time. + * So, do not forget to free the connection after use! + *
+ * + * @param jobID ID of the job/thread/process which has asked for this connection. note: The returned connection must then be identified thanks to this ID. + * + * @return A new and free connection to the database. MUST BE NOT NULL, or otherwise a TAPException should be returned. + * + * @throws TAPException If there is any error while getting a free connection. + * + * @since 2.0 + */ + public abstract DBConnection getConnection(final String jobID) throws TAPException; + + /** + *Free the given connection.
+ * + *+ * This function is called by the TAP library when a job/thread does not need this connection any more. It aims + * to free resources associated to the given database connection. + *
+ * + *Note: + * This function can just close definitely the connection OR give it back to a connection pool. The implementation is + * here totally free! + *
+ * + * @param conn The connection to close. + * + * @since 2.0 + */ + public abstract void freeConnection(final DBConnection conn); + + /** + *Destroy all resources (and particularly DB connections and JDBC driver) allocated in this factory.
+ * + *Note: + * This function is called when the TAP service is shutting down. + * After this call, the factory may not be able to provide any closed resources ; its behavior may be unpredictable. + *
+ * + * @since 2.0 + */ + public abstract void destroy(); + + /* *************** */ + /* ADQL MANAGEMENT */ + /* *************** */ + + /** + *Create the object able to execute an ADQL query and to write and to format its result.
+ * + *Note: + * A default implementation is provided by {@link AbstractTAPFactory} + *
+ * + * @return An ADQL executor. + * + * @throws TAPException If any error occurs while creating an ADQL executor. + */ + public abstract ADQLExecutor createADQLExecutor() throws TAPException; + + /** + *Create a parser of ADQL query.
+ * + *Warning: + * This parser can be created with a query factory and/or a query checker. + * {@link #createQueryFactory()} will be used only if the default query factory (or none) is set + * in the ADQL parser returned by this function. + * Idem for {@link #createQueryChecker(TAPSchema)}: it will used only if no query checker is set + * in the returned ADQL parser. + *
+ * + *Note: + * A default implementation is provided by {@link AbstractTAPFactory}. + *
+ * + * @return An ADQL query parser. + * + * @throws TAPException If any error occurs while creating an ADQL parser. + * + * @since 2.0 + */ + public abstract ADQLParser createADQLParser() throws TAPException; + + /** + *Create a factory able to build every part of an {@link ADQLQuery} object.
+ * + *Warning: + * This function is used only if the default query factory (or none) is set in the ADQL parser + * returned by {@link #createADQLParser()}. + *
+ * + *Note: + * A default implementation is provided by {@link AbstractTAPFactory} + *
+ * + * @return An {@link ADQLQuery} factory. + * + * @throws TAPException If any error occurs while creating the factory. + */ + public abstract ADQLQueryFactory createQueryFactory() throws TAPException; + + /** + *Create an object able to check the consistency between the ADQL query and the database. + * That's to say, it checks whether the tables and columns used in the query really exist + * in the database.
+ * + *Warning: + * This function is used only if no query checker is set in the ADQL parser + * returned by {@link #createADQLParser()}. + *
+ * + *Note: + * A default implementation is provided by {@link AbstractTAPFactory} + *
+ * + * @param uploadSchema ADQL schema containing the description of all uploaded tables. + * + * @return A query checker. + * + * @throws TAPException If any error occurs while creating a query checker. + */ + public abstract QueryChecker createQueryChecker(final TAPSchema uploadSchema) throws TAPException; + + /* ****** */ + /* UPLOAD */ + /* ****** */ + + /** + *Create an object able to manage the creation of submitted user tables (in VOTable) into the database.
+ * + *Note: + * A default implementation is provided by {@link AbstractTAPFactory}. + *
+ * + * @param dbConn The database connection which has requested an {@link Uploader}. + * + * @return An {@link Uploader}. + * + * @throws TAPException If any error occurs while creating an {@link Uploader} instance. + */ + public abstract Uploader createUploader(final DBConnection dbConn) throws TAPException; + + /* ************** */ + /* UWS MANAGEMENT */ + /* ************** */ + + /** + *Create the object which will manage the asynchronous resource of the TAP service. + * This resource is a UWS service.
+ * + *Note: + * A default implementation is provided by {@link AbstractTAPFactory}. + *
+ * + * @return A UWS service which will be the asynchronous resource of this TAP service. + * + * @throws TAPException If any error occurs while creating this UWS service. + */ + public abstract UWSService createUWS() throws TAPException; + + /** + *Create the object which will manage the backup and restoration of all asynchronous jobs.
+ * + *Note: + * This function may return NULL. If it does, asynchronous jobs won't be backuped. + *
+ * + *Note: + * A default implementation is provided by {@link AbstractTAPFactory}. + *
+ * + * @param uws The UWS service which has to be backuped and restored. + * + * @return The backup manager to use. MAY be NULL + * + * @throws TAPException If any error occurs while creating this backup manager. + */ + public abstract UWSBackupManager createUWSBackupManager(final UWSService uws) throws TAPException; -import adql.translator.ADQLTranslator; + /** + *Creates a (PENDING) UWS job from the given HTTP request.
+ * + *+ * This implementation just call {@link #createTAPJob(HttpServletRequest, JobOwner)} + * with the given request, in order to ensure that the returned object is always a {@link TAPJob}. + *
+ * + * @see uws.service.AbstractUWSFactory#createJob(javax.servlet.http.HttpServletRequest, uws.job.user.JobOwner) + * @see #createTAPJob(HttpServletRequest, JobOwner) + */ + @Override + public final UWSJob createJob(HttpServletRequest request, JobOwner owner) throws UWSException{ + return createTAPJob(request, owner); + } -public interface TAPFactory< R > extends UWSFactory { + /** + *Create a PENDING asynchronous job from the given HTTP request.
+ * + *Note: + * A default implementation is provided by {@link AbstractTAPFactory}. + *
+ * + * @param request Request which contains all parameters needed to set correctly the asynchronous job to create. + * @param owner The user which has requested the job creation. + * + * @return A new PENDING asynchronous job. + * + * @throws UWSException If any error occurs while reading the parameters in the request or while creating the job. + */ + protected abstract TAPJob createTAPJob(final HttpServletRequest request, final JobOwner owner) throws UWSException; - public UWSService createUWS() throws TAPException, UWSException; + /** + *Creates a UWS job with the following attributes.
+ * + *+ * This implementation just call {@link #createTAPJob(String, JobOwner, TAPParameters, long, long, long, List, ErrorSummary)} + * with the given parameters, in order to ensure that the returned object is always a {@link TAPJob}. + *
+ * + *Note 1: + * This function is mainly used to restore a UWS job at the UWS initialization. + *
+ * + *Note 2: + * The job phase is chosen automatically from the given job attributes (i.e. no endTime => PENDING, no result and no error => ABORTED, ...). + *
+ * + * @see uws.service.AbstractUWSFactory#createJob(java.lang.String, uws.job.user.JobOwner, uws.job.parameters.UWSParameters, long, long, long, java.util.List, uws.job.ErrorSummary) + * @see #createTAPJob(String, JobOwner, TAPParameters, long, long, long, List, ErrorSummary) + */ + @Override + public final UWSJob createJob(String jobId, JobOwner owner, final UWSParameters params, long quote, long startTime, long endTime, ListCreate a PENDING asynchronous job with the given parameters.
+ * + *Note: + * A default implementation is provided in {@link AbstractTAPFactory}. + *
+ * + * @param jobId ID of the job (NOT NULL). + * @param owner Owner of the job. + * @param params List of all input job parameters. + * @param quote Its quote (in seconds). + * @param startTime Date/Time of the start of this job. + * @param endTime Date/Time of the end of this job. + * @param results All results of this job. + * @param error The error which ended the job to create. + * + * @return A new PENDING asynchronous job. + * + * @throws UWSException If there is an error while creating the job. + */ + protected abstract TAPJob createTAPJob(final String jobId, final JobOwner owner, final TAPParameters params, final long quote, final long startTime, final long endTime, final ListCreate the thread which will execute the task described by the given UWSJob instance.
+ * + *+ * This function is definitely implemented here and can not be overridden. The processing of + * an ADQL query must always be the same in a TAP service ; it is completely done by {@link AsyncThread}. + *
+ * + * @see uws.service.UWSFactory#createJobThread(uws.job.UWSJob) + * @see AsyncThread + */ + @Override + public final JobThread createJobThread(final UWSJob job) throws UWSException{ + try{ + return new AsyncThread((TAPJob)job, createADQLExecutor(), getErrorWriter()); + }catch(TAPException te){ + if (te.getCause() != null && te.getCause() instanceof UWSException) + throw (UWSException)te.getCause(); + else + throw new UWSException(UWSException.INTERNAL_SERVER_ERROR, te, "Impossible to create an AsyncThread !"); + } + } - public ADQLQueryFactory createQueryFactory() throws TAPException; + /** + *Extract the parameters from the given request (multipart or not).
+ * + *+ * This function is used only to create the set of parameters for a TAP job (synchronous or asynchronous). + * Thus, it just call {@link #createTAPParameters(HttpServletRequest)} with the given request, in order to ensure + * that the returned object is always a {@link TAPParameters}. + *
+ * + * @see #createTAPParameters(HttpServletRequest) + */ + @Override + public final UWSParameters createUWSParameters(HttpServletRequest request) throws UWSException{ + try{ + return createTAPParameters(request); + }catch(TAPException te){ + if (te.getCause() != null && te.getCause() instanceof UWSException) + throw (UWSException)te.getCause(); + else + throw new UWSException(te.getHttpErrorCode(), te); + } + } - public QueryChecker createQueryChecker(TAPSchema uploadSchema) throws TAPException; + /** + *Extract all the TAP parameters from the given HTTP request (multipart or not) and return them.
+ * + *Note: + * A default implementation is provided by {@link AbstractTAPFactory}. + *
+ * + * @param request The HTTP request containing the TAP parameters to extract. + * + * @return An object gathering all successfully extracted TAP parameters. + * + * @throws TAPException If any error occurs while extracting the parameters. + */ + public abstract TAPParameters createTAPParameters(final HttpServletRequest request) throws TAPException; - public ADQLTranslator createADQLTranslator() throws TAPException; + /** + *Identify and gather all identified parameters of the given map inside a {@link TAPParameters} object.
+ * + *+ * This implementation just call {@link #createTAPParameters(Map)} with the given map, in order to ensure + * that the returned object is always a {@link TAPParameters}. + *
+ * + * @see uws.service.AbstractUWSFactory#createUWSParameters(java.util.Map) + * @see #createTAPParameters(Map) + */ + @Override + public final UWSParameters createUWSParameters(MapIdentify all TAP parameters and gather them inside a {@link TAPParameters} object.
+ * + *Note: + * A default implementation is provided by {@link AbstractTAPFactory}. + *
+ * + * @param params Map containing all parameters. + * + * @return An object gathering all successfully identified TAP parameters. + * + * @throws TAPException If any error occurs while creating the {@link TAPParameters} object. + */ + public abstract TAPParameters createTAPParameters(final MapDescription of a TAP job. This class is used for asynchronous but also synchronous queries.
+ * + *+ * On the contrary to {@link UWSJob}, it is loading parameters from {@link TAPParameters} instances rather than {@link UWSParameters}. + * However, {@link TAPParameters} is an extension of {@link UWSParameters}. That's what allow the UWS library to use both {@link TAPJob} and {@link TAPParameters}. + *
+ * + * @author Grégory Mantelet (CDS;ARI) + * @version 2.0 (04/2015) + */ public class TAPJob extends UWSJob { - private static final long serialVersionUID = 1L; + /** Name of the standard TAP parameter which specifies the type of request to execute: "REQUEST". */ public static final String PARAM_REQUEST = "request"; + /** REQUEST value meaning an ADQL query must be executed: "doQuery". */ public static final String REQUEST_DO_QUERY = "doQuery"; + /** REQUEST value meaning VO service capabilities must be returned: "getCapabilities". */ public static final String REQUEST_GET_CAPABILITIES = "getCapabilities"; + /** Name of the standard TAP parameter which specifies the query language: "LANG". (only the ADQL language is supported by default in this version of the library) */ public static final String PARAM_LANGUAGE = "lang"; + /** LANG value meaning ADQL language: "ADQL". */ public static final String LANG_ADQL = "ADQL"; + /** LANG value meaning PQL language: "PQL". (this language is not supported in this version of the library) */ public static final String LANG_PQL = "PQL"; + /** Name of the standard TAP parameter which specifies the version of the TAP protocol that must be used: "VERSION". (only the version 1.0 is supported in this version of the library) */ public static final String PARAM_VERSION = "version"; + /** VERSION value meaning the version 1.0 of TAP: "1.0". */ public static final String VERSION_1_0 = "1.0"; + /** Name of the standard TAP parameter which specifies the output format (format of a query result): "FORMAT". */ public static final String PARAM_FORMAT = "format"; + /** FORMAT value meaning the VOTable format: "votable". */ public static final String FORMAT_VOTABLE = "votable"; + /** Name of the standard TAP parameter which specifies the maximum number of rows that must be returned in the query result: "MAXREC". */ public static final String PARAM_MAX_REC = "maxRec"; + /** Special MAXREC value meaning the number of output rows is not limited. */ public static final int UNLIMITED_MAX_REC = -1; + /** Name of the standard TAP parameter which specifies the query to execute: "QUERY". */ public static final String PARAM_QUERY = "query"; + + /** Name of the standard TAP parameter which defines the tables to upload in the database for the query execution: "UPLOAD". */ public static final String PARAM_UPLOAD = "upload"; + /** Name of the library parameter which informs about a query execution progression: "PROGRESSION". (this parameter is removed once the execution is finished) */ public static final String PARAM_PROGRESSION = "progression"; - protected TAPExecutionReport execReport; + /** Internal query execution report. */ + protected TAPExecutionReport execReport = null; + /** Parameters of this job for its execution. */ protected final TAPParameters tapParams; - public TAPJob(final JobOwner owner, final TAPParameters tapParams) throws UWSException, TAPException{ + /** + *Build a pending TAP job with the given parameters.
+ * + *Note: if the parameter {@link #PARAM_PHASE} (phase) is given with the value {@link #PHASE_RUN} + * the job execution starts immediately after the job has been added to a job list or after {@link #applyPhaseParam(JobOwner)} is called.
+ * + * @param owner User who owns this job. MAY BE NULL + * @param tapParams Set of parameters. + * + * @throws TAPException If one of the given parameters has a forbidden or wrong value. + */ + public TAPJob(final JobOwner owner, final TAPParameters tapParams) throws TAPException{ super(owner, tapParams); this.tapParams = tapParams; tapParams.check(); - //progression = ExecutionProgression.PENDING; - //loadTAPParams(tapParams); } - public TAPJob(final String jobID, final JobOwner owner, final TAPParameters params, final long quote, final long startTime, final long endTime, final ListRestore a job in a state defined by the given parameters. + * The phase must be set separately with {@link #setPhase(uws.job.ExecutionPhase, boolean)}, where the second parameter is true.
+ * + * @param jobID ID of the job. + * @param owner User who owns this job. + * @param params Set of not-standard UWS parameters (i.e. what is called by {@link UWSJob} as additional parameters ; they includes all TAP parameters). + * @param quote Quote of this job. + * @param startTime Date/Time at which this job started. (if not null, it means the job execution was finished, so a endTime should be provided) + * @param endTime Date/Time at which this job finished. + * @param results List of results. NULL if the job has not been executed, has been aborted or finished with an error. + * @param error Error with which this job ends. + * + * @throws TAPException If one of the given parameters has a forbidden or wrong value. + */ + public TAPJob(final String jobID, final JobOwner owner, final TAPParameters params, final long quote, final long startTime, final long endTime, final ListGet the value of the REQUEST parameter.
+ * + *This value must be {@value #REQUEST_DO_QUERY}.
+ * + * @return REQUEST value. + */ public final String getRequest(){ return tapParams.getRequest(); } + /** + * Get the value of the FORMAT parameter. + * + * @return FORMAT value. + */ public final String getFormat(){ return tapParams.getFormat(); } + /** + *Get the value of the LANG parameter.
+ * + *This value should always be {@value #LANG_ADQL} in this version of the library
+ * + * @return LANG value. + */ public final String getLanguage(){ return tapParams.getLang(); } + /** + *Get the value of the MAXREC parameter.
+ * + *If this value is negative, it means the number of output rows is not limited.
+ * + * @return MAXREC value. + */ public final int getMaxRec(){ return tapParams.getMaxRec(); } + /** + * Get the value of the QUERY parameter (i.e. the query, in the language returned by {@link #getLanguage()}, to execute). + * + * @return QUERY value. + */ public final String getQuery(){ return tapParams.getQuery(); } + /** + *Get the value of the VERSION parameter.
+ * + *This value should be {@value #VERSION_1_0} in this version of the library.
+ * + * @return VERSION value. + */ public final String getVersion(){ return tapParams.getVersion(); } + /** + *Get the value of the UPLOAD parameter.
+ * + *This value must be formatted as specified by the TAP standard (= a semicolon separated list of DALI uploads).
+ * + * @return UPLOAD value. + */ public final String getUpload(){ return tapParams.getUpload(); } - public final TableLoader[] getTablesToUpload(){ - return tapParams.getTableLoaders(); + /** + *Get the list of tables to upload in the database for the query execution.
+ * + *The returned array is an interpretation of the UPLOAD parameter.
+ * + * @return List of tables to upload. + */ + public final DALIUpload[] getTablesToUpload(){ + return tapParams.getUploadedTables(); } /** + *Get the execution report.
+ * + *+ * This report is available only during or after the job execution. + * It tells in which step the execution is, and how long was the previous steps. + * It can also give more information about the number of resulting rows and columns. + *
+ * * @return The execReport. */ public final TAPExecutionReport getExecReport(){ @@ -138,63 +241,120 @@ public class TAPJob extends UWSJob { } /** - * @param execReport The execReport to set. + *Set the execution report.
+ * + *IMPORTANT: + * This function can be called only if the job is running or is being restored, otherwise an exception would be thrown. + * It should not be used by implementors, but only by the internal library processing. + *
+ * + * @param execReport An execution report. + * + * @throws UWSException If this job has never been restored and is not running. */ - public final void setExecReport(TAPExecutionReport execReport) throws UWSException{ - if (getRestorationDate() == null && !isRunning()) + public final void setExecReport(final TAPExecutionReport execReport) throws UWSException{ + if (getRestorationDate() == null && (thread == null || thread.isFinished())) throw new UWSException("Impossible to set an execution report if the job is not in the EXECUTING phase ! Here, the job \"" + jobId + "\" is in the phase " + getPhase()); this.execReport = execReport; } - /* - *Starts in an asynchronous manner this ADQLExecutor.
- *The execution will stop after the duration specified in the given {@link TAPJob} - * (see {@link TAPJob#getExecutionDuration()}).
- * - * @param output - * @return - * @throws IllegalStateException - * @throws InterruptedException - * - public synchronized final boolean startSync(final OutputStream output) throws IllegalStateException, InterruptedException, UWSException { - // TODO Set the output stream so that the result is written directly in the given output ! - start(); - System.out.println("Joining..."); - thread.join(getExecutionDuration()); - System.out.println("Aborting..."); - thread.interrupt(); - thread.join(getTimeToWaitForEnd()); - return thread.isInterrupted(); - }*/ + /** + *Create the thread to use for the execution of this job.
+ * + *Note: If the job already exists, this function does nothing.
+ * + * @throws NullPointerException If the factory returned NULL rather than the asked {@link JobThread}. + * @throws UWSException If the thread creation fails. + * + * @see TAPFactory#createJobThread(UWSJob) + * + * @since 2.0 + */ + private final void createThread() throws NullPointerException, UWSException{ + if (thread == null){ + thread = getFactory().createJobThread(this); + if (thread == null) + throw new NullPointerException("Missing job work! The thread created by the factory is NULL => The job can't be executed!"); + } + } + + /** + *Check whether this job is able to start right now.
+ * + *+ * Basically, this function try to get a database connection. If none is available, + * then this job can not start and this function return FALSE. In all the other cases, + * TRUE is returned. + *
+ * + *Warning: This function will indirectly open and keep a database connection, so that the job can be started just after its call. + * If it turns out that the execution won't start just after this call, the DB connection should be closed in some way in order to save database resources.
+ * + * @return true if this job can start right now, false otherwise. + * + * @since 2.0 + */ + public final boolean isReadyForExecution(){ + return thread != null && ((AsyncThread)thread).isReadyForExecution(); + } @Override - protected void stop(){ - if (!isStopped()){ - //try { - stopping = true; - // TODO closeDBConnection(); - super.stop(); - /*} catch (TAPException e) { - getLogger().error("Impossible to cancel the query execution !", e); - return; - }*/ + public final void start(final boolean useManager) throws UWSException{ + // This job must know its jobs list and this jobs list must know its UWS: + if (getJobList() == null || getJobList().getUWS() == null) + throw new IllegalStateException("A TAPJob can not start if it is not linked to a job list or if its job list is not linked to a UWS."); + + // If already running do nothing: + else if (isRunning()) + return; + + // If asked propagate this request to the execution manager: + else if (useManager){ + // Create its corresponding thread, if not already existing: + createThread(); + // Ask to the execution manager to test whether the job is ready for execution, and if, execute it (by calling this function with "false" as parameter): + getJobList().getExecutionManager().execute(this); + + }// Otherwise start directly the execution: + else{ + // Create its corresponding thread, if not already existing: + createThread(); + if (!isReadyForExecution()){ + UWSException ue = new NoDBConnectionAvailableException(); + ((TAPLog)getLogger()).logDB(LogLevel.ERROR, null, "CONNECTION_LACK", "No more database connection available for the moment!", ue); + getLogger().logJob(LogLevel.ERROR, this, "ERROR", "Asynchronous job " + jobId + " execution aborted: no database connection available!", null); + throw ue; + } + + // Change the job phase: + setPhase(ExecutionPhase.EXECUTING); + + // Set the start time: + setStartTime(new Date()); + + // Run the job: + thread.start(); + (new JobTimeOut()).start(); + + // Log the start of this job: + getLogger().logJob(LogLevel.INFO, this, "START", "Job \"" + jobId + "\" started.", null); } } - /*protected boolean deleteResultFiles(){ - try{ - // TODO service.deleteResults(this); - return true; - }catch(TAPException ex){ - service.log(LogType.ERROR, "Job "+getJobId()+" - Can't delete results files: "+ex.getMessage()); - return false; + /** + * This exception is thrown by a job execution when no database connection are available anymore. + * + * @author Grégory Mantelet (ARI) + * @version 2.0 (02/2015) + * @since 2.0 + */ + public static class NoDBConnectionAvailableException extends UWSException { + private static final long serialVersionUID = 1L; + + public NoDBConnectionAvailableException(){ + super("Service momentarily too busy! Please try again later."); } - }*/ - @Override - public void clearResources(){ - super.clearResources(); - // TODO deleteResultFiles(); } } diff --git a/src/tap/TAPRequestParser.java b/src/tap/TAPRequestParser.java new file mode 100644 index 0000000000000000000000000000000000000000..91c24c6099e227ee1212fbdaccf8738ac06359ac --- /dev/null +++ b/src/tap/TAPRequestParser.java @@ -0,0 +1,216 @@ +package tap; + +/* + * This file is part of TAPLibrary. + * + * TAPLibrary is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * TAPLibrary is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with TAPLibrary. If not, seeThis parser adapts the request parser to use in function of the request content-type:
+ *+ * The request body size is limited for the multipart AND the no-encoding parsers. If you want to change this limit, + * you MUST do it for each of these parsers, setting the following static attributes: resp. {@link MultipartParser#SIZE_LIMIT} + * and {@link NoEncodingParser#SIZE_LIMIT}. + *
+ * + *Note: + * If you want to change the support other request parsing, you will have to write your own {@link RequestParser} implementation. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 2.0 (12/2014) + * @since 2.0 + */ +public class TAPRequestParser implements RequestParser { + + /** File manager to use to create {@link UploadFile} instances. + * It is required by this new object to execute open, move and delete operations whenever it could be asked. */ + private final UWSFileManager fileManager; + + /** {@link RequestParser} to use when a application/x-www-form-urlencoded request must be parsed. This attribute is set by {@link #parse(HttpServletRequest)} + * only when needed, by calling the function {@link #getFormParser()}. */ + private RequestParser formParser = null; + + /** {@link RequestParser} to use when a multipart/form-data request must be parsed. This attribute is set by {@link #parse(HttpServletRequest)} + * only when needed, by calling the function {@link #getMultipartParser()}. */ + private RequestParser multipartParser = null; + + /** {@link RequestParser} to use when none of the other parsers can be used ; it will then transform the whole request body in a parameter called "JDL" + * (Job Description Language). This attribute is set by {@link #parse(HttpServletRequest)} only when needed, by calling the function + * {@link #getNoEncodingParser()}. */ + private RequestParser noEncodingParser = null; + + /** + * Build a {@link RequestParser} able to choose the most appropriate {@link RequestParser} in function of the request content-type. + * + * @param fileManager The file manager to use in order to store any eventual upload. MUST NOT be NULL + */ + public TAPRequestParser(final UWSFileManager fileManager){ + if (fileManager == null) + throw new NullPointerException("Missing file manager => can not create a TAPRequestParser!"); + this.fileManager = fileManager; + } + + @Override + public MapThis class represent a TAP synchronous job. + * A such job must execute an ADQL query and return immediately its result.
+ * + *+ * The execution of a such job is limited to a short time. Once this time elapsed, the job is stopped. + * For a longer job, an asynchronous job should be used. + *
+ * + *+ * If an error occurs it must be propagated ; it will be written later in the HTTP response on a top level. + *
+ * + * @author Grégory Mantelet (CDS;ARI) + * @version 2.0 (02/2015) + */ public class TAPSyncJob { /** The time (in ms) to wait the end of the thread after an interruption. */ protected long waitForStop = 1000; + /** Last generated ID of a synchronous job. */ protected static String lastId = null; - protected final ServiceConnection service; + /** Description of the TAP service in charge of this synchronous job. */ + protected final ServiceConnection service; + /** ID of this job. This ID is also used to identify the thread. */ protected final String ID; + + /** Parameters of the execution. It mainly contains the ADQL query to execute. */ protected final TAPParameters tapParams; + /** The thread in which the query execution will be done. */ protected SyncThread thread; + /** Report of the query execution. It stays NULL until the execution ends. */ protected TAPExecutionReport execReport = null; + /** Date at which this synchronous job has really started. It is NULL when the job has never been started. + * + *Note: A synchronous job can be run just once ; so if an attempt of executing it again, the start date will be tested: + * if NULL, the second starting is not considered and an exception is thrown.
*/ private Date startedAt = null; - public TAPSyncJob(final ServiceConnection service, final TAPParameters params) throws NullPointerException{ + /** + * Create a synchronous TAP job. + * + * @param service Description of the TAP service which is in charge of this synchronous job. + * @param params Parameters of the query to execute. It must mainly contain the ADQL query to execute. + * + * @throws NullPointerException If one of the parameters is NULL. + */ + public TAPSyncJob(final ServiceConnection service, final TAPParameters params) throws NullPointerException{ if (params == null) throw new NullPointerException("Missing TAP parameters ! => Impossible to create a synchronous TAP job."); tapParams = params; @@ -63,8 +105,8 @@ public class TAPSyncJob { * *By default: "S"+System.currentTimeMillis()+UpperCharacter (UpperCharacter: one upper-case character: A, B, C, ....)
* - *note: DO NOT USE in this function any of the following functions: {@link #getLogger()}, - * {@link #getFileManager()} and {@link #getFactory()}. All of them will return NULL, because this job does not + *
note: DO NOT USE in this function any of the following functions: {@link ServiceConnection#getLogger()}, + * {@link ServiceConnection#getFileManager()} and {@link ServiceConnection#getFactory()}. All of them will return NULL, because this job does not * yet know its jobs list (which is needed to know the UWS and so, all of the objects returned by these functions).
* * @return A unique job identifier. @@ -79,94 +121,214 @@ public class TAPSyncJob { return generatedId; } + /** + * Get the ID of this synchronous job. + * + * @return The job ID. + */ public final String getID(){ return ID; } + /** + * Get the TAP parameters provided by the user and which will be used for the execution of this job. + * + * @return Job parameters. + */ public final TAPParameters getTapParams(){ return tapParams; } + /** + * Get the report of the execution of this job. + * This report is NULL if the execution has not yet started. + * + * @return Report of this job execution. + */ public final TAPExecutionReport getExecReport(){ return execReport; } - public synchronized boolean start(final HttpServletResponse response) throws IllegalStateException, UWSException, TAPException{ + /** + *Start the execution of this job in order to execute the given ADQL query.
+ * + *The execution itself will be processed by an {@link ADQLExecutor} inside a thread ({@link SyncThread}).
+ * + *Important: + * No error should be written in this function. If any error occurs it should be thrown, in order to be manager on a top level. + *
+ * + * @param response Response in which the result must be written. + * + * @return true if the execution was successful, false otherwise. + * + * @throws IllegalStateException If this synchronous job has already been started before. + * @throws IOException If any error occurs while writing the query result in the given {@link HttpServletResponse}. + * @throws TAPException If any error occurs while executing the ADQL query. + * + * @see SyncThread + */ + public synchronized boolean start(final HttpServletResponse response) throws IllegalStateException, IOException, TAPException{ if (startedAt != null) - throw new IllegalStateException("Impossible to restart a synchronous TAP query !"); + throw new IllegalStateException("Impossible to restart a synchronous TAP query!"); + + // Log the start of this sync job: + service.getLogger().logTAP(LogLevel.INFO, this, "START", "Synchronous job " + ID + " is starting!", null); - ADQLExecutor executor; + // Create the object having the knowledge about how to execute an ADQL query: + ADQLExecutor executor = service.getFactory().createADQLExecutor(); try{ - executor = service.getFactory().createADQLExecutor(); - }catch(TAPException e){ - // TODO Log this error ! - return true; + executor.initDBConnection(ID); + }catch(TAPException te){ + service.getLogger().logDB(LogLevel.ERROR, null, "CONNECTION_LACK", "No more database connection available for the moment!", te); + service.getLogger().logTAP(LogLevel.ERROR, this, "END", "Synchronous job " + ID + " execution aborted: no database connection available!", null); + throw new TAPException("TAP service too busy! No connection available for the moment. You should try later or create an asynchronous query (which will be executed when enough resources will be available again).", UWSException.SERVICE_UNAVAILABLE); } + + // Give to a thread which will execute the query: thread = new SyncThread(executor, ID, tapParams, response); thread.start(); - boolean timeout = false; + // Wait the end of the thread until the maximum execution duration is reached: + boolean timeout = false; try{ - System.out.println("Joining..."); - thread.join(tapParams.getExecutionDuration()); + // wait the end: + thread.join(tapParams.getExecutionDuration() * 1000); + // if still alive after this duration, interrupt it: if (thread.isAlive()){ timeout = true; - System.out.println("Aborting..."); thread.interrupt(); thread.join(waitForStop); } }catch(InterruptedException ie){ - ; + /* Having a such exception here, is not surprising, because we may have interrupted the thread! */ }finally{ + // Whatever the way the execution stops (normal, cancel or error), an execution report must be fulfilled: execReport = thread.getExecutionReport(); } - if (!thread.isSuccess()){ + // Report any error that may have occurred while the thread execution: + Throwable error = thread.getError(); + // CASE: TIMEOUT + if (timeout && error != null && error instanceof InterruptedException){ + // Log the timeout: if (thread.isAlive()) - throw new TAPException("Time out (=" + tapParams.getExecutionDuration() + "ms) ! However, the thread (synchronous query) can not be stopped !", HttpServletResponse.SC_INTERNAL_SERVER_ERROR); - else if (timeout) - throw new TAPException("Time out ! The execution of this synchronous TAP query was limited to " + tapParams.getExecutionDuration() + "ms.", HttpServletResponse.SC_INTERNAL_SERVER_ERROR); + service.getLogger().logTAP(LogLevel.WARNING, this, "TIME_OUT", "Time out (after " + tapParams.getExecutionDuration() + "ms) for the synchonous job " + ID + ", but the thread can not be interrupted!", null); + else + service.getLogger().logTAP(LogLevel.INFO, this, "TIME_OUT", "Time out (after " + tapParams.getExecutionDuration() + "ms) for the synchonous job " + ID + ".", null); + + // Report the timeout to the user: + throw new TAPException("Time out! The execution of this synchronous TAP query was limited to " + tapParams.getExecutionDuration() + "ms. You should try again but in asynchronous execution.", UWSException.ACCEPTED_BUT_NOT_COMPLETE); + } + // CASE: ERRORS + else if (!thread.isSuccess()){ + // INTERRUPTION: + if (error instanceof InterruptedException){ + // log the unexpected interruption (unexpected because not caused by a timeout): + service.getLogger().logTAP(LogLevel.ERROR, this, "END", "The execution of the synchronous job " + ID + " has been unexpectedly interrupted!", error); + // report the unexpected interruption to the user: + throw new TAPException("The execution of this synchronous job " + ID + " has been unexpectedly aborted!", UWSException.ACCEPTED_BUT_NOT_COMPLETE); + } + // REQUEST ABORTION: + else if (error instanceof IOException){ + // log the unexpected interruption (unexpected because not caused by a timeout): + service.getLogger().logTAP(LogLevel.INFO, this, "END", "Abortion of the synchronous job " + ID + "! Cause: connection with the HTTP client unexpectedly closed.", null); + // throw the error until the TAP instance to notify it about the abortion: + throw (IOException)error; + } + // TAP EXCEPTION: + else if (error instanceof TAPException){ + // log the error: + service.getLogger().logTAP(LogLevel.ERROR, this, "END", "The following error interrupted the execution of the synchronous job " + ID + ".", error); + // report the error to the user: + throw (TAPException)error; + } + // ANY OTHER EXCEPTION: else{ - Throwable t = thread.getError(); - if (t instanceof InterruptedException) - throw new TAPException("The execution of this synchronous TAP query has been unexpectedly aborted !"); - else if (t instanceof UWSException) - throw (UWSException)t; + // log the error: + service.getLogger().logTAP(LogLevel.FATAL, this, "END", "The following GRAVE error interrupted the execution of the synchronous job " + ID + ".", error); + // report the error to the user: + if (error instanceof Error) + throw (Error)error; else - throw new TAPException(t); + throw new TAPException(error); } - } + }else + service.getLogger().logTAP(LogLevel.INFO, this, "END", "Success of the synchronous job " + ID + ".", null); - return thread.isInterrupted(); + return thread.isSuccess(); } - public class SyncThread extends Thread { + /** + *Thread which will process the job execution.
+ * + *+ * Actually, it will basically just call {@link ADQLExecutor#start(Thread, String, TAPParameters, HttpServletResponse)} + * with the given {@link ADQLExecutor} and TAP parameters (containing the ADQL query to execute). + *
+ * + * @author Grégory Mantelet (CDS;ARI) + * @version 2.0 (04/2015) + */ + protected class SyncThread extends Thread { - private final String taskDescription; - public final ADQLExecutor executor; + /** Object knowing how to execute an ADQL query and which will execute it by calling {@link ADQLExecutor#start(Thread, String, TAPParameters, HttpServletResponse)}. */ + protected final ADQLExecutor executor; + /** Response in which the query result must be written. No error should be written in it directly at this level ; + * the error must be propagated and it will be written in this HTTP response later on a top level. */ protected final HttpServletResponse response; + /** ID of this thread. It is also the ID of the synchronous job owning this thread. */ protected final String ID; + /** Parameters containing the ADQL query to execute and other execution parameters/options. */ protected final TAPParameters tapParams; + + /** Exception that occurs while executing this thread. NULL if the execution was a success. */ protected Throwable exception = null; + /** Query execution report. NULL if the execution has not yet started. */ protected TAPExecutionReport report = null; - public SyncThread(final ADQLExecutor executor, final String ID, final TAPParameters tapParams, final HttpServletResponse response){ + /** + * Create a thread that will run the given executor with the given parameters. + * + * @param executor Object to execute and which knows how to execute an ADQL query. + * @param ID ID of the synchronous job owning this thread. + * @param tapParams TAP parameters to use to get the query to execute and the execution parameters. + * @param response HTTP response in which the ADQL query result must be written. + */ + public SyncThread(final ADQLExecutor executor, final String ID, final TAPParameters tapParams, final HttpServletResponse response){ super(JobThread.tg, ID); - taskDescription = "Executing the synchronous TAP query " + ID; this.executor = executor; this.ID = ID; this.tapParams = tapParams; this.response = response; } + /** + * Tell whether the execution has ended with success. + * + * @return true if the query has been successfully executed, + * false otherwise (or if this thread is still executed). + */ public final boolean isSuccess(){ return !isAlive() && report != null && exception == null; } + /** + * Get the error that has interrupted/stopped this thread. + * This function returns NULL if the query has been successfully executed. + * + * @return Error that occurs while executing the query + * or NULL if the execution was a success. + */ public final Throwable getError(){ return exception; } + /** + * Get the report of the query execution. + * + * @return Query execution report. + */ public final TAPExecutionReport getExecutionReport(){ return report; } @@ -174,17 +336,30 @@ public class TAPSyncJob { @Override public void run(){ // Log the start of this thread: - executor.getLogger().threadStarted(this, taskDescription); + executor.getLogger().logThread(LogLevel.INFO, thread, "START", "Synchronous thread \"" + ID + "\" started.", null); try{ + // Execute the ADQL query: report = executor.start(this, ID, tapParams, response); - executor.getLogger().threadFinished(this, taskDescription); + + // Log the successful end of this thread: + executor.getLogger().logThread(LogLevel.INFO, thread, "END", "Synchronous thread \"" + ID + "\" successfully ended.", null); + }catch(Throwable e){ + + // Save the exception for later reporting: exception = e; - if (e instanceof InterruptedException){ - // Log the abortion: - executor.getLogger().threadInterrupted(this, taskDescription, e); - } + + // Log the end of the job: + if (e instanceof InterruptedException || e instanceof IOException) + // Abortion: + executor.getLogger().logThread(LogLevel.INFO, this, "END", "Synchronous thread \"" + ID + "\" cancelled.", null); + else if (e instanceof TAPException) + // Error: + executor.getLogger().logThread(LogLevel.ERROR, this, "END", "Synchronous thread \"" + ID + "\" ended with an error.", null); + else + // GRAVE error: + executor.getLogger().logThread(LogLevel.FATAL, this, "END", "Synchronous thread \"" + ID + "\" ended with a FATAL error.", null); } } diff --git a/src/tap/backup/DefaultTAPBackupManager.java b/src/tap/backup/DefaultTAPBackupManager.java index 4e4a86f48d84062a03a162b2a9fd8e1bc00c1366..5e740ddd33e13f8630969c81fd05ae4ec7ba05b1 100644 --- a/src/tap/backup/DefaultTAPBackupManager.java +++ b/src/tap/backup/DefaultTAPBackupManager.java @@ -16,105 +16,288 @@ package tap.backup; * You should have received a copy of the GNU Lesser General Public License * along with TAPLibrary. If not, seeLet backup all TAP asynchronous jobs.
+ * + *note: Basically the saved data are the same, but in addition some execution statistics are also added.
+ * + * @author Grégory Mantelet (CDS;ARI) + * @version 2.0 (12/2014) + * + * @see DefaultUWSBackupManager + */ public class DefaultTAPBackupManager extends DefaultUWSBackupManager { + /** + * Build a default TAP jobs backup manager. + * + * @param uws The UWS containing all the jobs to backup. + * + * @see DefaultUWSBackupManager#DefaultUWSBackupManager(UWS) + */ public DefaultTAPBackupManager(UWS uws){ super(uws); } + /** + * Build a default TAP jobs backup manager. + * + * @param uws The UWS containing all the jobs to backup. + * @param frequency The backup frequency (in ms ; MUST BE positive and different from 0. + * If negative or 0, the frequency will be automatically set to DEFAULT_FREQUENCY). + * + * @see DefaultUWSBackupManager#DefaultUWSBackupManager(UWS, long) + */ public DefaultTAPBackupManager(UWS uws, long frequency){ super(uws, frequency); } + /** + * Build a default TAP jobs backup manager. + * + * @param uws The UWS containing all the jobs to backup. + * @param byUser Backup mode. + * + * @see DefaultUWSBackupManager#DefaultUWSBackupManager(UWS, boolean) + */ public DefaultTAPBackupManager(UWS uws, boolean byUser) throws UWSException{ super(uws, byUser); } + /** + * Build a default TAP jobs backup manager. + * + * @param uws The UWS containing all the jobs to backup. + * @param byUser Backup mode. + * @param frequency The backup frequency (in ms ; MUST BE positive and different from 0. + * If negative or 0, the frequency will be automatically set to DEFAULT_FREQUENCY). + * + * @see DefaultUWSBackupManager#DefaultUWSBackupManager(UWS, boolean, long) + */ public DefaultTAPBackupManager(UWS uws, boolean byUser, long frequency) throws UWSException{ super(uws, byUser, frequency); } @Override protected JSONObject getJSONJob(UWSJob job, String jlName) throws UWSException, JSONException{ - JSONObject json = super.getJSONJob(job, jlName); + JSONObject jsonJob = Json4Uws.getJson(job); + + // Re-Build the parameters map, by separating the uploads and the "normal" parameters: + JSONArray uploads = new JSONArray(); + JSONObject params = new JSONObject(); + Object val; + for(String name : job.getAdditionalParameters()){ + // get the raw value: + val = job.getAdditionalParameterValue(name); + // if no value, skip this item: + if (val == null) + continue; + // if an array, build a JSON array of strings: + else if (val.getClass().isArray()){ + JSONArray array = new JSONArray(); + for(Object o : (Object[])val){ + if (o != null && o instanceof DALIUpload) + array.put(getDALIUploadJson((DALIUpload)o)); + else if (o != null) + array.put(o.toString()); + } + params.put(name, array); + } + // if upload file: + else if (val instanceof UploadFile) + uploads.put(getUploadJson((UploadFile)val)); + // if DALIUpload: + else if (val instanceof DALIUpload) + params.put(name, getDALIUploadJson((DALIUpload)val)); + // otherwise, just put the value: + else + params.put(name, val); + } + // Deal with the execution report of the job: if (job instanceof TAPJob && ((TAPJob)job).getExecReport() != null){ TAPExecutionReport execReport = ((TAPJob)job).getExecReport(); + // Build the JSON representation of the execution report of this job: JSONObject jsonExecReport = new JSONObject(); jsonExecReport.put("success", execReport.success); - jsonExecReport.put("sql", execReport.sqlTranslation); jsonExecReport.put("uploadduration", execReport.getUploadDuration()); jsonExecReport.put("parsingduration", execReport.getParsingDuration()); - jsonExecReport.put("translationduration", execReport.getTranslationDuration()); jsonExecReport.put("executionduration", execReport.getExecutionDuration()); jsonExecReport.put("formattingduration", execReport.getFormattingDuration()); jsonExecReport.put("totalduration", execReport.getTotalDuration()); - JSONObject params = json.getJSONObject(UWSJob.PARAM_PARAMETERS); - if (params == null) - params = new JSONObject(); + // Add the execution report into the parameters list: params.put("tapexecreport", jsonExecReport); - - json.put(UWSJob.PARAM_PARAMETERS, params); } - return json; + // Add the parameters and the uploads inside the JSON representation of the job: + jsonJob.put(UWSJob.PARAM_PARAMETERS, params); + jsonJob.put("uwsUploads", uploads); + + // Add the job owner: + jsonJob.put(UWSJob.PARAM_OWNER, (job != null && job.getOwner() != null) ? job.getOwner().getID() : null); + + // Add the name of the job list owning the given job: + jsonJob.put("jobListName", jlName); + + return jsonJob; + } + + /** + * Get the JSON representation of the given {@link DALIUpload}. + * + * @param upl The DALI upload specification to serialize in JSON. + * + * @return Its JSON representation. + * + * @throws JSONException If there is an error while building the JSON object. + * + * @since 2.0 + */ + protected JSONObject getDALIUploadJson(final DALIUpload upl) throws JSONException{ + if (upl == null) + return null; + JSONObject o = new JSONObject(); + o.put("label", upl.label); + o.put("uri", upl.uri); + o.put("file", (upl.file == null ? null : upl.file.paramName)); + return o; } @Override protected void restoreOtherJobParams(JSONObject json, UWSJob job) throws UWSException{ - if (job != null && json != null && job instanceof TAPJob){ - TAPJob tapJob = (TAPJob)job; - Object obj = job.getAdditionalParameterValue("tapexecreport"); - if (obj != null){ - if (obj instanceof JSONObject){ - JSONObject jsonExecReport = (JSONObject)obj; - TAPExecutionReport execReport = new TAPExecutionReport(job.getJobId(), false, tapJob.getTapParams()); - String[] keys = JSONObject.getNames(jsonExecReport); - for(String key : keys){ + // 0. Nothing to do in this function if the job is missing OR if it is not an instance of TAPJob: + if (job == null || !(job instanceof TAPJob)) + return; + + // 1. Build correctly the TAP UPLOAD parameter (the value of this parameter should be an array of DALIUpload): + if (json != null && json.has(TAPJob.PARAM_PARAMETERS)){ + try{ + // Retrieve the whole list of parameters: + JSONObject params = json.getJSONObject(TAPJob.PARAM_PARAMETERS); + // If there is an UPLOAD parameter, convert the JSON array into a DALIUpload[] and add it to the job: + if (params.has(TAPJob.PARAM_UPLOAD)){ + // retrieve the JSON array: + JSONArray uploads = params.getJSONArray(TAPJob.PARAM_UPLOAD); + // for each item of this array, build the corresponding DALIUpload and add it into an ArrayList: + DALIUpload upl; + ArrayListConcrete implementation of {@link ServiceConnection}, fully parameterized with a TAP configuration file.
+ * + *+ * Every aspects of the TAP service are configured here. This instance is also creating the {@link TAPFactory} using the + * TAP configuration file thanks to the implementation {@link ConfigurableTAPFactory}. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 2.0 (04/2015) + * @since 2.0 + */ +public final class ConfigurableServiceConnection implements ServiceConnection { + + /** File manager to use in the TAP service. */ + private UWSFileManager fileManager; + + /** Object to use in the TAP service in order to log different types of messages (e.g. DEBUG, INFO, WARNING, ERROR, FATAL). */ + private TAPLog logger; + + /** Factory which can create different types of objects for the TAP service (e.g. database connection). */ + private TAPFactory tapFactory; + + /** Object gathering all metadata of this TAP service. */ + private final TAPMetadata metadata; + + /** Name of the organization/person providing the TAP service. */ + private final String providerName; + /** Description of the TAP service. */ + private final String serviceDescription; + + /** Indicate whether the TAP service is available or not. */ + private boolean isAvailable = false; // the TAP service must be disabled until the end of its connection initialization + /** Description of the available or unavailable state of the TAP service. */ + private String availability = "TAP service not yet initialized."; + + /** Maximum number of asynchronous jobs that can run simultaneously. */ + private int maxAsyncJobs = DEFAULT_MAX_ASYNC_JOBS; + + /** Array of 2 integers: resp. default and maximum execution duration. + * Both duration are expressed in milliseconds. */ + private int[] executionDuration = new int[2]; + /** Array of 2 integers: resp. default and maximum retention period. + * Both period are expressed in seconds. */ + private int[] retentionPeriod = new int[2]; + + /** List of all available output formatters. */ + private final ArrayListResolve the given file name/path.
+ * + *Only the URI protocol "file:" is allowed. If the protocol is different a {@link TAPException} is thrown.
+ * + *+ * If not an absolute URI, the given path may be either relative or absolute. A relative path is always considered + * as relative from the Web Application directory (supposed to be given in 2nd parameter). + *
+ * + * @param filePath URI/Path/Name of the file to get. + * @param webAppRootPath Web Application directory local path. + * @param propertyName Name of the property which gives the given file path. + * + * @return The specified File instance. + * + * @throws TAPException If the given URI is malformed or if the used URI scheme is different from "file:". + */ + protected static final File getFile(final String filePath, final String webAppRootPath, final String propertyName) throws TAPException{ + if (filePath == null) + return null; + + try{ + URI uri = new URI(filePath); + if (uri.isAbsolute()){ + if (uri.getScheme().equalsIgnoreCase("file")) + return new File(uri); + else + throw new TAPException("Incorrect file URI for the property \"" + propertyName + "\": \"" + filePath + "\"! Only URI with the protocol \"file:\" are allowed."); + }else{ + File f = new File(filePath); + if (f.isAbsolute()) + return f; + else + return new File(webAppRootPath, filePath); + } + }catch(URISyntaxException use){ + throw new TAPException("Incorrect file URI for the property \"" + propertyName + "\": \"" + filePath + "\"! Bad syntax for the given file URI.", use); + } + } + + /** + * Initialize the TAP logger with the given TAP configuration file. + * + * @param tapConfig The content of the TAP configuration file. + */ + private void initLogger(final Properties tapConfig){ + // Create the logger: + logger = new DefaultTAPLog(fileManager); + + StringBuffer buf = new StringBuffer("Logger initialized"); + + // Set the minimum log level: + String propValue = getProperty(tapConfig, KEY_MIN_LOG_LEVEL); + if (propValue != null){ + try{ + ((DefaultTAPLog)logger).setMinLogLevel(LogLevel.valueOf(propValue.toUpperCase())); + }catch(IllegalArgumentException iae){} + } + buf.append(" (minimum log level: ").append(((DefaultTAPLog)logger).getMinLogLevel()); + + // Set the log rotation period, if any: + if (fileManager instanceof LocalUWSFileManager){ + propValue = getProperty(tapConfig, KEY_LOG_ROTATION); + if (propValue != null) + ((LocalUWSFileManager)fileManager).setLogRotationFreq(propValue); + buf.append(", log rotation: ").append(((LocalUWSFileManager)fileManager).getLogRotationFreq()); + } + + // Log the successful initialization with set parameters: + buf.append(")."); + logger.info(buf.toString()); + } + + /** + *Initialize the {@link TAPFactory} to use.
+ * + *+ * The built factory is either a {@link ConfigurableTAPFactory} instance (by default) or + * an instance of the class specified in the TAP configuration file. + *
+ * + * @param tapConfig The content of the TAP configuration file. + * + * @throws TAPException If an error occurs while building the specified {@link TAPFactory}. + * + * @see ConfigurableTAPFactory + */ + private void initFactory(final Properties tapConfig) throws TAPException{ + String propValue = getProperty(tapConfig, KEY_TAP_FACTORY); + if (propValue == null) + tapFactory = new ConfigurableTAPFactory(this, tapConfig); + else + tapFactory = newInstance(propValue, KEY_TAP_FACTORY, TAPFactory.class, new Class[]{ServiceConnection.class}, new Object[]{this}); + } + + /** + * Initialize the TAP metadata (i.e. database schemas, tables and columns and their attached metadata). + * + * @param tapConfig The content of the TAP configuration file. + * @param webAppRootDir Web Application directory local path. + * This directory may be used if a relative path is given for an XML metadata file. + * + * @return The extracted TAP metadata. + * + * @throws TAPException If some TAP configuration file properties are wrong or missing, + * or if an error has occurred while extracting the metadata from the database or the XML file. + * + * @see DBConnection#getTAPSchema() + * @see TableSetParser + */ + private TAPMetadata initMetadata(final Properties tapConfig, final String webAppRootDir) throws TAPException{ + // Get the fetching method to use: + String metaFetchType = getProperty(tapConfig, KEY_METADATA); + if (metaFetchType == null) + throw new TAPException("The property \"" + KEY_METADATA + "\" is missing! It is required to create a TAP Service. Three possible values: " + VALUE_XML + " (to get metadata from a TableSet XML document), " + VALUE_DB + " (to fetch metadata from the database schema TAP_SCHEMA) or the name (between {}) of a class extending TAPMetadata."); + + TAPMetadata metadata = null; + + // GET METADATA FROM XML & UPDATE THE DATABASE (schema TAP_SCHEMA only): + if (metaFetchType.equalsIgnoreCase(VALUE_XML)){ + // Get the XML file path: + String xmlFilePath = getProperty(tapConfig, KEY_METADATA_FILE); + if (xmlFilePath == null) + throw new TAPException("The property \"" + KEY_METADATA_FILE + "\" is missing! According to the property \"" + KEY_METADATA + "\", metadata must be fetched from an XML document. The local file path of it MUST be provided using the property \"" + KEY_METADATA_FILE + "\"."); + + // Parse the XML document and build the corresponding metadata: + try{ + metadata = (new TableSetParser()).parse(getFile(xmlFilePath, webAppRootDir, KEY_METADATA_FILE)); + }catch(IOException ioe){ + throw new TAPException("A grave error occurred while reading/parsing the TableSet XML document: \"" + xmlFilePath + "\"!", ioe); + } + + // Update the database: + DBConnection conn = null; + try{ + conn = tapFactory.getConnection("SET_TAP_SCHEMA"); + conn.setTAPSchema(metadata); + }finally{ + if (conn != null) + tapFactory.freeConnection(conn); + } + } + // GET METADATA FROM DATABASE (schema TAP_SCHEMA): + else if (metaFetchType.equalsIgnoreCase(VALUE_DB)){ + DBConnection conn = null; + try{ + conn = tapFactory.getConnection("GET_TAP_SCHEMA"); + metadata = conn.getTAPSchema(); + }finally{ + if (conn != null) + tapFactory.freeConnection(conn); + } + } + // MANUAL ~ TAPMETADATA CLASS + else if (isClassName(metaFetchType)){ + /* 1. Get the metadata */ + // get the class: + Class metaClass = fetchClass(metaFetchType, KEY_METADATA, TAPMetadata.class); + if (metaClass == TAPMetadata.class) + throw new TAPException("Wrong class for the property \"" + KEY_METADATA + "\": \"" + metaClass.getName() + "\"! The class provided in this property MUST EXTEND tap.metadata.TAPMetadata."); + try{ + // get one of the expected constructors: + try{ + // (UWSFileManager, TAPFactory, TAPLog): + Constructor constructor = metaClass.getConstructor(UWSFileManager.class, TAPFactory.class, TAPLog.class); + // create the TAP metadata: + metadata = constructor.newInstance(fileManager, tapFactory, logger); + }catch(NoSuchMethodException nsme){ + // () (empty constructor): + Constructor constructor = metaClass.getConstructor(); + // create the TAP metadata: + metadata = constructor.newInstance(); + } + }catch(NoSuchMethodException nsme){ + throw new TAPException("Missing constructor tap.metadata.TAPMetadata() or tap.metadata.TAPMetadata(uws.service.file.UWSFileManager, tap.TAPFactory, tap.log.TAPLog)! See the value \"" + metaFetchType + "\" of the property \"" + KEY_METADATA + "\"."); + }catch(InstantiationException ie){ + throw new TAPException("Impossible to create an instance of an abstract class: \"" + metaClass.getName() + "\"! See the value \"" + metaFetchType + "\" of the property \"" + KEY_METADATA + "\"."); + }catch(InvocationTargetException ite){ + if (ite.getCause() != null){ + if (ite.getCause() instanceof TAPException) + throw (TAPException)ite.getCause(); + else + throw new TAPException(ite.getCause()); + }else + throw new TAPException(ite); + }catch(Exception ex){ + throw new TAPException("Impossible to create an instance of tap.metadata.TAPMetadata as specified in the property \"" + KEY_METADATA + "\": \"" + metaFetchType + "\"!", ex); + } + + /* 2. Update the database */ + DBConnection conn = null; + try{ + conn = tapFactory.getConnection("SET_TAP_SCHEMA"); + conn.setTAPSchema(metadata); + }finally{ + if (conn != null) + tapFactory.freeConnection(conn); + } + } + // INCORRECT VALUE => ERROR! + else + throw new TAPException("Unsupported value for the property \"" + KEY_METADATA + "\": \"" + metaFetchType + "\"! Only two values are allowed: " + VALUE_XML + " (to get metadata from a TableSet XML document) or " + VALUE_DB + " (to fetch metadata from the database schema TAP_SCHEMA)."); + + return metadata; + } + + /** + * Initialize the maximum number of asynchronous jobs. + * + * @param tapConfig The content of the TAP configuration file. + * + * @throws TAPException If the corresponding TAP configuration property is wrong. + */ + private void initMaxAsyncJobs(final Properties tapConfig) throws TAPException{ + // Get the property value: + String propValue = getProperty(tapConfig, KEY_MAX_ASYNC_JOBS); + try{ + // If a value is provided, cast it into an integer and set the attribute: + maxAsyncJobs = (propValue == null) ? DEFAULT_MAX_ASYNC_JOBS : Integer.parseInt(propValue); + }catch(NumberFormatException nfe){ + throw new TAPException("Integer expected for the property \"" + KEY_MAX_ASYNC_JOBS + "\", instead of: \"" + propValue + "\"!"); + } + } + + /** + * Initialize the default and maximum retention period. + * + * @param tapConfig The content of the TAP configuration file. + * + * @throws TAPException If the corresponding TAP configuration properties are wrong. + */ + private void initRetentionPeriod(final Properties tapConfig) throws TAPException{ + retentionPeriod = new int[2]; + + // Set the default period: + String propValue = getProperty(tapConfig, KEY_DEFAULT_RETENTION_PERIOD); + try{ + retentionPeriod[0] = (propValue == null) ? DEFAULT_RETENTION_PERIOD : Integer.parseInt(propValue); + }catch(NumberFormatException nfe){ + throw new TAPException("Integer expected for the property \"" + KEY_DEFAULT_RETENTION_PERIOD + "\", instead of: \"" + propValue + "\"!"); + } + + // Set the maximum period: + propValue = getProperty(tapConfig, KEY_MAX_RETENTION_PERIOD); + try{ + retentionPeriod[1] = (propValue == null) ? DEFAULT_RETENTION_PERIOD : Integer.parseInt(propValue); + }catch(NumberFormatException nfe){ + throw new TAPException("Integer expected for the property \"" + KEY_MAX_RETENTION_PERIOD + "\", instead of: \"" + propValue + "\"!"); + } + + // The maximum period MUST be greater or equals than the default period. + // If not, the default period is set (so decreased) to the maximum period. + if (retentionPeriod[1] > 0 && retentionPeriod[1] < retentionPeriod[0]) + retentionPeriod[0] = retentionPeriod[1]; + } + + /** + * Initialize the default and maximum execution duration. + * + * @param tapConfig The content of the TAP configuration file. + * + * @throws TAPException If the corresponding TAP configuration properties are wrong. + */ + private void initExecutionDuration(final Properties tapConfig) throws TAPException{ + executionDuration = new int[2]; + + // Set the default duration: + String propValue = getProperty(tapConfig, KEY_DEFAULT_EXECUTION_DURATION); + try{ + executionDuration[0] = (propValue == null) ? DEFAULT_EXECUTION_DURATION : Integer.parseInt(propValue); + }catch(NumberFormatException nfe){ + throw new TAPException("Integer expected for the property \"" + KEY_DEFAULT_EXECUTION_DURATION + "\", instead of: \"" + propValue + "\"!"); + } + + // Set the maximum duration: + propValue = getProperty(tapConfig, KEY_MAX_EXECUTION_DURATION); + try{ + executionDuration[1] = (propValue == null) ? DEFAULT_EXECUTION_DURATION : Integer.parseInt(propValue); + }catch(NumberFormatException nfe){ + throw new TAPException("Integer expected for the property \"" + KEY_MAX_EXECUTION_DURATION + "\", instead of: \"" + propValue + "\"!"); + } + + // The maximum duration MUST be greater or equals than the default duration. + // If not, the default duration is set (so decreased) to the maximum duration. + if (executionDuration[1] > 0 && executionDuration[1] < executionDuration[0]) + executionDuration[0] = executionDuration[1]; + } + + /** + *Initialize the list of all output format that the TAP service must support.
+ * + *+ * This function ensures that at least one VOTable format is part of the returned list, + * even if none has been specified in the TAP configuration file. Indeed, the VOTable format is the only + * format required for a TAP service. + *
+ * + * @param tapConfig The content of the TAP configuration file. + * + * @throws TAPException If the corresponding TAP configuration properties are wrong. + */ + private void addOutputFormats(final Properties tapConfig) throws TAPException{ + // Fetch the value of the property for additional output formats: + String formats = getProperty(tapConfig, KEY_OUTPUT_FORMATS); + + // SPECIAL VALUE "ALL": + if (formats == null || formats.equalsIgnoreCase(VALUE_ALL)){ + outputFormats.add(new VOTableFormat(this, DataFormat.BINARY)); + outputFormats.add(new VOTableFormat(this, DataFormat.BINARY2)); + outputFormats.add(new VOTableFormat(this, DataFormat.TABLEDATA)); + outputFormats.add(new VOTableFormat(this, DataFormat.FITS)); + outputFormats.add(new FITSFormat(this)); + outputFormats.add(new JSONFormat(this)); + outputFormats.add(new SVFormat(this, ",", true)); + outputFormats.add(new SVFormat(this, "\t", true)); + outputFormats.add(new TextFormat(this)); + outputFormats.add(new HTMLFormat(this)); + return; + } + + // LIST OF FORMATS: + // Since it is a comma separated list of output formats, a loop will parse this list comma by comma: + String f; + int indexSep, indexLPar, indexRPar; + boolean hasVotableFormat = false; + while(formats != null && formats.length() > 0){ + // Get a format item from the list: + indexSep = formats.indexOf(','); + // if a comma is after a left parenthesis + indexLPar = formats.indexOf('('); + if (indexSep > 0 && indexLPar > 0 && indexSep > indexLPar){ + indexRPar = formats.indexOf(')', indexLPar); + if (indexRPar > 0) + indexSep = formats.indexOf(',', indexRPar); + else + throw new TAPException("Missing right parenthesis in: \"" + formats + "\"!"); + } + // no comma => only one format + if (indexSep < 0){ + f = formats; + formats = null; + } + // comma at the first position => empty list item => go to the next item + else if (indexSep == 0){ + formats = formats.substring(1).trim(); + continue; + } + // else => get the first format item, and then remove it from the list for the next iteration + else{ + f = formats.substring(0, indexSep).trim(); + formats = formats.substring(indexSep + 1).trim(); + } + + // Identify the format and append it to the output format list of the service: + // FITS + if (f.equalsIgnoreCase(VALUE_FITS)) + outputFormats.add(new FITSFormat(this)); + // JSON + else if (f.equalsIgnoreCase(VALUE_JSON)) + outputFormats.add(new JSONFormat(this)); + // HTML + else if (f.equalsIgnoreCase(VALUE_HTML)) + outputFormats.add(new HTMLFormat(this)); + // TEXT + else if (f.equalsIgnoreCase(VALUE_TEXT)) + outputFormats.add(new TextFormat(this)); + // CSV + else if (f.equalsIgnoreCase(VALUE_CSV)) + outputFormats.add(new SVFormat(this, ",", true)); + // TSV + else if (f.equalsIgnoreCase(VALUE_TSV)) + outputFormats.add(new SVFormat(this, "\t", true)); + // any SV (separated value) format + else if (f.toLowerCase().startsWith(VALUE_SV)){ + // get the separator: + int endSep = f.indexOf(')'); + if (VALUE_SV.length() < f.length() && f.charAt(VALUE_SV.length()) == '(' && endSep > VALUE_SV.length() + 1){ + String separator = f.substring(VALUE_SV.length() + 1, f.length() - 1); + // get the MIME type and its alias, if any of them is provided: + String mimeType = null, shortMimeType = null; + if (endSep + 1 < f.length() && f.charAt(endSep + 1) == ':'){ + int endMime = f.indexOf(':', endSep + 2); + if (endMime < 0) + mimeType = f.substring(endSep + 2, f.length()); + else if (endMime > 0){ + mimeType = f.substring(endSep + 2, endMime); + shortMimeType = f.substring(endMime + 1); + } + } + // add the defined SV(...) format: + outputFormats.add(new SVFormat(this, separator, true, mimeType, shortMimeType)); + }else + throw new TAPException("Missing separator char/string for the SV output format: \"" + f + "\"!"); + } + // VOTABLE + else if (f.toLowerCase().startsWith(VALUE_VOTABLE) || f.toLowerCase().startsWith(VALUE_VOT)){ + // Parse the format: + VOTableFormat votFormat = parseVOTableFormat(f); + + // Add the VOTable format: + outputFormats.add(votFormat); + + // Determine whether the MIME type is the VOTable expected one: + if (votFormat.getShortMimeType().equals("votable") || votFormat.getMimeType().equals("votable")) + hasVotableFormat = true; + } + // custom OutputFormat + else if (isClassName(f)) + outputFormats.add(TAPConfiguration.newInstance(f, KEY_OUTPUT_FORMATS, OutputFormat.class, new Class[]{ServiceConnection.class}, new Object[]{this})); + // unknown format + else + throw new TAPException("Unknown output format: " + f); + } + + // Add by default VOTable format if none is specified: + if (!hasVotableFormat) + outputFormats.add(new VOTableFormat(this)); + } + + /** + *Parse the given VOTable format specification.
+ * + *This specification is expected to be an item of the property {@link TAPConfiguration#KEY_OUTPUT_FORMATS}.
+ * + * @param propValue A single VOTable format specification. + * + * @return The corresponding configured {@link VOTableFormat} instance. + * + * @throws TAPException If the syntax of the given specification is incorrect, + * or if the specified VOTable version or serialization does not exist. + */ + private VOTableFormat parseVOTableFormat(final String propValue) throws TAPException{ + DataFormat serialization = null; + VOTableVersion votVersion = null; + String mimeType = null, shortMimeType = null; + + // Get the parameters, if any: + int beginSep = propValue.indexOf('('); + if (beginSep > 0){ + int endSep = propValue.indexOf(')'); + if (endSep <= beginSep) + throw new TAPException("Wrong output format specification syntax in: \"" + propValue + "\"! A VOTable parameters list must end with ')'."); + // split the parameters: + String[] params = propValue.substring(beginSep + 1, endSep).split(","); + if (params.length > 2) + throw new TAPException("Wrong number of parameters for the output format VOTable: \"" + propValue + "\"! Only two parameters may be provided: serialization and version."); + else if (params.length >= 1){ + // resolve the serialization format: + params[0] = params[0].trim().toLowerCase(); + if (params[0].length() == 0 || params[0].equals("b") || params[0].equals("binary")) + serialization = DataFormat.BINARY; + else if (params[0].equals("b2") || params[0].equals("binary2")) + serialization = DataFormat.BINARY2; + else if (params[0].equals("td") || params[0].equals("tabledata")) + serialization = DataFormat.TABLEDATA; + else if (params[0].equals("fits")) + serialization = DataFormat.FITS; + else + throw new TAPException("Unsupported VOTable serialization: \"" + params[0] + "\"! Accepted values: 'binary' (or 'b'), 'binary2' (or 'b2'), 'tabledata' (or 'td') and 'fits'."); + // resolve the version: + if (params.length == 2){ + params[1] = params[1].trim(); + if (params[1].equals("1.0") || params[1].equalsIgnoreCase("v1.0")) + votVersion = VOTableVersion.V10; + else if (params[1].equals("1.1") || params[1].equalsIgnoreCase("v1.1")) + votVersion = VOTableVersion.V11; + else if (params[1].equals("1.2") || params[1].equalsIgnoreCase("v1.2")) + votVersion = VOTableVersion.V12; + else if (params[1].equals("1.3") || params[1].equalsIgnoreCase("v1.3")) + votVersion = VOTableVersion.V13; + else + throw new TAPException("Unsupported VOTable version: \"" + params[1] + "\"! Accepted values: '1.0' (or 'v1.0'), '1.1' (or 'v1.1'), '1.2' (or 'v1.2') and '1.3' (or 'v1.3')."); + } + } + } + + // Get the MIME type and its alias, if any: + beginSep = propValue.indexOf(':'); + if (beginSep > 0){ + int endSep = propValue.indexOf(':', beginSep + 1); + if (endSep < 0) + endSep = propValue.length(); + // extract the MIME type, if any: + mimeType = propValue.substring(beginSep + 1, endSep).trim(); + if (mimeType.length() == 0) + mimeType = null; + // extract the short MIME type, if any: + if (endSep < propValue.length()){ + beginSep = endSep; + endSep = propValue.indexOf(':', beginSep + 1); + if (endSep >= 0) + throw new TAPException("Wrong output format specification syntax in: \"" + propValue + "\"! After a MIME type and a short MIME type, no more information is expected."); + else + endSep = propValue.length(); + shortMimeType = propValue.substring(beginSep + 1, endSep).trim(); + if (shortMimeType.length() == 0) + shortMimeType = null; + } + } + + // Create the VOTable format: + VOTableFormat votFormat = new VOTableFormat(this, serialization, votVersion); + votFormat.setMimeType(mimeType, shortMimeType); + + return votFormat; + } + + /** + * Initialize the default and maximum output limits. + * + * @param tapConfig The content of the TAP configuration file. + * + * @throws TAPException If the corresponding TAP configuration properties are wrong. + */ + private void initOutputLimits(final Properties tapConfig) throws TAPException{ + Object[] limit = parseLimit(getProperty(tapConfig, KEY_DEFAULT_OUTPUT_LIMIT), KEY_DEFAULT_OUTPUT_LIMIT, false); + outputLimitTypes[0] = (LimitUnit)limit[1]; // it should be "rows" since the parameter areBytesAllowed of parseLimit =false + setDefaultOutputLimit((Integer)limit[0]); + + limit = parseLimit(getProperty(tapConfig, KEY_MAX_OUTPUT_LIMIT), KEY_DEFAULT_OUTPUT_LIMIT, false); + outputLimitTypes[1] = (LimitUnit)limit[1]; // it should be "rows" since the parameter areBytesAllowed of parseLimit =false + setMaxOutputLimit((Integer)limit[0]); + } + + /** + * Initialize the fetch size for the synchronous and for the asynchronous resources. + * + * @param tapConfig The content of the TAP configuration file. + * + * @throws TAPException If the corresponding TAP configuration properties are wrong. + */ + private void initFetchSize(final Properties tapConfig) throws TAPException{ + fetchSize = new int[2]; + + // Set the fetch size for asynchronous queries: + String propVal = getProperty(tapConfig, KEY_ASYNC_FETCH_SIZE); + if (propVal == null) + fetchSize[0] = DEFAULT_ASYNC_FETCH_SIZE; + else{ + try{ + fetchSize[0] = Integer.parseInt(propVal); + if (fetchSize[0] < 0) + fetchSize[0] = 0; + }catch(NumberFormatException nfe){ + throw new TAPException("Integer expected for the property " + KEY_ASYNC_FETCH_SIZE + ": \"" + propVal + "\"!"); + } + } + + // Set the fetch size for synchronous queries: + propVal = getProperty(tapConfig, KEY_SYNC_FETCH_SIZE); + if (propVal == null) + fetchSize[1] = DEFAULT_SYNC_FETCH_SIZE; + else{ + try{ + fetchSize[1] = Integer.parseInt(propVal); + if (fetchSize[1] < 0) + fetchSize[1] = 0; + }catch(NumberFormatException nfe){ + throw new TAPException("Integer expected for the property " + KEY_SYNC_FETCH_SIZE + ": \"" + propVal + "\"!"); + } + } + } + + /** + * Initialize the default and maximum upload limits. + * + * @param tapConfig The content of the TAP configuration file. + * + * @throws TAPException If the corresponding TAP configuration properties are wrong. + */ + private void initUploadLimits(final Properties tapConfig) throws TAPException{ + Object[] limit = parseLimit(getProperty(tapConfig, KEY_DEFAULT_UPLOAD_LIMIT), KEY_DEFAULT_UPLOAD_LIMIT, true); + uploadLimitTypes[0] = (LimitUnit)limit[1]; + setDefaultUploadLimit((Integer)limit[0]); + + limit = parseLimit(getProperty(tapConfig, KEY_MAX_UPLOAD_LIMIT), KEY_MAX_UPLOAD_LIMIT, true); + if (!((LimitUnit)limit[1]).isCompatibleWith(uploadLimitTypes[0])) + throw new TAPException("The default upload limit (in " + uploadLimitTypes[0] + ") and the maximum upload limit (in " + limit[1] + ") MUST be expressed in the same unit!"); + else + uploadLimitTypes[1] = (LimitUnit)limit[1]; + setMaxUploadLimit((Integer)limit[0]); + } + + /** + * Initialize the maximum size (in bytes) of a VOTable files set upload. + * + * @param tapConfig The content of the TAP configuration file. + * + * @throws TAPException If the corresponding TAP configuration property is wrong. + */ + private void initMaxUploadSize(final Properties tapConfig) throws TAPException{ + String propValue = getProperty(tapConfig, KEY_UPLOAD_MAX_FILE_SIZE); + // If a value is specified... + if (propValue != null){ + // ...parse the value: + Object[] limit = parseLimit(propValue, KEY_UPLOAD_MAX_FILE_SIZE, true); + if (((Integer)limit[0]).intValue() <= 0) + limit[0] = new Integer(TAPConfiguration.DEFAULT_UPLOAD_MAX_FILE_SIZE); + // ...check that the unit is correct (bytes): + if (!LimitUnit.bytes.isCompatibleWith((LimitUnit)limit[1])) + throw new TAPException("The maximum upload file size " + KEY_UPLOAD_MAX_FILE_SIZE + " (here: " + propValue + ") can not be expressed in a unit different from bytes (B, kB, MB, GB)!"); + // ...set the max file size: + int value = (int)((Integer)limit[0] * ((LimitUnit)limit[1]).bytesFactor()); + setMaxUploadSize(value); + } + } + + /** + * Initialize the TAP user identification method. + * + * @param tapConfig The content of the TAP configuration file. + * + * @throws TAPException If the corresponding TAP configuration property is wrong. + */ + private void initUserIdentifier(final Properties tapConfig) throws TAPException{ + // Get the property value: + String propValue = getProperty(tapConfig, KEY_USER_IDENTIFIER); + if (propValue != null) + userIdentifier = newInstance(propValue, KEY_USER_IDENTIFIER, UserIdentifier.class); + } + + /** + * Initialize the list of all allowed coordinate systems. + * + * @param tapConfig The content of the TAP configuration file. + * + * @throws TAPException If the corresponding TAP configuration properties are wrong. + */ + private void initCoordSys(final Properties tapConfig) throws TAPException{ + // Get the property value: + String propValue = getProperty(tapConfig, KEY_COORD_SYS); + + // NO VALUE => ALL COORD SYS ALLOWED! + if (propValue == null) + lstCoordSys = null; + + // "NONE" => ALL COORD SYS FORBIDDEN (= no coordinate system expression is allowed)! + else if (propValue.equalsIgnoreCase(VALUE_NONE)) + lstCoordSys = new ArrayListSet the default retention period.
+ * + *This period is set by default if the user did not specify one before the execution of his query.
+ * + *Important note: + * This function will apply the given retention period only if legal compared to the currently set maximum value. + * In other words, if the given value is less or equals to the current maximum retention period. + *
+ * + * @param period New default retention period (in seconds). + * + * @return true if the given retention period has been successfully set, false otherwise. + */ + public boolean setDefaultRetentionPeriod(final int period){ + if ((retentionPeriod[1] <= 0) || (period > 0 && period <= retentionPeriod[1])){ + retentionPeriod[0] = period; + return true; + }else + return false; + } + + /** + *Set the maximum retention period.
+ * + *This period limits the default retention period and the retention period specified by a user.
+ * + *Important note: + * This function may reduce the default retention period if the current default retention period is bigger + * to the new maximum retention period. In a such case, the default retention period is set to the + * new maximum retention period. + *
+ * + * @param period New maximum retention period (in seconds). + */ + public void setMaxRetentionPeriod(final int period){ + // Decrease the default retention period if it will be bigger than the new maximum retention period: + if (period > 0 && (retentionPeriod[0] <= 0 || period < retentionPeriod[0])) + retentionPeriod[0] = period; + // Set the new maximum retention period: + retentionPeriod[1] = period; + } + + @Override + public int[] getExecutionDuration(){ + return executionDuration; + } + + /** + *Set the default execution duration.
+ * + *This duration is set by default if the user did not specify one before the execution of his query.
+ * + *Important note: + * This function will apply the given execution duration only if legal compared to the currently set maximum value. + * In other words, if the given value is less or equals to the current maximum execution duration. + *
+ * + * @param duration New default execution duration (in milliseconds). + * + * @return true if the given execution duration has been successfully set, false otherwise. + */ + public boolean setDefaultExecutionDuration(final int duration){ + if ((executionDuration[1] <= 0) || (duration > 0 && duration <= executionDuration[1])){ + executionDuration[0] = duration; + return true; + }else + return false; + } + + /** + *Set the maximum execution duration.
+ * + *This duration limits the default execution duration and the execution duration specified by a user.
+ * + *Important note: + * This function may reduce the default execution duration if the current default execution duration is bigger + * to the new maximum execution duration. In a such case, the default execution duration is set to the + * new maximum execution duration. + *
+ * + * @param duration New maximum execution duration (in milliseconds). + */ + public void setMaxExecutionDuration(final int duration){ + // Decrease the default execution duration if it will be bigger than the new maximum execution duration: + if (duration > 0 && (executionDuration[0] <= 0 || duration < executionDuration[0])) + executionDuration[0] = duration; + // Set the new maximum execution duration: + executionDuration[1] = duration; + } + + @Override + public IteratorAdd the given {@link OutputFormat} in the list of output formats supported by the TAP service.
+ * + *Warning: + * No verification is done in order to avoid duplicated output formats in the list. + * NULL objects are merely ignored silently. + *
+ * + * @param newOutputFormat New output format. + */ + public void addOutputFormat(final OutputFormat newOutputFormat){ + if (newOutputFormat != null) + outputFormats.add(newOutputFormat); + } + + /** + * Remove the specified output format. + * + * @param mimeOrAlias Full or short MIME type of the output format to remove. + * + * @return true if the specified format has been found and successfully removed from the list, + * false otherwise. + */ + public boolean removeOutputFormat(final String mimeOrAlias){ + OutputFormat of = getOutputFormat(mimeOrAlias); + if (of != null) + return outputFormats.remove(of); + else + return false; + } + + @Override + public int[] getOutputLimit(){ + return outputLimits; + } + + /** + *Set the default output limit.
+ * + *This limit is set by default if the user did not specify one before the execution of his query.
+ * + *Important note: + * This function will apply the given output limit only if legal compared to the currently set maximum value. + * In other words, if the given value is less or equals to the current maximum output limit. + *
+ * + * @param limit New default output limit (in number of rows). + * + * @return true if the given output limit has been successfully set, false otherwise. + */ + public boolean setDefaultOutputLimit(final int limit){ + if ((outputLimits[1] <= 0) || (limit > 0 && limit <= outputLimits[1])){ + outputLimits[0] = limit; + return true; + }else + return false; + } + + /** + *Set the maximum output limit.
+ * + *This output limit limits the default output limit and the output limit specified by a user.
+ * + *Important note: + * This function may reduce the default output limit if the current default output limit is bigger + * to the new maximum output limit. In a such case, the default output limit is set to the + * new maximum output limit. + *
+ * + * @param limit New maximum output limit (in number of rows). + */ + public void setMaxOutputLimit(final int limit){ + // Decrease the default output limit if it will be bigger than the new maximum output limit: + if (limit > 0 && (outputLimits[0] <= 0 || limit < outputLimits[0])) + outputLimits[0] = limit; + // Set the new maximum output limit: + outputLimits[1] = limit; + } + + @Override + public final LimitUnit[] getOutputLimitType(){ + return new LimitUnit[]{LimitUnit.rows,LimitUnit.rows}; + } + + @Override + public CollectionSet the default upload limit.
+ * + *Important note: + * This function will apply the given upload limit only if legal compared to the currently set maximum value. + * In other words, if the given value is less or equals to the current maximum upload limit. + *
+ * + * @param limit New default upload limit. + * + * @return true if the given upload limit has been successfully set, false otherwise. + */ + public boolean setDefaultUploadLimit(final int limit){ + try{ + if ((uploadLimits[1] <= 0) || (limit > 0 && LimitUnit.compare(limit, uploadLimitTypes[0], uploadLimits[1], uploadLimitTypes[1]) <= 0)){ + uploadLimits[0] = limit; + return true; + } + }catch(TAPException e){} + return false; + } + + /** + *Set the maximum upload limit.
+ * + *This upload limit limits the default upload limit.
+ * + *Important note: + * This function may reduce the default upload limit if the current default upload limit is bigger + * to the new maximum upload limit. In a such case, the default upload limit is set to the + * new maximum upload limit. + *
+ * + * @param limit New maximum upload limit. + */ + public void setMaxUploadLimit(final int limit){ + try{ + // Decrease the default output limit if it will be bigger than the new maximum output limit: + if (limit > 0 && (uploadLimits[0] <= 0 || LimitUnit.compare(limit, uploadLimitTypes[1], uploadLimits[0], uploadLimitTypes[0]) < 0)) + uploadLimits[0] = limit; + // Set the new maximum output limit: + uploadLimits[1] = limit; + }catch(TAPException e){} + } + + @Override + public int getMaxUploadSize(){ + return maxUploadSize; + } + + /** + *Set the maximum size of a VOTable files set that can be uploaded in once.
+ * + *Warning: + * This size can not be negative or 0. If the given value is in this case, nothing will be done + * and false will be returned. + * On the contrary to the other limits, no "unlimited" limit is possible here ; only the + * maximum value can be set (i.e. maximum positive integer value). + *
+ * + * @param maxSize New maximum size (in bytes). + * + * @return true if the size has been successfully set, false otherwise. + */ + public boolean setMaxUploadSize(final int maxSize){ + // No "unlimited" value possible there: + if (maxSize <= 0) + return false; + + // Otherwise, set the maximum upload file size: + maxUploadSize = maxSize; + return true; + } + + @Override + public int getNbMaxAsyncJobs(){ + return maxAsyncJobs; + } + + @Override + public UserIdentifier getUserIdentifier(){ + return userIdentifier; + } + + @Override + public TAPMetadata getTAPMetadata(){ + return metadata; + } + + @Override + public CollectionConcrete implementation of a {@link TAPFactory} which is parameterized by a TAP configuration file.
+ * + *+ * All abstract or NULL-implemented methods/functions left by {@link AbstractTAPFactory} are implemented using values + * of a TAP configuration file. The concerned methods are: {@link #getConnection(String)}, {@link #freeConnection(DBConnection)}, + * {@link #destroy()}, {@link #createADQLTranslator()} and {@link #createUWSBackupManager(UWSService)}. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 2.0 (04/2015) + * @since 2.0 + */ +public final class ConfigurableTAPFactory extends AbstractTAPFactory { + + /* ADQL to SQL translation: */ + /** The {@link JDBCTranslator} to use when a ADQL query must be executed in the database. + * This translator is also used to convert ADQL types into database types. */ + private Class translator; + + /* JNDI DB access: */ + /** The {@link DataSource} to use in order to access the database. + * This attribute is actually used only if the chosen database access method is JNDI. */ + private final DataSource datasource; + + /* Simple JDBC access: */ + /** Classpath of the JDBC driver to use in order to access the database. + * This attribute is actually used only if the chosen database access method is JDBC. */ + private final String driverPath; + /** JDBC URL of the database to access. + * This attribute is actually used only if the chosen database access method is JDBC. */ + private final String dbUrl; + /** Name of the database user to use in order to access the database. + * This attribute is actually used only if the chosen database access method is JDBC. */ + private final String dbUser; + /** Password of the database user to use in order to access the database. + * This attribute is actually used only if the chosen database access method is JDBC. */ + private final String dbPassword; + + /* UWS's jobs backup: */ + /** Indicate whether the jobs must be backuped gathered by user or just all mixed together. */ + private boolean backupByUser; + /** Frequency at which the jobs must be backuped. */ + private long backupFrequency; + + /** + * Build a {@link TAPFactory} using the given TAP service description and TAP configuration file. + * + * @param service The TAP service description. + * @param tapConfig The TAP configuration file containing particularly information about the database access. + * + * @throws NullPointerException If one of the parameter is NULL. + * @throws TAPException If some properties of the TAP configuration file are wrong. + */ + public ConfigurableTAPFactory(ServiceConnection service, final Properties tapConfig) throws NullPointerException, TAPException{ + super(service); + + if (tapConfig == null) + throw new NullPointerException("Missing TAP properties! "); + + /* 1. Configure the database access */ + final String dbAccessMethod = getProperty(tapConfig, KEY_DATABASE_ACCESS); + + // Case a: Missing access method => error! + if (dbAccessMethod == null) + throw new TAPException("The property \"" + KEY_DATABASE_ACCESS + "\" is missing! It is required to connect to the database. Two possible values: \"" + VALUE_JDBC + "\" and \"" + VALUE_JNDI + "\"."); + + // Case b: JDBC ACCESS + else if (dbAccessMethod.equalsIgnoreCase(VALUE_JDBC)){ + // Extract the DB type and deduce the JDBC Driver path: + String jdbcDriver = getProperty(tapConfig, KEY_JDBC_DRIVER); + String dbUrl = getProperty(tapConfig, KEY_JDBC_URL); + if (jdbcDriver == null){ + if (dbUrl == null) + throw new TAPException("The property \"" + KEY_JDBC_URL + "\" is missing! Since the choosen database access method is \"" + VALUE_JDBC + "\", this property is required."); + else if (!dbUrl.startsWith(JDBCConnection.JDBC_PREFIX + ":")) + throw new TAPException("JDBC URL format incorrect! It MUST begins with " + JDBCConnection.JDBC_PREFIX + ":"); + else{ + String dbType = dbUrl.substring(JDBCConnection.JDBC_PREFIX.length() + 1); + if (dbType.indexOf(':') <= 0) + throw new TAPException("JDBC URL format incorrect! Database type name is missing."); + dbType = dbType.substring(0, dbType.indexOf(':')); + + jdbcDriver = VALUE_JDBC_DRIVERS.get(dbType); + if (jdbcDriver == null) + throw new TAPException("No JDBC driver known for the DBMS \"" + dbType + "\"!"); + } + } + // Set the DB connection parameters: + this.driverPath = jdbcDriver; + this.dbUrl = dbUrl; + this.dbUser = getProperty(tapConfig, KEY_DB_USERNAME); + this.dbPassword = getProperty(tapConfig, KEY_DB_PASSWORD); + // Set the other DB connection parameters: + this.datasource = null; + } + // Case c: JNDI ACCESS + else if (dbAccessMethod.equalsIgnoreCase(VALUE_JNDI)){ + // Get the datasource JDNI name: + String dsName = getProperty(tapConfig, KEY_DATASOURCE_JNDI_NAME); + if (dsName == null) + throw new TAPException("The property \"" + KEY_DATASOURCE_JNDI_NAME + "\" is missing! Since the choosen database access method is \"" + VALUE_JNDI + "\", this property is required."); + try{ + // Load the JNDI context: + InitialContext cxt = new InitialContext(); + // Look for the specified datasource: + datasource = (DataSource)cxt.lookup(dsName); + if (datasource == null) + throw new TAPException("No datasource found with the JNDI name \"" + dsName + "\"!"); + // Set the other DB connection parameters: + this.driverPath = null; + this.dbUrl = null; + this.dbUser = null; + this.dbPassword = null; + }catch(NamingException ne){ + throw new TAPException("No datasource found with the JNDI name \"" + dsName + "\"!"); + } + } + // Case d: unsupported value + else + throw new TAPException("Unsupported value for the property " + KEY_DATABASE_ACCESS + ": \"" + dbAccessMethod + "\"! Allowed values: \"" + VALUE_JNDI + "\" or \"" + VALUE_JDBC + "\"."); + + /* 2. Set the ADQLTranslator to use in function of the sql_translator property */ + String sqlTranslator = getProperty(tapConfig, KEY_SQL_TRANSLATOR); + // case a: no translator specified + if (sqlTranslator == null) + throw new TAPException("The property \"" + KEY_SQL_TRANSLATOR + "\" is missing! ADQL queries can not be translated without it. Allowed values: \"" + VALUE_POSTGRESQL + "\", \"" + VALUE_PGSPHERE + "\" or a class path of a class implementing SQLTranslator."); + + // case b: PostgreSQL translator + else if (sqlTranslator.equalsIgnoreCase(VALUE_POSTGRESQL)) + translator = PostgreSQLTranslator.class; + + // case c: PgSphere translator + else if (sqlTranslator.equalsIgnoreCase(VALUE_PGSPHERE)) + translator = PgSphereTranslator.class; + + // case d: a client defined ADQLTranslator (with the provided class name) + else if (TAPConfiguration.isClassName(sqlTranslator)) + translator = TAPConfiguration.fetchClass(sqlTranslator, KEY_SQL_TRANSLATOR, JDBCTranslator.class); + + // case e: unsupported value + else + throw new TAPException("Unsupported value for the property " + KEY_SQL_TRANSLATOR + ": \"" + sqlTranslator + "\" !"); + + /* 3. Test the construction of the ADQLTranslator */ + createADQLTranslator(); + + /* 4. Test the DB connection (note: a translator is needed to create a connection) */ + DBConnection dbConn = getConnection("0"); + freeConnection(dbConn); + + /* 5. Set the UWS Backup Parameter */ + // Set the backup frequency: + String propValue = getProperty(tapConfig, KEY_BACKUP_FREQUENCY); + // determine whether the value is a time period ; if yes, set the frequency: + if (propValue != null){ + try{ + backupFrequency = Long.parseLong(propValue); + if (backupFrequency <= 0) + backupFrequency = DEFAULT_BACKUP_FREQUENCY; + }catch(NumberFormatException nfe){ + // if the value was not a valid numeric time period, try to identify the different textual options: + if (propValue.equalsIgnoreCase(VALUE_NEVER)) + backupFrequency = DefaultTAPBackupManager.MANUAL; + else if (propValue.equalsIgnoreCase(VALUE_USER_ACTION)) + backupFrequency = DefaultTAPBackupManager.AT_USER_ACTION; + else + throw new TAPException("Long expected for the property \"" + KEY_BACKUP_FREQUENCY + "\", instead of: \"" + propValue + "\"!"); + } + }else + backupFrequency = DEFAULT_BACKUP_FREQUENCY; + // Specify whether the backup must be organized by user or not: + propValue = getProperty(tapConfig, KEY_BACKUP_BY_USER); + backupByUser = (propValue == null) ? DEFAULT_BACKUP_BY_USER : Boolean.parseBoolean(propValue); + } + + /** + * Build a {@link JDBCTranslator} instance with the given class ({@link #translator} ; + * specified by the property sql_translator). If the instance can not be build, + * whatever is the reason, a TAPException MUST be thrown. + * + * Note: This function is called at the initialization of {@link ConfigurableTAPFactory} + * in order to check that a translator can be created. + */ + protected JDBCTranslator createADQLTranslator() throws TAPException{ + try{ + return translator.getConstructor().newInstance(); + }catch(Exception ex){ + if (ex instanceof TAPException) + throw (TAPException)ex; + else + throw new TAPException("Impossible to create a JDBCTranslator instance with the empty constructor of \"" + translator.getName() + "\" (see the property " + KEY_SQL_TRANSLATOR + ") for the following reason: " + ex.getMessage()); + } + } + + /** + * Build a {@link JDBCConnection} thanks to the database parameters specified + * in the TAP configuration file (the properties: jdbc_driver_path, db_url, db_user, db_password). + * + * @see JDBCConnection#JDBCConnection(java.sql.Connection, JDBCTranslator, String, tap.log.TAPLog) + * @see JDBCConnection#JDBCConnection(String, String, String, String, JDBCTranslator, String, tap.log.TAPLog) + */ + @Override + public DBConnection getConnection(String jobID) throws TAPException{ + if (datasource != null){ + try{ + return new JDBCConnection(datasource.getConnection(), createADQLTranslator(), jobID, this.service.getLogger()); + }catch(SQLException se){ + throw new TAPException("Impossible to establish a connection to the database using the set up datasource!", se); + } + }else + return new JDBCConnection(driverPath, dbUrl, dbUser, dbPassword, createADQLTranslator(), jobID, this.service.getLogger()); + } + + @Override + public void freeConnection(DBConnection conn){ + try{ + ((JDBCConnection)conn).getInnerConnection().close(); + }catch(SQLException se){ + service.getLogger().error("Can not close properly the connection \"" + conn.getID() + "\"!", se); + } + } + + @Override + public void destroy(){ + // Unregister the JDBC driver, only if registered by the library (i.e. database_access=jdbc): + if (dbUrl != null){ + // Now deregister JDBC drivers in this context's ClassLoader: + // Get the webapp's ClassLoader + ClassLoader cl = Thread.currentThread().getContextClassLoader(); + // Loop through all drivers + EnumerationHTTP servlet fully configured with a TAP configuration file.
+ * + *+ * This configuration file may be specified in the initial parameter named {@link TAPConfiguration#TAP_CONF_PARAMETER} + * of this servlet inside the WEB-INF/web.xml file. If none is specified, the file {@link TAPConfiguration#DEFAULT_TAP_CONF_FILE} + * will be searched inside the directories of the classpath, and inside WEB-INF and META-INF. + *
+ * + * @author Grégory Mantelet (ARI) + * @version 2.0 (04/2015) + * @since 2.0 + */ +public class ConfigurableTAPServlet extends HttpServlet { + private static final long serialVersionUID = 1L; + + /** TAP object representing the TAP service. */ + private TAP tap = null; + + @Override + public void init(final ServletConfig config) throws ServletException{ + // Nothing to do, if TAP is already initialized: + if (tap != null) + return; + + /* 1. GET THE FILE PATH OF THE TAP CONFIGURATION FILE */ + String tapConfPath = config.getInitParameter(TAP_CONF_PARAMETER); + if (tapConfPath == null || tapConfPath.trim().length() == 0) + tapConfPath = null; + //throw new ServletException("Configuration file path missing! You must set a servlet init parameter whose the name is \"" + TAP_CONF_PARAMETER + "\"."); + + /* 2. OPEN THE CONFIGURATION FILE */ + InputStream input = null; + // CASE: No file specified => search in the classpath for a file having the default name "tap.properties". + if (tapConfPath == null) + input = searchFile(DEFAULT_TAP_CONF_FILE, config); + else{ + File f = new File(tapConfPath); + // CASE: The given path matches to an existing local file. + if (f.exists()){ + try{ + input = new FileInputStream(f); + }catch(IOException ioe){ + throw new ServletException("Impossible to read the TAP configuration file (" + tapConfPath + ")!", ioe); + } + } + // CASE: The given path seems to be relative to the servlet root directory. + else + input = searchFile(tapConfPath, config); + } + // If no file has been found, cancel the servlet loading: + if (input == null) + throw new ServletException("Configuration file not found with the path: \"" + ((tapConfPath == null) ? DEFAULT_TAP_CONF_FILE : tapConfPath) + "\"! Please provide a correct file path in servlet init parameter (\"" + TAP_CONF_PARAMETER + "\") or put your configuration file named \"" + DEFAULT_TAP_CONF_FILE + "\" in a directory of the classpath or in WEB-INF or META-INF."); + + /* 3. PARSE IT INTO A PROPERTIES SET */ + Properties tapConf = new Properties(); + try{ + tapConf.load(input); + }catch(IOException ioe){ + throw new ServletException("Impossible to read the TAP configuration file (" + tapConfPath + ")!", ioe); + }finally{ + try{ + input.close(); + }catch(IOException ioe2){} + } + + /* 4. CREATE THE TAP SERVICE */ + ServiceConnection serviceConn = null; + try{ + // Create the service connection: + serviceConn = new ConfigurableServiceConnection(tapConf, config.getServletContext().getRealPath("")); + // Create all the TAP resources: + tap = new TAP(serviceConn); + }catch(Exception ex){ + tap = null; + if (ex instanceof TAPException) + throw new ServletException(ex.getMessage(), ex.getCause()); + else + throw new ServletException("Impossible to initialize the TAP service!", ex); + } + + /* 4Bis. SET THE HOME PAGE */ + String propValue = getProperty(tapConf, KEY_HOME_PAGE); + if (propValue != null){ + // If it is a class path, replace the current home page by an instance of this class: + if (isClassName(propValue)){ + try{ + tap.setHomePage(newInstance(propValue, KEY_HOME_PAGE, HomePage.class, new Class[]{TAP.class}, new Object[]{tap})); + }catch(TAPException te){ + throw new ServletException(te.getMessage(), te.getCause()); + } + } + // If it is a file URI (null, file inside WebContent, file://..., http://..., etc...): + else{ + // ...set the given URI: + tap.setHomePageURI(propValue); + // ...and its MIME type (if any): + propValue = getProperty(tapConf, KEY_HOME_PAGE_MIME_TYPE); + if (propValue != null) + tap.setHomePageMimeType(propValue); + } + } + + /* 5. SET ADDITIONAL TAP RESOURCES */ + propValue = getProperty(tapConf, KEY_ADD_TAP_RESOURCES); + if (propValue != null){ + // split all list items: + String[] lstResources = propValue.split(","); + for(String addRes : lstResources){ + addRes = addRes.trim(); + // ignore empty items: + if (addRes.length() > 0){ + try{ + // create an instance of the resource: + TAPResource newRes = newInstance(addRes, KEY_ADD_TAP_RESOURCES, TAPResource.class, new Class[]{TAP.class}, new Object[]{tap}); + if (newRes.getName() == null || newRes.getName().trim().length() == 0) + throw new TAPException("TAP resource name missing for the new resource \"" + addRes + "\"! The function getName() of the new TAPResource must return a non-empty and not NULL name. See the property \"" + KEY_ADD_TAP_RESOURCES + "\"."); + // add it into TAP: + tap.addResource(newRes); + }catch(TAPException te){ + throw new ServletException(te.getMessage(), te.getCause()); + } + } + } + } + + /* 6. DEFAULT SERVLET INITIALIZATION */ + super.init(config); + + /* 7. FINALLY MAKE THE SERVICE AVAILABLE */ + serviceConn.setAvailable(true, "TAP service available."); + } + + /** + * Search the given file name/path in the directories of the classpath, then inside WEB-INF and finally inside META-INF. + * + * @param filePath A file name/path. + * @param config Servlet configuration (containing also the context class loader - link with the servlet classpath). + * + * @return The input stream toward the specified file, or NULL if no file can be found. + * + * @since 2.0 + */ + protected final InputStream searchFile(String filePath, final ServletConfig config){ + InputStream input = null; + + // Try to search in the classpath (with just a file name or a relative path): + input = Thread.currentThread().getContextClassLoader().getResourceAsStream(filePath); + + // If not found, try searching in WEB-INF and META-INF (as this fileName is a file path relative to one of these directories): + if (input == null){ + if (filePath.startsWith("/")) + filePath = filePath.substring(1); + // ...try at the root of WEB-INF: + input = config.getServletContext().getResourceAsStream("/WEB-INF/" + filePath); + // ...and at the root of META-INF: + if (input == null) + input = config.getServletContext().getResourceAsStream("/META-INF/" + filePath); + } + + return input; + } + + @Override + public void destroy(){ + // Free all resources used by TAP: + if (tap != null){ + tap.destroy(); + tap = null; + } + super.destroy(); + } + + @Override + protected void service(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException{ + if (tap != null){ + try{ + tap.executeRequest(req, resp); + }catch(Throwable t){ + resp.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, t.getMessage()); + } + }else + resp.sendError(HttpServletResponse.SC_SERVICE_UNAVAILABLE, "TAP service not yet initialized!"); + } + +} diff --git a/src/tap/config/TAPConfiguration.java b/src/tap/config/TAPConfiguration.java new file mode 100644 index 0000000000000000000000000000000000000000..11934ed55bd9b8760ae462c9ca3083bd190c348f --- /dev/null +++ b/src/tap/config/TAPConfiguration.java @@ -0,0 +1,539 @@ +package tap.config; + +/* + * This file is part of TAPLibrary. + * + * TAPLibrary is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * TAPLibrary is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with TAPLibrary. If not, seeUtility class gathering tool functions and properties' names useful to deal with a TAP configuration file.
+ * + *