wiki:RasdamanTestSuites

Version 6 (modified by pcampalani, 4 years ago) (diff)

--

NOTE: this page is work in an ongoing effort to document the test system used to test rasdaman

Overview

"make check" will automatically find all the tests inside the four test case directories, specifically, testcases_mandatory, testcases_petascope, testcases_fixed and testcases_open.

  1. Whenever a bug is found, there should be a corresponding test be created in the testcases_open directory;
  2. when the bug is fixed, the test should be moved to the testcases_fixed directory;
  3. testcases_petascope holds the test cases for petascope;
  4. testcases_mandatory  holds the mandatory test cases.
  1. Each test should have a folder which is inside one of the above mentioned directories;
  2. the test should be invoked by a shell script inside the folder;
  3. the test result should be told by its EXIT code;
  4. the test can be a pure shell script or a shell script with corresponding test queries and/or test oracles;
  5. test detail should be log in the same folder.

Adding a test to the suite

Howto with a sample test

System test

Let us assume that we want to add a test case for our Rasdaman building, you can do as below:

  • how to add a test case?
      1. copy the test package to rasdaman/systemtest/test_open
      2. rename the test packet to test_<testcaseName> 
      3. rename the main shell script in rasdaman/systemtest/test_open/test_<testcaseName> to test.sh 
    
  • how to add a test query?
      1. save the test query as <queryName>.rasql 
      2. copy the query to rasdaman/systemtest/test_mandatory/test_select/test_rasql
      3. save the related query result as a test oracle with the name <queryName>.rasql (the same name with test query file)
      4. copy the oracle to rasdaman/systemtest/test_mandatory/test_select/oracle 
    
  • how to generate a test oracle?
      1. check the result correctness 
      2. if the result is a scalar, run
    rasql -q  "$QUERY" --out string|grep Result > oracleName
      3. if the result is an array, run
     rasql -q  "$QUERY" --out file --outfile oracleName
    
  • how to run the test?
      1. run 'make check' under rasdaman or rasdaman/systemtest directory 
    

Petascope tests

Before submitting any patch which modifies the source of Petascope, both WCS and WCPS smoke tests should be checked before and after applying the changes.

The scripts for WCS and WCPS testing can be found respectively in:

  • rasdaman/systemtest/testcases_petascope/test_wcs
  • rasdaman/systemtest/testcases_petascope/test_wcps

To run the tests, simply execute in both cases:

./test.sh

from inside the respective directories. Do not execute sh test.sh as the script is written for bash, and sh is often linked to a restricted version of bash like dash, or similar.

Variables like Tomcat port, host, rasdaman connection details, etc. may need to be adapted before running the tests by editing the file:

rasdaman/systemtest/testcases_petascope/test_wcps/test.cfg

which currently is used for both WCS and WCPS tests.

NOTE: currently (jan2013, sha1_HEAD=b04bdb9), 8/8 WCS tests and 49/51 WCPS tests pass with no error.

Testdata

The following coverages are available for the tests:

+----------+-----+-------+-----------------+-------+-----------------+------------+
| coverage | dim | type  | pixel extent    | axes  | geo-boundingbox |    CRS     |
+----------+-----+-------+-----------------+-------+-----------------+------------+
| rgb      | 2D  | rgb   | 0:399,0:343     | x/y   |        -        |     -      |   
| mr       | 2D  | char  | 0:255,0:210     | x/y   |        -        |     -      |   
| eobstest | 3D  | short | 0:5,0:100,0:231 | t/x/y | 25,-40 - 75,75  | EPGSG:4326 |
+----------+-----+-------+-----------------+-------+-----------------+------+

These coverages are automatically inserted if missing.

Adding tests

To add new tests to the test suite, simply add new WCS or WCPS queries to the queries directory. Please adhere to the naming convention, continuing from the last number:

  • WCS KVP : number-meaningful_name.kvp
  • WCS XML : number-meaningful_name.xml
  • WCS SOAP : number-meaningful_name.soap
  • WCS REST : number-meaningful_name.rest
  • WCPS abstract: number-meaningful_name.test
  • WCPS XML : number-meaningful_name.xml
  • rasql : number-meaningful_name.rasql