wiki:PetascopeUserGuide_8_5

Version 6 (modified by pbaumann, 21 months ago) (diff)

--

NOTE: this guide is for maintenance purposes and refers to rasdaman version 8.5.x (and prior); it is spawned from the latest user guide at version 37. Feel free to contribute to this if you are working with rasdaman 8.5 to improve it.

Petascope User guide

The petascope component of rasdaman implements the OGC interface standards WCS 2.0, WCS-T 1.4, WCPS 1.0, and WPS 1.0. For this purpose, petascope maintains its additional metadata (such as georeferencing) which is kept in separate relational tables. Note that not all rasdaman raster objects and collections are available through petascope by default; rather, they need to be registered through the petascope administration interface.

Petascope is implemented as a war file of servlets which give access to coverages (in the OGC sense) stored in rasdaman. Internally, incoming requests requiring coverage evaluation are translated into rasql queries by petascope. These queries are passed on to rasdaman, which constitutes the central workhorse. Results returned from rasdaman are forwarded to the client, finally.

N.B. Please note that Petascope adopts the minimum bounding-box policy: the bounding box in the WCS request -- when covers an area which partially falls outside the coverage extents -- is trimmed in output to fit the coverage bounding box.

Miscellanea

  • For the installation, a .war file needs to be deployed and configured for use with some rasdaman server installation. See the installation section? for details.
  • For a developer's introduction to petascope, see the presentation and the related developer guide.
  • For the interpretation of subsets in petascope, see this page.
  • For the handling of time series see this page.

WCS

"The OpenGIS Web Coverage Service Interface Standard (WCS) defines a standard interface and operations that enables interoperable access to geospatial coverages." (WCS standards)

Petascope supports WCS 2.0; see this demo.

Metadata

gmlcov:metadata

Additional arbitrary metadata for a coverage "myCoverage" can be manually injected into the output of a DescribeCoverage response by properly filling the petascopedb:ps_metadata table:

INSERT INTO ps_metadata (coverage, metadata) 
VALUES ((SELECT id FROM ps_coverage WHERE name='myCoverage'), 'myMetadata'); 

After this operation, a new XML element will appear in the coverage description:

<gmlcov:metadata>myMetadata</gmlcov:metadata>

As by definition (http://schemas.opengis.net/gmlcov/1.0/coverage.xsd), the metadata content should make use of the XML extension mechanism, although this does not represent a restriction:

This is a hook for adding any further information to a coverage, such as domain-specific metadata. Recommended use is to use the XML extension mechanism, such as in a WCS extension or Application Profile, to define the desired metadata structure.

In this case, remember to escape those XML predefined entities inside elements' values and attributes' values that could break the whole XML document. Additionally the prefixes should be bound (embed XML namespaces 'xmlsn:<prefix>=<uri>' in your root elements).

Service capabilities metadata

It is possible to customize the templates in petascope.wcs2.templates for the service capabilities, e.g. for adding further information/extended capabilities about the service. These templates are

  • ServiceIdentification.templ
    Encoding of the common ServiceIdentification section of the GetCapabilities operation response (.xsd).
  • ServiceProvider.templ
    Encoding of the the common ServiceProvider section of the GetCapabilities operation response (.xsd).
  • ServiceMetadata.templ
    Encoding of the information describing the WCS service on hand. Extension elements allow WCS extension standards to define their individual extra service metadata (.xsd).

Modifying the information on existing XML elements in the templates is straightforward, nonetheless it is possible to add new optional elements like ows:Fees or ows:AccessConstraints that are not defined by the templates, and this is still doable starting from changeset:de7ef92 and namesake commits in branch release_8.4.

An other possible case is adding a custom wcs:extension to the wcs:ServiceMetadata: you can just manually insert it into the ServiceMetadata.templ template, then Petascope will properly embed the further metadata which are not hard-wired in the templates.

See , and comment:1:ticket:366 for practical examples.

Feature space description

Metadata regarding the data types of the feature space for a coverage "myCoverage" can also be injected in the DescribeCoverage response. Name, type and UoM of each band can be set by proper filling of the petascopedb tables ps_range and ps_uom.

For instance, if we want to publish that "myCoverage" has 1 band of [0,255] integer values (uchar) which represent mass concentrations (ug/m3 UCUM code):

petascopedb=# INSERT INTO ps_uom (uom) VALUES ('ug/m3') RETURNING id;
INSERT 0 1
petascopedb=# INSERT INTO ps_range (coverage, i, name, type, uom) VALUES (
petascopedb-#    (SELECT id FROM ps_coverage WHERE name='myCoverage'),        -- coverage
petascopedb-#    0,                                                           -- i
petascopedb-#    'PM10 mass concentrations',                                  -- name
petascopedb-#    (SELECT id FROM ps_datatype WHERE datatype='unsigned char'), -- type
petascopedb-#    (SELECT id FROM ps_uom WHERE uom='ug/m3'));                  -- uom

..and similarly for updating. The i represents the order of the bands (starting from 0). The type is to be set by pointing to the proper row in ps_datatype (see this link for details):

petascopedb=# SELECT * FROM ps_datatype ORDER BY datatype;
 id |    datatype    
----+----------------
  1 | boolean         
  2 | char
 12 | complex         
 13 | complex2
 11 | double
 10 | float
  6 | int
  8 | long
  4 | short
  3 | unsigned char
  7 | unsigned int
  9 | unsigned long
  5 | unsigned short
(13 rows)

To verify the inserted metadata, you can execute:

petascopedb=# SELECT ps_coverage.name AS coverage, i, ps_range.name AS band, ps_datatype.datatype, ps_uom.uom 
petascopedb-# FROM ps_coverage, ps_range, ps_datatype, ps_uom 
petascopedb-# WHERE ps_coverage.id=ps_range.coverage 
petascopedb-#   AND ps_datatype.id=ps_range.type 
petascopedb-#   AND ps_uom.id=ps_range.uom 
petascopedb-#   AND ps_coverage.name='myCoverage';
  coverage  | i |           band           |   datatype    |  uom  
------------+---+--------------------------+---------------+-------
 myCoverage | 0 | PM10 mass concentrations | unsigned char | ug/m3
(1 row)

After doing so, a DescribeCoverage response will then report the inserted metadata:

<gmlcov:rangeType>
  <swe:DataRecord>
    <swe:field name="PM10 mass concentrations">
      <swe:Quantity definition="urn:ogc:def:dataType:OGC:1.1:unsigned char">
        <swe:description/>
        <swe:uom code="ug/m3"/>
        <swe:constraint>
          <swe:AllowedValues>
            <swe:interval>0 255</swe:interval>
          </swe:AllowedValues>
        </swe:constraint>
      </swe:Quantity>
    </swe:field>
  </swe:DataRecord>
</gmlcov:rangeType>

NOTE1: Petascope requires at least 1 row in ps_range per coverage.
NOTE2: ps_range is automatically filled if the coverage is ingested via rasimport utility.

Current Deviations

  • 0D coverages can be retrieved through a d-dimensional slicing on a d-dimensional coverage. The resulting coverage is disallowed by GML, however petascope delivers it for user convenience. Should that not be desired then a trimming with lower bound = upper bound in all d dimensions delivers a valid coverage with the same value.

WCPS

"The OpenGIS Web Coverage Service Interface Standard (WCS) defines a protocol-independent language for the extraction, processing, and analysis of multi-dimensional gridded coverages representing sensor, image, or statistics data. Services implementing this language provide access to original or derived sets of geospatial coverage information, in forms that are useful for client-side rendering, input into scientific models, and other client applications. Further information about WPCS can be found at the WCPS Service page of the OGC Network. (http://www.opengeospatial.org/standards/wcps)

The WCPS language is independent from any particular request and response encoding, allowing embedding of WCPS into different target service frameworks like WCS and WPS. The following documents are relevant for WCPS; they can be downloaded from www.opengeospatial.org/standards/wcps:

  • OGC 08-068r2: The protocol-independent ("abstract") syntax definition; this is the core document. Document type: IS (Interface Standard.
  • OGC 08-059r3: This document defines the embedding of WCPS into WCS by specifying a concrete protocol which adds an optional ProcessCoverages request type to WCS. Document type: IS (Interface Standard.
  • OGC 09-045: This draft document defines the embedding of WCPS into WPS as an application profile by specifying a concrete subtype of the Execute request type.

There is a demo sandbox and webinars plus WCPS manual and tutorial.

The petascope implementation supports both XML and Abstract Syntax.

  • To send a abstract syntax request: A post request must be sent. The query string must be sent in a post parameter named query.
  • To send a request as a XML file, a multi-part post request must be sent. Only one file must be sent. The Mime type has to be text/xml.
  • To send a request as XML: A post request must be sent. A post parameter named xml must be attached.

If you intend to implement forms-based access: To obtain a proper query from an XML page a form must be used. The method should be post and action should be an URL pointing to the servlet access point. The form should contain either a file field or a text/hidden field that has the name xml or query. An error in the access method will generate an HTML query input form.

For guidelines on how to safely build and troubleshoot WCPS query with Petascope, see this topic in the mailing-list.

Current Deviations

  • WCPS coverage variables in petascope must not have a $ prefix, in contrast to the WCPS specification.
  • imageCrsDomain can be used in the OVER clause of a coverage constructor (cf. #475) and the scale operation (cf. #387)
  • SWITCH statement. It can be used as in the following example:
    for c in mean_summer_airtemp 
    return 
      encode( 
        switch 
          case c>0 return log(c)
          case c=0 return (char)0
          default return (char)1,
        "csv")
    

WPS

"The OpenGIS Web Processing Service (WPS) Interface Standard provides rules for standardizing how inputs and outputs (requests and responses) for geospatial processing services, such as polygon overlay. The standard also defines how a client can request the execution of a process, and how the output from the process is handled. It defines an interface that facilitates the publishing of geospatial processes and clients’ discovery of and binding to those processes. The data required by the WPS can be delivered across a network or they can be available at the server." (OGC Web Processing Service)

Petascope integrates with the 52°north WPS allowing to submit a WCPS request via WPS and deliver sets of coverages or scalars as results. The WCPS query can be in either XML or abstract syntax.

Deployment

First, the original war file from 52 north needs to be adapted for this to work. This can be done with make wps from the petascope directory:

  • make wps wpswar=/path/to/52n-wps-webapp-2.0.war

The 52n admin WPS page can be accessed then at http://localhost:8080/wps/webAdmin/index.jsp with default user/pass: wps/wps

To integrate a newly deployed custom n52 war with petascope WCPS

  1. go to Upload Process and upload petascope/src/main/resources/n52/ProcessCoverages.java and petascope/src/main/resources/n52/ProcessCoverages.xml
  2. Save configuration
  3. In Algorithm Repositories a property "Algorithm=petascope.wps.n52.ProcessCoverages" for the UploadedAlgorithmRepository needs to be added.
  4. Deactivate the LocalAlgorithmRepository

Internals

Incoming WPS Execute requests invoking the ProcessCoverages process are forwarded by the 52°north to petascope, stripped off the WPS specifics. The resulting WCPS request is processed, and the result is returned to WPS.

Technically, the coupling consists of an adapter class (a Process). The process identifier is petascope.wps.n52.ProcessCoverages.

Demo

See this small earthlook we have set a demo too, available EarthLook WPS demo. There is a simple test client done by n52.

WMS

"The OpenGIS Web Map Service Interface Standard (WMS) provides a simple HTTP interface for requesting geo-registered map images from one or more distributed geospatial databases. A WMS request defines the geographic layer(s) and area of interest to be processed. The response to the request is one or more geo-registered map images (returned as JPEG, PNG, etc) that can be displayed in a browser application. The interface also supports the ability to specify whether the returned images should be transparent so that layers from multiple servers can be combined or not."

Petascope supports WMS 1.0.0/1.1.0. How to publish a WMS layer.

Limitations

  • as the WCS CRS extension is not yet approved by OGC (although submitted by the WCS.SWG) CRS handling is still limited.

Tickets reporting deviations from standards

Tickets reporting deviations should be tagged with the keyword "deviation" so they can get listed here and assessed for documentation.

No results