Opened 3 years ago

Closed 14 months ago

#732 closed defect (fixed)

GML CRS definitions are invalid and need to validate all definitions

Reported by: jpass Owned by: bphamhuu
Priority: critical Milestone: Future
Component: secore Version: development
Keywords: Cc:
Complexity: Medium

Description

The output of a EPSG GML definition are schema invalid, checking against the EPSG schema (http://www.epsg.org/RegistrySchemas/EPSG-and-GML-3.2.1-schemas_2012-06-20.zip) for example:

http://kahlua.eecs.jacobs-university.de:8080/def/crs/EPSG/0/5711.

A valid response would be:

<?xml version="1.0" encoding="ISO-8859-1"?>
<gml:VerticalCRS xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.opengis.net/gml/3.2 file:/E:/xml/EPSG-and-GML-3.2.1-schemas_2012-06-20/epsg/EPSG.xsd"
  xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gco="http://www.isotc211.org/2005/gco"
  xmlns:epsg="urn:x-ogp:spec:schema-xsd:EPSG:1.0:dataset" xmlns:xlink="http://www.w3.org/1999/xlink"
  xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:rim="urn:oasis:names:tc:ebxml-regrep:xsd:rim:3.0"
  gml:id="epsg-crs-5711">
  <gml:metaDataProperty>
    <epsg:CommonMetaData>
      <epsg:type>vertical</epsg:type>
      <epsg:alias alias="Australian Height Datum height" code="271"
        codeSpace="http://www.opengis.net/def/naming-system/EPSG/0/7301">
        <epsg:remarks>Australian Height Datum height</epsg:remarks>
      </epsg:alias>
      <epsg:informationSource>http://www.auslig.gov.au/geodesy</epsg:informationSource>
      <epsg:revisionDate>2008-03-14</epsg:revisionDate>
      <epsg:changes>
        <epsg:changeID xlink:href="http://www.opengis.net/def/change-request/EPSG/0/2008.010"/>
      </epsg:changes>
      <epsg:show>true</epsg:show>
      <epsg:isDeprecated>false</epsg:isDeprecated>
    </epsg:CommonMetaData>
  </gml:metaDataProperty>
  <gml:identifier codeSpace="OGP">http://www.opengis.net/def/crs/EPSG/0/5711</gml:identifier>
  <gml:name>AHD height</gml:name>
  <gml:domainOfValidity>
    <epsg:AreaOfUse xmlns:gmd="http://www.isotc211.org/2005/gmd"
      xmlns:gco="http://www.isotc211.org/2005/gco">
      <gmd:description>
        <gco:CharacterString>Australia - Australian Capital Territory; New South Wales; Northern Territory; Queensland; South Australia; Western Australia; Victoria.</gco:CharacterString>
      </gmd:description>
      <gmd:geographicElement>
        <gmd:EX_GeographicBoundingBox>
          <gmd:westBoundLongitude>
            <gco:Decimal>112.85</gco:Decimal>
          </gmd:westBoundLongitude>
          <gmd:eastBoundLongitude>
            <gco:Decimal>153.68</gco:Decimal>
          </gmd:eastBoundLongitude>
          <gmd:southBoundLatitude>
            <gco:Decimal>-39.2</gco:Decimal>
          </gmd:southBoundLatitude>
          <gmd:northBoundLatitude>
            <gco:Decimal>-10.66</gco:Decimal>
          </gmd:northBoundLatitude>
        </gmd:EX_GeographicBoundingBox>
      </gmd:geographicElement>
      <gml:metaDataProperty>
        <epsg:CommonMetaData>
          <epsg:informationSource>OGP</epsg:informationSource>
          <epsg:revisionDate>2012-02-19</epsg:revisionDate>
          <epsg:changes>
            <epsg:changeID xlink:href="http://www.opengis.net/def/change-request/EPSG/0/2007.009"/>
            <epsg:changeID xlink:href="http://www.opengis.net/def/change-request/EPSG/0/2012.027"/>
          </epsg:changes>
          <epsg:show>false</epsg:show>
          <epsg:isDeprecated>false</epsg:isDeprecated>
        </epsg:CommonMetaData>
      </gml:metaDataProperty>
      <gml:identifier codeSpace="OGP">http://www.opengis.net/def/area/EPSG/0/1281</gml:identifier>
      <gml:name>Australia - mainland</gml:name>
      <gml:remarks/>
    </epsg:AreaOfUse>
  </gml:domainOfValidity>
  <gml:scope>Geodetic and engineering surveying.</gml:scope>
  <gml:verticalCS>
    <gml:VerticalCS gml:id="epsg-cs-6499">
      <gml:metaDataProperty>
        <epsg:CommonMetaData>
          <epsg:type>vertical</epsg:type>
          <epsg:informationSource>OGP</epsg:informationSource>
          <epsg:revisionDate>2006-11-28</epsg:revisionDate>
          <epsg:isDeprecated>false</epsg:isDeprecated>
        </epsg:CommonMetaData>
      </gml:metaDataProperty>
      <gml:identifier codeSpace="OGP">http://www.opengis.net/def/cs/EPSG/0/6499</gml:identifier>
      <gml:name>Vertical CS. Axis: height (H). Orientation: up. UoM: m.</gml:name>
      <gml:remarks>Used in vertical coordinate reference systems.</gml:remarks>
      <gml:axis>
        <gml:CoordinateSystemAxis gml:id="epsg-axis-114"
          uom="http://www.opengis.net/def/uom/EPSG/0/9001">
          <gml:descriptionReference xlink:href="http://www.opengis.net/def/axis-name/EPSG/0/9904"/>
          <gml:identifier codeSpace="OGP">http://www.opengis.net/def/axis/EPSG/0/114</gml:identifier>
          <gml:axisAbbrev>H</gml:axisAbbrev>
          <gml:axisDirection codeSpace="EPSG">up</gml:axisDirection>
        </gml:CoordinateSystemAxis>
      </gml:axis>
    </gml:VerticalCS>
  </gml:verticalCS>
  <gml:verticalDatum>
    <gml:VerticalDatum gml:id="epsg-datum-5111">
      <gml:metaDataProperty>
        <epsg:CommonMetaData>
          <epsg:type>vertical</epsg:type>
          <epsg:alias alias="AHD" code="253"
            codeSpace="http://www.opengis.net/def/naming-system/EPSG/0/7302"/>
          <epsg:informationSource>Australian Land Information Group  www.auslig.gov.au</epsg:informationSource>
          <epsg:revisionDate>2005-09-06</epsg:revisionDate>
          <epsg:changes>
            <epsg:changeID xlink:href="http://www.opengis.net/def/change-request/EPSG/0/2005.460"/>
          </epsg:changes>
          <epsg:isDeprecated>false</epsg:isDeprecated>
        </epsg:CommonMetaData>
      </gml:metaDataProperty>
      <gml:identifier codeSpace="OGP">http://www.opengis.net/def/datum/EPSG/0/5111</gml:identifier>
      <gml:name>Australian Height Datum</gml:name>
      <gml:remarks/>
      <gml:domainOfValidity>
      <epsg:AreaOfUse>
        <gmd:description xmlns:gmd="http://www.isotc211.org/2005/gmd">
          <gco:CharacterString xmlns:gco="http://www.isotc211.org/2005/gco">Australia - Australian Capital Territory; New South Wales; Northern Territory; Queensland; South Australia; Western Australia; Victoria.</gco:CharacterString>
        </gmd:description>
        <gmd:geographicElement xmlns:gmd="http://www.isotc211.org/2005/gmd">
          <gmd:EX_GeographicBoundingBox>
            <gmd:westBoundLongitude>
              <gco:Decimal xmlns:gco="http://www.isotc211.org/2005/gco">112.85</gco:Decimal>
            </gmd:westBoundLongitude>
            <gmd:eastBoundLongitude>
              <gco:Decimal xmlns:gco="http://www.isotc211.org/2005/gco">153.68</gco:Decimal>
            </gmd:eastBoundLongitude>
            <gmd:southBoundLatitude>
              <gco:Decimal xmlns:gco="http://www.isotc211.org/2005/gco">-39.2</gco:Decimal>
            </gmd:southBoundLatitude>
            <gmd:northBoundLatitude>
              <gco:Decimal xmlns:gco="http://www.isotc211.org/2005/gco">-10.66</gco:Decimal>
            </gmd:northBoundLatitude>
          </gmd:EX_GeographicBoundingBox>
        </gmd:geographicElement>
        <gml:metaDataProperty>
          <epsg:CommonMetaData>
            <epsg:informationSource>OGP</epsg:informationSource>
            <epsg:revisionDate>2012-02-19</epsg:revisionDate>
            <epsg:changes>
              <epsg:changeID xlink:href="http://www.opengis.net/def/change-request/EPSG/0/2007.009" />
              <epsg:changeID xlink:href="http://www.opengis.net/def/change-request/EPSG/0/2012.027"/>
            </epsg:changes>
            <epsg:show>false</epsg:show>
            <epsg:isDeprecated>false</epsg:isDeprecated>
          </epsg:CommonMetaData>
        </gml:metaDataProperty>
        <gml:identifier codeSpace="OGP">http://www.opengis.net/def/area/EPSG/0/1281</gml:identifier>
        <gml:name>Australia - mainland</gml:name>
        <epsg:geometryFile xlink:href="http://www.opengis.net/def/extent-polygon/EPSG/0/1281"
          xlink:type="simple"/>
      </epsg:AreaOfUse>
      </gml:domainOfValidity>
      <gml:scope>Topographic mapping, geodetic survey.</gml:scope>
      <gml:anchorDefinition>MSL 1966-68 at 30 gauges around coast.</gml:anchorDefinition>
      <gml:realizationEpoch>1968-01-01</gml:realizationEpoch>
    </gml:VerticalDatum>
  </gml:verticalDatum>
</gml:VerticalCRS>

The principal issues are:

  • <epsg:AreaOfUse> should be nested inside an <gml:domainOfValidity> element
  • <gml:VerticalCS gml:id="epsg-cs-6499"> should be nested inside an <gml:verticalCS> element
  • <gml:VerticalDatum gml:id="epsg-datum-5111"> should be nested inside an </gml:verticalDatum> element
  • the <empty/> element should probably be <gml:remarks/>

Change History (15)

comment:2 Changed 2 years ago by dmisev

  • Milestone set to Future
  • Owner changed from pcampalani to dmisev
  • Status changed from new to assigned

comment:3 Changed 16 months ago by bphamhuu

  • Owner changed from dmisev to bphamhuu

As Dimitar has suggested before, so I will take this ticket and try to understand SECORE.

comment:4 Changed 16 months ago by bphamhuu

Ok, now when I've known how to valid GML from SECORE, I will try to know why SECORE resolved from EPSG dataset is not valid with GML 3.2.1 schema soon.

For example in this URI, the problem is:

cvc-complex-type.2.4.a: Invalid content was found starting with element 'epsg:ChangeRequest'. One of '{"urn:x-ogp:spec:schema-xsd:EPSG:1.0:dataset":changeID}' is expected.
cvc-complex-type.2.4.a: Invalid content was found starting with element 'epsg:AreaOfUse'. One of '{"http://www.opengis.net/gml/3.2":name, "http://www.opengis.net/gml/3.2":remarks, "http://www.opengis.net/gml/3.2":domainOfValidity, "http://www.opengis.net/gml/3.2":scope}' is expected.
cvc-complex-type.2.4.a: Invalid content was found starting with element 'epsg:ChangeRequest'. One of '{"urn:x-ogp:spec:schema-xsd:EPSG:1.0:dataset":changeID}' is expected.
cvc-complex-type.2.4.a: Invalid content was found starting with element 'empty'. One of '{"http://www.opengis.net/gml/3.2":name, "http://www.opengis.net/gml/3.2":remarks, "urn:x-ogp:spec:schema-xsd:EPSG:1.0:dataset":isoA2CountryCode, "urn:x-ogp:spec:schema-xsd:EPSG:1.0:dataset":isoA3CountryCode, "urn:x-ogp:spec:schema-xsd:EPSG:1.0:dataset":isoNumericCountryCode, "urn:x-ogp:spec:schema-xsd:EPSG:1.0:dataset":geometryFile}' is expected.
cvc-complex-type.2.4.a: Invalid content was found starting with element 'epsg:AxisName'. One of '{"http://www.opengis.net/gml/3.2":metaDataProperty, "http://www.opengis.net/gml/3.2":description, "http://www.opengis.net/gml/3.2":descriptionReference, "http://www.opengis.net/gml/3.2":identifier}' is expected.
cvc-complex-type.2.4.a: Invalid content was found starting with element 'epsg:ChangeRequest'. One of '{"urn:x-ogp:spec:schema-xsd:EPSG:1.0:dataset":changeID}' is expected.
cvc-complex-type.2.4.a: Invalid content was found starting with element 'epsg:AreaOfUse'. One of '{"http://www.opengis.net/gml/3.2":domainOfValidity, "http://www.opengis.net/gml/3.2":scope}' is expected.

comment:5 Changed 16 months ago by dmisev

How about this URL?
I think it may not validate the earlier because the hrefs are flattened by default, which I guess is not valid by the schema.

comment:6 Changed 15 months ago by bphamhuu

up to now, I've found an important problem with SECORE when resolving xlink:href is "you just cannot resolve every xlink:href". Based on GML Schema, some reference is allowed to do this, but some will make resolved definition invalid.

+ if element is referenced to another CRS definition like (domainOfValidity, VerticalDatum?, VerticalCS) like this

<gml:verticalDatum xlink:href="urn:ogc:def:datum:EPSG::1028"/>

must have <gml:verticalDatum></gml:verticalDatum> outside of the content.

+ Every information element like changeID, descriptionReference should not be resolved because if you put <changeID> or <descptionReference> outside of the content, it will notice the element should have no children.

+ If the definition does not exist anymore, return <empty/> is very confusing, example

http://www.opengis.net/def/area/EPSG/0/1281/browse.jsp?update=true

which has a reference to not exist definition.

<epsg:geometryFile xlink:type="simple" xlink:href="http://www.opengis.net/def/extent-polygon/EPSG/0/1281"/>

It should return in <gml:remarks></gml:remarks> with content like this, as <gml:remarks> cannot have children.

<gml:remarks>
   empty: xlink:href="http://www.opengis.net/def/extent-polygon/EPSG/0/1281"
<gml:remakrs>

Here is the diff between 2 cases expand=full does not valid and expand=full which is modified to valid

https://www.diffchecker.com/dtjdcnt5


Last edited 15 months ago by bphamhuu (previous) (diff)

comment:7 Changed 15 months ago by dmisev

I see, that makes sense. It wasn't clear that not all hrefs can be expanded, from the specification documents it sounded as if they can all be expanded and still yield a valid definition.

You are right, then I think we should fix the expansion.

comment:8 Changed 15 months ago by bphamhuu

I've solved this ticket with the solution but to make sure everything is all valid (except ParameteriszedCRS in userdb) then all definition will need to be validated to make sure no thing is ignored.

this is the most important, if one element should not be expanded then can add it to where not contains (nodeName)

this is an example of how to do this.

declare function local:getid($db as document-node(), $id as xs:string) as element() {
let $retUserDB := $db//gml:identifier[fn:ends-with(text(), $id)]/..
let $retValue := ""
return if (empty($retUserDB)) then
             let $retGML := collection('gml')//gml:identifier[fn:ends-with(text(), $id)]/..
             return if (empty($retGML)) then
                       (: return empty node inside <remarks></remarks> with the unresolveable link :) 
                       let $retValue := <remarks>empty_xlink:href="{$id}"</remarks>
                       return $retValue  
                   else
                       let $retValue := $retGML[last()]
                       return $retValue      
       else
             let $retValue := $retUserDB[last()]
             return $retValue};


declare function local:flatten($db as document-node(), $id as xs:string, $depth as xs:integer) as element()* {
  copy $el := local:getid($db, $id)
  modify
  (
  for $c in $el//*[@xlink:href]
  let $elName := $c/name()
  let $test := node-name($c)
  where     not (contains($elName, "changeID"))
        and not (contains($elName, "descriptionReference"))
        and not (contains($elName, "projectionConversion")) 
        and not (contains($elName, "sourceGeographicCRS"))
        and not (contains($elName, "geometryFile"))
  return if ($depth < 2) then
        (: This will add a parent node around imported definition from xlink:href :)
  	replace node $c with element {$test} {
                 ( local:flatten($db, $c/@xlink:href, $depth + 1) )
             }
	  else replace node $c with $c
  )
  return $el
};

comment:9 Changed 15 months ago by bphamhuu

  • Summary changed from GML CRS definitions are invalid to GML CRS definitions are invalid and need to validate all definitions

comment:10 Changed 15 months ago by dmisev

Perhaps we could implement a button in the admin pages -> "Validate all definitions"?

comment:11 Changed 15 months ago by bphamhuu

It takes really long time to validate GML definition, I'm running the test, if you like to see web browser running in few hours then I can do it later with another ticket.

I check all the definition to make sure don't miss some cases as in above.

comment:12 Changed 15 months ago by dmisev

Ok nevermind that, ignore my question.

comment:13 Changed 15 months ago by dmisev

Can this ticket be closed now?

comment:14 Changed 15 months ago by bphamhuu

:) I would like to do as it lived to old but only when patch is accepted.

comment:15 Changed 15 months ago by bphamhuu

this ticket will have effect on systemtest/test_secore and somes parameterized like Index2D in test_wcs, test_wcps as I've seen in log file. So still till need to work more.

comment:16 Changed 14 months ago by bphamhuu

  • Resolution set to fixed
  • Status changed from assigned to closed

patch was accepted, close ticket here.

Note: See TracTickets for help on using tickets.