Opened 4 years ago

Closed 4 years ago

#335 closed defect (fixed)

Coverage iterator not translated properly to rasql on WCPS trimmings

Reported by: ungarj Owned by: dmisev
Priority: critical Milestone: 9.0
Component: petascope Version: 8.4
Keywords: coverage iterator condense construct Cc: dirk.daems@…, pbaumann
Complexity: Medium

Description (last modified by ungarj)

Just found out that some of the WCPS queries in our WCPS Tutorial cause an error now though having worked in earlier versions.

It seems after some testing the last version the queries worked was rasdaman 8.3.1-3.

Not sure whether it's relevant but the only queries affected (just tested our example queries though) are the ones with 1D output.

catalina.out indicates the "Specified domain does not intersect with spatial domain of MDD."

Example Query:

for c in (FSC_Scandinavia)
  return
    encode(
      coverage temp
        over
        $pt t(150190:150200)
      values (count(c[t($pt)]=30)) / (count(c[t($pt)]!=0)) * 100,
      "csv"
    )

The two files attached contain the tomcat output of the same query running with rasdaman 8.3.1-3 and 8.4.0-rc2.2.

Maybe I'm missing a schema or a syntax change. Do you have any idea what could be wrong here?

Thanks, Joachim

Attachments (2)

8.3.1-3.log (48.5 KB) - added by ungarj 4 years ago.
catalina.out with rasdaman 8.3.1-3
8.4.0.rc2.log (52.6 KB) - added by ungarj 4 years ago.
catalina.out with rasdaman 8.4.0-rc2

Download all attachments as: .zip

Change History (20)

Changed 4 years ago by ungarj

catalina.out with rasdaman 8.3.1-3

Changed 4 years ago by ungarj

catalina.out with rasdaman 8.4.0-rc2

comment:1 Changed 4 years ago by abeccati

  • Component changed from DEB to petascope
  • Milestone set to 8.4
  • Owner changed from dmisev to abeccati

comment:2 Changed 4 years ago by ungarj

  • Description modified (diff)

comment:3 Changed 4 years ago by dmisev

  • Owner changed from abeccati to arezaeim
  • Status changed from new to assigned

Yes indeed, the coverage expression is not properly translated to rasql. Alireza can you look at this?

Old

select csv(marray i_i in [150190:150200] values ((count_cells((((c) [*:*,*:*,i_i[0]])=(30))))/(count_cells((((c) [*:*,*:*,i_i[0]])!=(0)))))*(100)) from FSC_Scandinavia AS c

New (notice the iterator variable is not used, only a 0 is used instead:

select csv(marray i_i in [150190:150200] values ((count_cells((((c) [*:*,*:*,0])=(30))))/(count_cells((((c) [*:*,*:*,0])!=(0)))))*(100)) from FSC_Scandinavia AS c

comment:4 Changed 4 years ago by abeccati

  • Milestone changed from 8.4 to 8.4.1

comment:5 Changed 4 years ago by dirk.daems@…

I can confirm that we are experiencing the same problem: iterator variables are not replaced but a zero is used when transforming the WCPS query into a RASQL query.

comment:6 Changed 4 years ago by dmisev

  • Resolution set to fixed
  • Status changed from assigned to closed

comment:7 Changed 4 years ago by dmisev

  • Cc dirk.daems@… added
  • Resolution fixed deleted
  • Status changed from closed to reopened

New bug discovered by dirk:

I have a 3 dimensional (t,x,y) coverage NDVI_SPOT_VGT.
The temporal resolution on axis t is a dekad, which is about 10 days. Each month has 3 dekads so one year has 36 dekads.

When I execute query:

for c in (NDVI_SPOT_VGT)
return encode(
  coverage averagesOverTime
    over $T t(0:10)
      values ( avg(c[x(15:20),y(0:5),t(2+$T*36)] ) ),
  "csv")

it is translated into RASQL command:

select csv(marray i_i in [0:10] values avg_cells((c) [(2)+((i_i[0])*(36)),3696:4256,3696:4256])) from NDVI_SPOT_VGT AS c

This seems to be OK.
But when I specify a range for the time axis (t) as well:

for c in (NDVI_SPOT_VGT)
return encode(
  coverage averagesOverTime
    over $T t(0:10)
      values ( avg(c[x(15:20),y(0:5),t(2+$T*36:4+$T*36)] ) ),
  "csv")

it is translated into RASQL command:

select csv(marray i_i in [0:10] values avg_cells(c[0:0,3696:4256,3696:4256])) from NDVI_SPOT_VGT AS c

So the range specified for axis t (2+$T*36:4+$T*36) is mapped to 0:0 which looks like a bug to me.
Is this a known issue?

comment:8 Changed 4 years ago by dmisev

  • Milestone changed from 8.4.1 to 8.5
  • Summary changed from WCPS queries fail after update to WCPS coverage iterator not translated properly to rasql

comment:9 Changed 4 years ago by pcampalani

  • Keywords coverage iterator condense construct added
  • Summary changed from WCPS coverage iterator not translated properly to rasql to Coverage iterator not translated properly to rasql on WCPS trimmings

Some more info can be found as well in #360 (closed as duplicate).

comment:10 follow-up: Changed 4 years ago by dmisev

Dirk, can you test this query again?

comment:11 in reply to: ↑ 10 Changed 4 years ago by dirk.daems@…

Replying to dmisev:

Dirk, can you test this query again?

Dimitar,

Just tested with the code I fetched from the git repository yesterday, but the problem remains.

Dirk

comment:12 Changed 4 years ago by arezaeim

DimensionIntervalElement? does not handle non-numeric variables, it takes a ScalarExpr? which resolve to a NumericScalarExpression? from which it takes the numeric value "dvalue".
Setting `transformedCoordinates' to false works in case of GRID_CRS slice but but not in this case. Moreover slicing with an arithmetic operation involving a variable reference doesn't work either. This patch deserves a more comprehensive solution on Dimension*Element.java to admit a non-numeric outcome in the subset

comment:13 Changed 4 years ago by pcampalani

transformedCoordinates is meant to differentiate between values already converted for RasQL, or not.
So, setting transformedCoordinates to false as in changeset:63a177a works for this specific case but doesn't make much sense to me :)

comment:14 Changed 4 years ago by dmisev

  • Milestone changed from 8.5 to 9.0

So what's the conclusion, Piero/Alireza?? How do we proceed?

comment:15 Changed 4 years ago by dmisev

  • Cc pbaumann added
  • Owner changed from arezaeim to dmisev
  • Status changed from reopened to assigned

I'll take over the ticket, but I need details on where did you stuck last time (let's sync next week).

comment:16 Changed 4 years ago by dmisev

  • Priority changed from major to critical

comment:17 Changed 4 years ago by dmisev

I submitted patch that fixes it. All WCPS/WCS tests pass so it's good. I had to change the lo/hi in DimensionIntervalElement from BigInteger to String in order to get this working. This also fixed the test for #371 actually, which was only partially correct in the rasql translation.

comment:18 Changed 4 years ago by dmisev

  • Resolution set to fixed
  • Status changed from assigned to closed
Note: See TracTickets for help on using tickets.