#1231 closed defect (invalid)

Rasql_Wrong negative value in calculation

Reported by: bphamhuu Owned by:
Priority: major Milestone: 10.0
Component: undecided Version: development
Keywords: WCPS, wrong value, Rasql, negative number Cc: dmisev
Complexity: Medium

Description

WCPS query

for c in (rgb) return encode( (unsigned char)(c.red - c.blue)[i(0:0), j(0:0)] , "csv")

with c.red[0:0,0:0] = 119 and c.blue[0:0,0:0] = 248

then (c.red - c.blue)[i(0:0), j(0:0)] = 119 - 248 = 127 ?

Change History (3)

comment:1 Changed 20 months ago by dmisev

But you do a cast to (unsigned char), what should be the proper result?

comment:2 Changed 20 months ago by bphamhuu

Sorry, actually the query does not need encode in "unsigned char", example below

for c in (rgb) return encode(  (c.red  - 248)[i(0:0), j(0:0)] , "csv")

will return {-129}

but

for c in (rgb) return encode(  (c.red  - c.blue)[i(0:0), j(0:0)] , "csv")

will return {127}

and I expected it should be negative number.

comment:3 Changed 17 months ago by dmisev

  • Resolution set to invalid
  • Status changed from new to closed

The original types are unsigned char of c.red and c.blue, the minus operation doesn't automatically expand the types. You should cast both operands to short for example for this exapmle to work.

for c in (rgb) return encode(  ((short)c.red  - (short)c.blue)[i(0:0), j(0:0)] , "csv")
Note: See TracTickets for help on using tickets.