Opened 5 months ago

Closed 5 months ago

#1537 closed defect (fixed)

Profile petascope, rasj for excessive RAM usage

Reported by: dmisev Owned by: dmisev
Priority: critical Milestone: 9.4
Component: petascope Version: development
Keywords: Cc: vmerticariu
Complexity: Medium

Description

It seems that petascope uses more memory than it theoretically should. The following query selects a 1GB PNG from rasdaman:

for b4 in(s2a_l2a_t32tps_B04_10m), b8 in (s2a_l2a_t32tps_B08_10m)
return encode(
 (b8[ansi("2016-11-08T15:47:44.000Z")] - b4[ansi("2016-11-08T15:47:44.000Z")]) / 
 (b8[ansi("2016-11-08T15:47:44.000Z")] + b4[ansi("2016-11-08T15:47:44.000Z")])
,"png")

however, Tomcat consumes at least 4GB, even though petascope needs to transfer only 1GB in total from rasdaman to the client.

Change History (8)

comment:1 Changed 5 months ago by bphamhuu

  • Summary changed from Profile petascope to Rasserver takes 4.5 GB for a request with result in 40 MB.

The problem is not from Petascope as Rasserver takes 4.5 GB Ram (monitored by htop and System Monitor) and the result returns only 40 MB.

the coverages were imported with 1 band for 1 coverage (band04: 113.6 MB and band08: 135.3 MB) JPG2 and the WCPS query is

for c in (sentinel2_10m_B04), d in (sentinel2_10m_B08)
return encode(
 (c - d) / 
 (c + d)
,"png")

and Rasql query is

SELECT encode( ( c - d )  /  ( c + d ) , "PNG" , "{\"geoReference\":{\"crs\":\"EPSG:32633\",\"bbox\":{\"xmin\":300000.0,\"ymin\":5290200.0,\"xmax\":409800.0,\"ymax\":5400000.0}}}") FROM sentinel2_10m_B04 AS c,sentinel2_10m_B08 AS d

comment:2 Changed 5 months ago by bphamhuu

  • Owner changed from bphamhuu to dmisev
  • Status changed from new to assigned

comment:3 Changed 5 months ago by bphamhuu

  • Summary changed from Rasserver takes 4.5 GB for a request with result in 40 MB. to Petascope profile

comment:4 Changed 5 months ago by dmisev

  • Owner changed from dmisev to bphamhuu
  • Summary changed from Petascope profile to Profile petascope for excessive RAM usage

comment:5 Changed 5 months ago by dmisev

  • Milestone changed from 10.0 to 9.4
  • Priority changed from major to critical

comment:6 Changed 5 months ago by bphamhuu

There is a problem with memory usage in rasj library for Tomcat which occupies the byte array multiple times from the rasql result (e.g: a rasql returns 600 MB Tiff file, Tomcat will need to set heap maximum memory at least 2.4 GB). It will need to remove the initialization unnecessarily in rasj for storing result.

comment:7 Changed 5 months ago by bphamhuu

  • Owner changed from bphamhuu to dmisev

comment:8 Changed 5 months ago by bphamhuu

  • Resolution set to fixed
  • Status changed from assigned to closed
  • Summary changed from Profile petascope for excessive RAM usage to Profile petascope, rasj for excessive RAM usage
Note: See TracTickets for help on using tickets.