Opened 5 years ago

Last modified 3 years ago

#246 reopened enhancement

Getting information about arrays

Reported by: dmisev Owned by: dmisev
Priority: minor Milestone: Future
Component: qlparser Version: 8.3
Keywords: Cc: pbaumann
Complexity: Medium

Description (last modified by dmisev)

A function which returns information about an array in the database would be very useful. The information we can extract right now is limited to:

  • array dimensionality and extents
  • array type by specifying --type to rasql

Further info that would be highly useful:

  • tiling scheme and configuration, number of tiles
  • the index used
  • storage format
  • size in bytes of an array; often arrays are only partially materialized, so this would be useful information.

This could be implemented as an info function, which similarly to RAS_COLLECTIONNAMES returns a char array with the above (and possibly further) information.

Change History (16)

comment:1 Changed 5 years ago by dmisev

  • Description modified (diff)

comment:2 Changed 5 years ago by dmisev

  • Description modified (diff)

comment:3 Changed 5 years ago by dmisev

  • Description modified (diff)

comment:4 Changed 5 years ago by dmisev

Proposed solution:

$ rasql -q 'select info(c) from mr as c' --out string
rasql: rasdaman query tool v1.0, rasdaman v8.3.1-g1ecc2e1 -- generated on 07.12.2012 16:06:36.
opening database RASBASE at localhost:7001...ok
Executing retrieval query...ok
Query result collection has 3 element(s):
  Result object 1: Oid: 342529; Tiles: 1; Object size: 54016B; Tiling scheme: no tiling; Tile size: 2097152; Tile configuration: [0:511,0:511]; Index type: rpt_index; Index size: 0; PCT max: 4096B; PCT min: 2048B; Base type: marray <char, 2>
  Result object 2: Oid: 343041; Tiles: 1; Object size: 54016B; Tiling scheme: no tiling; Tile size: 2097152; Tile configuration: [0:511,0:511]; Index type: rpt_index; Index size: 0; PCT max: 4096B; PCT min: 2048B; Base type: marray <char, 2>
  Result object 3: Oid: 343553; Tiles: 1; Object size: 54016B; Tiling scheme: no tiling; Tile size: 2097152; Tile configuration: [0:511,0:511]; Index type: rpt_index; Index size: 0; PCT max: 4096B; PCT min: 2048B; Base type: marray <char, 2>
rasql done.

comment:5 Changed 5 years ago by dmisev

Alternative JSON output encoding:

$ rasql -q 'select info(c) from mr as c' --out string
rasql: rasdaman query tool v1.0, rasdaman v8.3.1-g8897c51 -- generated on 07.12.2012 16:06:36.
opening database RASBASE at localhost:7001...ok
Executing retrieval query...ok
Query result collection has 4 element(s):
  Result object 1: {
 "oid": "150529",
 "baseType": "marray <char>",
 "tileNo": "1",
 "totalSize": "54016B",
 "tiling": {
	"tilingScheme": "no_tiling",
	"tileSize": "2097152",
	"tileConfiguration": "[0:511,0:511]"
 },
 "index": {
	"type": "rpt_index",
	"indexSize": "0",
	"PCTmax": "4096B",
	"PCTmin": "2048B"
 }
}

comment:6 Changed 5 years ago by pbaumann

  • Milestone set to 8.4

comment:7 Changed 5 years ago by dmisev

  • Resolution set to fixed
  • Status changed from new to closed

Fixed in changeset:ce08b30e549711685a044384f867831352c51276

Example use:

   $ rasql -q 'select dbinfo(eobs) from eobs' --out string
   rasql: rasdaman query tool v1.0, rasdaman osgeolive.5-24-g3ee4345 -- generated on 12.02.2013 09:39:11.
   opening database RASBASE at localhost:7001...ok
   Executing retrieval query...ok
   Query result collection has 1 element(s):
      Result object 1: {
     "oid": "111617",
     "baseType": "marray <short, 5>",
     "tileNo": "6344",
     "totalSize": "1063437888",
     "tiling": {
        "tilingScheme": "aligned",
        "tileSize": "4194304",
        "tileConfiguration": "[0:1023,0:1023]"
     },
     "index": {
        "type": "rpt_index",
        "PCTmax": "4096",
        "PCTmin": "2048"
     }
   }
   rasql done. 

comment:8 Changed 5 years ago by pbaumann

  • Resolution fixed deleted
  • Status changed from closed to reopened

Let us reconsider syntax to make it more database style. Generalizing the idea, we might think of cases like:

DESCRIBE COLLECTION collName
DESCRIBE ARRAY oid
DESCRIBE TYPE typeName

comment:9 Changed 5 years ago by pbaumann

  • Milestone changed from 8.4 to 9.0
  • Priority changed from major to minor

comment:10 Changed 5 years ago by pbaumann

  • Component changed from applications to qlparser

comment:11 Changed 4 years ago by dirk.daems@…

  • Complexity set to Medium

The output of the tile configuration doesn't seem to be correct in version 8.4 rc2:

[rasdaman@epod8 Biopar]$ rasql -q "select dbinfo(c) from Biopar_Africa_LAI as c" --out string
rasql: rasdaman query tool v1.0, rasdaman v8.4.0rc.2-3-g27a914b -- generated on 20.03.2013 11:53:11.
opening database RASBASE at localhost:7001...ok
Executing retrieval query...ok
Query result collection has 1 element(s):
  Result object 1: {
 "oid": "3585",
 "baseType": "marray <char, 3>",
 "tileNo": "2592",
 "totalSize": "2890782756",
 "tiling": {
	"tilingScheme": "aligned",
	"tileSize": "4194304",
	"tileConfiguration": "[0:1023,0:1023]"
 },
 "index": {
	"type": "rpt_index",
	"PCTmax": "4096",
	"PCTmin": "2048"
 }
}
rasql done.

In this case, it should be:

...
"tileConfiguration": "[0:3,0:1023,0:1023]"
...

Which would also match the tile size: 4*1021*1024 = 4194304.

comment:12 Changed 4 years ago by dmisev

Did you specify the tiling yourself when initializing the object?

The above tiling conf matches better the totalSize / tileNo

comment:13 Changed 4 years ago by dirk.daems@…

We used initmdd to initialize the collection and to specify the tiling strategy, but the tile configuration shown by dbinfo() doesn't match the one used with initmdd.

comment:14 Changed 4 years ago by dmisev

initmdd may not be completely compatible with rasql. You should switch to using rasql for the initialization too (tiling has been fixed in the meantime), see wiki:Tiling

comment:15 Changed 3 years ago by dmisev

  • Milestone changed from 9.0.x to 9.1

comment:16 Changed 3 years ago by dmisev

  • Milestone changed from 9.1 to Future
Note: See TracTickets for help on using tickets.