#1100 closed defect (fixed)

Test aligned tiling with 3D schemes

Reported by: dmisev Owned by: dmisev
Priority: major Milestone: 9.2
Component: rasodmg Version: development
Keywords: Cc: pbaumann, vmerticariu, mdumitru
Complexity: Medium

Description

It appears that aligned tiling doesn't quite work as expected, creating more tiles than expected when using a 3D scheme.

Change History (8)

comment:1 Changed 16 months ago by dmisev

  • Cc mdumitru added

I think this has nothing to do with dimension, but with the tile size. If it is not specified, by default it is

"tileSize": "131072"

which takes precedence and restricts the tiling domain given. So for a 1000x1000x1 floating point array with default tile size we get this:

 "baseType": "marray <float>",
 "tileNo": "16",
 "totalSize": "4000000",
 "tiling": {
	"tilingScheme": "aligned",
	"tileSize": "131072",
	"tileConfiguration": "[0:999,0:999,0:0]"",
	"tileDomains":
	[
		"[320:639,960:999,0:0]",
		"[640:959,0:319,0:0]",
		"[640:959,320:639,0:0]",
		"[640:959,640:959,0:0]",
		"[640:959,960:999,0:0]",
		"[960:999,0:319,0:0]",
		"[960:999,320:639,0:0]",
		"[960:999,640:959,0:0]",
		"[960:999,960:999,0:0]",
		"[320:639,640:959,0:0]",
		"[320:639,320:639,0:0]",
		"[320:639,0:319,0:0]",
		"[0:319,960:999,0:0]",
		"[0:319,640:959,0:0]",
		"[0:319,320:639,0:0]",
		"[0:319,0:319,0:0]"
	]
 },

This tileSize is misleading, because it's not really size as in bytes, but number of cells covered by the domain.. as such we have no way to specify a default limit of 4MB for example that is independent of the cell type. We could set a default limit of 2^20 which would translate to 1MB tiles for char arrays, 2MB for short, 4MB for floats, etc.

comment:2 Changed 16 months ago by dmisev

I'm guessing wcst_import.sh specifies by default the tiling scheme domain, but not tile size?

comment:3 Changed 16 months ago by mdumitru

WCSTImport supports any tiling that rasdaman supports, it basically sends the string mentioned in the tiling parameter directly to rasdaman. Usually the parameter is set to something like "ALIGNED [0:1000, 0:1000, 10]"
We can indicate to our users that they need to specify the tile size as well or any other option if we come to the conclusion that this is the expected behavior in rasdaman.

comment:4 Changed 16 months ago by dmisev

Best if it is "ALIGNED .. TILE SIZE 2097152" I guess.

I don't even get it where is the default 131072 coming from though (no references in the code), must be something dynamically calculated.

comment:5 Changed 16 months ago by dmisev

So the ingredients specify by default

"tiling": "ALIGNED [0:500, 0:500]"

I'd suggest to change it to

"tiling": "ALIGNED [0:1023, 0:1023] TILE SIZE 4194304"

comment:6 Changed 16 months ago by mdumitru

I will do this as a first step to alleviate the problems. Thanks for investigating. I'm assuming no solution for 3D objects using ALIGNED tiling, right?

comment:7 Changed 16 months ago by dmisev

Yes, and I have to figure out where is the 131072 coming from.

Then I'll look at making it work right for 3D objects, but this will have to be similar to the regular tiling where the full tiles are initialized as soon as any pixel is touched.

comment:8 Changed 16 months ago by dmisev

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.