Opened 10 months ago

Closed 9 months ago

#1409 closed task (fixed)

Test nectdf import + export

Reported by: vmerticariu Owned by: bphamhuu
Priority: major Milestone: 9.3
Component: rasserver Version: development
Keywords: Cc: abadoi, dmisev, bphamhuu
Complexity: Medium

Description

Task is:

  • download the attached netcdf file and inspect them in a netcdf client (e.g. panoply)
  • import the netcdfs into rasdaman (either 1 band at a time, or several bands)
  • export the data from rasdaman in png and netcdf
  • inspect the results using the same client, check that the files resulting from rasdaman are the same as the original

Examples on how to import netcdf data in rasaman can be found in systemtests/testcases_mandatory/test_netcdf

Attachments (3)

upsize_down.png (103.5 KB) - added by bphamhuu 10 months ago.
netcdf compare to png
netcdf_mirror.png (144.0 KB) - added by bphamhuu 10 months ago.
diffrent_netcdfs.png (67.5 KB) - added by bphamhuu 10 months ago.

Download all attachments as: .zip

Change History (16)

comment:2 Changed 10 months ago by dmisev

Here's some more sample data: https://rsg.pml.ac.uk/shared_files/olcl/data_for_vlad/

Incl. wcst_import ingredients :

comment:3 Changed 10 months ago by bphamhuu

  • Cc bphamhuu added
  • Milestone set to 9.3

Changed 10 months ago by bphamhuu

netcdf compare to png

comment:4 Changed 10 months ago by bphamhuu

  • Owner changed from drusu to bphamhuu
  • Status changed from new to assigned

I see there is a patch for this ticket was uploaded in patch manager and it was failed. The change in this patch with calculation of the Lat (negative direction) leads to different subsets (netCDF is top and PNG is bottom).

http://rasdaman.org/attachment/ticket/1409/upsize_down.png

So I'll take this ticket and check it a bit more careful, thanks Drusu for doing it first.

comment:5 Changed 10 months ago by bphamhuu

So the problem with the reversed image is invalid, sorry, (because the netcdf was flipped then I saw wrong direction, checked by Vlad).

There is a problem now is the netCDF when do subsets (e.g: Lat(-45:45) is missing value 0, then it is shifted by 1 pixel).
e.g: subsets

45, 35, 25, 15, 5, -5, -15, -25 ;

e.g: the unsubsetted result:

 90, 80, 70, 60, 50, 40, 30, 20, 10, 0, -10, -20, -30, -40, -50, -60, -70, -80 ;

Changed 10 months ago by bphamhuu

comment:6 Changed 10 months ago by bphamhuu

@Vlad: if we write the Lat axis top-down (90, 88, ... -90) it will be mirrored in Panoply. If write bottom up (-90, -88, ... 90) then it display correctly.

http://rasdaman.org/attachment/ticket/1409/netcdf_mirror.png

comment:7 Changed 10 months ago by vmerticariu

It doesn't matter. As you see in the left-hand side image, panoply believes that the north pole is at -90 degrees, which is false.

Bottom up is correct for linear negative axes.

comment:8 Changed 10 months ago by bphamhuu

Yes, but in the patch for this ticket it writes top-down (which is in the right-hand side image), then I fixed it to write bottom-up (left-hand side image), you see in both case Panoly displays bottom-up (North is -90 degree in axis number).

maybe I'm confusing with the axis, sorry.

Last edited 10 months ago by bphamhuu (previous) (diff)

comment:9 follow-up: Changed 10 months ago by vmerticariu

The way the image is written by the patch is correct.

Right-hand side = correct

Left-hand side = incorrect

comment:10 in reply to: ↑ 9 Changed 10 months ago by bphamhuu

Replying to vmerticariu:

The way the image is written by the patch is correct.

Right-hand side = correct

Left-hand side = incorrect

yeah, I just realize what you meant after comment, you are rightthanks.

Changed 10 months ago by bphamhuu

comment:11 Changed 10 months ago by bphamhuu

I imported the file http://flanche.net/geodata/nc-201003.nc with this recipe http://pastebin.com/C9bhSiB7

then compare in Panoply (left is encoded netCDF output, right is the input file). There are different

http://rasdaman.org/attachment/ticket/1409/diffrent_netcdfs.png

+ the left image is shifted by 0.21 degree.
+ the left image null value should be -999999 and in recipe set nilValue.
+ also it has an extended bound > -90 (Lat) and > 180 (Long), (not sure it was fixed).

    Lat (measured in degree ) with domain extent from -123.33333333347065 to 90.00000000003335.

    Long (measured in degree ) with domain extent from -180.00000000033336 to 203.99999999997385.

comment:12 Changed 10 months ago by bphamhuu

@Vlad: in http://rasdaman.org/ticket/1409#comment:5 I meant it missed the 0 value, actually, due to the formula current in NetCdfDecodeFactory?:

for (i = 0l i < points; i++) {
 if (axisDirection > 0)
     coord = geoDomMax.subtract(resolution * i)
 else 
     coord = geoDomMin.add(resolution * i) 
}

So if I have a coverage with offset vector is (Long: 10,Lat:-10) and bounding box: (Lat: -90: 90, Long: -180:180), then I subset: Lat(-20:15), it returns a coverage with data: Lat = 15, 5, -5 ;

this is wrong coordinates and what need to do is shifted the input subset to correct geo coordinate (http://codereview.rasdaman.org/D249, wait for your review), so Lat(-20:15) will be shifted to (-20:20) and the netCDF formula above will return data: Lat = 20, 10, 0, -10 ; which is correct points.

comment:13 Changed 9 months ago by dmisev

  • Resolution set to fixed
  • Status changed from assigned to closed

I guess can be closed.

Note: See TracTickets for help on using tickets.