Opened 5 years ago

Closed 3 years ago

#169 closed task (fixed)

rasimport to be synchronized with incoming new petascopedb schema

Reported by: pcampalani Owned by: aherzig
Priority: major Milestone: 9.0
Component: rasgeo Version: 8.3
Keywords: rasimport petascopedb schema Cc: herziga@…, pbaumann
Complexity: Medium

Description

petascopedb is going to apply some modification in its tables structure: rasimport will need to correctly fill out the metadata of the imported collections.

Attachments (1)

0001-rasgeo-preview-for-rasdaman-v9.patch (390.3 KB) - added by aherzig 4 years ago.
rasgeo preview patch for rasdaman v9 (created against master 2013-11-26)

Download all attachments as: .zip

Change History (62)

comment:1 Changed 5 years ago by dmisev

Also we need to add command-line options, which will allow to override metadata information, in case rasimport can't figure them out from the input files.

comment:2 Changed 5 years ago by pcampalani

There are several GML-related aspect to be solved before these modifications can be applied, but meanwhile I can post the relevant modifications to the petascopedb schema, so that some implementation can get started, in case.

Any comment and suggestion is appreciated, of course!

  • NEW TABLES
    • ps_irrSeries : explicit mapping of each domain-value of an irregularly spaced axis to its cell index.
      psql#> \d ps_irrseries
                               Table "public.ps_irrseries"
       Column |  Type   |                         Modifiers                         
      --------+---------+-----------------------------------------------------------
       id     | integer | not null default nextval('ps_irrseries_id_seq'::regclass)
       axis   | integer | not null
       value  | text    | not null
       cell   | integer | not null
      Indexes:
          "ps_irrseries_pkey" PRIMARY KEY, btree (id, axis, value, cell)
      Foreign-key constraints:
          "ps_irrseries_axis_fkey" FOREIGN KEY (axis) REFERENCES ps_domain(id) ON DELETE CASCADE
      
      The axis field links the tuple to a certain axis in ps_domain, then cell and value map the association to the rasdaman index. For instance, in case of an irregular time-series of, say, three images:
      psql#> select * from ps_irrSeries where axis=23;
       id | axis |       value      | cell |
      ----+------+------------------+------+
        1 |   23 | 2012-01-01 12:00 |   0  |
        2 |   23 | 2012-01-05 16:30 |   1  |  
        3 |   23 | 2012-01-18 22:05 |   2  |
      
  • MODIFIED TABLES
    • ps_coverage : crs field turned to foreign key (as it was meant to be) imported from ps_crs; this field, previously left empty, will now be used as unique CRS reference of the coverage, independently of its number of axes: a unique compound URI will be used in case. The URI must be stored in ps_crs and referenced by ps_coverage.
      ALTER TABLE ps_coverage DROP COLUMN     crs; -- cast integer from text is not possible: need to drop the column.
      ALTER TABLE ps_coverage ADD  COLUMN     crs integer;
      ALTER TABLE ps_coverage ADD  CONSTRAINT ps_coverage_crs_fkey FOREIGN KEY (crs) REFERENCES ps_crs (id) MATCH FULL;
      
    • ps_domain : removed redundant metadata present in the GML definition of a CRS (i.e. uom, name, type) and rely on a generic alphanumeric value for the extents (now called minValue and maxValue), whether it is a numeric coordinate or a timestamp.
      psql#> \d ps_domain
                                  Table "public.ps_domain"
          Column    |  Type   |                       Modifiers                        
      --------------+---------+--------------------------------------------------------
       id           | integer | not null default nextval('ps_domain_id_seq'::regclass)
       coverage     | integer | not null
       i            | integer | not null
       minvalue     | text    | 
       maxvalue     | text    | 
       isirregular  | boolean | not null default false
      Indexes:
          "ps_domain_pkey" PRIMARY KEY, btree (id)
          "ps_domain_coverage_key" UNIQUE, btree (coverage, i)
      Foreign-key constraints:
          "ps_domain_coverage_fkey" FOREIGN KEY (coverage) REFERENCES ps_coverage(id) ON DELETE CASCADE
      Referenced by:
          TABLE "ps_irrseries" CONSTRAINT "ps_irrseries_axis_fkey" FOREIGN KEY (axis) REFERENCES ps_domain(id) ON DELETE CASCADE
      
      NOTE-1: the i-order is now the only metadata of an axis, and it must reflect the order which an axis is stored in rasdaman. Moreover the easting component should always be stored relatively before the northing axis. In case the direction is not clear (e.g. antarctic imagery?), the axis defined as x (or longitude, etc.), in the GML definition should be put first. This is to make metadata handling less prone to misunderstandings, see http://docs.geotools.org/latest/userguide/library/referencing/order.html for explanations.

NOTE-2: In case a collection has a compound CRS (CCRS), the i-order must also be reflected in the order of concatenation of the single atomic CRSs: e.g. in case of a spatio-temporal collection where the t dimension is stored as first axis in rasdaman, the ISO:8601 URI (e.g. http://.../def/crs/ISO/0/8601) must be put before the spatial CRS in the compound URI.

  • DROPPED TABLES
    • ps_crsDetails : really harmful table, limiting the bboxes to 2D and providing redundant information already visible in ps_domain.
    • ps_cellDomain : the cell-domain extents are now retrieved by Petascope via RasQL queries (sdom).
    • ps_crsSet : the CRS URI is now unique for a coverage, and does not need to be linked to single axes.
    • ps_axisType: this kind of metadata is now in the GML definitions of the CRSs and in Petascope.

comment:3 Changed 5 years ago by dmisev

Piero, I created a dedicated wiki page for this now (and petascope development in general): http://kahlua.eecs.jacobs-university.de/trac/rasdaman/wiki/PetascopeDevGuide#Newdatabaseschema

comment:4 Changed 5 years ago by dmisev

Comment by Alex (I'll allow posting as soon as I figure out how to block the spam btw!):

I'm happy to adjust rasimport to the new schema as well as the coding guidelines, and account for the suggestions Peter made a while ago. There is one aspect, I'd like to add to the discussion: I need support for raster attribute tables (i.e. a reference from individual images (not collections) to tables in the / a data base) for my particular use-case, that's why I came up with the nm_meta table in the first place. What's your opinion on that? For example my rasdaman ImageIO for Orfeo expects this kind of information as well.

comment:5 Changed 5 years ago by dmisev

And my reply:

We have already thought about using image OIDs, instead of collection names, as the connection with coverages in petascope (or perhaps allow both). I actually thought that nm_meta was needed for GDAL-specific info that couldn't be stored in petascopedb.

Piero, do you think this could be incorporated easily in the above schema changes (OIDs along with collection names)?

comment:6 Changed 5 years ago by dmisev

  • Cc beccati@… pbaumann added

And also we could put nm_meta into petascopedb (update_petascopedb.sh), I don't see any reason for them being separate.

Let me include Alan in the discussion, he's upgrading rasgeo with ODBC support, so we should merge all efforts into the new (1.0?) release of rasgeo :)

comment:7 follow-ups: Changed 5 years ago by pcampalani

Hi Alex,
if you need that nm_meta table, we can add it to petascopedb directly as well, of course. There is no image-specific table currently, coverages as meant as collections.
(How should we name it?)

I then will add the OID column to ps_coverage table: is that going to be mandatory? Should still the name of the coverage be unique, or the pair (name,OID) should?

comment:8 in reply to: ↑ 7 ; follow-up: Changed 5 years ago by anonymous

Replying to pcampalani:

(How should we name it?)

maybe ps_rasgeo?

I then will add the OID column to ps_coverage table: is that going to be mandatory? Should still the name of the coverage be unique, or the pair (name,OID) should?

It seems impossible to determine the rasdaman collection name given the MDD OID, so I'd suggest to store the collection name. The ps_coverage then would have:

name (coverage name)
oid  (rasdaman MDD identifier)
coll (rasdaman collection name)

so that we can use then this info when coverage name is requested in WCS/WCPS, in rasql query like

select c
from coll as c
where oid(c) == oid

In ps_coverage, name only must be unique, for the other columns it doesn't matter.

comment:9 in reply to: ↑ 7 Changed 5 years ago by dmisev

coll and oid should be optional for backwards compatibility and simplifying imports. When coll is left out, we could assume that coll = name.

comment:10 Changed 5 years ago by dmisev

Message from Alex (@Alex: you can comment on the trac now, you just need to specify valid email):

I think we should consider two options of how to include nm_meta into
petascope:

a) Just rename nm_meta into ps_rasgeo and make it part of petascope, but
don't have any other changes, such as adding "oid" to ps_coverage.

b) Including nm_meta as ps_rasgeo and only keep the meta information,
which is not provided by other ps-tables.

ps_rasgeo could then look like this

  oid  (rasdaman MDD identifier)
  imagetype (rasdaman MDD type; renamed from "pixeltype")
  attrtable_name (raster attribute table name)
  minx
  maxx
  miny
  maxy
  minz
  maxz
  cellsize_x
  cellsize_y
  cellsize_z

whereas the minx ... cellsize_z fields are not really required, since
the information can
be derived or read from other ps-/ras-tables, they're just handy to have
and save some queries I guess. The stats fields I've never really used,
and having them meant updating them properly as soon as parts of an
image gets updated, so could be dropped. The image type is quite handy
to know for updating especially multi-band images.

And given a ps_coverage structure like

  name (coverage name)
  oid  (rasdaman MDD identifier)
  coll (rasdaman collection name)

would even make each individual image available as coverage, which
wouldn't be the case for a), right?. Only question I have is, is there a
way of configuring which collections/images are actually available via
WCS/WCPS, or are all data sets stored in the petascope DB are
automatically available?
In general I favour option b), especially, if we could configure which
data sets are offered as WCS/WCPS.

comment:11 follow-up: Changed 5 years ago by dmisev

I'm for option b), along with removing the columns from nm_meta which aren't necessary (i.e. information which can be derived from other tables in petascope).

So after removing oid and minx - cellsize_z we're left with imagetype and attrtable_name in the new ps_rasgeo.

Regarding the visibility of coverages, others need this functionality too. So what about adding a column to ps_coverage, to control this (e.g. a boolean visible)?

comment:12 follow-up: Changed 5 years ago by pcampalani

Ok, I agree to add a visible bool in ps_coverageand for the b) solution to ps_rasgeo as well.
Do we want to import the OID from ps_coverage or, Alex, do you prefer to keep it untied and to read the OID directly in ps_rasgeo?

comment:13 Changed 5 years ago by beccati@…

I see just one issue with the approach of keeping the OIDs:

As far as I know a coverage should map to a rasdaman collection, which is a set of "homogeneous" arrays and retrieved data (if you do not specify an OID) is taken form all arrays in the collection.
With rasgeo, each array object is allowed to keep its own, possibly different, CRS information. If we keep that into petascope we have a potential issue of a coverage holding data in different CRS, possibly with different offsets and resolutions.

Two possibilities here:
1) Keep the feature and let petascope publish the pairs (collection, OID) as single coverage;
2) "Remove the feature" in the sense that we have to ensure all array objects in a coverage conform to its single CRS defintion;

PS: Extension to odbc is taking a little longer than expected since unixODBC and Oracle driver seems to somewhat dislike each other, especially with metadata operations.

Alan

comment:14 Changed 5 years ago by pcampalani

I was going to ask why the EPSG/CRS information was not mentioned as field to be kept in ps_rasgeo ?

Regarding the OIDs, this would inherently mean that the petascope collection name would now be not necessarily equal to the rasdaman collection name, so in practice what you, Alan, put as possibility 1).

comment:15 in reply to: ↑ 12 ; follow-up: Changed 5 years ago by dmisev

Replying to pcampalani:

Do we want to import the OID from ps_coverage or, Alex, do you prefer to keep it untied and to read the OID directly in ps_rasgeo?

Let's not duplicate information, if there's oid in ps_coverage there's no reason for oid in ps_rasgeo.

I was going to ask why the EPSG/CRS information was not mentioned as field to be kept in ps_rasgeo ?

Because it's stored elsewhere (in ps_coverage with the schema changes)?

As far as I know a coverage should map to a rasdaman collection, which is a set of "homogeneous" arrays and retrieved data (if you do not specify an OID) is taken form all arrays in the collection.

We discussed this with Piero, and the conclusion is that a coverage can only be one particular MDD, not a collection of MDDs (according to the coverage model in WCS/WCPS). So it is better to tie the petascope coverage to the OID in rasdaman, rather than the collection name. So it's option 1)

comment:16 in reply to: ↑ 15 Changed 5 years ago by pcampalani

Let's not duplicate information, if there's oid in ps_coverage there's no reason
for oid in ps_rasgeo.

Ok.. so OID will be unique in ps_coverage, and will be imported in ps_rasgeo. Should OID (and rasName) be mandatory ?

I was going to ask why the EPSG/CRS information was not mentioned as field to be kept in ps_rasgeo ?

Because it's stored elsewhere (in ps_coverage with the schema changes)?

Yes, but the atomic CRS of a single image/OID is not stored. E.g. in a space-time cube of images you have the compound CRS in ps_coverage, not the spatial CRS of the projected images.

comment:17 Changed 5 years ago by dmisev

A space-time cube is usually one image in rasdaman, not a collection of images.

comment:18 Changed 5 years ago by herziga@…

I agree with what Dimitar suggested, i.e. reducing ps_rasgeo to the data which isn't stored elsewhere, so importing oid from ps_coverage. This means equally for CRS that as long as we can derive individual CRS information for space and time, we shouldn't store separate crs info in ps_rasgeo. The only problem I see is how to appropriately map GDAL crs info to CRS URIs. As Dimitar suggested, we should probably provide rasimport parameters for specifying the CRS info. Disadvantage is that this would probably mean to specify the URI each time you're importing a new image unless there is something smart which could do the mapping. For lazy users and those not worried about correct CRS information, would there be a default URI specifying 'pixel space'? Is the CRS info mandatory for WCS/WCPS? I presume it is.

comment:19 follow-up: Changed 5 years ago by pcampalani

Yes, the individual CRSs can be derived from the compound, I thought it was handy for you but of course we should avoid redundancy.
A CRS info is required for WCS/WCPS, even though until now we used this CRS:1 label for i,j referencing.
Alex, how would a practical case of uncertain gdal2uri CRS mapping?

I see these couple of issues:

  • nD coverage with 1 or more (not temporal) axes not defined by the ESPG CRS: I'm thinking of e.g. a 3D dataset with images piled up on different pressure levels. In this case we thought we could create some template-CRS like:
    http://www.opengis.net/def/crs-compound?
        1=http://www.opengis.net/def/crs/EPSG/0/<whatever>&
        2=http://www.opengis.net/def/crs?
           authority=OGC&
           version=0.1&
           code=Linear1D&
           axisName=Pressure&
           abbreviation=P&
           uom=<EPSG-code-of-hPa-UoM>&
    
    in this case the semantic of this Linear1D axis should be set. For simplicity a Linear1D axis with no further parameters (axisName, uom, etc.) could be set as default. For pixel-space (not just linear) CRSs Dimitar found we could define an Image2D CRS of type ImageCRS [1][2]. http://www.opengis.net/def/crs/OGC/0.1/Image2D could then be our default URI for images without CRS.
  • for spatio-temporal cubes just compound the spatial CRS with ISO:8601 (e.g. http://www.opengis.net/def/crs/ISO/2004/8601) but I am a bit worried we will need to define TemporalCRS definitions as well, GML does not seem to contemplate the use of timestamps that much, see [3].

Specifying the CRS from commandline might be optional. ?

  1. http://www.schemacentral.com/sc/niem21/e-gml32_ImageCRS.html
  2. http://www.schemacentral.com/sc/niem21/e-gml32_cartesianCS-a.html
  3. http://212.201.49.163/trac/wiki/OgcStandardsIssues

comment:20 in reply to: ↑ 19 ; follow-up: Changed 5 years ago by herziga@…

Replying to pcampalani:

I was more concerned about non-standardised crs descriptions/codes by different vendors, i.e. vendor A uses a different description for say WGS84 than vendor B.

comment:21 in reply to: ↑ 20 Changed 5 years ago by pcampalani

Replying to herziga@…:

Ok.. well, what is done in Petascope is to force the XY order either when using the GeoTools library for reprojections and when parsing CRSs definitions. This way an EPSG:4326 with latitude before longitude would still have longitude first, and this should be done by rasimport as well, that is to store the easting as first axis in rasdaman (see NOTE1 of ps_domain section above).

That was my idea actually, so surely we can discuss about it if it is not satisfying!

Are there other problems that may arise on this side?

comment:22 Changed 5 years ago by pcampalani

ISSUE: use the "Kahlua" host as default SECORE reference or not?

It depends on whether we want to 'suggest' a local SECORE installation or not.
This means that with:

http://kahlua.eecs.jacobs-university.de:8080/def/crs/EPSG/0/4326

Petascope can startup, as soon as Kahlua server is running at startup; whereas with:

http://localhost:8080/def/crs/EPSG/0/4326

Petascope can startup (a little faster as well) as soon as SECORE is installed and deployed in the host.

comment:23 Changed 5 years ago by pbaumann

First, kahlua must not be hardcoded. This just happens to be one of our deployments.
Second, why assume a default anyway? The URL provider has to ensure that it works, and it may want to use OGC in the first place, or any other.
What you actually suggest is to split the URL in a normative path and an independent domain location. IMHO we will run into conceptual problems - it is not always a clear splitting; for example, a homebrewn URI (with own server) may choose to not use /def/.
So if we have a fully qualified domain name AFAICS there is no need for any such setting.
However, your discussion shows that the caching mechanism needs more thoughts: how is a URI cached, actually? Remember, as per discussion, this should be a SECORE feature that is transparent to users - they invoke an OGC URL and get it from the local store. (As this is persistent, there is no startup penalty.)

comment:24 Changed 5 years ago by pcampalani

URIs and the relative definition java objects are cached when first time read. The caching system extracts single CRS from compound URI and also recognizes different URIs pointing to a same authority:code CRS avoiding redundancy.

Ok not to hardcode kahlua.

There is no startup penalty, it can come when a unregistered CRS definition needs to be fetched and the server hosting the definition is down.

comment:25 Changed 5 years ago by pbaumann

ok, sounds good.
When a remote server is down we cannot do anything - it is the authoritative source, and we have no rationale for replacing a URI with something else. So perfectly OK to then throw an exception (following a resolver 404 or alike).

comment:26 Changed 5 years ago by pcampalani

Ok, that's fine.

comment:27 follow-up: Changed 5 years ago by pcampalani

Alex,
let me know if you need me to send you a patch for your tests.
When, approximately, do you think you will start to work on this task?

thanks!
Piero

comment:28 in reply to: ↑ 27 Changed 5 years ago by herziga@…

Yeah, it would be good, if you could send me a patch, so I could start working on it.

Alex

Replying to pcampalani:

Alex,
let me know if you need me to send you a patch for your tests.
When, approximately, do you think you will start to work on this task?

thanks!
Piero

comment:29 Changed 5 years ago by pcampalani

A note: collecting information of RASBASE from petascopedb to keep the content of NM_META table is possible, but with an additional psql module called dblink:

[1] http://www.postgresql.org/docs/8.4/static/dblink.html

comment:30 Changed 5 years ago by dmisev

Piero don't spend much time on this, I've just realized it's probably not that critical. The information in nm_meta is only used by rasimport when importing data, and is only relevant when updating existing data (e.g. adding another slice to a 3D cube). We can add an update to migrate it later, or just provide a separate script to do it.

comment:31 Changed 5 years ago by dmisev

I think most of the information stored in nm_meta (like geo bounding box) should be in petascopedb already anyway.

comment:32 Changed 5 years ago by dmisev

  • Cc beccati@… removed

comment:33 follow-up: Changed 5 years ago by pcampalani

Yes, but I believe that updating capabilities are pretty critical.
Anyway, the script is ready, just let's decide quickly if use that in the SQL updates of petascedb or maybe as a separate bash script that does not require that module.

I would update seamlessy from SQL.. opinions?

comment:34 in reply to: ↑ 33 Changed 5 years ago by herziga@…

I agree with Piero to integrate the migration of nm_meta data to ps_* tables into the SQL updates. Also, I'm using another rasdaman client (ImageIO for the Orfeo Toolbox) which makes use of rasgeo's RasdamanHelper2 to read and write image meta data.

comment:35 follow-up: Changed 5 years ago by pcampalani

Anyway I am looking at real cases and I can see sometimes a same NM_META::coll_name (= PS_COVERAGE::rasname) for several NM_META::img_id (= PS_COVERAGE::oid).

That is, I have N oids for one collection name in rasdaman and one coverage name in petascope.

We hereby decided to associated a unique OID to a coverage, so what should I do when this does not happen: just overwrite OID/rasname in ps_coverage as soon as I am reading new records?

One more: can it happen now to find a NM_META::rasname which is different from PS_COVERAGE::name? It does not seems so until now, both looking at the real databases and at rasimport sources.

comment:36 Changed 5 years ago by dmisev

Is the dblink easy to install (apt-get)? If yes let's put it in the regular updates.

comment:37 in reply to: ↑ 35 ; follow-up: Changed 5 years ago by herziga@…

Replying to pcampalani:

We hereby decided to associated a unique OID to a coverage, so what should I do when this does not happen: just overwrite OID/rasname in ps_coverage as soon as I am reading new records?

I suggest you leave the oid/rasname/name combination for the one image already referenced by petascope and add any remaining images in the same collection to petascope with a constructed ps_coverage::name (e.g. '$rasname_$oid') and set the visibility to 'false'. This way, from a client point of view (e.g. WCPS, rasimport), everything appears to be the same as it was before the schema change. Additionally, the user/admin could easily expose the other (additional) images as coverages by simply changing the visibility and perhaps the name in ps_coverage.

One more: can it happen now to find a NM_META::rasname which is different from PS_COVERAGE::name? It does not seems so until now, both looking at the real databases and at rasimport sources.

You're right, you shouldn't find that.

comment:38 in reply to: ↑ 37 Changed 5 years ago by pcampalani

Thanks Alex,
however where is the petascope-coverage-name/OID binding?

And anyway, all, with a unique OID for each coverage, could a coverage in WCS/WCPS still represent e.g. a whole mosaic or a whole time series (of images/cubes)?

If the answer is no, we may move OID and rasName to ps_rasgeo, importing ps_coverage::name as foreign key, letting a coverage be associated with N OIDs and a collection to be spread to M coverages, in case.

(As Alan said, this would mean to additionally ensure that a coverage with N OIDs has the same CRS in each MDD)

comment:39 Changed 5 years ago by dmisev

Please leave oid and rasname in ps_coverage, they are very essential to every coverage so putting them in another table makes less sense.

In WCS/WCPS a coverage is one image/cube, there's no mosaics etc, so one coverage = one oid. There can not be one coverage with N oids.

comment:40 Changed 5 years ago by dmisev

Or were you talking maybe about some pseudo-mosaic that perhaps rasimport creates in the table when -mosaic is used? That should go to ps_rasgeo, but you'd reference the coverage id in ps_coverage.

comment:41 follow-up: Changed 5 years ago by pcampalani

Ok,
I was talking about a group of picture with the same CRS covering different spatial extents and grouped together as a one coverage.
Example 2D:

    y
    .
    | o-----------------o
    | |   .---.         |
    | |   | 1 |         |  
    | |   '---'   .---. |
    | |           | 2 | |
    | |           '---' |
    | o-----------------o
    L_____________________. x

Anyway, I will just let a coverage be bound to one OID and one rasname.

Regarding the suggestion of Alex, the question is where to find the binding between an OID and the name of the coverage: in nm_meta you can have N OIDs linked to the same name.

comment:42 in reply to: ↑ 41 Changed 5 years ago by herziga@…

Piero, you're right, I was a little mistaken with my earlier post/suggestion and I had to go back to my code and old data to find out how it actually works: Currently, rasimport only writes meaningful petascope meta data for single image collections (including updates with -oid), in which case the oid-coverage-name-binding is straight forward. But as soon as a collection contains more than one image, rasimport messes the petascope data up (for the particular collection/image). Therefore, for the migration of nm_meta referenced multi image collections to the new schema, I'd simply ignore all current petascope meta data and re-generate the data based on nm_meta, which contains the correct geospatial information for each image. The coverage names would have to be constructed, e.g. as suggested earlier.

comment:43 Changed 5 years ago by pcampalani

Ok,
so when I read multiple OIDs for one coverage I am going to create N coverages from the single <rasname> to N <rasname>_<oid> in PS_COVERAGE, one for each OID.
That's all folks.

comment:44 Changed 5 years ago by p.baumann@…

...before jumping into schema changes: we are going to be more hesitant with schema changes in future, and will establish a clear overall concept before any patch that affects the petascope schema. Should a schema change be required, as outcome of this discussion, then this will lead to a major release increment and detailed migration documentation. For now, as you can see, we are in the planning phase.
So nothing for tomorrow morning, don't worry about your existing installations.

comment:45 Changed 5 years ago by dmisev

rasimport has been synced, so we can close this ticket?

comment:46 Changed 5 years ago by dmisev

  • Resolution set to fixed
  • Status changed from new to closed

to be reopened in case of further changes are necessary.

comment:47 Changed 5 years ago by pcampalani

  • Resolution fixed deleted
  • Status changed from closed to reopened

petascopedb needs a new model, so we shall postpone this after the new schema is established.

comment:48 Changed 4 years ago by abeccati

  • Milestone set to 9.0

comment:49 in reply to: ↑ 11 ; follow-up: Changed 4 years ago by pcampalani

  • Complexity set to Medium

Hi Dimitar, Alex,

regarding this functionality:

Replying to dmisev:

Regarding the visibility of coverages, others need this functionality too. So what about adding a column to ps_coverage, to control this (e.g. a boolean visible)?

it seems it will not be accepted for the upcoming rasdaman release.
Please start a topic on the mailing list(s) if you wish to discuss on this further. !

comment:50 in reply to: ↑ 49 Changed 4 years ago by aherzig

Replying to pcampalani:

Piero, do you mean it wouldn't be accepted just for the upcoming release (8.4.x) or it wouldn't be accepted at all (i.e. not even for 9.0 and beyond)?

comment:51 Changed 4 years ago by pcampalani

Hi Alex,

I mean "at all" (though I was supporting this visibility switch).
If you have strong opinions about this, you could start a discussion on the mailing list, we are still in time.

comment:52 in reply to: ↑ 8 Changed 4 years ago by pcampalani

Replying to anonymous:

so that we can use then this info when coverage name is requested in WCS/WCPS, in rasql query like

select c
from coll as c
where oid(c) == oid

A little note: here you must use '=' instead of '=='.

comment:53 follow-up: Changed 4 years ago by pcampalani

  • Owner set to aherzig
  • Status changed from reopened to assigned

Hi Alex, can I assign this to you officially?
Then you can accept it when you have to time start on it.

comment:54 in reply to: ↑ 53 Changed 4 years ago by aherzig

Hi Piero,

I'm happy to take the assignment. I've got a running implementation for the feature_petascope branch but I haven't tried it on current master yet. The implementation also addresses ticket #125. Only major thing I haven't worked on yet is the required tests for the patch.
Alex

comment:55 follow-up: Changed 4 years ago by pcampalani

Hi Alex,
did you already implemented rasimport to work with PS9_ tables.. ?
Branch feature_PetascopeSecore has been merged already, so you can work on master now.

Checkout the quick upgrade guide, and feel free to contact me for help (Cc-ing the mailing list(s) if you feel it is appropriate to share, thanks).

comment:56 in reply to: ↑ 55 ; follow-up: Changed 4 years ago by aherzig

did you already implemented rasimport to work with PS9_ tables.. ?

Yes.

Branch feature_PetascopeSecore has been merged already, so you can work on master now.

If you don't mind, I could prepare a preview patch based on current master for you to check, whether it's roughly working as expected.

comment:57 in reply to: ↑ 56 Changed 4 years ago by pcampalani

Replying to aherzig:

did you already implemented rasimport to work with PS9_ tables.. ?

Yes.

Branch feature_PetascopeSecore has been merged already, so you can work on master now.

If you don't mind, I could prepare a preview patch based on current master for you to check, whether it's roughly working as expected.

That's perfect, let me have the patch via email whenever you're fine with it.
(Thanks for your work)

Changed 4 years ago by aherzig

rasgeo preview patch for rasdaman v9 (created against master 2013-11-26)

comment:58 Changed 4 years ago by aherzig

Just uploaded a preview patch for rasgeo for rasdaman v9. The patch addresses tickets #125, #169, and #319.
The patch does not contain any tests.
Alex

comment:59 Changed 4 years ago by pcampalani

Further discussions moved to the mailing list.

comment:60 Changed 3 years ago by pcampalani

A first patch has been applied in the repo: changeset:bed4424.

Several things need to be addressed (see link in comment:59): I think it is better if I close this fat ticket as soon as I can test the patch, and then create new tickets for each single TODO task.

comment:61 Changed 3 years ago by pcampalani

  • Resolution set to fixed
  • Status changed from assigned to closed

rasgeo updated by Alex in changeset:bed4424, changeset:5e74380 and changeset:a79d86c.
rasimport partially integrated in our systemtest (changeset:81011a4).

Several review iteration have been done for this patch, which now reasonably works for 2D rectified datasets and as well 3D irregular series.

See RasgeoUserGuide for further info.

Note: See TracTickets for help on using tickets.