wiki:RasdamanStorageBackend

Version 7 (modified by pbaumann, 20 months ago) (diff)

--

Rasdaman storage backend

rasdaman community can store array data in two different ways:

  1. arrays in a file system directory, array metadata in SQLite
  2. all in PostgreSQL: arrays in BLOBs, array metadata in tables

Note: rasdaman enterprise additionally supports access to pre-existing archives of any structure.

The array storage location can be chosen during the configuration step by setting ./configure parameter --with-default-basedb when installing from source; it is fixed in the RPMs and VMs distributed.

Starting with rasdaman v9.1, file system storage is the default to ease work with rasdaman.

Storing arrays in a file system directory

In this storage variant, a particular directory gets designated to hold rasdaman arrays (maintained by rasdaman) and their metadata (maintained by an SQLite instance embedded in rasdaman).

The recommended directory location is $RMANHOME/data/. Administrators may configure this to be a symbolic link to some other location, possibly another filesystem than where $RMANHOMEresides (so as to keep programs and data separate). Further (not recommended) ways of changing the default directory are:

  • by ./configure parameter --with-filedatadir=PATH where PATH is an absolute path
  • by setting variable $RASDATA which must be set in the environment of the rasdaman executables

The data directory will contain the named database, identified by a subdirectory of the same name. Currently only one database is supported, but this may change in future. Default database name, assumed by all tools, is RASBASE. While it can be changed this is not recommended as all tools will need to receive an extra parameter indicating the changed name.

The database name needs to be communicated to rasdaman in the $RMANHOME/etc/rasmgr.conf configuration file. Specifically, the connect string should be an absolute path to the RASBASE database (note that variables are not recognized in the script, therefore $RMANHOMEhas to be spelt out). Assuming the default values described above and a rasdaman installation directory of $RMANHOME=/opt/rasdaman, the corresponding configuration line might look like this:

define dbh rasdaman_host -connect /opt/rasdaman/data/RASBASE

Such an entry gets established automatically when running the create_db.sh script.

Again, for a customized data directory location it is recommended to use a symbolic link, but not modify defaults.

Note: Postgresql currently is still needed by petascope and (deprecated) rasgeo. This dependency is planned to be removed in future.

Storing arrays in PostgreSQL BLOBs

In this storage variant, rasdaman arrays and their metadata are stored in a PostgreSQL database.

First, install and configure PostgreSQL for use with rasdaman.

To deploy rasdaman for using PostgreSQL use

./configure ... --with-default-basedb=postgresql

In $RMANHOME/etc/rasmgr.conf the connect string should be the name of the RASBASE database:

define dbh rasdaman_host -connect RASBASE

The create_db.sh script sets this automatically. It is recommended to keep this value because otherwise this name has to be changed in many places.