Merge pull request #1813 from rouault/rfc4_network

[RFC4_dev] Add networking capabilities

7 files changed, 163 insertions, 3 deletions

diff --git a/docs/source/apps/cs2cs.rst b/docs/source/apps/cs2cs.rst
index d7f0e9ad..a4069472 100644
--- a/docs/source/apps/cs2cs.rst
+++ b/docs/source/apps/cs2cs.rst
@@ -175,6 +175,14 @@ normally be in DMS format (use ``-f %.12f`` for decimal degrees with 12 decimal
 places), while projected (cartesian) coordinates will be in linear
 (meter, feet) units.
 
+Use of remote grids
+-------------------
+
+.. versionadded:: 7.0.0
+
+If the :envvar:`PROJ_NETWORK` environment variable is set to ``ON``,
+:program:`cs2cs` will attempt to use remote grids stored on CDN (Content
+Delivery Network) storage, when they are not available locally.
 
 Examples
 ********
diff --git a/docs/source/apps/projinfo.rst b/docs/source/apps/projinfo.rst
index 6236056d..a9b6aecc 100644
--- a/docs/source/apps/projinfo.rst
+++ b/docs/source/apps/projinfo.rst
@@ -20,7 +20,7 @@ Synopsis
     |    [[--area name_or_code] | [--bbox west_long,south_lat,east_long,north_lat]]
     |    [--spatial-test contains|intersects]
     |    [--crs-extent-use none|both|intersection|smallest]
-    |    [--grid-check none|discard_missing|sort] [--show-superseded]
+    |    [--grid-check none|discard_missing|sort|known_available] [--show-superseded]
     |    [--pivot-crs always|if_no_direct_transformation|never|{auth:code[,auth:code]*}]
     |    [--boundcrs-to-wgs84]
     |    [--main-db-path path] [--aux-db-path path]*
@@ -148,12 +148,13 @@ The following control parameters can appear in any order:
 
     .. note:: only used for coordinate operation computation
 
-.. option:: --grid-check none|discard_missing|sort
+.. option:: --grid-check none|discard_missing|sort|known_available
 
     Specify how the presence or absence of a horizontal or vertical shift grid
     required for a coordinate operation affects the results returned when
     researching coordinate operations between 2 CRS.
-    The default strategy is ``sort``: in that case, all candidate
+    The default strategy is ``sort`` (if :envvar:`PROJ_NETWORK` is not defined).
+    In that case, all candidate
     operations are returned, but the actual availability of the grids is used
     to determine the sorting order. That is, if a coordinate operation involves
     using a grid that is not available in the PROJ resource directories
@@ -163,6 +164,9 @@ The following control parameters can appear in any order:
     this returns the results as if all the grids where available.
     The ``discard_missing`` strategy discards results that involve grids not
     present in the PROJ resource directories.
+    The ``known_available`` strategy discards results that involve grids not
+    present in the PROJ resource directories and that are not known of the CDN.
+    This is the default strategy is :envvar:`PROJ_NETWORK` is set to ``ON``.
 
     .. note:: only used for coordinate operation computation
 
diff --git a/docs/source/operations/transformations/index.rst b/docs/source/operations/transformations/index.rst
index d4bbf4e0..6bd503d4 100644
--- a/docs/source/operations/transformations/index.rst
+++ b/docs/source/operations/transformations/index.rst
@@ -19,3 +19,4 @@ systems are based on different datums.
    molobadekas
    hgridshift
    vgridshift
+   xyzgridshift
diff --git a/docs/source/operations/transformations/xyzgridshift.rst b/docs/source/operations/transformations/xyzgridshift.rst
new file mode 100644
index 00000000..c49b7592
--- /dev/null
+++ b/docs/source/operations/transformations/xyzgridshift.rst
@@ -0,0 +1,91 @@
+.. _xyzgridshift:
+
+================================================================================
+Geocentric grid shift
+================================================================================
+
+.. versionadded:: 7.0.0
+
+Geocentric translation using a grid shift
+
++-----------------+-------------------------------------------------------------------+
+| **Domain**      | 3D                                                                |
++-----------------+-------------------------------------------------------------------+
+| **Input type**  | Cartesian coordinates                                             |
++-----------------+-------------------------------------------------------------------+
+| **Output type** | Cartesian coordinates                                             |
++-----------------+-------------------------------------------------------------------+
+
+Perform a geocentric translation by bilinear interpolation of dx, dy, dz
+translation values from a grid. The grid is referenced against either the
+2D geographic CRS corresponding to the input (or sometimes output) CRS.
+
+This method is described (in French) in :cite:`NTF_88`
+and as EPSG operation methode code 9655 in :cite:`IOGP2018` (§2.4.4.1.1
+France geocentric interpolation).
+
+The translation in the grids are added to the input coordinates in the forward direction,
+and subtracted in the reverse direction.
+By default (if grid_ref=input_crs), in the forward direction, the input coordinates
+are converted to their geographic equivalent to directly read and interpolate from
+the grid. In the reverse direction, an iterative method is used to be able to find
+the grid locations to read.
+If grid_ref=output_crs is used, then the reverse strategy is applied: iterative
+method in the forward direction, and direct read in the reverse direction.
+
+Example
+-------------------------------------------------------------------------------
+
+NTF to RGF93 transformation using gr3df97a.tif grid
+
+::
+
+    +proj=pipeline
+        +step +proj=push +v_3
+        +step +proj=cart +ellps=clrk80ign
+        +step +proj=xyzgridshift +grids=gr3df97a.tif +grid_ref=output_crs +ellps=GRS80
+        +step +proj=cart +ellps=GRS80 +inv
+        +step +proj=pop +v_3
+
+Parameters
+################################################################################
+
+The ellipsoid parameters should be the ones consistent with ``grid_ref``.
+They are used to perform a geocentric to geographic conversion to find the
+translation parameters.
+
+
+Required
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+.. include:: ../options/ellps.rst
+
+.. option:: +grids=<list>
+
+    Comma-separated list of grids to load. If a grid is prefixed by an `@` the
+    grid is considered optional and PROJ will the not complain if the grid is
+    not available.
+
+    Grids are expected to be in GeoTIFF format. If no metadata is provided,
+    the first, second and third samples are assumed to be the geocentric
+    translation along X, Y and Z axis respectively, in metres.
+
+Optional
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+.. option:: +grid_ref=input_crs/output_crs
+
+    Specify in which CRS the grid is referenced to. The default value is
+    input_crs. That is the grid is referenced in the geographic CRS corresponding
+    to the input geocentric CRS.
+
+    If output_crs is specified, the grid is referenced in the geographic CRS corresponding
+    to the output geocentric CRS. This is for example the case for the French
+    gr3df97a.tif grid converting from NTF to RGF93, but referenced against RGF93.
+    Thus in the forward direction (NTF->RGF93), an iterative method is used to find
+    the appropriate shift.
+
+.. option:: +multiplier=<value>
+
+    Specify the multiplier to apply to the grid values. Defaults to 1.0
+
diff --git a/docs/source/references.bib b/docs/source/references.bib
index 0668e10d..8e527ea1 100644
--- a/docs/source/references.bib
+++ b/docs/source/references.bib
@@ -230,6 +230,15 @@
   Doi                      = {10.2312/PE/PG/PG2012short/005-010}
 }
 
+@TechReport{NTF_88,
+  Title                    = {Grille de parametres de transformation de coordonnees - GR3DF97A - Notice d'utilisation},
+  Author                   = {IGN},
+  Institution              = {Service de Geodesie et Nivellement, Institut Geographique National},
+  Year                     = {1997},
+
+  Url                      = {https://geodesie.ign.fr/contenu/fichiers/documentation/algorithmes/notice/NTG_88.pdf}
+}
+
 @TechReport{ONeilLaubscher1976,
   Title                    = {Extended Studies of a Quadrilateralized Spherical Cube Earth Data Base},
   Author                   = {E. M. O'Neill and R. E. Laubscher},
diff --git a/docs/source/resource_files.rst b/docs/source/resource_files.rst
index 13b9386d..ea02fd4b 100644
--- a/docs/source/resource_files.rst
+++ b/docs/source/resource_files.rst
@@ -40,6 +40,11 @@ The following paths are checked in order:
   PROJ installation is not moved somewhere else.
 - The current directory
 
+When networking capabilities are enabled, either by API with the
+:c:func:`proj_context_set_enable_network` function or when the
+:envvar:`PROJ_NETWORK` environment variable is set to ``ON``, PROJ will
+attempt to use remote grids stored on CDN (Content Delivery Network) storage.
+
 .. _proj-db:
 
 proj.db
@@ -49,6 +54,30 @@ A proj installation includes a SQLite database of transformation information
 that must be accessible for the library to work properly.  The library will
 print an error if the database can't be found.
 
+proj.ini
+-------------------------------------------------------------------------------
+
+.. versionadded:: 7.0
+
+proj.ini is a text configuration file, mostly dedicated at setting up network
+related parameters.
+
+Its default content is:
+
+::
+
+    [general]
+    ; Lines starting by ; are commented lines.
+    ;
+
+    ; Network capabilities disabled by default.
+    ; Can be overriden with the PROJ_NETWORK=ON environment variable.
+    ; network = on
+
+    ; Can be overriden with the PROJ_NETWORK_ENDPOINT environment variable.
+    cdn_endpoint = https://cdn.proj.org
+
+
 Transformation grids
 -------------------------------------------------------------------------------
 
diff --git a/docs/source/usage/environmentvars.rst b/docs/source/usage/environmentvars.rst
index 457432a0..24ae4a45 100644
--- a/docs/source/usage/environmentvars.rst
+++ b/docs/source/usage/environmentvars.rst
@@ -56,3 +56,21 @@ done by setting the variable with no content::
     Set the debug level of PROJ. The default debug level is zero, which results
     in no debug output when using PROJ. A number from 1-3, whit 3 being the most
     verbose setting.
+
+.. envvar:: PROJ_NETWORK
+
+    .. versionadded:: 7.0.0
+
+    If set to ON, enable the capability to use remote grids stored on CDN
+    (Content Delivery Network) storage, when grids are not available locally.
+    Alternatively, the :c:func:`proj_context_set_enable_network` function can
+    be used.
+
+.. envvar:: PROJ_NETWORK_ENDPOINT
+
+    .. versionadded:: 7.0.0
+
+    Define the endpoint of the CDN storage. Normally defined through the proj.ini
+    configuration file locale in PROJ_LIB.
+    Alternatively, the :c:func:`proj_context_set_url_endpoint` function can
+    be used.