Merge RFC4 (#1865)

This commit is the result of the squashing of rfc4_dev branch in a single
commit. It implements mostly RFC 4 related work.

* Grid handling:
  - remove obsolete and presumably unfinished implementation of grid catalog functionality
  - all grid functionality is in grids.cpp/.hpp
  - vertical and horizontal grid shift: rework to no longer load whole grid into memory
  - remove hgrids and vgrids member from PJ structure, and store them in hgridshift/vgridshift/deformation structures
  - build systems: add optional libtiff dependency. Must be explicitly disabled if not desired
  - add support for horizontal and vertical grids in GeoTIFF, if libtiff is available
  - add GenericShiftGridSet and GenericShiftGrid classes, relying on TIFF grids, that can be used for generic purpose grid-based adjustment
  - add a +proj=xyzgridshift method to perform geocentric translation by grid. Used for French NTF to RGF93 transformation using gr3df97a.tif grid
  - deformation: add support for +grids= for GeoTIFF grids
  - horizontal grid shift: fix failures on points slightly outside a subgrid (fixes #209)

* File management:
  - add a filemanager.cpp/.hpp to deal with file related work
  - test for legacy proj_api.h fileapi
  - proj.h: add proj_context_set_fileapi() and proj_context_set_sqlite3_vfs_name() (fixes #866)
  - add capability to read resource files from the user writable directory

* Network access:
  - build systems: add optional curl dependency
  - add a curl-based default implementation for network related functionality
  - proj.h: add C API to control network functionality, and optionaly provide network callbacks
  - add data/proj.ini with default settings
  - add a SQLite3 local cache of downloaded chunks
  - add proj_is_download_needed() and proj_download_file()

* Use Win32 Unicode APIs and expect all strings to be UTF-8 (fixes #1765)
  For backward compatibility, if PROJ_LIB content is found to be not UTF-8 or
  pointing to a non existing directory, then an attempt at interpretating it
  in the ANSI page encoding is done.
  proj_context_set_search_paths() now assumes strings to be in UTF-8, and
  functions returning paths will also return values in UTF-8.

1 files changed, 102 insertions, 0 deletions

diff --git a/docs/source/usage/network.rst b/docs/source/usage/network.rst
new file mode 100644
index 00000000..295232b0
--- /dev/null
+++ b/docs/source/usage/network.rst
@@ -0,0 +1,102 @@
+.. _network:
+
+================================================================================
+Network capabilities
+================================================================================
+
+.. versionadded:: 7.0
+
+PROJ 7.0 has introduced, per :ref:`rfc4`, the capability to work with grid files
+that are not installed on the local machine where PROJ is executed.
+
+This enables to transparently download the parts of grids that are needed to
+perform a coordinate transformation.
+
+.. _cdn_grids:
+
+CDN of GeoTIFF grids
+--------------------
+
+Files are accessed by default through a CDN (Content Delivery Network),
+accessible through https://cdn.proj.org, that contains :ref:`geodetictiffgrids`
+datasets which are mirrored and managed by the
+https://github.com/OSGeo/proj-datumgrid-geotiff/ GitHub project.
+Files in the CDN are designed to be used by PROJ 7 or later, but any software
+project wishing to use the CDN for shifting support are encouraged to
+participate in the project and leverage the CDN. 
+
+How to enable network capabilities ?
+------------------------------------
+
+This capability assumes that PROJ has been build against `libcurl`, and that
+the user authorizes network access.
+
+Authorizing network access can be done in multiple ways:
+
+  - enabling / uncommenting the ``network = on`` line of :ref:`proj-ini`
+  - definiting the :envvar:`PROJ_NETWORK` environment variable to ON
+  - or using the :cpp:func:`proj_context_set_enable_network` with a
+    ``enabled`` = TRUE value.
+
+.. note::
+
+    Instead of using the `libcurl` implementation, an application using the PROJ
+    API can supply its own network implementation through C function callbacks
+    with :cpp:func:`proj_context_set_network_callbacks`. Enabling network use
+    must still be done with one of the above mentionned method.
+
+Setting endpoint
+----------------
+
+When this is enabled, and a grid is not found in the various locations where
+:ref:`resource files are looked for <resource_file_paths>`, PROJ will then
+attempt at loading the file from a remote server, which defaults to
+https://cdn.proj.org in :ref:`proj-ini`. This location can be changed with
+the :envvar:`PROJ_NETWORK_ENDPOINT` environment variable or with
+:cpp:func:`proj_context_set_url_endpoint`.
+
+Caching
+-------
+
+To avoid repeated access to network, a local cache of downloaded chunks of grids
+is implemented as SQLite3 database, ``cache.db``, stored in the
+:ref:`PROJ user writable directory <user_writable_directory>`.
+
+This local caching is enabled by default (can be changed in :ref:`proj-ini` or
+with :cpp:func:`proj_grid_cache_set_enable`). The default maximum size of the
+cache is 300 MB, which is more than half of the total size of grids available,
+at time of writing. This size can also be customized in :ref:`proj-ini` or
+with :cpp:func:`proj_grid_cache_set_max_size`
+
+Download API
+------------
+
+When on-demand loading of grid is not desirable, the PROJ API also offers the
+capability to download whole grids in the
+:ref:`PROJ user writable directory <user_writable_directory>` by using the
+:cpp:func:`proj_is_download_needed` and :cpp:func:`proj_download_file` functions.
+
+
+Mirroring
+---------
+
+If you are able, you are encouraged to mirror the grids via AWS S3 command line:
+
+::
+
+    aws s3 sync s3://cdn.proj.org .
+
+If direct S3 access is not possible, you can also use wget to locally mirror the
+data:
+
+::
+
+    wget --mirror https://cdn.proj.org/
+
+Acknowledgments
+---------------
+
+The s3://cdn.proj.org bucket is hosted by the
+`Amazon Public Datasets program <https://aws.amazon.com/opendata/public-datasets/>`_.
+CDN services are provided by the AWS Public Dataset team via
+`CloudFront <https://aws.amazon.com/cloudfront/>`_