[vcpkg manifest] Add documentation! (#13488)

* [vcpkg docs] add docs for manifest files

These are just for the maintainer docs,
not user docs.

* [vcpkg] EBNF-ify platform expression parsing

this modifies nothing about what strings are accepted or rejected,
it just moves stuff around.

also adds tests.

* [vcpkg docs] add manifest mode example

* [wip] docs for augustin

also fix tabs

* [vcpkg manifest] switch to using maps for features

* Apply suggestions from code review

* un-experimentize format-manifest

* flesh out the user manifest mode docs

* CRs

* billy CRs

* final personal pass-thru

4 files changed, 261 insertions, 43 deletions

diff --git a/docs/examples/manifest-mode-cmake.md b/docs/examples/manifest-mode-cmake.md
new file mode 100644
index 000000000..7087da020
--- /dev/null
+++ b/docs/examples/manifest-mode-cmake.md
@@ -0,0 +1,199 @@
+# Manifest Mode: CMake Example
+
+We would like to add vcpkg manifest support to an existing cmake project!
+Let's create a simple project that prints the fibonacci sequence up to a certain number,
+using some common dependencies.
+
+## Initial Layout
+
+Let's create the following file layout:
+
+```no-highlight
+fibo/
+  src/
+    main.cxx
+  CMakeLists.txt
+```
+
+And we wish to use [fmt](https://github.com/fmtlib/fmt), [range-v3](https://github.com/ericniebler/range-v3),
+and [cxxopts](https://github.com/jarro2783/cxxopts).
+
+Let's write our `CMakeLists.txt` first:
+
+```cmake
+cmake_minimum_required(VERSION 3.15)
+
+project(fibo CXX)
+
+find_package(fmt REQUIRED)
+find_package(range-v3 REQUIRED)
+find_package(cxxopts REQUIRED)
+
+add_executable(fibo src/main.cxx)
+
+target_link_libraries(fibo
+  PRIVATE
+    fmt::fmt
+    range-v3::range-v3
+    cxxopts::cxxopts)
+```
+
+And then we should add `main.cxx`:
+
+```cxx
+#include <cxxopts.hpp>
+#include <fmt/format.h>
+#include <range/v3/view.hpp>
+
+namespace view = ranges::view;
+
+int fib(int x) {
+  int a = 0, b = 1;
+
+  for (int it : view::repeat(0) | view::take(x)) {
+    (void)it;
+    int tmp = a;
+    a += b;
+    b = tmp;
+  }
+
+  return a;
+}
+
+int main(int argc, char** argv) {
+  cxxopts::Options options("fibo", "Print the fibonacci sequence up to a value 'n'");
+    options.add_options()
+      ("n,value", "The value to print to", cxxopts::value<int>()->default_value("10"));
+
+  auto result = options.parse(argc, argv);
+  auto n = result["value"].as<int>();
+
+  for (int x : view::iota(1) | view::take(n)) {
+    fmt::print("fib({}) = {}\n", x, fib(x));
+  }
+}
+```
+
+This is a simple project of course, but it should give us a clean project to start with.
+Let's try it out!
+
+Let's assume you have `fmt`, `range-v3`, and `cxxopts` installed with vcpkg classic mode;
+then, you can just do a simple:
+
+```cmd
+D:\src\fibo> cmake -B build -S . -DCMAKE_TOOLCHAIN_FILE=D:\src\vcpkg\scripts\buildsystems\vcpkg.cmake
+-- Building for: Visual Studio 16 2019
+-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19041.
+-- The CXX compiler identification is MSVC 19.27.29111.0
+-- Detecting CXX compiler ABI info
+-- Detecting CXX compiler ABI info - done
+-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/cl.exe - skipped
+-- Detecting CXX compile features
+-- Detecting CXX compile features - done
+-- Configuring done
+-- Generating done
+-- Build files have been written to: D:/src/fibo/build
+D:\src\fibo> cmake --build build
+Microsoft (R) Build Engine version 16.7.0+b89cb5fde for .NET Framework
+Copyright (C) Microsoft Corporation. All rights reserved.
+
+  Checking Build System
+  Building Custom Rule D:/src/fibo/CMakeLists.txt
+  main.cxx
+  The contents of <span> are available only with C++20 or later.
+  fibo.vcxproj -> D:\src\fibo\build\Debug\fibo.exe
+  Building Custom Rule D:/src/fibo/CMakeLists.txt
+```
+
+And now we can try out the `fibo` binary!
+
+```cmd
+D:\src\fibo> .\build\Debug\fibo.exe -n 7 
+fib(1) = 1
+fib(2) = 1
+fib(3) = 2
+fib(4) = 3
+fib(5) = 5
+fib(6) = 8
+fib(7) = 13
+```
+
+it works!
+
+## Converting to Manifest Mode
+
+We now wish to use manifest mode, so all of our dependencies are managed for us! Let's write a `vcpkg.json`:
+
+```json
+{
+  "name": "fibo",
+  "version-string": "0.1.0",
+  "dependencies": [
+    "cxxopts",
+    "fmt",
+    "range-v3"
+  ]
+}
+```
+
+Let's delete the build directory and rerun the build:
+
+```cmd
+D:\src\fibo> rmdir /S /Q build
+D:\src\fibo> cmake -B build -S . -DCMAKE_TOOLCHAIN_FILE=D:\src\vcpkg\scripts\buildsystems\vcpkg.cmake
+-- Running vcpkg install
+Detecting compiler hash for triplet x64-windows...
+The following packages will be built and installed:
+    cxxopts[core]:x64-windows
+    fmt[core]:x64-windows
+    range-v3[core]:x64-windows
+Starting package 1/3: cxxopts:x64-windows
+Building package cxxopts[core]:x64-windows...
+Using cached binary package: C:\Users\me\AppData\Local\vcpkg/archives\d2\d2d1e5302cdfefef2fd090d8eda84cc0c1fbe6f1.zip
+Building package cxxopts[core]:x64-windows... done
+Installing package cxxopts[core]:x64-windows...
+Installing package cxxopts[core]:x64-windows... done
+Elapsed time for package cxxopts:x64-windows: 50.64 ms
+Starting package 2/3: fmt:x64-windows
+Building package fmt[core]:x64-windows...
+Using cached binary package: C:\Users\me\AppData\Local\vcpkg/archives\bf\bf00d5214e912d71414b545b241f54ef87fdf6e5.zip
+Building package fmt[core]:x64-windows... done
+Installing package fmt[core]:x64-windows...
+Installing package fmt[core]:x64-windows... done
+Elapsed time for package fmt:x64-windows: 225 ms
+Starting package 3/3: range-v3:x64-windows
+Building package range-v3[core]:x64-windows...
+Using cached binary package: C:\Users\me\AppData\Local\vcpkg/archives\fe\fe2cdedef6953bf954e8ddca471bf3cc8d9b06d7.zip
+Building package range-v3[core]:x64-windows... done
+Installing package range-v3[core]:x64-windows...
+Installing package range-v3[core]:x64-windows... done
+Elapsed time for package range-v3:x64-windows: 1.466 s
+
+Total elapsed time: 1.742 s
+
+-- Running vcpkg install - done
+-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19041.
+-- The CXX compiler identification is MSVC 19.27.29111.0
+-- Detecting CXX compiler ABI info
+-- Detecting CXX compiler ABI info - done
+-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/cl.exe - skipped
+-- Detecting CXX compile features
+-- Detecting CXX compile features - done
+-- Configuring done
+-- Generating done
+-- Build files have been written to: D:/src/fibo/build
+D:\src\fibo> cmake --build build
+Microsoft (R) Build Engine version 16.7.0+b89cb5fde for .NET Framework
+Copyright (C) Microsoft Corporation. All rights reserved.
+
+  Checking Build System
+  Building Custom Rule D:/src/fibo/CMakeLists.txt
+  main.cxx
+  The contents of <span> are available only with C++20 or later.
+  fibo.vcxproj -> D:\src\fibo\build\Debug\fibo.exe
+  Building Custom Rule D:/src/fibo/CMakeLists.txt
+```
+
+You can see that with just a _single file_, we've changed over to manifests without _any_ trouble.
+The build system doesn't change _at all_! We just add a `vcpkg.json` file, delete the build directory,
+and reconfigure. And we're done!
diff --git a/docs/examples/packaging-github-repos.md b/docs/examples/packaging-github-repos.md
index 61e1f9a5d..287354ff2 100644
--- a/docs/examples/packaging-github-repos.md
+++ b/docs/examples/packaging-github-repos.md
@@ -1,18 +1,23 @@
 ## Packaging Github Repos Example: libogg

-### Create the CONTROL file

-The `CONTROL` file is a simple set of fields describing the package's metadata.

-

-*For libogg, we'll create the file `ports\libogg\CONTROL` with the following contents:*

-```no-highlight

-Source: libogg

-Version: 1.3.3

-Description: Ogg is a multimedia container format, and the native file and stream format for the Xiph.org multimedia codecs.

+### Create the manifest file

+The manifest file (called `vcpkg.json`) is a json file describing the package's metadata.

+

+For libogg, we'll create the file `ports/libogg/vcpkg.json` with the following content:

+

+```json

+{

+  "name": "libogg",

+  "version-string": "1.3.3",

+  "description": "Ogg is a multimedia container format, and the native file and stream format for the Xiph.org multimedia codecs."

+}

 ```

 

+You can format the manifest file to our specifications with `vcpkg format-manifest ports/libogg/vcpkg.json`.

+

 ### Create the portfile

 `portfile.cmake` describes how to build and install the package. First we download the project from Github with [`vcpkg_from_github`](../maintainers/vcpkg_from_github.md):

 

-```no-highlight

+```cmake

 vcpkg_from_github(

     OUT_SOURCE_PATH SOURCE_PATH

     REPO xiph/ogg

@@ -26,7 +31,7 @@ The important parts to update are `REPO` for the GitHub repository path, `REF` f
 

 Finally, we configure the project with CMake, install the package, and copy over the license file:

 

-```no-highlight

+```cmake

 vcpkg_configure_cmake(

     SOURCE_PATH ${SOURCE_PATH}

     PREFER_NINJA

@@ -40,14 +45,14 @@ Check the documentation for [`vcpkg_configure_cmake`](../maintainers/vcpkg_confi
 Now you can run `vcpkg install libogg` to build and install the package.

 

 ### Suggested example portfiles

-In the `ports\` directory are many libraries that can be used as examples, including many that are not based on CMake.

+In the `ports/` directory are many libraries that can be used as examples, including many that are not based on CMake.

 

 - Header only libraries

-    - rapidjson

-    - range-v3

+  - rapidjson

+  - range-v3

 - MSBuild-based

-    - cppunit

-    - mpg123

+  - cppunit

+  - mpg123

 - Non-CMake, custom buildsystem

-    - openssl

-    - ffmpeg

+  - openssl

+  - ffmpeg

diff --git a/docs/examples/packaging-zipfiles.md b/docs/examples/packaging-zipfiles.md
index 056eb5287..9ece22d93 100644
--- a/docs/examples/packaging-zipfiles.md
+++ b/docs/examples/packaging-zipfiles.md
@@ -13,21 +13,23 @@ Finally, if the server's name for the archive is not very descriptive (such as d
 
 *`zlib1211.zip` is a fine name, so no change needed.*
 
-All this information can then be passed into the `create` command, which will download the sources and bootstrap the packaging process inside `ports\<packagename>`.
+All this information can then be passed into the `create` command, which will download the sources and bootstrap the packaging process inside `ports/<packagename>`.
 
 ```no-highlight
 PS D:\src\vcpkg> .\vcpkg create zlib2 http://zlib.net/zlib-1.2.11.tar.gz zlib1211.tar.gz
 -- Generated portfile: D:/src/vcpkg/ports/zlib2/portfile.cmake
 ```
 
-### Create the CONTROL file
-In addition to the generated `ports\<package>\portfile.cmake`, we also need a `ports\<package>\CONTROL` file. This file is a simple set of fields describing the package's metadata.
+### Create the manifest file
+In addition to the generated `ports/<package>/portfile.cmake`, we also need a `ports/<package>/vcpkg.json` file. This file is a simple set of fields describing the package's metadata.
 
-*For zlib2, we'll create the file `ports\zlib2\CONTROL` with the following contents:*
-```no-highlight
-Source: zlib2
-Version: 1.2.11
-Description: A Massively Spiffy Yet Delicately Unobtrusive Compression Library
+*For zlib2, we'll create the file `ports/zlib2/vcpkg.json` with the following contents:*
+```json
+{
+  "name": "zlib2",
+  "version-string": "1.2.11",
+  "description": "A Massively Spiffy Yet Delicately Unobtrusive Compression Library"
+}
 ```
 
 ### Tweak the generated portfile
@@ -62,7 +64,7 @@ Found 3 error(s). Please correct the portfile:
 At this point, it is a matter of reading the error messages and log files while steadily improving the quality of the portfile. Zlib required providing a discrete copy of the LICENSE to copy into the package, suppressing the build and installation of executables and headers, and removing the static libraries after they were installed.
 
 ### Suggested example portfiles
-In the `ports\` directory are many libraries that can be used as examples, including many that are not based on CMake.
+In the `ports/` directory are many libraries that can be used as examples, including many that are not based on CMake.
 
 - Header only libraries
     - rapidjson
diff --git a/docs/examples/patching.md b/docs/examples/patching.md
index f55cdea04..a02a40ce5 100644
--- a/docs/examples/patching.md
+++ b/docs/examples/patching.md
@@ -163,9 +163,9 @@ To be completely sure this works from scratch, we need to remove the package and
 PS D:\src\vcpkg> vcpkg remove libpng:x64-uwp
 Package libpng:x64-uwp was successfully removed
 ```
-and complete delete the building directory: D:\src\vcpkg\buildtrees\libpng
 
 Now we try a fresh, from scratch install.
+
 ```no-highlight
 PS D:\src\vcpkg> vcpkg install libpng:x64-uwp
 Computing installation plan...
@@ -173,36 +173,48 @@ The following packages will be built and installed:
     libpng[core]:x64-uwp
 Starting package 1/1: libpng:x64-uwp
 Building package libpng[core]:x64-uwp...
--- Using cached C:/src/vcpkg/downloads/glennrp-libpng-v1.6.37.tar.gz
--- Cleaning sources at C:/src/vcpkg/buildtrees/libpng/src/v1.6.37-c993153cdf.clean. Pass --editable to vcpkg to reuse sources.
--- Extracting source C:/src/vcpkg/downloads/glennrp-libpng-v1.6.37.tar.gz
--- Applying patch use-abort-on-all-platforms.patch
--- Using source at C:/src/vcpkg/buildtrees/libpng/src/v1.6.37-c993153cdf.clean
+Could not locate cached archive: C:\Users\me\AppData\Local\vcpkg/archives\f4\f44b54f818f78b9a4ccd34b3666f566f94286850.zip
+-- Using cached D:/src/vcpkg/downloads/glennrp-libpng-v1.6.37.tar.gz
+-- Extracting source D:/src/vcpkg/downloads/glennrp-libpng-v1.6.37.tar.gz
+-- Applying patch use_abort.patch
+-- Applying patch cmake.patch
+-- Applying patch pkgconfig.patch
+-- Applying patch pkgconfig.2.patch
+-- Using source at D:/src/vcpkg/buildtrees/libpng/src/v1.6.37-10db9f58e4.clean
 -- Configuring x64-uwp
 -- Building x64-uwp-dbg
 -- Building x64-uwp-rel
--- Installing: C:/src/vcpkg/packages/libpng_x64-uwp/share/libpng/copyright
+-- Fixing pkgconfig file: D:/src/vcpkg/packages/libpng_x64-uwp/lib/pkgconfig/libpng.pc
+-- Fixing pkgconfig file: D:/src/vcpkg/packages/libpng_x64-uwp/lib/pkgconfig/libpng16.pc
+-- Fixing pkgconfig file: D:/src/vcpkg/packages/libpng_x64-uwp/debug/lib/pkgconfig/libpng.pc
+-- Fixing pkgconfig file: D:/src/vcpkg/packages/libpng_x64-uwp/debug/lib/pkgconfig/libpng16.pc
+-- Installing: D:/src/vcpkg/packages/libpng_x64-uwp/share/libpng/copyright
 -- Performing post-build validation
 -- Performing post-build validation done
+Stored binary cache: C:\Users\me\AppData\Local\vcpkg/archives\f4\f44b54f818f78b9a4ccd34b3666f566f94286850.zip
 Building package libpng[core]:x64-uwp... done
 Installing package libpng[core]:x64-uwp...
 Installing package libpng[core]:x64-uwp... done
-Elapsed time for package libpng:x64-uwp: 15.31 s
+Elapsed time for package libpng:x64-uwp: 11.94 s
 
-Total elapsed time: 15.35 s
+Total elapsed time: 11.95 s
 
 The package libpng:x64-uwp provides CMake targets:
 
     find_package(libpng CONFIG REQUIRED)
     target_link_libraries(main PRIVATE png)
-
 ```
 
-Finally, to fully commit and publish the changes, we need to bump the internal release number and add the patch file to source control, then make a Pull Request!
-
-```no-highlight
-# ports\libpng\CONTROL
-Source: libpng
-Version: 1.6.37-1
-Build-Depends: zlib
+Finally, to fully commit and publish the changes, we need to bump the port version in `vcpkg.json`,
+and add the patch file to source control, then make a Pull Request!
+
+```json
+{
+  "name": "libpng",
+  "version": "1.6.37",
+  "port-version": 1,
+  "dependencies": [
+    "zlib"
+  ]
+}
 ```