[vcpkg manifest] Manifest Implementation (#11757)

==== Changes Related to manifests ====

* Add the `manifests` feature flag
  * This only says whether we look for a `vcpkg.json` in the cwd, not
    whether we support parsing manifests (for ports, for example)
* Changes to the manifests RFC
  * `"authors"` -> `"maintainers"`
  * `--x-classic-mode` -> `-manifests` \in `vcpkg_feature_flags`
  * reserve `"core"` in addition to `"default"`, since that's already
    reserved for features
  * Add a small helper note about what identifiers must look like
  * `<license-string>`: SPDX v3.8 -> v3.9
  * `"feature"."description"` is allowed to be an array of strings as well
  * `"version"` -> `"version-string"` for forward-compat with versions
    RFC
* Add the `--feature-flags` option
* Add the ability to turn off feature flags via passing
  `-<feature-flag>` to `VCPKG_FEATURE_FLAGS` or `--feature-flags`
* Add CMake toolchain support for manifests
  * Requires either:
    * a feature flag of `manifests` in either `Env{VCPKG_FEATURE_FLAGS}`
      or `VCPKG_FEATURE_FLAGS`
    * Passing the `VCPKG_ENABLE_MANIFESTS` option
  * The toolchain will install your packages to
    `${VCPKG_MANIFEST_DIR}/vcpkg_installed`.
* Add MSBuild `vcpkg integrate install` support for manifests
  * Requires `VcpkgEnableManifest` to be true
* `vcpkg create` creates a port that has a `vcpkg.json` instead of a
  `CONTROL`
* argparse, abseil, 3fd, and avisynthplus ports switched to manifest
  from CONTROL
* Add support for `--x-manifest-root`, as well as code for finding it if
  not passed
* Add support for parsing manifests!
* Add a filesystem lock!

==== Important Changes which are somewhat unrelated to manifests ====

* Rename `logicexpression.{h,cpp}` to `platform-expression.{h,cpp}`
* Add `PlatformExpression` type which takes the place of the old logic
  expression
  * Split the parsing of platform expressions from checking whether
    they're true or not
  * Eagerly parse PlatformExpressions as opposed to leaving them as
    strings
* Add checking for feature flag consistency
  * i.e., if `-binarycaching` is passed, you shouldn't be passing
    `--binarysource`
* Add the `Json::Reader` type which, with the help of user-defined
  visitors, converts JSON to your internal type
* VcpkgArgParser: place the switch names into a constant as opposed to
  using magic constants
  * In general update the parsing code so that this ^ works
* Add `Port-Version` fields to CONTROL files
  * This replaces the existing practice of
    `Version: <my-version>-<port-version>`

==== Smaller changes ====
* small drive-by cleanups to some CMake
  * `${_VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}` ->
    `${CURRENT_INSTALLED_DIR}`
  * Remove `-analyze` when compiling with clang-cl, since that's not a
    supported flag (vcpkg's build system)
  * Add a message about which compiler is detected by vcpkg's build
    system machinery
* Fix `Expected::then`
* Convert `""` to `{}` for `std::string` and `fs::path`, to avoid a
  `strlen` (additionally, `.empty()` instead of `== ""`, and `.clear()`)
* Add `Strings::strto` which converts strings to numeric types
* Support built-in arrays and `StringView` for `Strings::join`
* Add `operator<` and friends to `StringView`
* Add `substr` to `StringView`
* SourceParagraphParser gets some new errors

3 files changed, 61 insertions, 71 deletions

diff --git a/docs/about/privacy.md b/docs/about/privacy.md
index 76661a2c8..15ef69d76 100644
--- a/docs/about/privacy.md
+++ b/docs/about/privacy.md
@@ -41,7 +41,8 @@ We collect various telemetry events such as the command line used, the time of i
 
 You can see the telemetry events any command by appending `--printmetrics` after the vcpkg command line.
 
-In the source code (included in `toolsrc\`), you can search for calls to the functions `track_property()` and `track_metric()` to see every specific data point we collect.
+In the source code (included in `toolsrc\`), you can search for calls to the functions `track_property()`, `track_feature()`, `track_metric()`, and `track_buildtime()`
+to see every specific data point we collect.
 
 ## Avoid inadvertent disclosure information
 
diff --git a/docs/specifications/manifests.md b/docs/specifications/manifests.md
index 469126158..350cfd0c7 100644
--- a/docs/specifications/manifests.md
+++ b/docs/specifications/manifests.md
@@ -34,12 +34,6 @@ rather than JSON5 or JSON with comments because JSON is the everywhere-supported
 standard. That is not necessarily true of JSON with comments. Additionally, if one needs
 to write a comment, they can do so via `"$reason"` or `"$comment"` fields.
 
-### Why are `<platform-specification>`s so verbose?
-
-In the initial implementation, we didn't want to do more parsing than is strictly necessary,
-especially parsing languages which aren't defined anywhere. We may add a shorter way of
-defining platform specifications in the future (more similar to those in control files).
-
 ## Specification
 
 A manifest file shall have the name `vcpkg.json`, and shall be in the root directory of a package.
@@ -54,26 +48,28 @@ to specify the shape of a value. Note that any object may contain any directives
 a field key that starts with a `$`; these directive shall be ignored by `vcpkg`. Common
 directives may include `"$schema"`, `"$comment"`, `"$reason"`.
 
-A manifest must be a top-level object, and must have at least the following properties:
+A manifest must be a top-level object, and must have at least:
 
 * `"name"`: a `<package-name>`
-* `"version"`: A `string`. This will be defined further later.
-  * [Semver](https://semver.org) is recommended but not required.
+* One (and only one) of the following version fields:
+  * `"version-string"`: A `string`. Has no semantic meaning.
+    Equivalent to `CONTROL`'s `Version:` field.
+  * Other version fields will be defined by the Versions RFC
 
 The simplest vcpkg.json looks like this:
 
 ```json
 {
   "name": "mypackage",
-  "version": "0.1.0-dev"
+  "version-string": "0.1.0-dev"
 }
 ```
 
 Additionally, it may contain the following properties:
 * `"port-version"`: A non-negative integer. If this field doesn't exist, it's assumed to be `0`.
   * Note that this is a change from existing CONTROL files, where versions were a part of the version string
-* `"authors"`: An array of `string`s which contain the authors of a package
-  * `"authors": [ "Nicole Mazzuca <nicole@example.com>", "שלום עליכם <shalom@example.com>" ]`
+* `"maintainers"`: An array of `string`s which contain the authors of a package
+  * `"maintainers": [ "Nicole Mazzuca <nicole@example.com>", "שלום עליכם <shalom@example.com>" ]`
 * `"description"`: A string or array of strings containing the description of a package
   * `"description": "mypackage is a package of mine"`
 * `"homepage"`: A url which points to the homepage of a package
@@ -86,8 +82,8 @@ Additionally, it may contain the following properties:
 * `"dev-dependencies"`: An array of `<dependency>`s which are required only for developers (testing and the like)
 * `"features"`: An array of `<feature>`s that the package supports
 * `"default-features"`: An array of `<identifier>`s that correspond to features, which will be used by default.
-* `"supports"`: A `<platform-specification>`
-  * `"supports": { "and": [ "win", { "not": "arm" } ] }`
+* `"supports"`: A `<platform-expression>`
+  * `"supports": "windows & !arm"`
 
 Any properties which are not listed, and which do not start with a `$`,
 will be warned against and are reserved for future use.
@@ -105,7 +101,7 @@ Build-Depends: glib, gettext, cairo, fontconfig, freetype, harfbuzz[glib] (!(win
 ```json
 {
   "name": "pango",
-  "version": "1.40.11",
+  "version-string": "1.40.11",
   "port-version": 6,
   "homepage": "https://ftp.gnome.org/pub/GNOME/sources/pango/",
   "description": "Text and font handling library.",
@@ -118,22 +114,15 @@ Build-Depends: glib, gettext, cairo, fontconfig, freetype, harfbuzz[glib] (!(win
     {
       "name": "harfbuzz",
       "features": [ "glib" ],
-      "platform": {
-        "and": [
-          { "not": { "and": [ "windows", "static" ] } },
-          { "not": "osx" }
-        ]
-      }
+      "platform": "!(windows & static) & !osx"
     }
   ]
 }
 ```
 
-You may notice that the platform specification is fairly wordy. See [reasoning](#why-are-platform-specifications-so-verbose) for why.
-
 ## Behavior of the Tool
 
-There will be two "modes" for vcpkg from this point forward: "classic", and "modern".
+There will be two "modes" for vcpkg from this point forward: "classic", and "manifest".
 The former will act exactly like the existing vcpkg workflow, so as to avoid breaking
 anyone. The latter will be the mode only when the user either:
 
@@ -146,11 +135,10 @@ anyone. The latter will be the mode only when the user either:
     * The environment variable `VCPKG_FEATURE_FLAGS`
     * The option `--feature-flags`
       * (e.g., `--feature-flags=binarycaching,manifests`)
+    * If someone wants to use classic mode and silence the warning, they can add the
+      `-manifests` feature flag to disable the mode.
 
-Additionally, we'll add the `--x-classic-mode` flag to allow someone to force classic
-mode.
-
-When in "modern" mode, the `installed` directory will be changed to
+When in "manifest" mode, the `installed` directory will be changed to
 `<manifest-root>/vcpkg_installed` (name up for bikeshedding).
 The following commands will change behavior:
 
@@ -158,49 +146,38 @@ The following commands will change behavior:
   the manifest file, and will remove any dependencies
   which are no longer in the dependency tree implied by the manifest file.
 * `vcpkg install` with port arguments will give an error.
-* `vcpkg x-clean` will be added, and will delete your `vcpkg_installed` directory.
 
-The following commands will not work in modern mode, at least initially:
+The following commands will not work in manifest mode, at least initially:
 
 * `vcpkg x-set-installed`: `vcpkg install` serves the same function
 * `vcpkg remove`
 * `vcpkg export`
-* `vcpkg import`
-* `vcpkg create`
 
-We may add these features back for modern mode once we understand how best to
+We may add these features back for manifest mode once we understand how best to
 implement them.
 
 ### Behavior of the Toolchain
 
-Mostly, the toolchain file stays the same; however, we shall add one public cache variable:
-
-```cmake
-VCPKG_MANIFEST_ROOT:PATH=<path to the directory containing the vcpkg.json file>
-```
-
-and one function:
+Mostly, the toolchain file stays the same; however, we shall add
+two public options:
 
 ```cmake
-vcpkg_acquire_dependencies(
-  [TRIPLET <triplet>]
-  [MANIFEST <path to manifest>]
-  [INSTALL_DIRECTORY <install directory>])
+VCPKG_MANIFEST_MODE:BOOL=<we found a manifest>
+VCPKG_MANIFEST_INSTALL:BOOL=ON
 ```
 
-which installs the dependencies required by the manifest file.
-
-The default for `TRIPLET` is `VCPKG_TARGET_TRIPLET`
-(which is the default triplet for the configured system).
-For example, on x64 Windows, it defaults to `x64-windows`.
+The first option either explicitly turns on, or off, manifest mode;
+otherwise, we default to looking for a manifest file in the directory
+tree upwards from the source directory.
 
-The default for `INSTALL_DIRECTORY` is `${CMAKE_BINARY_DIR}/vcpkg_installed`.
+The `VCPKG_MANIFEST_INSTALL` option tells the toolchain whether to
+install the packages or not -- if you wish to install the manifest
+dependencies manually, you can set this to off, and we also turn it
+off for packages installed by vcpkg.
 
-Additionally, in the course of implementation, we would like to
-look at adding the following function, but may not be able to:
-
-It is almost certain that one should guard any use of this function
-by `if(EXISTS CACHE{VCPKG_MANIFEST_FILE})`.
+Additionally, if `-manifests` is set in the feature flags environment
+variable, we turn off manifest mode in the toolchain, and we act like
+the classic toolchain.
 
 ### Example - CMake Integration
 
@@ -232,7 +209,7 @@ Therefore, in `vcpkg.json`, we'll need to depend on `fmt`:
 ```json
 {
   "name": "example",
-  "version": "0.0.1",
+  "version-string": "0.0.1",
   "dependencies": [
     "fmt"
   ]
@@ -246,11 +223,6 @@ cmake_minimum_required(VERSION 3.14)
 
 project(example CXX)
 
-if(EXISTS CACHE{VCPKG_MANIFEST_FILE})
-  vcpkg_acquire_dependencies()
-endif()
-
-
 add_executable(example src/main.cxx)
 
 find_package(fmt REQUIRED)
@@ -285,7 +257,9 @@ Hello, world!
   * Does not have multiple consecutive hyphens
   * Does not begin nor end with a hyphen
   * Is not a Windows filesystem reserved name
-  * Is not a vcpkg reserved name: "default".
+  * Is not a vcpkg reserved name: "default" or "core".
+  * In other words, it must follow the regex `[a-z0-9]+(-[a-z0-9]+)*`, and must not be any of:
+    * `{ prn, aux, nul, con, lpt[1-9], com[1-9], core, default }`
 * `<package-name>`: A `string` consisting of a non-zero number of `<identifier>`s, separated by `.`.
   * `a.b.c` is valid
   * `a` is valid
@@ -296,15 +270,29 @@ Hello, world!
     * `"name"`: A `<package-name>`
     * Optionally, `"features"`: an array of `<identifier>`s corresponding to features in the package.
     * Optionally, `"default-features"`: a `boolean`. If this is false, then don't use the default features of the package; equivalent to core in existing CONTROL files. If this is true, do the default thing of including the default features.
-    * Optionally, `"platform"`: a `<platform-specification>`
+    * Optionally, `"platform"`: a `<platform-expression>`
   * `<dependency.port>`: No extra fields are required.
-* `<license-string>`: An SPDX license expression at version 3.8.
-* `<platform-specification>`: A specification of a set of platforms; used in platform-specific dependencies and supports fields. One of:
-  * `<platform-specification.exact>`: A string denoting a triplet tag like “windows”, “osx”, etc.
-  * `<platform-specification.not>`: An object containing a member with key "not" and value `<platform-specification>`.
-  * `<platform-specification.and>`: An object containing a member with key "and" and value array of `<platform-specification>`s.
-  * `<platform-specification.or>`: An object containing a member with key "or" and value array of `<platform-specification>`s.
+* `<license-string>`: An SPDX license expression at version 3.9.
+* `<platform-expression>`: A specification of a set of platforms; used in platform-specific dependencies and supports fields. A string that is parsed as follows:
+  * `<platform-expression>`:
+    * `<platform-expression.not>`
+    * `<platform-expression.and>`
+    * `<platform-expression.or>`
+  * `<platform-expression.simple>`:
+    * `( <platform-expression> )`
+    * `<platform-expression.identifier>`
+  * `<platform-expression.identifier>`:
+    * regex: `/^[a-z0-9]+$/`
+  * `<platform-expression.not>`:
+    * `<platform-expression.simple>`
+    * `! <platform-expression.simple>`
+  * `<platform-expression.and>`
+    * `<platform-expression.not>`
+    * `<platform-expression.and> & <platform-expression.not>`
+  * `<platform-expression.or>`
+    * `<platform-expression.not>`
+    * `<platform-expression.or> | <platform-expression.not>`
 * `<feature>`: An object containing the following:
   * `"name"`: An `<identifier>`, the name of the feature
-  * `"description"`: A `string`, the description of the feature
+  * `"description"`: A `string` or array of `string`s, the description of the feature
   * Optionally, `"dependencies"`: An array of `<dependency>`s, the dependencies used by this feature
diff --git a/docs/users/config-environment.md b/docs/users/config-environment.md
index bffa07278..ffa988cbc 100644
--- a/docs/users/config-environment.md
+++ b/docs/users/config-environment.md
@@ -15,6 +15,7 @@ subject to change without notice and should be considered highly unstable.
 Non-exhaustive list of off-by-default features:

 

 - `binarycaching`

+- `manifest`

 

 #### EDITOR