diff options
| author | ras0219 <533828+ras0219@users.noreply.github.com> | 2021-01-15 12:35:48 -0800 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2021-01-15 12:35:48 -0800 |
| commit | 4f8fb510ba03f195a49f6353b97fabf5bb20d450 (patch) | |
| tree | 7564b1db60e1086ce5cf587846ea28d628735d05 | |
| parent | a8e97d4a4b22d489123dc6d673ceee2c203dc046 (diff) | |
| download | vcpkg-4f8fb510ba03f195a49f6353b97fabf5bb20d450.tar.gz vcpkg-4f8fb510ba03f195a49f6353b97fabf5bb20d450.zip | |
[vcpkg] Add initial versioning documentation (#15565)
* [vcpkg] Improve efficiency and tests of versioning
* [vcpkg] Add initial versioning documentation and rename x-default-baseline to builtin-baseline
* [vcpkg] Enable metrics for builtin-baseline & overrides
* [vcpkg] Address PR comments
* [vcpkg] Add support for syntax in version>=
* [vcpkg] Remove port-version from dependency syntax
* [vcpkg] Address CR comment
* [vcpkg] Minor docs fixup
28 files changed, 676 insertions, 312 deletions
diff --git a/docs/users/manifests.md b/docs/users/manifests.md index 427d336c3..e1bc0c58b 100644 --- a/docs/users/manifests.md +++ b/docs/users/manifests.md @@ -64,6 +64,12 @@ Additionally, the `"port-version"` field is used by registries of packages, as a way to version "the package gotten from `vcpkg install`" differently from the upstream package version. You shouldn't need to worry about this at all. +#### Additional version fields + +**Experimental behind the `versions` feature flag** + +See [versioning.md](versioning.md#version%20schemes) for additional version types. + ### `"description"` This is where you describe your project. Give it a good description to help in searching for it! @@ -71,6 +77,14 @@ This can be a single string, or it can be an array of strings; in the latter case, the first string is treated as a summary, while the remaining strings are treated as the full description. +### `"builtin-baseline"` + +**Experimental behind the `versions` feature flag** + +This field indicates the commit of vcpkg which provides global minimum version information for your manifest. It is required for top-level manifest files using versioning. + +See also [versioning](versioning.md#builtin-baseline) for more semantic details. + ### `"dependencies"` This field lists all the dependencies you'll need to build your library (as well as any your dependents might need, @@ -131,6 +145,16 @@ The common identifiers are: although one can define their own. +#### `"version>="` + +**Experimental behind the `versions` feature flag** + +A minimum version constraint on the dependency. + +This field specifies the minimum version of the dependency using a '#' suffix to denote port-version if non-zero. + +See also [versioning](versioning.md#constraints) for more semantic details. + #### Example: ```json @@ -151,6 +175,26 @@ although one can define their own. } ``` +### `"overrides"` + +**Experimental behind the `versions` feature flag** + +This field enables version resolution to be ignored for certain dependencies and to use specific versions instead. + +See also [versioning](versioning.md#overrides) for more semantic details. + +#### Example: + +```json +{ + "overrides": [ + { + "name": "arrow", "version": "1.2.3", "port-version": 7 + } + ] +} +``` + ### `"supports"` If your project doesn't support common platforms, you can tell your users this with the `"supports"` field. @@ -180,8 +224,9 @@ and that's the `"default-features"` field, which is an array of feature names. "name": "libdb", "description": [ "An example database library.", - "Optionally uses one of CBOR, JSON, or CSV as a backend." + "Optionally can build with CBOR, JSON, or CSV as backends." ], + "$default-features-explanation": "Users using this library transitively will get all backends automatically", "default-features": [ "cbor", "csv", "json" ], "features": { "cbor": { @@ -189,8 +234,7 @@ and that's the `"default-features"` field, which is an array of feature names. "dependencies": [ { "$explanation": [ - "This is currently how you tell vcpkg that the cbor feature depends on the json feature of this package", - "We're looking into making this easier" + "This is how you tell vcpkg that the cbor feature depends on the json feature of this package" ], "name": "libdb", "default-features": false, diff --git a/docs/users/versioning.md b/docs/users/versioning.md new file mode 100644 index 000000000..86840422b --- /dev/null +++ b/docs/users/versioning.md @@ -0,0 +1,85 @@ +# Versioning + +**This feature is experimental and requires `--feature-flags=versions`** + +Versioning allows you to deterministically control the precise revisions of dependencies used by +your project from within your manifest file. + +## Version schemes + +### Schemes +Versions in vcpkg come in four primary flavors: + +#### version +A dot-separated sequence of numbers (1.2.3.4) + +#### version-date +A date (2021-01-01.5) + +#### version-semver +A Semantic Version 2.0 (2.1.0-rc2) + +See https://semver.org/ for a full specification. + +#### version-string +An exact, incomparable version (Vista) + +### Port Versions +Each version additionally has a "port-version" which is a nonnegative integer. When rendered as text, the +port version (if nonzero) is added as a suffix to the primary version text separated by a hash (#). +Port-versions are sorted lexographically after the primary version text, for example: + + 1.0.0 < 1.0.0#1 < 1.0.1 < 1.0.1#5 < 2.0.0 + +## Constraints + +Manifests can place three kinds of constraints upon the versions used: + +### builtin-baseline +The baseline references a commit within the vcpkg repository that +establishes a minimum version on every dependency in the graph. If +no other constraints are specified (directly or transitively), +then the version from the baseline of the top level manifest will +be used. + +You can get the current commit of your vcpkg instance either by adding an empty `"builtin-baseline"` field, installing, and examining the error message or by running `git rev-parse HEAD` in the root of the vcpkg instance. + +Baselines provide stability and ease of development for top-level manifest files. They are not considered from ports consumed as a dependency. If a minimum version constraint is required during transitive version resolution, the port should use `version>=`. + +### version>= +Within the "dependencies" field, each dependency can have a +minimum constraint listed. These minimum constraints will be used +when transitively depending upon this library. A minimum +port-version can additionally be specified with a '#' suffix. + +This constraint must refer to an existing, valid version (including port-version). + +### overrides +When used as the top-level manifest (such as when running `vcpkg +install` in the directory), overrides allow a manifest to +short-circuit dependency resolution and specify exactly the +version to use. These can be used to handle version conflicts, +such as with `version-string` dependencies. + +Overrides are not considered from ports consumed as a dependency. + +## Example top-level manifest: +```json +{ + "name": "example", + "version": "1.0", + "builtin-baseline": "a14a6bcb27287e3ec138dba1b948a0cdbc337a3a", + "dependencies": [ + { "name": "zlib", "version>=": "1.2.11#8" }, + "rapidjson" + ], + "overrides": [ + { "name": "rapidjson", "version": "2020-09-14" } + ] +} +``` +See also the [manifest documentation](manifests.md) for more syntax information. + +## Original Specification + +See also the [original specification](https://github.com/vicroms/vcpkg/blob/versioning-spec/docs/specifications/versioning.md) diff --git a/scripts/azure-pipelines/end-to-end-tests-dir/registries.ps1 b/scripts/azure-pipelines/end-to-end-tests-dir/registries.ps1 index 777c0901f..ef7023c85 100644 --- a/scripts/azure-pipelines/end-to-end-tests-dir/registries.ps1 +++ b/scripts/azure-pipelines/end-to-end-tests-dir/registries.ps1 @@ -14,12 +14,13 @@ Throw-IfNotFailed Run-Vcpkg install @builtinRegistryArgs --feature-flags=registries 'zlib' Throw-IfFailed -# Test git and filesystem registries +Write-Trace "Test git and filesystem registries" Refresh-TestRoot $filesystemRegistry = "$TestingRoot/filesystem-registry" $gitRegistryUpstream = "$TestingRoot/git-registry-upstream" # build a filesystem registry +Write-Trace "build a filesystem registry" New-Item -Path $filesystemRegistry -ItemType Directory $filesystemRegistry = (Get-Item $filesystemRegistry).FullName @@ -51,6 +52,7 @@ New-Item ` # build a git registry +Write-Trace "build a git registry" New-Item -Path $gitRegistryUpstream -ItemType Directory $gitRegistryUpstream = (Get-Item $gitRegistryUpstream).FullName @@ -82,6 +84,7 @@ finally } # actually test the registries +Write-Trace "actually test the registries" $vcpkgJson = @{ "name" = "manifest-test"; "version-string" = "1.0.0"; @@ -90,37 +93,8 @@ $vcpkgJson = @{ ) } -$manifestDir = "$TestingRoot/builtin-registry-test-manifest-dir" - -New-Item -Path $manifestDir -ItemType Directory -$manifestDir = (Get-Item $manifestDir).FullName - -Push-Location $manifestDir - -try -{ - $vcpkgJsonWithBaseline = $vcpkgJson.Clone() - $vcpkgJsonWithBaseline['$x-default-baseline'] = 'default' - - New-Item -Path 'vcpkg.json' -ItemType File ` - -Value (ConvertTo-Json -Depth 5 -InputObject $vcpkgJson) - - Run-Vcpkg install @builtinRegistryArgs '--feature-flags=registries,manifests' - Throw-IfNotFailed - - New-Item -Path 'vcpkg.json' -ItemType File -Force ` - -Value (ConvertTo-Json -Depth 5 -InputObject $vcpkgJsonWithBaseline) - - Run-Vcpkg install @builtinRegistryArgs '--feature-flags=registries,manifests' - Throw-IfFailed -} -finally -{ - Pop-Location -} - - # test the filesystem registry +Write-Trace "test the filesystem registry" $manifestDir = "$TestingRoot/filesystem-registry-test-manifest-dir" New-Item -Path $manifestDir -ItemType Directory @@ -154,6 +128,7 @@ finally } # test the git registry +Write-Trace "test the git registry" $manifestDir = "$TestingRoot/git-registry-test-manifest-dir" New-Item -Path $manifestDir -ItemType Directory diff --git a/scripts/azure-pipelines/end-to-end-tests-dir/versions.ps1 b/scripts/azure-pipelines/end-to-end-tests-dir/versions.ps1 index f5f9de20b..11550d7bd 100644 --- a/scripts/azure-pipelines/end-to-end-tests-dir/versions.ps1 +++ b/scripts/azure-pipelines/end-to-end-tests-dir/versions.ps1 @@ -59,17 +59,40 @@ Throw-IfFailed Throw-IfFailed
$CurrentTest = "default baseline"
-$out = ./vcpkg $commonArgs "--feature-flags=versions" install --x-manifest-root=scripts/testing/version-files/default-baseline-1
+$out = ./vcpkg $commonArgs "--feature-flags=versions" install --x-manifest-root=scripts/testing/version-files/default-baseline-1 2>&1 | Out-String
Throw-IfNotFailed
-# if ($out -notmatch "Error: while checking out baseline" -or $out -notmatch " does not exist in ")
-# {
-# $out
-# throw "Expected to fail due to missing baseline"
-# }
+if ($out -notmatch ".*Error: while checking out baseline.*")
+{
+ $out
+ throw "Expected to fail due to missing baseline"
+}
git fetch https://github.com/vicroms/test-registries
-$CurrentTest = "default baseline"
-./vcpkg $commonArgs --debug "--feature-flags=versions" install `
- "--x-manifest-root=scripts/testing/version-files/default-baseline-2" `
- "--x-builtin-port-versions-dir=scripts/testing/version-files/default-baseline-2/port_versions"
-Throw-IfFailed
+foreach ($opt_registries in @("",",registries"))
+{
+ Write-Trace "testing baselines: $opt_registries"
+ Refresh-TestRoot
+ $CurrentTest = "without default baseline 2 -- enabling versions should not change behavior"
+ Remove-Item -Recurse $buildtreesRoot/versioning -ErrorAction SilentlyContinue
+ ./vcpkg $commonArgs "--feature-flags=versions$opt_registries" install `
+ "--dry-run" `
+ "--x-manifest-root=scripts/testing/version-files/without-default-baseline-2" `
+ "--x-builtin-port-versions-dir=scripts/testing/version-files/default-baseline-2/port_versions"
+ Throw-IfFailed
+ Require-FileNotExists $buildtreesRoot/versioning
+
+ $CurrentTest = "default baseline 2"
+ ./vcpkg $commonArgs "--feature-flags=versions$opt_registries" install `
+ "--dry-run" `
+ "--x-manifest-root=scripts/testing/version-files/default-baseline-2" `
+ "--x-builtin-port-versions-dir=scripts/testing/version-files/default-baseline-2/port_versions"
+ Throw-IfFailed
+ Require-FileExists $buildtreesRoot/versioning
+
+ $CurrentTest = "using version features fails without flag"
+ ./vcpkg $commonArgs "--feature-flags=-versions$opt_registries" install `
+ "--dry-run" `
+ "--x-manifest-root=scripts/testing/version-files/default-baseline-2" `
+ "--x-builtin-port-versions-dir=scripts/testing/version-files/default-baseline-2/port_versions"
+ Throw-IfNotFailed
+}
\ No newline at end of file diff --git a/scripts/azure-pipelines/end-to-end-tests-prelude.ps1 b/scripts/azure-pipelines/end-to-end-tests-prelude.ps1 index deadb210c..29718bf76 100644 --- a/scripts/azure-pipelines/end-to-end-tests-prelude.ps1 +++ b/scripts/azure-pipelines/end-to-end-tests-prelude.ps1 @@ -63,6 +63,10 @@ function Throw-IfNotFailed { }
}
+function Write-Trace ([string]$text) {
+ Write-Host (@($MyInvocation.ScriptName, ":", $MyInvocation.ScriptLineNumber, ": ", $text) -join "")
+}
+
function Run-Vcpkg {
Param(
[Parameter(ValueFromRemainingArguments)]
diff --git a/scripts/generatePortVersionsDb.py b/scripts/generatePortVersionsDb.py index 3b7de6942..3be4a939e 100644 --- a/scripts/generatePortVersionsDb.py +++ b/scripts/generatePortVersionsDb.py @@ -35,7 +35,7 @@ def generate_port_versions_file(port_name): env = os.environ.copy()
env['GIT_OPTIONAL_LOCKS'] = '0'
output = subprocess.run(
- [os.path.join(SCRIPT_DIRECTORY, '../vcpkg.exe'),
+ [os.path.join(SCRIPT_DIRECTORY, '../vcpkg'),
'x-history', port_name, '--x-json', f'--output={output_file_path}'],
capture_output=True, encoding='utf-8', env=env)
if output.returncode != 0:
diff --git a/scripts/testing/version-files/default-baseline-1/vcpkg.json b/scripts/testing/version-files/default-baseline-1/vcpkg.json index f6d902393..51e65c82c 100644 --- a/scripts/testing/version-files/default-baseline-1/vcpkg.json +++ b/scripts/testing/version-files/default-baseline-1/vcpkg.json @@ -1,7 +1,7 @@ { "name": "default-baseline-test", "version-string": "0", - "$x-default-baseline": "fca18ba3572f8aebe3b8158c359db62a7e26134e", + "builtin-baseline": "fca18ba3572f8aebe3b8158c359db62a7e26134e", "dependencies": [ "zlib" ] diff --git a/scripts/testing/version-files/default-baseline-2/vcpkg.json b/scripts/testing/version-files/default-baseline-2/vcpkg.json index af353c09d..f540ccba0 100644 --- a/scripts/testing/version-files/default-baseline-2/vcpkg.json +++ b/scripts/testing/version-files/default-baseline-2/vcpkg.json @@ -1,7 +1,7 @@ { - "name": "default-baseline-test", + "name": "default-baseline-test-2", "version-string": "0", - "$x-default-baseline": "16002d9c2318dec4c69e02d9af8c0e11dca0d4c6", + "builtin-baseline": "16002d9c2318dec4c69e02d9af8c0e11dca0d4c6", "dependencies": [ "zlib" ] diff --git a/scripts/testing/version-files/without-default-baseline-2/port_versions/z-/zlib.json b/scripts/testing/version-files/without-default-baseline-2/port_versions/z-/zlib.json new file mode 100644 index 000000000..f5ee7cb9d --- /dev/null +++ b/scripts/testing/version-files/without-default-baseline-2/port_versions/z-/zlib.json @@ -0,0 +1,14 @@ +{ + "versions": [ + { + "git-tree": "7bb2b2f3783303a4dd41163553fe4cc103dc9262", + "version-string": "1.2.11", + "port-version": 9 + }, + { + "git-tree": "4927735fa9baca564ebddf6e6880de344b20d7a8", + "version-string": "1.2.11", + "port-version": 8 + } + ] +}
\ No newline at end of file diff --git a/scripts/testing/version-files/without-default-baseline-2/vcpkg.json b/scripts/testing/version-files/without-default-baseline-2/vcpkg.json new file mode 100644 index 000000000..839418fb7 --- /dev/null +++ b/scripts/testing/version-files/without-default-baseline-2/vcpkg.json @@ -0,0 +1,7 @@ +{ + "name": "without-default-baseline-test-2", + "version-string": "0", + "dependencies": [ + "zlib" + ] +}
\ No newline at end of file diff --git a/scripts/vcpkg.schema.json b/scripts/vcpkg.schema.json index 9df362086..c485aa415 100644 --- a/scripts/vcpkg.schema.json +++ b/scripts/vcpkg.schema.json @@ -136,12 +136,9 @@ "$ref": "#/definitions/platform-expression" }, "version>=": { - "description": "Minimum required port version", - "$ref": "#/definitions/version-text" - }, - "port-version": { - "description": "Minimum port revision (defaults to 0)", - "$ref": "#/definitions/port-version" + "description": "Minimum required version", + "type": "string", + "pattern": "^[^#]+(#\\d+)?$" } }, "patternProperties": { @@ -256,6 +253,10 @@ "description": "An SPDX license expression at version 3.9.", "type": "string" }, + "builtin-baseline": { + "description": "A vcpkg repository commit for version control.", + "type": "string" + }, "dependencies": { "description": "Dependencies that are always required.", "type": "array", diff --git a/toolsrc/include/vcpkg/base/expected.h b/toolsrc/include/vcpkg/base/expected.h index 13e7b4bcb..20c23f077 100644 --- a/toolsrc/include/vcpkg/base/expected.h +++ b/toolsrc/include/vcpkg/base/expected.h @@ -226,11 +226,10 @@ namespace vcpkg void exit_if_error(const LineInfo& line_info) const { // This is used for quick value_or_exit() calls, so always put line_info in the error message. - Checks::check_exit(line_info, - !m_s.has_error(), - "Failed at [%s] with message:\n%s", - line_info.to_string(), - m_s.to_string()); + if (m_s.has_error()) + { + Checks::exit_with_message(line_info, "Failed at [%s] with message:\n%s", line_info, m_s.to_string()); + } } ErrorHolder<S> m_s; diff --git a/toolsrc/include/vcpkg/base/files.h b/toolsrc/include/vcpkg/base/files.h index dd1b3a6c6..5f2e96e91 100644 --- a/toolsrc/include/vcpkg/base/files.h +++ b/toolsrc/include/vcpkg/base/files.h @@ -282,45 +282,34 @@ namespace vcpkg::Files }; ExclusiveFileLock() = default; - ExclusiveFileLock(ExclusiveFileLock&& other) - : fs_(other.fs_), handle_(std::exchange(handle_, fs::SystemHandle{})) - { - } - ExclusiveFileLock& operator=(ExclusiveFileLock&& other) - { - if (this == &other) return *this; - - clear(); - fs_ = other.fs_; - handle_ = std::exchange(other.handle_, fs::SystemHandle{}); - return *this; - } + ExclusiveFileLock(ExclusiveFileLock&&) = delete; + ExclusiveFileLock& operator=(ExclusiveFileLock&&) = delete; - ExclusiveFileLock(Wait wait, Filesystem& fs, const fs::path& path_, std::error_code& ec) : fs_(&fs) + ExclusiveFileLock(Wait wait, Filesystem& fs, const fs::path& path_, std::error_code& ec) : m_fs(&fs) { switch (wait) { - case Wait::Yes: handle_ = fs_->take_exclusive_file_lock(path_, ec); break; - case Wait::No: handle_ = fs_->try_take_exclusive_file_lock(path_, ec); break; + case Wait::Yes: m_handle = m_fs->take_exclusive_file_lock(path_, ec); break; + case Wait::No: m_handle = m_fs->try_take_exclusive_file_lock(path_, ec); break; } } ~ExclusiveFileLock() { clear(); } - explicit operator bool() const { return handle_.is_valid(); } - bool has_lock() const { return handle_.is_valid(); } + explicit operator bool() const { return m_handle.is_valid(); } + bool has_lock() const { return m_handle.is_valid(); } void clear() { - if (fs_ && handle_.is_valid()) + if (m_fs && m_handle.is_valid()) { std::error_code ignore; - fs_->unlock_file_lock(std::exchange(handle_, fs::SystemHandle{}), ignore); + m_fs->unlock_file_lock(std::exchange(m_handle, fs::SystemHandle{}), ignore); } } private: - fs::SystemHandle handle_; - Filesystem* fs_; + Filesystem* m_fs; + fs::SystemHandle m_handle; }; } diff --git a/toolsrc/include/vcpkg/base/json.h b/toolsrc/include/vcpkg/base/json.h index e7f3076f7..31696b757 100644 --- a/toolsrc/include/vcpkg/base/json.h +++ b/toolsrc/include/vcpkg/base/json.h @@ -291,7 +291,9 @@ namespace vcpkg::Json ExpectedT<std::pair<Value, JsonStyle>, std::unique_ptr<Parse::IParseError>> parse_file( const Files::Filesystem&, const fs::path&, std::error_code& ec) noexcept; ExpectedT<std::pair<Value, JsonStyle>, std::unique_ptr<Parse::IParseError>> parse( - StringView text, const fs::path& filepath = {}) noexcept; + StringView text, const fs::path& filepath) noexcept; + ExpectedT<std::pair<Value, JsonStyle>, std::unique_ptr<Parse::IParseError>> parse(StringView text, + StringView origin = {}) noexcept; std::pair<Value, JsonStyle> parse_file(vcpkg::LineInfo linfo, const Files::Filesystem&, const fs::path&) noexcept; std::string stringify(const Value&, JsonStyle style); diff --git a/toolsrc/include/vcpkg/base/jsonreader.h b/toolsrc/include/vcpkg/base/jsonreader.h index da086fa02..4aca02998 100644 --- a/toolsrc/include/vcpkg/base/jsonreader.h +++ b/toolsrc/include/vcpkg/base/jsonreader.h @@ -278,7 +278,7 @@ namespace vcpkg::Json struct NaturalNumberDeserializer final : IDeserializer<int> { - virtual StringView type_name() const override { return "a natural number"; } + virtual StringView type_name() const override { return "a nonnegative integer"; } virtual Optional<int> visit_integer(Reader&, int64_t value) override { diff --git a/toolsrc/include/vcpkg/sourceparagraph.h b/toolsrc/include/vcpkg/sourceparagraph.h index f2dd0798b..52ce53980 100644 --- a/toolsrc/include/vcpkg/sourceparagraph.h +++ b/toolsrc/include/vcpkg/sourceparagraph.h @@ -69,6 +69,7 @@ namespace vcpkg std::vector<DependencyOverride> overrides; std::vector<std::string> default_features; std::string license; // SPDX license expression + Optional<std::string> builtin_baseline; Type type; PlatformExpression::Expr supports_expression; diff --git a/toolsrc/include/vcpkg/vcpkgpaths.h b/toolsrc/include/vcpkg/vcpkgpaths.h index 36447b9f3..d36ebfb7c 100644 --- a/toolsrc/include/vcpkg/vcpkgpaths.h +++ b/toolsrc/include/vcpkg/vcpkgpaths.h @@ -118,8 +118,12 @@ namespace vcpkg const std::string& get_tool_version(const std::string& tool) const; // Git manipulation in the vcpkg directory - fs::path git_checkout_baseline(Files::Filesystem& filesystem, StringView commit_sha) const; - fs::path git_checkout_port(Files::Filesystem& filesystem, StringView port_name, StringView git_tree) const; + ExpectedS<std::string> get_current_git_sha() const; + std::string get_current_git_sha_message() const; + ExpectedS<fs::path> git_checkout_baseline(StringView commit_sha) const; + ExpectedS<fs::path> git_checkout_port(StringView port_name, + StringView git_tree, + const fs::path& dot_git_dir) const; ExpectedS<std::string> git_show(const std::string& treeish, const fs::path& dot_git_dir) const; ExpectedS<std::map<std::string, std::string, std::less<>>> git_get_local_port_treeish_map() const; @@ -168,12 +172,5 @@ namespace vcpkg const fs::path& destination, const fs::path& dot_git_dir, const fs::path& work_tree); - - static void git_checkout_object(const VcpkgPaths& paths, - StringView git_object, - const fs::path& local_repo, - const fs::path& destination, - const fs::path& dot_git_dir, - const fs::path& work_tree); }; } diff --git a/toolsrc/include/vcpkg/versions.h b/toolsrc/include/vcpkg/versions.h index 19b5546ea..b26c90dfb 100644 --- a/toolsrc/include/vcpkg/versions.h +++ b/toolsrc/include/vcpkg/versions.h @@ -81,8 +81,7 @@ namespace vcpkg::Versions enum class Type { None, - Minimum, - Exact + Minimum }; }; } diff --git a/toolsrc/src/vcpkg-test/dependencies.cpp b/toolsrc/src/vcpkg-test/dependencies.cpp index 2831ea5a1..8f936abc3 100644 --- a/toolsrc/src/vcpkg-test/dependencies.cpp +++ b/toolsrc/src/vcpkg-test/dependencies.cpp @@ -196,7 +196,7 @@ private: static const MockOverlayProvider s_empty_mock_overlay; -ExpectedS<Dependencies::ActionPlan> create_versioned_install_plan( +static ExpectedS<Dependencies::ActionPlan> create_versioned_install_plan( const PortFileProvider::IVersionedPortfileProvider& provider, const PortFileProvider::IBaselineProvider& bprovider, const CMakeVars::CMakeVarProvider& var_provider, @@ -335,7 +335,7 @@ TEST_CASE ("basic version install scheme baseline missing success", "[versionpla bp, var_provider, { - Dependency{"a", {}, {}, {Constraint::Type::Exact, "2"}}, + Dependency{"a", {}, {}, {Constraint::Type::Minimum, "2"}}, }, {}, toplevel_spec())); @@ -375,7 +375,7 @@ TEST_CASE ("version string baseline agree", "[versionplan]") MockCMakeVarProvider var_provider; auto install_plan = create_versioned_install_plan( - vp, bp, var_provider, {Dependency{"a", {}, {}, {Constraint::Type::Exact, "2"}}}, {}, toplevel_spec()); + vp, bp, var_provider, {Dependency{"a", {}, {}, {Constraint::Type::Minimum, "2"}}}, {}, toplevel_spec()); REQUIRE(install_plan.has_value()); } @@ -396,7 +396,7 @@ TEST_CASE ("version install scheme baseline conflict", "[versionplan]") bp, var_provider, { - Dependency{"a", {}, {}, {Constraint::Type::Exact, "3"}}, + Dependency{"a", {}, {}, {Constraint::Type::Minimum, "3"}}, }, {}, toplevel_spec()); @@ -421,7 +421,7 @@ TEST_CASE ("version install string port version", "[versionplan]") bp, var_provider, { - Dependency{"a", {}, {}, {Constraint::Type::Exact, "2", 1}}, + Dependency{"a", {}, {}, {Constraint::Type::Minimum, "2", 1}}, }, {}, toplevel_spec())); @@ -447,7 +447,7 @@ TEST_CASE ("version install string port version 2", "[versionplan]") bp, var_provider, { - Dependency{"a", {}, {}, {Constraint::Type::Exact, "2", 0}}, + Dependency{"a", {}, {}, {Constraint::Type::Minimum, "2", 0}}, }, {}, toplevel_spec())); @@ -463,10 +463,10 @@ TEST_CASE ("version install transitive string", "[versionplan]") MockVersionedPortfileProvider vp; vp.emplace("a", {"2", 0}).source_control_file->core_paragraph->dependencies = { - Dependency{"b", {}, {}, DependencyConstraint{Constraint::Type::Exact, "1"}}, + Dependency{"b", {}, {}, DependencyConstraint{Constraint::Type::Minimum, "1"}}, }; vp.emplace("a", {"2", 1}).source_control_file->core_paragraph->dependencies = { - Dependency{"b", {}, {}, DependencyConstraint{Constraint::Type::Exact, "2"}}, + Dependency{"b", {}, {}, DependencyConstraint{Constraint::Type::Minimum, "2"}}, }; vp.emplace("b", {"1", 0}); vp.emplace("b", {"2", 0}); @@ -478,7 +478,7 @@ TEST_CASE ("version install transitive string", "[versionplan]") bp, var_provider, { - Dependency{"a", {}, {}, {Constraint::Type::Exact, "2", 1}}, + Dependency{"a", {}, {}, {Constraint::Type::Minimum, "2", 1}}, }, {}, toplevel_spec())); @@ -1006,7 +1006,7 @@ TEST_CASE ("version install scheme change in port version", "[versionplan]") { MockVersionedPortfileProvider vp; vp.emplace("a", {"2", 0}).source_control_file->core_paragraph->dependencies = { - Dependency{"b", {}, {}, DependencyConstraint{Constraint::Type::Exact, "1"}}, + Dependency{"b", {}, {}, DependencyConstraint{Constraint::Type::Minimum, "1"}}, }; vp.emplace("a", {"2", 1}).source_control_file->core_paragraph->dependencies = { Dependency{"b", {}, {}, DependencyConstraint{Constraint::Type::Minimum, "1", 1}}, @@ -1026,7 +1026,7 @@ TEST_CASE ("version install scheme change in port version", "[versionplan]") bp, var_provider, { - Dependency{"a", {}, {}, {Constraint::Type::Exact, "2", 1}}, + Dependency{"a", {}, {}, {Constraint::Type::Minimum, "2", 1}}, }, {}, toplevel_spec())); @@ -1045,7 +1045,7 @@ TEST_CASE ("version install scheme change in port version", "[versionplan]") bp, var_provider, { - Dependency{"a", {}, {}, {Constraint::Type::Exact, "2", 0}}, + Dependency{"a", {}, {}, {Constraint::Type::Minimum, "2", 0}}, }, {}, toplevel_spec())); @@ -1305,7 +1305,8 @@ TEST_CASE ("version install transitive overrides", "[versionplan]") MockVersionedPortfileProvider vp; vp.emplace("b", {"1", 0}, Scheme::Relaxed) - .source_control_file->core_paragraph->dependencies.push_back({"c", {}, {}, {Constraint::Type::Exact, "2", 1}}); + .source_control_file->core_paragraph->dependencies.push_back( + {"c", {}, {}, {Constraint::Type::Minimum, "2", 1}}); vp.emplace("b", {"2", 0}, Scheme::Relaxed); vp.emplace("c", {"1", 0}, Scheme::String); vp.emplace("c", {"2", 1}, Scheme::String); diff --git a/toolsrc/src/vcpkg-test/manifests.cpp b/toolsrc/src/vcpkg-test/manifests.cpp index 39e470075..0af9f4f96 100644 --- a/toolsrc/src/vcpkg-test/manifests.cpp +++ b/toolsrc/src/vcpkg-test/manifests.cpp @@ -62,6 +62,7 @@ TEST_CASE ("manifest construct minimum", "[manifests]") REQUIRE(pgh.core_paragraph->maintainers.empty()); REQUIRE(pgh.core_paragraph->description.empty()); REQUIRE(pgh.core_paragraph->dependencies.empty()); + REQUIRE(!pgh.core_paragraph->builtin_baseline.has_value()); REQUIRE(!pgh.check_against_feature_flags({}, feature_flags_without_versioning)); } @@ -157,19 +158,31 @@ TEST_CASE ("manifest versioning", "[manifests]") } } -TEST_CASE ("manifest constraints error hash", "[manifests]") +TEST_CASE ("manifest constraints hash", "[manifests]") { + auto p = unwrap(test_parse_manifest(R"json({ + "name": "zlib", + "version-string": "abcd", + "dependencies": [ + { + "name": "d", + "version>=": "2018-09-01#1" + } + ] +})json")); + REQUIRE(p->core_paragraph->dependencies.at(0).constraint.value == "2018-09-01"); + REQUIRE(p->core_paragraph->dependencies.at(0).constraint.port_version == 1); + test_parse_manifest(R"json({ "name": "zlib", "version-string": "abcd", "dependencies": [ { - "name": "b", - "version=": "5#1" + "name": "d", + "version>=": "2018-09-01#0" } ] -} -)json", +})json", true); test_parse_manifest(R"json({ @@ -178,7 +191,20 @@ TEST_CASE ("manifest constraints error hash", "[manifests]") "dependencies": [ { "name": "d", - "version>=": "2018-09-01#1" + "version>=": "2018-09-01#-1" + } + ] +})json", + true); + + test_parse_manifest(R"json({ + "name": "zlib", + "version-string": "abcd", + "dependencies": [ + { + "name": "d", + "version>=": "2018-09-01", + "port-version": 1 } ] })json", @@ -238,14 +264,10 @@ TEST_CASE ("manifest constraints", "[manifests]") std::string raw = R"json({ "name": "zlib", "version-string": "abcd", + "builtin-baseline": "089fa4de7dca22c67dcab631f618d5cd0697c8d4", "dependencies": [ "a", { - "name": "b", - "port-version": 12, - "version=": "5" - }, - { "$extra": null, "name": "c" }, @@ -263,32 +285,17 @@ TEST_CASE ("manifest constraints", "[manifests]") REQUIRE(pgh.check_against_feature_flags({}, feature_flags_without_versioning)); REQUIRE(!pgh.check_against_feature_flags({}, feature_flags_with_versioning)); REQUIRE(Json::stringify(serialize_manifest(pgh), Json::JsonStyle::with_spaces(4)) == raw); - REQUIRE(pgh.core_paragraph->dependencies.size() == 4); + REQUIRE(pgh.core_paragraph->dependencies.size() == 3); REQUIRE(pgh.core_paragraph->dependencies[0].name == "a"); REQUIRE(pgh.core_paragraph->dependencies[0].constraint == DependencyConstraint{Versions::Constraint::Type::None, "", 0}); - REQUIRE(pgh.core_paragraph->dependencies[1].name == "b"); + REQUIRE(pgh.core_paragraph->dependencies[1].name == "c"); REQUIRE(pgh.core_paragraph->dependencies[1].constraint == - DependencyConstraint{Versions::Constraint::Type::Exact, "5", 12}); - REQUIRE(pgh.core_paragraph->dependencies[2].name == "c"); - REQUIRE(pgh.core_paragraph->dependencies[2].constraint == DependencyConstraint{Versions::Constraint::Type::None, "", 0}); - REQUIRE(pgh.core_paragraph->dependencies[3].name == "d"); - REQUIRE(pgh.core_paragraph->dependencies[3].constraint == + REQUIRE(pgh.core_paragraph->dependencies[2].name == "d"); + REQUIRE(pgh.core_paragraph->dependencies[2].constraint == DependencyConstraint{Versions::Constraint::Type::Minimum, "2018-09-01", 0}); - - test_parse_manifest(R"json({ - "name": "zlib", - "version-string": "abcd", - "dependencies": [ - { - "name": "d", - "version=": "2018-09-01", - "version>=": "2018-09-01" - } - ] - })json", - true); + REQUIRE(pgh.core_paragraph->builtin_baseline == "089fa4de7dca22c67dcab631f618d5cd0697c8d4"); test_parse_manifest(R"json({ "name": "zlib", @@ -303,6 +310,45 @@ TEST_CASE ("manifest constraints", "[manifests]") true); } +TEST_CASE ("manifest builtin-baseline", "[manifests]") +{ + SECTION ("valid baseline") + { + std::string raw = R"json({ + "name": "zlib", + "version-string": "abcd", + "builtin-baseline": "089fa4de7dca22c67dcab631f618d5cd0697c8d4" +} +)json"; + auto m_pgh = test_parse_manifest(raw); + + REQUIRE(m_pgh.has_value()); + auto& pgh = **m_pgh.get(); + REQUIRE(pgh.check_against_feature_flags({}, feature_flags_without_versioning)); + REQUIRE(!pgh.check_against_feature_flags({}, feature_flags_with_versioning)); + REQUIRE(pgh.core_paragraph->builtin_baseline.value_or("does not have a value") == + "089fa4de7dca22c67dcab631f618d5cd0697c8d4"); + } + + SECTION ("empty baseline") + { + std::string raw = R"json({ + "name": "zlib", + "version-string": "abcd", + "builtin-baseline": "" +} +)json"; + + auto m_pgh = test_parse_manifest(raw); + + REQUIRE(m_pgh.has_value()); + auto& pgh = **m_pgh.get(); + REQUIRE(pgh.check_against_feature_flags({}, feature_flags_without_versioning)); + REQUIRE(!pgh.check_against_feature_flags({}, feature_flags_with_versioning)); + REQUIRE(pgh.core_paragraph->builtin_baseline.value_or("does not have a value") == ""); + } +} + TEST_CASE ("manifest overrides", "[manifests]") { std::tuple<StringLiteral, Versions::Scheme, StringLiteral> data[] = { diff --git a/toolsrc/src/vcpkg/base/json.cpp b/toolsrc/src/vcpkg/base/json.cpp index 4a784235e..d99fa7392 100644 --- a/toolsrc/src/vcpkg/base/json.cpp +++ b/toolsrc/src/vcpkg/base/json.cpp @@ -1064,6 +1064,11 @@ namespace vcpkg::Json { return Parser::parse(json, fs::generic_u8string(filepath)); } + ExpectedT<std::pair<Value, JsonStyle>, std::unique_ptr<Parse::IParseError>> parse(StringView json, + StringView origin) noexcept + { + return Parser::parse(json, origin); + } // } auto parse() namespace diff --git a/toolsrc/src/vcpkg/commands.civerifyversions.cpp b/toolsrc/src/vcpkg/commands.civerifyversions.cpp index 84aab9c01..71ced9f76 100644 --- a/toolsrc/src/vcpkg/commands.civerifyversions.cpp +++ b/toolsrc/src/vcpkg/commands.civerifyversions.cpp @@ -404,7 +404,7 @@ namespace vcpkg::Commands::CIVerifyVersions System::printf(System::Color::error, "%s\n", error); } System::print2(System::Color::error, - "\nTo attempt to resolve all erros at once, run:\n\n" + "\nTo attempt to resolve all errors at once, run:\n\n" " vcpkg x-add-version --all\n\n"); Checks::exit_fail(VCPKG_LINE_INFO); } diff --git a/toolsrc/src/vcpkg/help.cpp b/toolsrc/src/vcpkg/help.cpp index ded6592fb..0556b4fa0 100644 --- a/toolsrc/src/vcpkg/help.cpp +++ b/toolsrc/src/vcpkg/help.cpp @@ -47,7 +47,69 @@ namespace vcpkg::Help nullptr, }; - static constexpr std::array<Topic, 15> topics = {{ + static void help_topic_versioning(const VcpkgPaths&) + { + HelpTableFormatter tbl; + tbl.text("Versioning allows you to deterministically control the precise revisions of dependencies used by " + "your project from within your manifest file."); + tbl.blank(); + tbl.blank(); + tbl.text("** This feature is experimental and requires `--feature-flags=versions` **"); + tbl.blank(); + tbl.blank(); + tbl.header("Versions in vcpkg come in four primary flavors"); + tbl.format("version", "A dot-separated sequence of numbers (1.2.3.4)"); + tbl.format("version-date", "A date (2021-01-01.5)"); + tbl.format("version-semver", "A Semantic Version 2.0 (2.1.0-rc2)"); + tbl.format("version-string", "An exact, incomparable version (Vista)"); + tbl.blank(); + tbl.text("Each version additionally has a \"port-version\" which is a nonnegative integer. When rendered as " + "text, the port version (if nonzero) is added as a suffix to the primary version text separated by a " + "hash (#). Port-versions are sorted lexographically after the primary version text, for example:"); + tbl.blank(); + tbl.blank(); + tbl.text(" 1.0.0 < 1.0.0#1 < 1.0.1 < 1.0.1#5 < 2.0.0"); + tbl.blank(); + tbl.blank(); + tbl.header("Manifests can place three kinds of constraints upon the versions used"); + tbl.format("builtin-baseline", + "The baseline references a commit within the vcpkg repository that establishes a minimum version on " + "every dependency in the graph. If no other constraints are specified (directly or transitively), " + "then the version from the baseline of the top level manifest will be used. Baselines of transitive " + "dependencies are ignored."); + tbl.blank(); + tbl.format("version>=", + "Within the \"dependencies\" field, each dependency can have a minimum constraint listed. These " + "minimum constraints will be used when transitively depending upon this library. A minimum " + "port-version can additionally be specified with a '#' suffix."); + tbl.blank(); + tbl.format( + "overrides", + "When used as the top-level manifest (such as when running `vcpkg install` in the directory), overrides " + "allow a manifest to short-circuit dependency resolution and specify exactly the version to use. These can " + "be used to handle version conflicts, such as with `version-string` dependencies. They will not be " + "considered when transitively depended upon."); + tbl.blank(); + tbl.text("Example manifest:"); + tbl.blank(); + tbl.text(R"({ + "name": "example", + "version": "1.0", + "builtin-baseline": "a14a6bcb27287e3ec138dba1b948a0cdbc337a3a", + "dependencies": [ + { "name": "zlib", "version>=": "1.2.11#8" }, + "rapidjson" + ], + "overrides": [ + { "name": "rapidjson", "version": "2020-09-14" } + ] +})"); + System::print2(tbl.m_str, + "\nExtended documentation is available at " + "https://github.com/Microsoft/vcpkg/tree/master/docs/users/versioning.md\n"); + } + + static constexpr std::array<Topic, 16> topics = {{ {"binarycaching", help_topic_binary_caching}, {"create", command_topic_fn<Commands::Create::COMMAND_STRUCTURE>}, {"depend-info", command_topic_fn<Commands::DependInfo::COMMAND_STRUCTURE>}, @@ -63,6 +125,7 @@ namespace vcpkg::Help {"search", command_topic_fn<Commands::Search::COMMAND_STRUCTURE>}, {"topics", help_topics}, {"triplet", help_topic_valid_triplet}, + {"versioning", help_topic_versioning}, }}; static void help_topics(const VcpkgPaths&) diff --git a/toolsrc/src/vcpkg/install.cpp b/toolsrc/src/vcpkg/install.cpp index 80b4eae3a..e86a78a76 100644 --- a/toolsrc/src/vcpkg/install.cpp +++ b/toolsrc/src/vcpkg/install.cpp @@ -17,6 +17,7 @@ #include <vcpkg/metrics.h> #include <vcpkg/paragraphs.h> #include <vcpkg/remove.h> +#include <vcpkg/tools.h> #include <vcpkg/vcpkglib.h> #include <vcpkg/vcpkgpaths.h> @@ -847,10 +848,25 @@ namespace vcpkg::Install if (args.versions_enabled() || args.registries_enabled()) { - if (auto p_baseline = manifest_scf.core_paragraph->extra_info.get("$x-default-baseline")) + if (!manifest_scf.core_paragraph->overrides.empty()) { - paths.get_configuration().registry_set.experimental_set_builtin_registry_baseline( - p_baseline->string()); + Metrics::g_metrics.lock()->track_property("manifest_overrides", "defined"); + } + if (auto p_baseline = manifest_scf.core_paragraph->builtin_baseline.get()) + { + Metrics::g_metrics.lock()->track_property("manifest_baseline", "defined"); + if (p_baseline->size() != 40 || !std::all_of(p_baseline->begin(), p_baseline->end(), [](char ch) { + return (ch >= 'a' || ch <= 'f') || Parse::ParserBase::is_ascii_digit(ch); + })) + { + Checks::exit_with_message( + VCPKG_LINE_INFO, + "Error: the top-level builtin-baseline (%s) was not a valid commit sha: " + "expected 40 lowercase hexadecimal characters.\n%s\n", + *p_baseline, + paths.get_current_git_sha_message()); + } + paths.get_configuration().registry_set.experimental_set_builtin_registry_baseline(*p_baseline); } } if (args.versions_enabled()) diff --git a/toolsrc/src/vcpkg/paragraphs.cpp b/toolsrc/src/vcpkg/paragraphs.cpp index 32103c76b..2d28c49d7 100644 --- a/toolsrc/src/vcpkg/paragraphs.cpp +++ b/toolsrc/src/vcpkg/paragraphs.cpp @@ -373,7 +373,14 @@ namespace vcpkg::Paragraphs auto error_info = std::make_unique<ParseControlErrorInfo>(); error_info->name = fs::u8string(path.filename()); - error_info->error = "Failed to find either a CONTROL file or vcpkg.json file."; + if (fs.exists(path)) + { + error_info->error = "Failed to find either a CONTROL file or vcpkg.json file."; + } + else + { + error_info->error = "The port directory (" + fs::u8string(path) + ") does not exist"; + } return error_info; } diff --git a/toolsrc/src/vcpkg/registries.cpp b/toolsrc/src/vcpkg/registries.cpp index b9afa089b..934e6537a 100644 --- a/toolsrc/src/vcpkg/registries.cpp +++ b/toolsrc/src/vcpkg/registries.cpp @@ -186,17 +186,6 @@ namespace DelayedInit<Baseline> m_baseline; }; - ExpectedS<fs::path> get_git_baseline_json_path(const VcpkgPaths& paths, StringView baseline_commit_sha) - { - auto baseline_path = paths.git_checkout_baseline(paths.get_filesystem(), baseline_commit_sha); - if (paths.get_filesystem().exists(baseline_path)) - { - return std::move(baseline_path); - } - return {Strings::concat("Error: Baseline database file does not exist: ", fs::u8string(baseline_path)), - expected_right_tag}; - } - struct VersionDbEntry { VersionT version; @@ -224,7 +213,7 @@ namespace // returns nullopt if the baseline is valid, but doesn't contain the specified baseline, // or (equivalently) if the baseline does not exist. - ExpectedS<Optional<Baseline>> parse_baseline_versions(StringView contents, StringView baseline); + ExpectedS<Optional<Baseline>> parse_baseline_versions(StringView contents, StringView baseline, StringView origin); ExpectedS<Optional<Baseline>> load_baseline_versions(const VcpkgPaths& paths, const fs::path& path_to_baseline, StringView identifier = {}); @@ -327,18 +316,20 @@ namespace if (baseline_identifier == "default") { - return Strings::format("Couldn't find explicitly specified baseline `\"default\"` in baseline file: %s", - fs::u8string(path_to_baseline)); + return Strings::format( + "Error: Couldn't find explicitly specified baseline `\"default\"` in baseline file: %s", + fs::u8string(path_to_baseline)); } // attempt to check out the baseline: - auto maybe_path = get_git_baseline_json_path(paths, baseline_identifier); + auto maybe_path = paths.git_checkout_baseline(baseline_identifier); if (!maybe_path.has_value()) { - return Strings::format("Couldn't find explicitly specified baseline `\"%s\"` in the baseline file, " - "and there was no baseline at that commit or the commit didn't exist.\n%s", + return Strings::format("Error: Couldn't find explicitly specified baseline `\"%s\"` in the baseline file, " + "and there was no baseline at that commit or the commit didn't exist.\n%s\n%s", baseline_identifier, - maybe_path.error()); + maybe_path.error(), + paths.get_current_git_sha_message()); } res_baseline = load_baseline_versions(paths, *maybe_path.get()); @@ -352,7 +343,7 @@ namespace return std::move(*p); } - return Strings::format("Couldn't find explicitly specified baseline `\"%s\"` in the baseline " + return Strings::format("Error: Couldn't find explicitly specified baseline `\"%s\"` in the baseline " "file, and the `\"default\"` baseline does not exist at that commit.", baseline_identifier); } @@ -549,7 +540,7 @@ namespace } auto contents = maybe_contents.get(); - res_baseline = parse_baseline_versions(*contents, {}); + res_baseline = parse_baseline_versions(*contents, "default", fs::u8string(path_to_baseline)); if (!res_baseline.has_value()) { Checks::exit_with_message(VCPKG_LINE_INFO, res_baseline.error()); @@ -599,12 +590,22 @@ namespace auto it = std::find(git_entry->port_versions.begin(), git_entry->port_versions.end(), version); if (it == git_entry->port_versions.end()) { - return Strings::concat( - "Error: No version entry for ", git_entry->port_name, " at version ", version, "."); + return { + Strings::concat("Error: No version entry for ", + git_entry->port_name, + " at version ", + version, + ". This may be fixed by updating vcpkg to the latest master via `git " + "pull`.\nAvailable versions:\n", + Strings::join("", + git_entry->port_versions, + [](const VersionT& v) { return Strings::concat(" ", v, "\n"); }), + "\nSee `vcpkg help versioning` for more information."), + expected_right_tag}; } const auto& git_tree = git_entry->git_trees[it - git_entry->port_versions.begin()]; - return paths.git_checkout_port(paths.get_filesystem(), git_entry->port_name, git_tree); + return paths.git_checkout_port(git_entry->port_name, git_tree, paths.root / fs::u8path(".git")); } if (scfl_version == version) @@ -1014,19 +1015,20 @@ namespace return db_entries; } - ExpectedS<Optional<Baseline>> parse_baseline_versions(StringView contents, StringView baseline) + ExpectedS<Optional<Baseline>> parse_baseline_versions(StringView contents, StringView baseline, StringView origin) { - auto maybe_value = Json::parse(contents); + auto maybe_value = Json::parse(contents, origin); if (!maybe_value.has_value()) { - return Strings::format("Error: failed to parse baseline file: %s", maybe_value.error()->format()); + return Strings::format( + "Error: failed to parse baseline file: %s\n%s", origin, maybe_value.error()->format()); } auto& value = *maybe_value.get(); if (!value.first.is_object()) { - return std::string("Error: baseline does not have a top-level object."); + return Strings::concat("Error: baseline does not have a top-level object: ", origin); } auto real_baseline = baseline.size() == 0 ? "default" : baseline; @@ -1047,8 +1049,7 @@ namespace } else { - Checks::exit_with_message( - VCPKG_LINE_INFO, "Error: failed to parse baseline:\n%s", Strings::join("\n", r.errors())); + return Strings::format("Error: failed to parse baseline: %s\n%s", origin, Strings::join("\n", r.errors())); } } @@ -1059,7 +1060,7 @@ namespace auto maybe_contents = paths.get_filesystem().read_contents(path_to_baseline); if (auto contents = maybe_contents.get()) { - return parse_baseline_versions(*contents, baseline); + return parse_baseline_versions(*contents, baseline, fs::u8string(path_to_baseline)); } else if (maybe_contents.error() == std::errc::no_such_file_or_directory) { @@ -1143,7 +1144,7 @@ namespace vcpkg if (!default_registry_is_builtin || registries_.size() != 0) { System::print2(System::Color::warning, - "Warning: when using the registries feature, one should not use `\"$x-default-baseline\"` " + "Warning: when using the registries feature, one should not use `\"builtin-baseline\"` " "to set the baseline.\n", " Instead, use the \"baseline\" field of the registry.\n"); } diff --git a/toolsrc/src/vcpkg/sourceparagraph.cpp b/toolsrc/src/vcpkg/sourceparagraph.cpp index c4ae3e855..61cc7f826 100644 --- a/toolsrc/src/vcpkg/sourceparagraph.cpp +++ b/toolsrc/src/vcpkg/sourceparagraph.cpp @@ -430,8 +430,6 @@ namespace vcpkg constexpr static StringLiteral FEATURES = "features"; constexpr static StringLiteral DEFAULT_FEATURES = "default-features"; constexpr static StringLiteral PLATFORM = "platform"; - constexpr static StringLiteral PORT_VERSION = "port-version"; - constexpr static StringLiteral VERSION_EQ = "version="; constexpr static StringLiteral VERSION_GE = "version>="; virtual Span<const StringView> valid_fields() const override @@ -441,8 +439,6 @@ namespace vcpkg FEATURES, DEFAULT_FEATURES, PLATFORM, - PORT_VERSION, - VERSION_EQ, VERSION_GE, }; @@ -488,33 +484,34 @@ namespace vcpkg r.optional_object_field(obj, PLATFORM, dep.platform, PlatformExprDeserializer::instance); - static auto version_deserializer = make_version_deserializer("a version"); + static Json::StringDeserializer version_deserializer("a version"); - auto has_eq_constraint = - r.optional_object_field(obj, VERSION_EQ, dep.constraint.value, *version_deserializer); auto has_ge_constraint = - r.optional_object_field(obj, VERSION_GE, dep.constraint.value, *version_deserializer); - auto has_port_ver = r.optional_object_field( - obj, PORT_VERSION, dep.constraint.port_version, Json::NaturalNumberDeserializer::instance); + r.optional_object_field(obj, VERSION_GE, dep.constraint.value, version_deserializer); - if (has_eq_constraint) + if (has_ge_constraint) { - dep.constraint.type = Versions::Constraint::Type::Exact; - if (has_ge_constraint) + dep.constraint.type = Versions::Constraint::Type::Minimum; + auto h = dep.constraint.value.find('#'); + if (h != std::string::npos) { - r.add_generic_error(type_name(), "cannot have both exact and minimum constraints simultaneously"); + auto opt = Strings::strto<int>(dep.constraint.value.c_str() + h + 1); + auto v = opt.get(); + if (v && *v > 0) + { + dep.constraint.port_version = *v; + } + else + { + r.add_generic_error(type_name(), + "embedded port-version ('#') in the primary " + "constraint (\"", + VERSION_GE, + "\") must be a positive integer"); + } + dep.constraint.value.erase(h); } } - else if (has_ge_constraint) - { - dep.constraint.type = Versions::Constraint::Type::Minimum; - } - else if (has_port_ver) // does not have a primary constraint - { - r.add_generic_error( - type_name(), - "\"port-version\" cannot be used without a primary constraint (\"version=\" or \"version>=\")"); - } return dep; } @@ -540,9 +537,7 @@ namespace vcpkg constexpr StringLiteral DependencyDeserializer::FEATURES; constexpr StringLiteral DependencyDeserializer::DEFAULT_FEATURES; constexpr StringLiteral DependencyDeserializer::PLATFORM; - constexpr StringLiteral DependencyDeserializer::VERSION_EQ; constexpr StringLiteral DependencyDeserializer::VERSION_GE; - constexpr StringLiteral DependencyDeserializer::PORT_VERSION; struct DependencyOverrideDeserializer : Json::IDeserializer<DependencyOverride> { @@ -860,6 +855,21 @@ namespace vcpkg }; LicenseExpressionDeserializer LicenseExpressionDeserializer::instance; + struct BaselineCommitDeserializer final : Json::IDeserializer<std::string> + { + virtual StringView type_name() const override { return "a vcpkg repository commit"; } + + virtual Optional<std::string> visit_string(Json::Reader&, StringView s) override + { + // We allow non-sha strings here to allow the core vcpkg code to provide better error + // messages including the current git commit + return s.to_string(); + } + + static BaselineCommitDeserializer instance; + }; + BaselineCommitDeserializer BaselineCommitDeserializer::instance; + struct ManifestDeserializer : Json::IDeserializer<std::unique_ptr<SourceControlFile>> { virtual StringView type_name() const override { return "a manifest"; } @@ -876,6 +886,7 @@ namespace vcpkg constexpr static StringLiteral DEFAULT_FEATURES = "default-features"; constexpr static StringLiteral SUPPORTS = "supports"; constexpr static StringLiteral OVERRIDES = "overrides"; + constexpr static StringLiteral BUILTIN_BASELINE = "builtin-baseline"; virtual Span<const StringView> valid_fields() const override { @@ -892,6 +903,7 @@ namespace vcpkg DEFAULT_FEATURES, SUPPORTS, OVERRIDES, + BUILTIN_BASELINE, }; static const auto t = Util::Vectors::concat<StringView>(schemed_deserializer_fields(), u); @@ -917,9 +929,9 @@ namespace vcpkg static Json::StringDeserializer url_deserializer{"a url"}; - constexpr static StringView type_name = "vcpkg.json"; + constexpr static StringView inner_type_name = "vcpkg.json"; DependencyOverrideDeserializer::visit_impl( - type_name, r, obj, spgh->name, spgh->version, spgh->version_scheme, spgh->port_version); + inner_type_name, r, obj, spgh->name, spgh->version, spgh->version_scheme, spgh->port_version); r.optional_object_field(obj, MAINTAINERS, spgh->maintainers, Json::ParagraphDeserializer::instance); r.optional_object_field(obj, DESCRIPTION, spgh->description, Json::ParagraphDeserializer::instance); @@ -933,8 +945,12 @@ namespace vcpkg if (obj.contains(DEV_DEPENDENCIES)) { - System::print2(System::Color::error, DEV_DEPENDENCIES, " are not yet supported"); - Checks::exit_fail(VCPKG_LINE_INFO); + r.add_generic_error(type_name(), DEV_DEPENDENCIES, " are not yet supported"); + } + std::string baseline; + if (r.optional_object_field(obj, BUILTIN_BASELINE, baseline, BaselineCommitDeserializer::instance)) + { + spgh->builtin_baseline = std::move(baseline); } r.optional_object_field(obj, SUPPORTS, spgh->supports_expression, PlatformExprDeserializer::instance); @@ -1007,6 +1023,7 @@ namespace vcpkg Optional<std::string> SourceControlFile::check_against_feature_flags(const fs::path& origin, const FeatureFlagSettings& flags) const { + static constexpr StringLiteral s_extended_help = "See `vcpkg help versioning` for more information."; if (!flags.versions) { auto check_deps = [&](View<Dependency> deps) -> Optional<std::string> { @@ -1014,11 +1031,12 @@ namespace vcpkg { if (dep.constraint.type != Versions::Constraint::Type::None) { - return Strings::concat(fs::u8string(origin), - " was rejected because it uses constraints and the `", - VcpkgCmdArguments::VERSIONS_FEATURE, - "` feature flag is disabled.\nThis can be fixed by removing uses of " - "\"version>=\" and \"version=\"."); + return Strings::concat( + fs::u8string(origin), + " was rejected because it uses constraints and the `", + VcpkgCmdArguments::VERSIONS_FEATURE, + "` feature flag is disabled.\nThis can be fixed by removing uses of \"version>=\".\n", + s_extended_help); } } return nullopt; @@ -1036,7 +1054,18 @@ namespace vcpkg return Strings::concat(fs::u8string(origin), " was rejected because it uses overrides and the `", VcpkgCmdArguments::VERSIONS_FEATURE, - "` feature flag is disabled.\nThis can be fixed by removing \"overrides\"."); + "` feature flag is disabled.\nThis can be fixed by removing \"overrides\".\n", + s_extended_help); + } + + if (core_paragraph->builtin_baseline.has_value()) + { + return Strings::concat( + fs::u8string(origin), + " was rejected because it uses builtin-baseline and the `", + VcpkgCmdArguments::VERSIONS_FEATURE, + "` feature flag is disabled.\nThis can be fixed by removing \"builtin-baseline\".\n", + s_extended_help); } } return nullopt; @@ -1242,19 +1271,14 @@ namespace vcpkg serialize_optional_array(dep_obj, DependencyDeserializer::FEATURES, features_copy); serialize_optional_string(dep_obj, DependencyDeserializer::PLATFORM, to_string(dep.platform)); - if (dep.constraint.port_version != 0) - { - dep_obj.insert(DependencyDeserializer::PORT_VERSION, - Json::Value::integer(dep.constraint.port_version)); - } - - if (dep.constraint.type == Versions::Constraint::Type::Exact) - { - dep_obj.insert(DependencyDeserializer::VERSION_EQ, Json::Value::string(dep.constraint.value)); - } - else if (dep.constraint.type == Versions::Constraint::Type::Minimum) + if (dep.constraint.type == Versions::Constraint::Type::Minimum) { - dep_obj.insert(DependencyDeserializer::VERSION_GE, Json::Value::string(dep.constraint.value)); + auto s = dep.constraint.value; + if (dep.constraint.port_version != 0) + { + Strings::append(s, '#', dep.constraint.port_version); + } + dep_obj.insert(DependencyDeserializer::VERSION_GE, Json::Value::string(std::move(s))); } } }; @@ -1294,6 +1318,11 @@ namespace vcpkg serialize_optional_string(obj, ManifestDeserializer::LICENSE, scf.core_paragraph->license); serialize_optional_string( obj, ManifestDeserializer::SUPPORTS, to_string(scf.core_paragraph->supports_expression)); + if (scf.core_paragraph->builtin_baseline.has_value()) + { + obj.insert(ManifestDeserializer::BUILTIN_BASELINE, + Json::Value::string(scf.core_paragraph->builtin_baseline.value_or_exit(VCPKG_LINE_INFO))); + } if (!scf.core_paragraph->dependencies.empty() || debug) { diff --git a/toolsrc/src/vcpkg/vcpkgpaths.cpp b/toolsrc/src/vcpkg/vcpkgpaths.cpp index afd724765..7926e7a5f 100644 --- a/toolsrc/src/vcpkg/vcpkgpaths.cpp +++ b/toolsrc/src/vcpkg/vcpkgpaths.cpp @@ -554,6 +554,33 @@ If you wish to silence this error and use classic mode, you can: } } + ExpectedS<std::string> VcpkgPaths::get_current_git_sha() const + { + auto cmd = git_cmd_builder(*this, this->root / fs::u8path(".git"), this->root); + cmd.string_arg("rev-parse").string_arg("HEAD"); + auto output = System::cmd_execute_and_capture_output(cmd); + if (output.exit_code != 0) + { + return {std::move(output.output), expected_right_tag}; + } + else + { + return {Strings::trim(std::move(output.output)), expected_left_tag}; + } + } + std::string VcpkgPaths::get_current_git_sha_message() const + { + auto maybe_cur_sha = get_current_git_sha(); + if (auto p_sha = maybe_cur_sha.get()) + { + return Strings::concat("The current commit is \"", *p_sha, '"'); + } + else + { + return Strings::concat("Failed to determine the current commit:\n", maybe_cur_sha.error()); + } + } + ExpectedS<std::string> VcpkgPaths::git_show(const std::string& treeish, const fs::path& dot_git_dir) const { // All git commands are run with: --git-dir={dot_git_dir} --work-tree={work_tree_temp} @@ -620,119 +647,148 @@ If you wish to silence this error and use classic mode, you can: return ret; } - void VcpkgPaths::git_checkout_object(const VcpkgPaths& paths, - StringView git_object, - const fs::path& local_repo, - const fs::path& destination, - const fs::path& dot_git_dir, - const fs::path& work_tree) - { - Files::Filesystem& fs = paths.get_filesystem(); - fs.remove_all(work_tree, VCPKG_LINE_INFO); - fs.remove_all(destination, VCPKG_LINE_INFO); - - if (!fs.exists(dot_git_dir)) - { - // All git commands are run with: --git-dir={dot_git_dir} --work-tree={work_tree_temp} - // git clone --no-checkout --local {vcpkg_root} {dot_git_dir} - System::CmdLineBuilder clone_cmd_builder = git_cmd_builder(paths, dot_git_dir, work_tree) - .string_arg("clone") - .string_arg("--no-checkout") - .string_arg("--local") - .string_arg("--no-hardlinks") - .path_arg(local_repo) - .path_arg(dot_git_dir); - const auto clone_output = System::cmd_execute_and_capture_output(clone_cmd_builder); - Checks::check_exit(VCPKG_LINE_INFO, - clone_output.exit_code == 0, - "Failed to clone temporary vcpkg instance.\n%s\n", - clone_output.output); - } - else - { - System::CmdLineBuilder fetch_cmd_builder = - git_cmd_builder(paths, dot_git_dir, work_tree).string_arg("fetch"); - const auto fetch_output = System::cmd_execute_and_capture_output(fetch_cmd_builder); - Checks::check_exit(VCPKG_LINE_INFO, - fetch_output.exit_code == 0, - "Failed to update refs on temporary vcpkg repository.\n%s\n", - fetch_output.output); - } - - if (!fs.exists(work_tree)) - { - fs.create_directories(work_tree, VCPKG_LINE_INFO); - } - - // git checkout {tree_object} . - System::CmdLineBuilder checkout_cmd_builder = git_cmd_builder(paths, dot_git_dir, work_tree) - .string_arg("checkout") - .string_arg(git_object) - .string_arg("."); - const auto checkout_output = System::cmd_execute_and_capture_output(checkout_cmd_builder); - Checks::check_exit(VCPKG_LINE_INFO, checkout_output.exit_code == 0, "Failed to checkout %s", git_object); - - const auto& containing_folder = destination.parent_path(); - if (!fs.exists(containing_folder)) - { - fs.create_directories(containing_folder, VCPKG_LINE_INFO); - } - - std::error_code ec; - fs.rename_or_copy(work_tree, destination, ".tmp", ec); - fs.remove_all(work_tree, VCPKG_LINE_INFO); - if (ec) - { - System::printf(System::Color::error, - "Error: Couldn't move checked out files from %s to destination %s", - fs::u8string(work_tree), - fs::u8string(destination)); - Checks::exit_fail(VCPKG_LINE_INFO); - } - } - - fs::path VcpkgPaths::git_checkout_baseline(Files::Filesystem& fs, StringView commit_sha) const + ExpectedS<fs::path> VcpkgPaths::git_checkout_baseline(StringView commit_sha) const { + Files::Filesystem& fs = get_filesystem(); const fs::path destination_parent = this->baselines_output / fs::u8path(commit_sha); - const fs::path destination = destination_parent / fs::u8path("baseline.json"); + fs::path destination = destination_parent / fs::u8path("baseline.json"); if (!fs.exists(destination)) { + const fs::path destination_tmp = destination_parent / fs::u8path("baseline.json.tmp"); auto treeish = Strings::concat(commit_sha, ":port_versions/baseline.json"); auto maybe_contents = git_show(treeish, this->root / fs::u8path(".git")); if (auto contents = maybe_contents.get()) { - fs.create_directories(destination_parent, VCPKG_LINE_INFO); - fs.write_contents(destination, *contents, VCPKG_LINE_INFO); + std::error_code ec; + fs.create_directories(destination_parent, ec); + if (ec) + { + return {Strings::format( + "Error: while checking out baseline %s\nError: while creating directories %s: %s", + commit_sha, + fs::u8string(destination_parent), + ec.message()), + expected_right_tag}; + } + fs.write_contents(destination_tmp, *contents, ec); + if (ec) + { + return {Strings::format("Error: while checking out baseline %s\nError: while writing %s: %s", + commit_sha, + fs::u8string(destination_tmp), + ec.message()), + expected_right_tag}; + } + fs.rename(destination_tmp, destination, ec); + if (ec) + { + return {Strings::format("Error: while checking out baseline %s\nError: while renaming %s to %s: %s", + commit_sha, + fs::u8string(destination_tmp), + fs::u8string(destination), + ec.message()), + expected_right_tag}; + } } else { - Checks::exit_with_message( - VCPKG_LINE_INFO, "Error: while checking out baseline '%s':\n%s", treeish, maybe_contents.error()); + return {Strings::format("Error: while checking out baseline '%s':\n%s\nThis may be fixed by updating " + "vcpkg to the latest master via `git pull`.", + treeish, + maybe_contents.error()), + expected_right_tag}; } } return destination; } - fs::path VcpkgPaths::git_checkout_port(Files::Filesystem& fs, StringView port_name, StringView git_tree) const + ExpectedS<fs::path> VcpkgPaths::git_checkout_port(StringView port_name, + StringView git_tree, + const fs::path& dot_git_dir) const { - /* Clone a new vcpkg repository instance using the local instance as base. - * - * The `--git-dir` directory will store all the Git metadata files, - * and the `--work-tree` is the directory where files will be checked out. + /* Check out a git tree into the versioned port recipes folder * * Since we are checking a git tree object, all files will be checked out to the root of `work-tree`. * Because of that, it makes sense to use the git hash as the name for the directory. */ - const fs::path& local_repo = this->root; - fs::path destination = this->versions_output / fs::u8path(git_tree) / fs::u8path(port_name); + Files::Filesystem& fs = get_filesystem(); + fs::path destination = this->versions_output / fs::u8path(port_name) / fs::u8path(git_tree); + if (fs.exists(destination)) + { + return destination; + } - if (!fs.exists(destination / "CONTROL") && !fs.exists(destination / "vcpkg.json")) + const fs::path destination_tmp = + this->versions_output / fs::u8path(port_name) / fs::u8path(Strings::concat(git_tree, ".tmp")); + const fs::path destination_tar = + this->versions_output / fs::u8path(port_name) / fs::u8path(Strings::concat(git_tree, ".tar")); +#define PRELUDE "Error: while checking out port ", port_name, " with git tree ", git_tree, "\n" + std::error_code ec; + fs::path failure_point; + fs.remove_all(destination_tmp, ec, failure_point); + if (ec) { - git_checkout_object( - *this, git_tree, local_repo, destination, this->versions_dot_git_dir, this->versions_work_tree); + return {Strings::concat(PRELUDE, "Error: while removing ", fs::u8string(failure_point), ": ", ec.message()), + expected_right_tag}; } + fs.create_directories(destination_tmp, ec); + if (ec) + { + return { + Strings::concat( + PRELUDE, "Error: while creating directories ", fs::u8string(destination_tmp), ": ", ec.message()), + expected_right_tag}; + } + + System::CmdLineBuilder tar_cmd_builder = git_cmd_builder(*this, dot_git_dir, dot_git_dir) + .string_arg("archive") + .string_arg(git_tree) + .string_arg("-o") + .path_arg(destination_tar); + const auto tar_output = System::cmd_execute_and_capture_output(tar_cmd_builder); + if (tar_output.exit_code != 0) + { + return {Strings::concat(PRELUDE, "Error: Failed to tar port directory\n", tar_output.output), + expected_right_tag}; + } + + System::CmdLineBuilder extract_cmd_builder; + extract_cmd_builder.path_arg(this->get_tool_exe(Tools::CMAKE)) + .string_arg("-E") + .string_arg("tar") + .string_arg("xf") + .path_arg(destination_tar); + + const auto extract_output = + System::cmd_execute_and_capture_output(extract_cmd_builder, System::InWorkingDirectory{destination_tmp}); + if (extract_output.exit_code != 0) + { + return {Strings::concat(PRELUDE, "Error: Failed to extract port directory\n", extract_output.output), + expected_right_tag}; + } + fs.remove(destination_tar, ec); + if (ec) + { + return { + Strings::concat(PRELUDE, "Error: while removing ", fs::u8string(destination_tar), ": ", ec.message()), + expected_right_tag}; + } + fs.rename(destination_tmp, destination, ec); + if (ec) + { + return {Strings::concat(PRELUDE, + "Error: while renaming ", + fs::u8string(destination_tmp), + " to ", + fs::u8string(destination), + ": ", + ec.message()), + expected_right_tag}; + } + return destination; +#undef PRELUDE } ExpectedS<std::string> VcpkgPaths::git_fetch_from_remote_registry(StringView repo, StringView treeish) const @@ -757,7 +813,7 @@ If you wish to silence this error and use classic mode, you can: auto lock_file = work_tree / fs::u8path(".vcpkg-lock"); std::error_code ec; - auto guard = Files::ExclusiveFileLock(Files::ExclusiveFileLock::Wait::Yes, fs, lock_file, ec); + Files::ExclusiveFileLock guard(Files::ExclusiveFileLock::Wait::Yes, fs, lock_file, ec); System::CmdLineBuilder fetch_git_ref = git_cmd_builder(*this, dot_git_dir, work_tree).string_arg("fetch").string_arg("--").string_arg(repo); |