From 2a42024b53ebb512fb5dd63c523338bf26c8489c Mon Sep 17 00:00:00 2001 From: Victor Romero Date: Thu, 7 Jan 2021 18:04:11 -0800 Subject: [vcpkg] Add commands to maintain and verify versions db integrity (#14999) * [vcpkg] Add x-ci-verify-versions command * Code cleanup * Remove port version splitting from x-history * Fix wrong message on success * Parallelize versions file generator * Use cpu_count()/2 to avoid crashes * Check db SHA against local SHA * Check baseline version with x-ci-verify-versions and make baseline generator much faster * Implement x-add-version to update version db files * Better checks for x-add-info and make x-ci-verify-versions silent on success * Use find() instead of [] on maps * Create version file if does not exist * Allow redirection of ports/ and port_versions/ * add test ports * WIP end-to-end tests * Change pats in e2e tests * Fix e2e args * e2e once more * Pass ersions feature flag to e2e * Exit with code 1 if there are errors * Files to test for failure cases * Update test files * Add test for x-add-version * fix redirected ports in last test * Add CI check (use dummy data) * Add feature-flags=versions * Ignore subdirectories inside ports * Add --verify-git-trees switch * [vcpkg] Fix build breaks * [x-ci-verify-versions] PR comments * [x-add-version] PR comments * Fix merge conflicts * Modify tests and pipeline * Baselines should only have version-string * Refactor x-add-version * [vcpkg] Fix help message * [vcpkg] Fix minor warnings * `x-add-version --all` doesn't stop on first failure and reduced default verbosity * [vcpkg] Fix default-baseline * Load file instead of using paths provider * Format * Remove ci test * Add fish port for testing * Update version files * Update fish port to cause SHA discrepancy * Test for discrepancy between local SHA and declared SHA * Missing = operator * Check for error message since x-add-version exits with code 0 * Make x-add-version fail with non-zero exit code if not run with --all Co-authored-by: Robert Schumacher Co-authored-by: Robert Schumacher --- .../version-files/default-baseline-1/vcpkg.json | 8 ++++++++ .../default-baseline-2/port_versions/z-/zlib.json | 14 ++++++++++++++ .../version-files/default-baseline-2/vcpkg.json | 8 ++++++++ .../version-files/port_versions/baseline.json | 20 ++++++++++++++++++++ .../testing/version-files/port_versions/c-/cat.json | 9 +++++++++ .../testing/version-files/port_versions/d-/dog.json | 9 +++++++++ .../testing/version-files/port_versions/d-/duck.json | 9 +++++++++ .../version-files/port_versions/m-/mouse.json | 9 +++++++++ .../port_versions_incomplete/baseline.json | 20 ++++++++++++++++++++ .../port_versions_incomplete/c-/cat.json | 9 +++++++++ .../port_versions_incomplete/d-/dog.json | 9 +++++++++ .../port_versions_incomplete/f-/fish.json | 9 +++++++++ .../port_versions_incomplete/m-/mouse.json | 9 +++++++++ .../testing/version-files/ports/cat/portfile.cmake | 1 + scripts/testing/version-files/ports/cat/vcpkg.json | 4 ++++ .../testing/version-files/ports/dog/portfile.cmake | 1 + scripts/testing/version-files/ports/dog/vcpkg.json | 4 ++++ .../testing/version-files/ports/duck/portfile.cmake | 1 + scripts/testing/version-files/ports/duck/vcpkg.json | 4 ++++ .../testing/version-files/ports/mouse/portfile.cmake | 1 + scripts/testing/version-files/ports/mouse/vcpkg.json | 4 ++++ .../ports_incomplete/cat/portfile.cmake | 1 + .../version-files/ports_incomplete/cat/vcpkg.json | 4 ++++ .../ports_incomplete/dog/portfile.cmake | 1 + .../version-files/ports_incomplete/dog/vcpkg.json | 5 +++++ .../ports_incomplete/duck/portfile.cmake | 1 + .../version-files/ports_incomplete/duck/vcpkg.json | 4 ++++ .../ports_incomplete/ferret/portfile.cmake | 1 + .../version-files/ports_incomplete/ferret/vcpkg.json | 4 ++++ .../ports_incomplete/fish/portfile.cmake | 1 + .../version-files/ports_incomplete/fish/vcpkg.json | 5 +++++ .../ports_incomplete/mouse/portfile.cmake | 1 + .../version-files/ports_incomplete/mouse/vcpkg.json | 4 ++++ 33 files changed, 194 insertions(+) create mode 100644 scripts/testing/version-files/default-baseline-1/vcpkg.json create mode 100644 scripts/testing/version-files/default-baseline-2/port_versions/z-/zlib.json create mode 100644 scripts/testing/version-files/default-baseline-2/vcpkg.json create mode 100644 scripts/testing/version-files/port_versions/baseline.json create mode 100644 scripts/testing/version-files/port_versions/c-/cat.json create mode 100644 scripts/testing/version-files/port_versions/d-/dog.json create mode 100644 scripts/testing/version-files/port_versions/d-/duck.json create mode 100644 scripts/testing/version-files/port_versions/m-/mouse.json create mode 100644 scripts/testing/version-files/port_versions_incomplete/baseline.json create mode 100644 scripts/testing/version-files/port_versions_incomplete/c-/cat.json create mode 100644 scripts/testing/version-files/port_versions_incomplete/d-/dog.json create mode 100644 scripts/testing/version-files/port_versions_incomplete/f-/fish.json create mode 100644 scripts/testing/version-files/port_versions_incomplete/m-/mouse.json create mode 100644 scripts/testing/version-files/ports/cat/portfile.cmake create mode 100644 scripts/testing/version-files/ports/cat/vcpkg.json create mode 100644 scripts/testing/version-files/ports/dog/portfile.cmake create mode 100644 scripts/testing/version-files/ports/dog/vcpkg.json create mode 100644 scripts/testing/version-files/ports/duck/portfile.cmake create mode 100644 scripts/testing/version-files/ports/duck/vcpkg.json create mode 100644 scripts/testing/version-files/ports/mouse/portfile.cmake create mode 100644 scripts/testing/version-files/ports/mouse/vcpkg.json create mode 100644 scripts/testing/version-files/ports_incomplete/cat/portfile.cmake create mode 100644 scripts/testing/version-files/ports_incomplete/cat/vcpkg.json create mode 100644 scripts/testing/version-files/ports_incomplete/dog/portfile.cmake create mode 100644 scripts/testing/version-files/ports_incomplete/dog/vcpkg.json create mode 100644 scripts/testing/version-files/ports_incomplete/duck/portfile.cmake create mode 100644 scripts/testing/version-files/ports_incomplete/duck/vcpkg.json create mode 100644 scripts/testing/version-files/ports_incomplete/ferret/portfile.cmake create mode 100644 scripts/testing/version-files/ports_incomplete/ferret/vcpkg.json create mode 100644 scripts/testing/version-files/ports_incomplete/fish/portfile.cmake create mode 100644 scripts/testing/version-files/ports_incomplete/fish/vcpkg.json create mode 100644 scripts/testing/version-files/ports_incomplete/mouse/portfile.cmake create mode 100644 scripts/testing/version-files/ports_incomplete/mouse/vcpkg.json (limited to 'scripts/testing/version-files') diff --git a/scripts/testing/version-files/default-baseline-1/vcpkg.json b/scripts/testing/version-files/default-baseline-1/vcpkg.json new file mode 100644 index 000000000..f6d902393 --- /dev/null +++ b/scripts/testing/version-files/default-baseline-1/vcpkg.json @@ -0,0 +1,8 @@ +{ + "name": "default-baseline-test", + "version-string": "0", + "$x-default-baseline": "fca18ba3572f8aebe3b8158c359db62a7e26134e", + "dependencies": [ + "zlib" + ] +} \ No newline at end of file diff --git a/scripts/testing/version-files/default-baseline-2/port_versions/z-/zlib.json b/scripts/testing/version-files/default-baseline-2/port_versions/z-/zlib.json new file mode 100644 index 000000000..f5ee7cb9d --- /dev/null +++ b/scripts/testing/version-files/default-baseline-2/port_versions/z-/zlib.json @@ -0,0 +1,14 @@ +{ + "versions": [ + { + "git-tree": "7bb2b2f3783303a4dd41163553fe4cc103dc9262", + "version-string": "1.2.11", + "port-version": 9 + }, + { + "git-tree": "4927735fa9baca564ebddf6e6880de344b20d7a8", + "version-string": "1.2.11", + "port-version": 8 + } + ] +} \ No newline at end of file diff --git a/scripts/testing/version-files/default-baseline-2/vcpkg.json b/scripts/testing/version-files/default-baseline-2/vcpkg.json new file mode 100644 index 000000000..1e41002ff --- /dev/null +++ b/scripts/testing/version-files/default-baseline-2/vcpkg.json @@ -0,0 +1,8 @@ +{ + "name": "default-baseline-test", + "version-string": "0", + "$x-default-baseline": "cbd5a68012471f820b7cf28d618199b4a4d89c58", + "dependencies": [ + "zlib" + ] +} \ No newline at end of file diff --git a/scripts/testing/version-files/port_versions/baseline.json b/scripts/testing/version-files/port_versions/baseline.json new file mode 100644 index 000000000..bfe49f984 --- /dev/null +++ b/scripts/testing/version-files/port_versions/baseline.json @@ -0,0 +1,20 @@ +{ + "default": { + "cat": { + "version-string": "1.0", + "port-version": 0 + }, + "dog": { + "version-string": "2001-01-01", + "port-version": 0 + }, + "duck": { + "version-string": "mallard", + "port-version": 0 + }, + "mouse": { + "version-string": "1.0.0", + "port-version": 0 + } + } +} diff --git a/scripts/testing/version-files/port_versions/c-/cat.json b/scripts/testing/version-files/port_versions/c-/cat.json new file mode 100644 index 000000000..e39cd6d4e --- /dev/null +++ b/scripts/testing/version-files/port_versions/c-/cat.json @@ -0,0 +1,9 @@ +{ + "versions": [ + { + "git-tree": "5dd257451526d5b9e560f5f35d7029ba40d88587", + "version": "1.0", + "port-version": 0 + } + ] +} diff --git a/scripts/testing/version-files/port_versions/d-/dog.json b/scripts/testing/version-files/port_versions/d-/dog.json new file mode 100644 index 000000000..49c086c38 --- /dev/null +++ b/scripts/testing/version-files/port_versions/d-/dog.json @@ -0,0 +1,9 @@ +{ + "versions": [ + { + "git-tree": "e170a2ed0da7ba5d434c4a0a98ffd7a3159e3200", + "version-date": "2001-01-01", + "port-version": 0 + } + ] +} diff --git a/scripts/testing/version-files/port_versions/d-/duck.json b/scripts/testing/version-files/port_versions/d-/duck.json new file mode 100644 index 000000000..4f4b209e2 --- /dev/null +++ b/scripts/testing/version-files/port_versions/d-/duck.json @@ -0,0 +1,9 @@ +{ + "versions": [ + { + "git-tree": "0a52a9d722c75b3bfe47d5f5db6c9eb1a64af156", + "version-string": "mallard", + "port-version": 0 + } + ] +} diff --git a/scripts/testing/version-files/port_versions/m-/mouse.json b/scripts/testing/version-files/port_versions/m-/mouse.json new file mode 100644 index 000000000..139c8450f --- /dev/null +++ b/scripts/testing/version-files/port_versions/m-/mouse.json @@ -0,0 +1,9 @@ +{ + "versions": [ + { + "git-tree": "55ed624191e0a1905bd97af29fdf6a1d7f4e6d7c", + "version-semver": "1.0.0", + "port-version": 0 + } + ] +} diff --git a/scripts/testing/version-files/port_versions_incomplete/baseline.json b/scripts/testing/version-files/port_versions_incomplete/baseline.json new file mode 100644 index 000000000..aadd634fc --- /dev/null +++ b/scripts/testing/version-files/port_versions_incomplete/baseline.json @@ -0,0 +1,20 @@ +{ + "default": { + "cat": { + "version-string": "1.0", + "port-version": 0 + }, + "dog": { + "version-string": "2001-01-01", + "port-version": 0 + }, + "duck": { + "version-string": "mallard", + "port-version": 0 + }, + "fish": { + "version-string": "1.0.0", + "port-version": 0 + } + } +} diff --git a/scripts/testing/version-files/port_versions_incomplete/c-/cat.json b/scripts/testing/version-files/port_versions_incomplete/c-/cat.json new file mode 100644 index 000000000..e39cd6d4e --- /dev/null +++ b/scripts/testing/version-files/port_versions_incomplete/c-/cat.json @@ -0,0 +1,9 @@ +{ + "versions": [ + { + "git-tree": "5dd257451526d5b9e560f5f35d7029ba40d88587", + "version": "1.0", + "port-version": 0 + } + ] +} diff --git a/scripts/testing/version-files/port_versions_incomplete/d-/dog.json b/scripts/testing/version-files/port_versions_incomplete/d-/dog.json new file mode 100644 index 000000000..49c086c38 --- /dev/null +++ b/scripts/testing/version-files/port_versions_incomplete/d-/dog.json @@ -0,0 +1,9 @@ +{ + "versions": [ + { + "git-tree": "e170a2ed0da7ba5d434c4a0a98ffd7a3159e3200", + "version-date": "2001-01-01", + "port-version": 0 + } + ] +} diff --git a/scripts/testing/version-files/port_versions_incomplete/f-/fish.json b/scripts/testing/version-files/port_versions_incomplete/f-/fish.json new file mode 100644 index 000000000..dc457aabd --- /dev/null +++ b/scripts/testing/version-files/port_versions_incomplete/f-/fish.json @@ -0,0 +1,9 @@ +{ + "versions": [ + { + "git-tree": "cf3be634f203c1b4152b65ec7700d5695a1fca5c", + "version-string": "1.0.0", + "port-version": 0 + } + ] +} \ No newline at end of file diff --git a/scripts/testing/version-files/port_versions_incomplete/m-/mouse.json b/scripts/testing/version-files/port_versions_incomplete/m-/mouse.json new file mode 100644 index 000000000..139c8450f --- /dev/null +++ b/scripts/testing/version-files/port_versions_incomplete/m-/mouse.json @@ -0,0 +1,9 @@ +{ + "versions": [ + { + "git-tree": "55ed624191e0a1905bd97af29fdf6a1d7f4e6d7c", + "version-semver": "1.0.0", + "port-version": 0 + } + ] +} diff --git a/scripts/testing/version-files/ports/cat/portfile.cmake b/scripts/testing/version-files/ports/cat/portfile.cmake new file mode 100644 index 000000000..0015715fb --- /dev/null +++ b/scripts/testing/version-files/ports/cat/portfile.cmake @@ -0,0 +1 @@ +set(VCPKG_POLICY_EMPTY_PACKAGE enabled) \ No newline at end of file diff --git a/scripts/testing/version-files/ports/cat/vcpkg.json b/scripts/testing/version-files/ports/cat/vcpkg.json new file mode 100644 index 000000000..b87e4a2da --- /dev/null +++ b/scripts/testing/version-files/ports/cat/vcpkg.json @@ -0,0 +1,4 @@ +{ + "name": "cat", + "version": "1.0" +} \ No newline at end of file diff --git a/scripts/testing/version-files/ports/dog/portfile.cmake b/scripts/testing/version-files/ports/dog/portfile.cmake new file mode 100644 index 000000000..0015715fb --- /dev/null +++ b/scripts/testing/version-files/ports/dog/portfile.cmake @@ -0,0 +1 @@ +set(VCPKG_POLICY_EMPTY_PACKAGE enabled) \ No newline at end of file diff --git a/scripts/testing/version-files/ports/dog/vcpkg.json b/scripts/testing/version-files/ports/dog/vcpkg.json new file mode 100644 index 000000000..028447b3e --- /dev/null +++ b/scripts/testing/version-files/ports/dog/vcpkg.json @@ -0,0 +1,4 @@ +{ + "name": "dog", + "version-date": "2001-01-01" +} \ No newline at end of file diff --git a/scripts/testing/version-files/ports/duck/portfile.cmake b/scripts/testing/version-files/ports/duck/portfile.cmake new file mode 100644 index 000000000..0015715fb --- /dev/null +++ b/scripts/testing/version-files/ports/duck/portfile.cmake @@ -0,0 +1 @@ +set(VCPKG_POLICY_EMPTY_PACKAGE enabled) \ No newline at end of file diff --git a/scripts/testing/version-files/ports/duck/vcpkg.json b/scripts/testing/version-files/ports/duck/vcpkg.json new file mode 100644 index 000000000..d780d01bf --- /dev/null +++ b/scripts/testing/version-files/ports/duck/vcpkg.json @@ -0,0 +1,4 @@ +{ + "name": "duck", + "version-string": "mallard" +} \ No newline at end of file diff --git a/scripts/testing/version-files/ports/mouse/portfile.cmake b/scripts/testing/version-files/ports/mouse/portfile.cmake new file mode 100644 index 000000000..0015715fb --- /dev/null +++ b/scripts/testing/version-files/ports/mouse/portfile.cmake @@ -0,0 +1 @@ +set(VCPKG_POLICY_EMPTY_PACKAGE enabled) \ No newline at end of file diff --git a/scripts/testing/version-files/ports/mouse/vcpkg.json b/scripts/testing/version-files/ports/mouse/vcpkg.json new file mode 100644 index 000000000..fd6cab451 --- /dev/null +++ b/scripts/testing/version-files/ports/mouse/vcpkg.json @@ -0,0 +1,4 @@ +{ + "name": "mouse", + "version-semver": "1.0.0" +} \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/cat/portfile.cmake b/scripts/testing/version-files/ports_incomplete/cat/portfile.cmake new file mode 100644 index 000000000..0015715fb --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/cat/portfile.cmake @@ -0,0 +1 @@ +set(VCPKG_POLICY_EMPTY_PACKAGE enabled) \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/cat/vcpkg.json b/scripts/testing/version-files/ports_incomplete/cat/vcpkg.json new file mode 100644 index 000000000..b87e4a2da --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/cat/vcpkg.json @@ -0,0 +1,4 @@ +{ + "name": "cat", + "version": "1.0" +} \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/dog/portfile.cmake b/scripts/testing/version-files/ports_incomplete/dog/portfile.cmake new file mode 100644 index 000000000..0015715fb --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/dog/portfile.cmake @@ -0,0 +1 @@ +set(VCPKG_POLICY_EMPTY_PACKAGE enabled) \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/dog/vcpkg.json b/scripts/testing/version-files/ports_incomplete/dog/vcpkg.json new file mode 100644 index 000000000..65465b940 --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/dog/vcpkg.json @@ -0,0 +1,5 @@ +{ + "name": "dog", + "version-date": "2001-01-01", + "port-version": 1 +} \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/duck/portfile.cmake b/scripts/testing/version-files/ports_incomplete/duck/portfile.cmake new file mode 100644 index 000000000..0015715fb --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/duck/portfile.cmake @@ -0,0 +1 @@ +set(VCPKG_POLICY_EMPTY_PACKAGE enabled) \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/duck/vcpkg.json b/scripts/testing/version-files/ports_incomplete/duck/vcpkg.json new file mode 100644 index 000000000..d780d01bf --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/duck/vcpkg.json @@ -0,0 +1,4 @@ +{ + "name": "duck", + "version-string": "mallard" +} \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/ferret/portfile.cmake b/scripts/testing/version-files/ports_incomplete/ferret/portfile.cmake new file mode 100644 index 000000000..0015715fb --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/ferret/portfile.cmake @@ -0,0 +1 @@ +set(VCPKG_POLICY_EMPTY_PACKAGE enabled) \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/ferret/vcpkg.json b/scripts/testing/version-files/ports_incomplete/ferret/vcpkg.json new file mode 100644 index 000000000..1b4b5c2c1 --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/ferret/vcpkg.json @@ -0,0 +1,4 @@ +{ + "name": "ferret", + "version": "1" +} \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/fish/portfile.cmake b/scripts/testing/version-files/ports_incomplete/fish/portfile.cmake new file mode 100644 index 000000000..0015715fb --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/fish/portfile.cmake @@ -0,0 +1 @@ +set(VCPKG_POLICY_EMPTY_PACKAGE enabled) \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/fish/vcpkg.json b/scripts/testing/version-files/ports_incomplete/fish/vcpkg.json new file mode 100644 index 000000000..f67c10a5a --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/fish/vcpkg.json @@ -0,0 +1,5 @@ +{ + "name": "fish", + "version-string": "1.0.0", + "description": "This description causes an intentional discrepancy between the local SHA and the SHA in fish.json for version 1.0.0" +} \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/mouse/portfile.cmake b/scripts/testing/version-files/ports_incomplete/mouse/portfile.cmake new file mode 100644 index 000000000..0015715fb --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/mouse/portfile.cmake @@ -0,0 +1 @@ +set(VCPKG_POLICY_EMPTY_PACKAGE enabled) \ No newline at end of file diff --git a/scripts/testing/version-files/ports_incomplete/mouse/vcpkg.json b/scripts/testing/version-files/ports_incomplete/mouse/vcpkg.json new file mode 100644 index 000000000..fd6cab451 --- /dev/null +++ b/scripts/testing/version-files/ports_incomplete/mouse/vcpkg.json @@ -0,0 +1,4 @@ +{ + "name": "mouse", + "version-semver": "1.0.0" +} \ No newline at end of file -- cgit v1.2.3