diff options
| author | Victor Romero <romerosanchezv@gmail.com> | 2019-06-21 23:50:05 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2019-06-21 23:50:05 -0700 |
| commit | f3db66b403840b24ea2612d09cca30a5285f5ea3 (patch) | |
| tree | eb3504205b87fa7ae5b7ccdba4820469c9e856c0 | |
| parent | d1b4e88d3c1bd714069f10009c6f9cef172cc822 (diff) | |
| download | vcpkg-f3db66b403840b24ea2612d09cca30a5285f5ea3.tar.gz vcpkg-f3db66b403840b24ea2612d09cca30a5285f5ea3.zip | |
Ports Overlay partial implementation (#6981)
* Ports Overlay feature spec
* Ports Overlay implementation
* [--overlay-ports] Refactor handling of additional paths
* Code cleanup
* [--overlay-ports] Add help
* [depend-info] Support --overlay-ports
* Add method to load all ports using PathsPortFileProvider
* Make PortFileProvider::load_all_control_files() const
* Remove unused code
* [vcpkg] Avoid double-load of source control file between Build::perform_and_exit and Build::perform_and_exit_ex
* [vcpkg] Clang format
* [vcpkg] Fixup build failure introduced in b069ceb2f231
* Report errors from Paragraphs::try_load_port()
30 files changed, 598 insertions, 123 deletions
diff --git a/docs/specifications/ports-overlay.md b/docs/specifications/ports-overlay.md new file mode 100644 index 000000000..e877f5010 --- /dev/null +++ b/docs/specifications/ports-overlay.md @@ -0,0 +1,178 @@ +# Ports Overlay (Jun 19, 2019)
+
+
+## 1. Motivation
+
+### A. Allow users to override ports with alternate versions
+
+It's a common scenario for `vcpkg` users to keep specific versions of libraries to use in their own projects. The current recommendation for users is to fork `vcpkg`'s repository and create tags for commits containing the specific versions of the ports they want to use.
+
+This proposal adds an alternative to solve this problem. By allowing `vcpkg` users to specify additional locations in their file system containing ports for:
+
+ * older or newer versions of libraries,
+ * modified libraries, or
+ * libraries not available in `vcpkg`.
+
+These locations will be searched when resolving port names during package installation, and override ports in `<vcpkg-root>/ports`.
+
+### B. Allow users to keep unmodified upstream ports
+
+Users will be able to keep unmodified versions of the ports shipped with `vcpkg` and update them via `vcpkg update` and `vcpkg upgrade` without having to solve merge conflicts.
+
+
+## 2. Other design concerns
+
+* Allow a set of `vcpkg` commands to optionally accept additional paths to be used when searching for ports.
+* Additional paths must take precedence when resolving names of ports to install.
+* Allow users to specify multiple additional paths.
+* Provide a simple disambiguation mechanism to resolve ambiguous port names.
+* After resolving a port name, the installation process has to work the same as for ports shipped by `vcpkg`.
+* This **DOES NOT ENABLE MULTIPLE VERSIONS** of a same library to be **INSTALLED SIDE-BY-SIDE**.
+
+
+## 3. Proposed solution
+
+This document proposes allowing additional locations to search for ports during package installation that will override and complement the set of ports provided by `vcpkg` (ports under the `<vcpkg_root>/ports` directory).`
+
+A new option `--overlay-ports` will be added to the `vcpkg install`, `vcpkg update`, `vcpkg upgrade`, `vcpkg export`, and `vcpkg depend-info` commands to specify additional paths containing ports.
+
+From a user experience perspective, a user expresses interest in adding additional lookup locations by passing the `--overlay-ports` option followed by a path to:
+
+* an individual port (directory containing a `CONTROL` file),
+ * `vcpkg install sqlite3 --overlay-ports="C:\custom-ports\sqlite3"`
+
+* a directory containing ports,
+ * `vcpkg install sqlite3 --overlay-ports=\\share\org\custom-ports`
+
+* a file listing paths to the former two.
+ * `vcpkg install sqlite3 --overlay-ports=..\port-repos.txt`
+
+ _port-repos.txt_
+
+ ```
+ .\experimental-ports\sqlite3
+ C:\custom-ports
+ \\share\team\custom-ports
+ \\share\org\custom-ports
+ ```
+ *Relative paths inside this file are resolved relatively to the file's location. In this case a `experimental-ports` directory should exist at the same level as the `port-repos.txt` file.*
+
+_NOTE: It is not the goal of this document to discuss library versioning or project dependency management solutions, which require the ability to install multiple versions of a same library side-by-side._
+
+### Multiple additional paths
+
+Users can provide multiple additional paths by repeating the `--overlay-ports` option.
+
+```
+vcpkg install sqlite3
+ --overlay-ports="..\experimental-ports\sqlite3"
+ --overlay-ports="C:\custom-ports"
+ --overlay-ports="\\share\team\custom-ports
+```
+
+### Overlaying ports
+
+Port name resolution follows the order in which additional paths are specified, with the first match being selected for installation, and falling back to `<vcpkg-root>/ports` if the port is not found in any of the additional paths.
+
+No effort is made to compare version numbers inside the `CONTROL` files, or to determine which port contains newer or older files.
+
+### Examples
+
+Given the following directory structure:
+
+ ```
+ team-ports/
+ |-- sqlite3/
+ |---- CONTROL
+ |-- rapidjson/
+ |---- CONTROL
+ |-- curl/
+ |---- CONTROL
+
+ my-ports/
+ |-- sqlite3/
+ |---- CONTROL
+ |-- rapidjson/
+ |---- CONTROL
+
+ vcpkg
+ |-- ports/
+ |---- <upstream ports>
+ |-- vcpkg.exe
+ |-- preferred-ports.txt
+ ```
+* #### Example #1:
+
+ Running:
+
+ ```
+ vcpkg/vcpkg.exe install sqlite3 --overlay-ports=my-ports --overlay-ports=team-ports
+ ```
+
+ Results in `my-ports/sqlite3` getting installed as that location appears first in the command line arguments.
+
+* #### Example #2:
+
+ A specific version of a port can be given priority by adding its path first in the list of arguments:
+
+ ```
+ vcpkg/vcpkg.exe install sqlite3 rapidjson curl
+ --overlay-ports=my-ports/rapidjson
+ --overlay-ports=vcpkg/ports/curl
+ --overlay-ports=team-ports
+ ```
+
+ Installs:
+ * `sqlite3` from `team-ports/sqlite3`
+ * `rapidjson` from `my-ports/rapidjson`
+ * `curl` from `vcpkg/ports/curl`
+
+* #### Example #3:
+
+ Given the content of `preferred-ports.txt` as:
+
+ ```
+ ./ports/curl
+ /my-ports/rapidjson
+ /team-ports
+ ```
+
+ A location can be appended or prepended to those included in `preferred-ports.txt` via the command line, like this:
+
+ ```
+ vcpkg/vcpkg.exe install sqlite3 curl --overlay-ports=my-ports --overlay-ports=vcpkg/preferred-ports.txt
+ ```
+
+ Which results in `my-ports/sqlite3` and `vcpkg/ports/curl` getting installed.
+
+
+## 4. Proposed User experience
+
+### i. User wants to preserve an older version of a port
+
+Developer Alice and her team use `vcpkg` to acquire **OpenCV** and some other packages. She has even contributed many patches to add features to the **OpenCV 3** port in `vcpkg`. But, one day, she notices that a PR to update **OpenCV** to the next major version has been merged.
+
+Alice wants to update some packages available in `vcpkg`. Unfortunately, updating her project to use the latest **OpenCV** is not immediately possible.
+
+Alice creates a private GitHub repository and checks in the set of ports that she wants to preserve. Then provides her teammates with the link to clone her private ports repository.
+
+```
+mkdir vcpkg-custom-ports
+cd vcpkg-custom-ports
+git init
+cp -r %VCPKG_ROOT%/ports/opencv .
+git add .
+git commit -m "[opencv] Add OpenCV 3 port"
+git remote add origin https://github.com/<Alice's GitHub username>/vcpkg-custom-ports.git
+git push -u origin master
+```
+
+Now her team is able to use:
+
+```
+git clone https://github.com/<Alice's GitHub username>/vcpkg-custom-ports.git
+vcpkg update --overlay-ports=./vcpkg-custom-ports
+vcpkg upgrade --no-dry-run --overlay-ports=./vcpkg-custom-ports
+```
+
+to upgrade their packages and preserve the old version of **OpenCV** they require.
diff --git a/toolsrc/include/vcpkg/build.h b/toolsrc/include/vcpkg/build.h index 9044cb556..04cd7cf87 100644 --- a/toolsrc/include/vcpkg/build.h +++ b/toolsrc/include/vcpkg/build.h @@ -20,7 +20,7 @@ namespace vcpkg::Build namespace Command { void perform_and_exit_ex(const FullPackageSpec& full_spec, - const fs::path& port_dir, + const SourceControlFileLocation& scfl, const ParsedArguments& options, const VcpkgPaths& paths); diff --git a/toolsrc/include/vcpkg/dependencies.h b/toolsrc/include/vcpkg/dependencies.h index 16fdb3f73..8c2050b3d 100644 --- a/toolsrc/include/vcpkg/dependencies.h +++ b/toolsrc/include/vcpkg/dependencies.h @@ -48,7 +48,7 @@ namespace vcpkg::Dependencies const RequestType& request_type); InstallPlanAction(const PackageSpec& spec, - const SourceControlFile& scf, + const SourceControlFileLocation& scfl, const std::set<std::string>& features, const RequestType& request_type, std::vector<PackageSpec>&& dependencies); @@ -57,7 +57,7 @@ namespace vcpkg::Dependencies PackageSpec spec; - Optional<const SourceControlFile&> source_control_file; + Optional<const SourceControlFileLocation&> source_control_file_location; Optional<InstalledPackageView> installed_package; InstallPlanType plan_type; @@ -129,26 +129,31 @@ namespace vcpkg::Dependencies struct PortFileProvider { - virtual Optional<const SourceControlFile&> get_control_file(const std::string& src_name) const = 0; + virtual Optional<const SourceControlFileLocation&> get_control_file(const std::string& src_name) const = 0; + virtual std::vector<const SourceControlFileLocation*> load_all_control_files() const = 0; }; struct MapPortFileProvider : Util::ResourceBase, PortFileProvider { - explicit MapPortFileProvider(const std::unordered_map<std::string, SourceControlFile>& map); - Optional<const SourceControlFile&> get_control_file(const std::string& src_name) const override; + explicit MapPortFileProvider(const std::unordered_map<std::string, SourceControlFileLocation>& map); + Optional<const SourceControlFileLocation&> get_control_file(const std::string& src_name) const override; + std::vector<const SourceControlFileLocation*> load_all_control_files() const override; private: - const std::unordered_map<std::string, SourceControlFile>& ports; + const std::unordered_map<std::string, SourceControlFileLocation>& ports; }; struct PathsPortFileProvider : Util::ResourceBase, PortFileProvider { - explicit PathsPortFileProvider(const VcpkgPaths& paths); - Optional<const SourceControlFile&> get_control_file(const std::string& src_name) const override; + explicit PathsPortFileProvider(const vcpkg::VcpkgPaths& paths, + const std::vector<std::string>* ports_dirs_paths); + Optional<const SourceControlFileLocation&> get_control_file(const std::string& src_name) const override; + std::vector<const SourceControlFileLocation*> load_all_control_files() const override; private: - const VcpkgPaths& ports; - mutable std::unordered_map<std::string, SourceControlFile> cache; + Files::Filesystem& filesystem; + std::vector<fs::path> ports_dirs; + mutable std::unordered_map<std::string, SourceControlFileLocation> cache; }; struct ClusterGraph; @@ -181,7 +186,7 @@ namespace vcpkg::Dependencies std::vector<ExportPlanAction> create_export_plan(const std::vector<PackageSpec>& specs, const StatusParagraphs& status_db); - std::vector<AnyAction> create_feature_install_plan(const std::unordered_map<std::string, SourceControlFile>& map, + std::vector<AnyAction> create_feature_install_plan(const std::unordered_map<std::string, SourceControlFileLocation>& map, const std::vector<FeatureSpec>& specs, const StatusParagraphs& status_db); diff --git a/toolsrc/include/vcpkg/postbuildlint.h b/toolsrc/include/vcpkg/postbuildlint.h index 5dcfeb8df..027619eb8 100644 --- a/toolsrc/include/vcpkg/postbuildlint.h +++ b/toolsrc/include/vcpkg/postbuildlint.h @@ -9,5 +9,6 @@ namespace vcpkg::PostBuildLint size_t perform_all_checks(const PackageSpec& spec, const VcpkgPaths& paths, const Build::PreBuildInfo& pre_build_info, - const Build::BuildInfo& build_info); + const Build::BuildInfo& build_info, + const fs::path& port_dir); } diff --git a/toolsrc/include/vcpkg/sourceparagraph.h b/toolsrc/include/vcpkg/sourceparagraph.h index d70fd4337..6232a3fd2 100644 --- a/toolsrc/include/vcpkg/sourceparagraph.h +++ b/toolsrc/include/vcpkg/sourceparagraph.h @@ -68,6 +68,15 @@ namespace vcpkg Optional<const FeatureParagraph&> find_feature(const std::string& featurename) const; }; + /// <summary> + /// Full metadata of a package: core and other features. As well as the location the SourceControlFile was loaded from. + /// </summary> + struct SourceControlFileLocation + { + std::unique_ptr<SourceControlFile> source_control_file; + fs::path source_location; + }; + void print_error_message(Span<const std::unique_ptr<Parse::ParseControlErrorInfo>> error_info_list); inline void print_error_message(const std::unique_ptr<Parse::ParseControlErrorInfo>& error_info_list) { diff --git a/toolsrc/include/vcpkg/vcpkgcmdarguments.h b/toolsrc/include/vcpkg/vcpkgcmdarguments.h index de65eec28..cad013eb8 100644 --- a/toolsrc/include/vcpkg/vcpkgcmdarguments.h +++ b/toolsrc/include/vcpkg/vcpkgcmdarguments.h @@ -15,6 +15,7 @@ namespace vcpkg { std::unordered_set<std::string> switches; std::unordered_map<std::string, std::string> settings; + std::unordered_map<std::string, std::vector<std::string>> multisettings; }; struct VcpkgPaths; @@ -41,10 +42,22 @@ namespace vcpkg StringLiteral short_help_text; }; + struct CommandMultiSetting + { + constexpr CommandMultiSetting(const StringLiteral& name, const StringLiteral& short_help_text) + : name(name), short_help_text(short_help_text) + { + } + + StringLiteral name; + StringLiteral short_help_text; + }; + struct CommandOptionsStructure { Span<const CommandSwitch> switches; Span<const CommandSetting> settings; + Span<const CommandMultiSetting> multisettings; }; struct CommandStructure @@ -74,6 +87,7 @@ namespace vcpkg std::unique_ptr<std::string> vcpkg_root_dir; std::unique_ptr<std::string> triplet; + std::unique_ptr<std::vector<std::string>> overlay_ports; Optional<bool> debug = nullopt; Optional<bool> sendmetrics = nullopt; Optional<bool> printmetrics = nullopt; @@ -88,6 +102,6 @@ namespace vcpkg ParsedArguments parse_arguments(const CommandStructure& command_structure) const; private: - std::unordered_map<std::string, Optional<std::string>> optional_command_arguments; + std::unordered_map<std::string, Optional<std::vector<std::string>>> optional_command_arguments; }; } diff --git a/toolsrc/include/vcpkg/vcpkgpaths.h b/toolsrc/include/vcpkg/vcpkgpaths.h index 42de40d9c..b09169b02 100644 --- a/toolsrc/include/vcpkg/vcpkgpaths.h +++ b/toolsrc/include/vcpkg/vcpkgpaths.h @@ -50,8 +50,6 @@ namespace vcpkg static Expected<VcpkgPaths> create(const fs::path& vcpkg_root_dir, const std::string& default_vs_path); fs::path package_dir(const PackageSpec& spec) const; - fs::path port_dir(const PackageSpec& spec) const; - fs::path port_dir(const std::string& name) const; fs::path build_info_file_path(const PackageSpec& spec) const; fs::path listfile_path(const BinaryParagraph& pgh) const; diff --git a/toolsrc/src/tests.plan.cpp b/toolsrc/src/tests.plan.cpp index 238aa7032..ab24266b4 100644 --- a/toolsrc/src/tests.plan.cpp +++ b/toolsrc/src/tests.plan.cpp @@ -47,7 +47,8 @@ namespace UnitTest1 Assert::AreEqual(plan.spec.triplet().to_string().c_str(), triplet.to_string().c_str()); - Assert::AreEqual(pkg_name.c_str(), plan.source_control_file.get()->core_paragraph->name.c_str()); + auto* scfl = plan.source_control_file_location.get(); + Assert::AreEqual(pkg_name.c_str(), scfl->source_control_file->core_paragraph->name.c_str()); Assert::AreEqual(size_t(vec.size()), feature_list.size()); for (auto&& feature_name : vec) @@ -79,7 +80,7 @@ namespace UnitTest1 /// </summary> struct PackageSpecMap { - std::unordered_map<std::string, SourceControlFile> map; + std::unordered_map<std::string, SourceControlFileLocation> map; Triplet triplet; PackageSpecMap(const Triplet& t = Triplet::X86_WINDOWS) noexcept { triplet = t; } @@ -94,7 +95,8 @@ namespace UnitTest1 { auto spec = PackageSpec::from_name_and_triplet(scf.core_paragraph->name, triplet); Assert::IsTrue(spec.has_value()); - map.emplace(scf.core_paragraph->name, std::move(scf)); + map.emplace(scf.core_paragraph->name, + SourceControlFileLocation{std::unique_ptr<SourceControlFile>(std::move(&scf)), ""}); return PackageSpec{*spec.get()}; } }; diff --git a/toolsrc/src/tests.update.cpp b/toolsrc/src/tests.update.cpp index b6e487c17..5e3f9f3e2 100644 --- a/toolsrc/src/tests.update.cpp +++ b/toolsrc/src/tests.update.cpp @@ -21,9 +21,9 @@ namespace UnitTest1 StatusParagraphs status_db(std::move(status_paragraphs)); - std::unordered_map<std::string, SourceControlFile> map; + std::unordered_map<std::string, SourceControlFileLocation> map; auto scf = unwrap(SourceControlFile::parse_control_file(Pgh{{{"Source", "a"}, {"Version", "0"}}})); - map.emplace("a", std::move(*scf)); + map.emplace("a", SourceControlFileLocation { std::move(scf), "" }); Dependencies::MapPortFileProvider provider(map); auto pkgs = SortedVector<OutdatedPackage>(Update::find_outdated_packages(provider, status_db), @@ -45,9 +45,9 @@ namespace UnitTest1 StatusParagraphs status_db(std::move(status_paragraphs)); - std::unordered_map<std::string, SourceControlFile> map; + std::unordered_map<std::string, SourceControlFileLocation> map; auto scf = unwrap(SourceControlFile::parse_control_file(Pgh{{{"Source", "a"}, {"Version", "0"}}})); - map.emplace("a", std::move(*scf)); + map.emplace("a", SourceControlFileLocation { std::move(scf), "" }); Dependencies::MapPortFileProvider provider(map); auto pkgs = SortedVector<OutdatedPackage>(Update::find_outdated_packages(provider, status_db), @@ -71,9 +71,9 @@ namespace UnitTest1 StatusParagraphs status_db(std::move(status_paragraphs)); - std::unordered_map<std::string, SourceControlFile> map; + std::unordered_map<std::string, SourceControlFileLocation> map; auto scf = unwrap(SourceControlFile::parse_control_file(Pgh{{{"Source", "a"}, {"Version", "0"}}})); - map.emplace("a", std::move(*scf)); + map.emplace("a", SourceControlFileLocation{ std::move(scf), "" }); Dependencies::MapPortFileProvider provider(map); auto pkgs = SortedVector<OutdatedPackage>(Update::find_outdated_packages(provider, status_db), @@ -92,9 +92,9 @@ namespace UnitTest1 StatusParagraphs status_db(std::move(status_paragraphs)); - std::unordered_map<std::string, SourceControlFile> map; + std::unordered_map<std::string, SourceControlFileLocation> map; auto scf = unwrap(SourceControlFile::parse_control_file(Pgh{{{"Source", "a"}, {"Version", "2"}}})); - map.emplace("a", std::move(*scf)); + map.emplace("a", SourceControlFileLocation{ std::move(scf), "" }); Dependencies::MapPortFileProvider provider(map); auto pkgs = SortedVector<OutdatedPackage>(Update::find_outdated_packages(provider, status_db), diff --git a/toolsrc/src/vcpkg/build.cpp b/toolsrc/src/vcpkg/build.cpp index daff7e6c8..059a09432 100644 --- a/toolsrc/src/vcpkg/build.cpp +++ b/toolsrc/src/vcpkg/build.cpp @@ -23,6 +23,7 @@ #include <vcpkg/vcpkglib.h>
using vcpkg::Build::BuildResult;
+using vcpkg::Dependencies::PathsPortFileProvider;
using vcpkg::Parse::ParseControlErrorInfo;
using vcpkg::Parse::ParseExpected;
@@ -34,34 +35,26 @@ namespace vcpkg::Build::Command static constexpr StringLiteral OPTION_CHECKS_ONLY = "--checks-only";
void perform_and_exit_ex(const FullPackageSpec& full_spec,
- const fs::path& port_dir,
+ const SourceControlFileLocation& scfl,
const ParsedArguments& options,
const VcpkgPaths& paths)
{
const PackageSpec& spec = full_spec.package_spec;
+ const auto& scf = *scfl.source_control_file;
if (Util::Sets::contains(options.switches, OPTION_CHECKS_ONLY))
{
const auto pre_build_info = Build::PreBuildInfo::from_triplet_file(paths, spec.triplet());
const auto build_info = Build::read_build_info(paths.get_filesystem(), paths.build_info_file_path(spec));
- const size_t error_count = PostBuildLint::perform_all_checks(spec, paths, pre_build_info, build_info);
+ const size_t error_count =
+ PostBuildLint::perform_all_checks(spec, paths, pre_build_info, build_info, scfl.source_location);
Checks::check_exit(VCPKG_LINE_INFO, error_count == 0);
Checks::exit_success(VCPKG_LINE_INFO);
}
- const ParseExpected<SourceControlFile> source_control_file =
- Paragraphs::try_load_port(paths.get_filesystem(), port_dir);
-
- if (!source_control_file.has_value())
- {
- print_error_message(source_control_file.error());
- Checks::exit_fail(VCPKG_LINE_INFO);
- }
-
- const auto& scf = source_control_file.value_or_exit(VCPKG_LINE_INFO);
Checks::check_exit(VCPKG_LINE_INFO,
- spec.name() == scf->core_paragraph->name,
+ spec.name() == scf.core_paragraph->name,
"The Source field inside the CONTROL file does not match the port directory: '%s' != '%s'",
- scf->core_paragraph->name,
+ scf.core_paragraph->name,
spec.name());
const StatusParagraphs status_db = database_load_check(paths);
@@ -80,7 +73,7 @@ namespace vcpkg::Build::Command features_as_set.emplace("core");
const Build::BuildPackageConfig build_config{
- *scf, spec.triplet(), fs::path{port_dir}, build_package_options, features_as_set};
+ scf, spec.triplet(), fs::path(scfl.source_location), build_package_options, features_as_set};
const auto build_timer = Chrono::ElapsedTimer::create_started();
const auto result = Build::build_package(paths, build_config, status_db);
@@ -128,10 +121,19 @@ namespace vcpkg::Build::Command // Build only takes a single package and all dependencies must already be installed
const ParsedArguments options = args.parse_arguments(COMMAND_STRUCTURE);
std::string first_arg = args.command_arguments.at(0);
+
const FullPackageSpec spec = Input::check_and_get_full_package_spec(
std::move(first_arg), default_triplet, COMMAND_STRUCTURE.example_text);
+
Input::check_triplet(spec.package_spec.triplet(), paths);
- perform_and_exit_ex(spec, paths.port_dir(spec.package_spec), options, paths);
+
+ PathsPortFileProvider provider(paths, args.overlay_ports.get());
+ const auto port_name = spec.package_spec.name();
+ const auto* scfl = provider.get_control_file(port_name).get();
+
+ Checks::check_exit(VCPKG_LINE_INFO, scfl != nullptr, "Error: Couldn't find port '%s'", port_name);
+
+ perform_and_exit_ex(spec, *scfl, options, paths);
}
}
@@ -360,6 +362,12 @@ namespace vcpkg::Build auto& fs = paths.get_filesystem();
const Triplet& triplet = spec.triplet();
+ if (!Strings::starts_with(Strings::ascii_to_lowercase(config.port_dir.u8string()),
+ Strings::ascii_to_lowercase(paths.ports.u8string())))
+ {
+ System::printf("-- Installing port from location: %s\n", config.port_dir.u8string());
+ }
+
#if !defined(_WIN32)
// TODO: remove when vcpkg.exe is in charge for acquiring tools. Change introduced in vcpkg v0.0.107.
// bootstrap should have already downloaded ninja, but making sure it is present in case it was deleted.
@@ -428,7 +436,8 @@ namespace vcpkg::Build }
const BuildInfo build_info = read_build_info(fs, paths.build_info_file_path(spec));
- const size_t error_count = PostBuildLint::perform_all_checks(spec, paths, pre_build_info, build_info);
+ const size_t error_count =
+ PostBuildLint::perform_all_checks(spec, paths, pre_build_info, build_info, config.port_dir);
auto bcf = create_binary_control_file(*config.scf.core_paragraph, triplet, build_info, abi_tag);
@@ -705,7 +714,7 @@ namespace vcpkg::Build }
}
- System::print2("Could not locate cached archive: ", archive_path.u8string(), "\n");
+ System::printf("Could not locate cached archive: %s\n", archive_path.u8string());
ExtendedBuildResult result = do_build_package_and_clean_buildtrees(
paths, pre_build_info, spec, maybe_abi_tag_and_file.value_or(AbiTagAndFile{}).tag, config);
diff --git a/toolsrc/src/vcpkg/commands.autocomplete.cpp b/toolsrc/src/vcpkg/commands.autocomplete.cpp index afef518eb..3cf4dd98f 100644 --- a/toolsrc/src/vcpkg/commands.autocomplete.cpp +++ b/toolsrc/src/vcpkg/commands.autocomplete.cpp @@ -90,7 +90,8 @@ namespace vcpkg::Commands::Autocomplete const auto port_name = match[2].str(); const auto triplet_prefix = match[3].str(); - auto maybe_port = Paragraphs::try_load_port(paths.get_filesystem(), paths.port_dir(port_name)); + // TODO: Support autocomplete for ports in --overlay-ports + auto maybe_port = Paragraphs::try_load_port(paths.get_filesystem(), paths.ports / port_name); if (maybe_port.error()) { Checks::exit_success(VCPKG_LINE_INFO); diff --git a/toolsrc/src/vcpkg/commands.buildexternal.cpp b/toolsrc/src/vcpkg/commands.buildexternal.cpp index 19b89c2f5..1c3c511c2 100644 --- a/toolsrc/src/vcpkg/commands.buildexternal.cpp +++ b/toolsrc/src/vcpkg/commands.buildexternal.cpp @@ -23,7 +23,12 @@ namespace vcpkg::Commands::BuildExternal std::string(args.command_arguments.at(0)), default_triplet, COMMAND_STRUCTURE.example_text); Input::check_triplet(spec.package_spec.triplet(), paths); - const fs::path port_dir = args.command_arguments.at(1); - Build::Command::perform_and_exit_ex(spec, port_dir, options, paths); + auto overlays = args.overlay_ports ? *args.overlay_ports : std::vector<std::string>(); + overlays.insert(overlays.begin(), args.command_arguments.at(1)); + Dependencies::PathsPortFileProvider provider(paths, &overlays); + auto maybe_scfl = provider.get_control_file(spec.package_spec.name()); + Checks::check_exit( + VCPKG_LINE_INFO, maybe_scfl.has_value(), "could not load control file for %s", spec.package_spec.name()); + Build::Command::perform_and_exit_ex(spec, maybe_scfl.value_or_exit(VCPKG_LINE_INFO), options, paths); } } diff --git a/toolsrc/src/vcpkg/commands.ci.cpp b/toolsrc/src/vcpkg/commands.ci.cpp index de66d917b..8cab79504 100644 --- a/toolsrc/src/vcpkg/commands.ci.cpp +++ b/toolsrc/src/vcpkg/commands.ci.cpp @@ -232,12 +232,17 @@ namespace vcpkg::Commands::CI { // determine abi tag std::string abi; - if (auto scf = p->source_control_file.get()) + if (auto scfl = p->source_control_file_location.get()) { auto triplet = p->spec.triplet(); const Build::BuildPackageConfig build_config{ - *scf, triplet, paths.port_dir(p->spec), build_options, p->feature_list}; + *scfl->source_control_file, + triplet, + static_cast<fs::path>(scfl->source_location), + build_options, + p->feature_list + }; auto dependency_abis = Util::fmap(p->computed_dependencies, [&](const PackageSpec& spec) -> Build::AbiEntry { @@ -351,7 +356,8 @@ namespace vcpkg::Commands::CI } StatusParagraphs status_db = database_load_check(paths); - const auto& paths_port_file = Dependencies::PathsPortFileProvider(paths); + + Dependencies::PathsPortFileProvider provider(paths, args.overlay_ports.get()); const Build::BuildPackageOptions install_plan_options = { Build::UseHeadVersion::NO, @@ -369,7 +375,10 @@ namespace vcpkg::Commands::CI XunitTestResults xunitTestResults; - std::vector<std::string> all_ports = Install::get_all_port_names(paths); + std::vector<std::string> all_ports = + Util::fmap(provider.load_all_control_files(), [](auto&& scfl) -> std::string { + return scfl->source_control_file.get()->core_paragraph->name; + }); std::vector<TripletAndSummary> results; auto timer = Chrono::ElapsedTimer::create_started(); for (const Triplet& triplet : triplets) @@ -378,13 +387,13 @@ namespace vcpkg::Commands::CI xunitTestResults.push_collection(triplet.canonical_name()); - Dependencies::PackageGraph pgraph(paths_port_file, status_db); + Dependencies::PackageGraph pgraph(provider, status_db); std::vector<PackageSpec> specs = PackageSpec::to_package_specs(all_ports, triplet); // Install the default features for every package auto all_feature_specs = Util::fmap(specs, [](auto& spec) { return FeatureSpec(spec, ""); }); auto split_specs = - find_unknown_ports_for_ci(paths, exclusions_set, paths_port_file, all_feature_specs, purge_tombstones); + find_unknown_ports_for_ci(paths, exclusions_set, provider, all_feature_specs, purge_tombstones); auto feature_specs = FullPackageSpec::to_feature_specs(split_specs->unknown); for (auto&& fspec : feature_specs) diff --git a/toolsrc/src/vcpkg/commands.dependinfo.cpp b/toolsrc/src/vcpkg/commands.dependinfo.cpp index 48a289fa5..c0800a4b5 100644 --- a/toolsrc/src/vcpkg/commands.dependinfo.cpp +++ b/toolsrc/src/vcpkg/commands.dependinfo.cpp @@ -6,6 +6,9 @@ #include <vcpkg/commands.h>
#include <vcpkg/help.h>
#include <vcpkg/paragraphs.h>
+#include <vcpkg/dependencies.h>
+
+using vcpkg::Dependencies::PathsPortFileProvider;
namespace vcpkg::Commands::DependInfo
{
@@ -35,7 +38,7 @@ namespace vcpkg::Commands::DependInfo return output;
}
- std::string create_dot_as_string(const std::vector<std::unique_ptr<SourceControlFile>>& source_control_files)
+ std::string create_dot_as_string(const std::vector<const SourceControlFile*>& source_control_files)
{
int empty_node_count = 0;
@@ -64,7 +67,7 @@ namespace vcpkg::Commands::DependInfo return s;
}
- std::string create_dgml_as_string(const std::vector<std::unique_ptr<SourceControlFile>>& source_control_files)
+ std::string create_dgml_as_string(const std::vector<const SourceControlFile*>& source_control_files)
{
std::string s;
s.append("<?xml version=\"1.0\" encoding=\"utf-8\"?>");
@@ -109,7 +112,7 @@ namespace vcpkg::Commands::DependInfo }
std::string create_graph_as_string(const std::unordered_set<std::string>& switches,
- const std::vector<std::unique_ptr<SourceControlFile>>& source_control_files)
+ const std::vector<const SourceControlFile*>& source_control_files)
{
if (Util::Sets::contains(switches, OPTION_DOT))
{
@@ -124,7 +127,7 @@ namespace vcpkg::Commands::DependInfo void build_dependencies_list(std::set<std::string>& packages_to_keep,
const std::string& requested_package,
- const std::vector<std::unique_ptr<SourceControlFile>>& source_control_files,
+ const std::vector<const SourceControlFile*>& source_control_files,
const std::unordered_set<std::string>& switches)
{
const auto source_control_file =
@@ -154,7 +157,11 @@ namespace vcpkg::Commands::DependInfo {
const ParsedArguments options = args.parse_arguments(COMMAND_STRUCTURE);
- auto source_control_files = Paragraphs::load_all_ports(paths.get_filesystem(), paths.ports);
+ // TODO: Optimize implementation, current implementation needs to load all ports from disk which is too slow.
+ PathsPortFileProvider provider(paths, args.overlay_ports.get());
+ auto source_control_files = Util::fmap(provider.load_all_control_files(), [](auto&& scfl) -> const SourceControlFile * {
+ return scfl->source_control_file.get();
+ });
if (args.command_arguments.size() >= 1)
{
@@ -178,7 +185,7 @@ namespace vcpkg::Commands::DependInfo for (auto&& source_control_file : source_control_files)
{
- const SourceParagraph& source_paragraph = *source_control_file->core_paragraph;
+ const SourceParagraph& source_paragraph = *source_control_file->core_paragraph.get();
const auto s = Strings::join(", ", source_paragraph.depends, [](const Dependency& d) { return d.name(); });
System::print2(source_paragraph.name, ": ", s, "\n");
}
diff --git a/toolsrc/src/vcpkg/commands.edit.cpp b/toolsrc/src/vcpkg/commands.edit.cpp index 98176b2b0..f2f0b1569 100644 --- a/toolsrc/src/vcpkg/commands.edit.cpp +++ b/toolsrc/src/vcpkg/commands.edit.cpp @@ -79,6 +79,7 @@ namespace vcpkg::Commands::Edit const auto& fs = paths.get_filesystem(); auto packages = fs.get_files_non_recursive(paths.packages); + // TODO: Support edit for --overlay-ports return Util::fmap(ports, [&](const std::string& port_name) -> std::string { const auto portpath = paths.ports / port_name; const auto portfile = portpath / "portfile.cmake"; diff --git a/toolsrc/src/vcpkg/commands.search.cpp b/toolsrc/src/vcpkg/commands.search.cpp index a0afd69e1..3d8387ee1 100644 --- a/toolsrc/src/vcpkg/commands.search.cpp +++ b/toolsrc/src/vcpkg/commands.search.cpp @@ -7,6 +7,9 @@ #include <vcpkg/paragraphs.h> #include <vcpkg/sourceparagraph.h> #include <vcpkg/vcpkglib.h> +#include <vcpkg/dependencies.h> + +using vcpkg::Dependencies::PathsPortFileProvider; namespace vcpkg::Commands::Search { @@ -63,7 +66,10 @@ namespace vcpkg::Commands::Search const ParsedArguments options = args.parse_arguments(COMMAND_STRUCTURE); const bool full_description = Util::Sets::contains(options.switches, OPTION_FULLDESC); - auto source_paragraphs = Paragraphs::load_all_ports(paths.get_filesystem(), paths.ports); + PathsPortFileProvider provider(paths, args.overlay_ports.get()); + auto source_paragraphs = Util::fmap(provider.load_all_control_files(), [](auto&& port) -> const SourceControlFile * { + return port->source_control_file.get(); + }); if (args.command_arguments.empty()) { diff --git a/toolsrc/src/vcpkg/commands.upgrade.cpp b/toolsrc/src/vcpkg/commands.upgrade.cpp index 29815ca94..77183ceaf 100644 --- a/toolsrc/src/vcpkg/commands.upgrade.cpp +++ b/toolsrc/src/vcpkg/commands.upgrade.cpp @@ -43,7 +43,8 @@ namespace vcpkg::Commands::Upgrade StatusParagraphs status_db = database_load_check(paths); - Dependencies::PathsPortFileProvider provider(paths); + // Load ports from ports dirs + Dependencies::PathsPortFileProvider provider(paths, args.overlay_ports.get()); Dependencies::PackageGraph graph(provider, status_db); // input sanitization @@ -85,12 +86,12 @@ namespace vcpkg::Commands::Upgrade not_installed.push_back(spec); } - auto maybe_scf = provider.get_control_file(spec.name()); - if (auto p_scf = maybe_scf.get()) + auto maybe_scfl = provider.get_control_file(spec.name()); + if (auto p_scfl = maybe_scfl.get()) { if (it != status_db.end()) { - if (p_scf->core_paragraph->version != (*it)->package.version) + if (p_scfl->source_control_file->core_paragraph->version != (*it)->package.version) { to_upgrade.push_back(spec); } diff --git a/toolsrc/src/vcpkg/dependencies.cpp b/toolsrc/src/vcpkg/dependencies.cpp index f4a05e7dc..df472515f 100644 --- a/toolsrc/src/vcpkg/dependencies.cpp +++ b/toolsrc/src/vcpkg/dependencies.cpp @@ -22,7 +22,7 @@ namespace vcpkg::Dependencies struct ClusterSource { - const SourceControlFile* scf = nullptr; + const SourceControlFileLocation* scfl = nullptr; std::unordered_map<std::string, std::vector<FeatureSpec>> build_edges; }; @@ -92,12 +92,12 @@ namespace vcpkg::Dependencies if (it == m_graph.end()) { // Load on-demand from m_provider - auto maybe_scf = m_provider.get_control_file(spec.name()); + auto maybe_scfl = m_provider.get_control_file(spec.name()); auto& clust = m_graph[spec]; clust.spec = spec; - if (auto p_scf = maybe_scf.get()) + if (auto p_scfl = maybe_scfl.get()) { - clust.source = cluster_from_scf(*p_scf, clust.spec.triplet()); + clust.source = cluster_from_scf(*p_scfl, clust.spec.triplet()); } return clust; } @@ -105,15 +105,17 @@ namespace vcpkg::Dependencies } private: - static ClusterSource cluster_from_scf(const SourceControlFile& scf, Triplet t) + static ClusterSource cluster_from_scf(const SourceControlFileLocation& scfl, Triplet t) { ClusterSource ret; - ret.build_edges.emplace("core", filter_dependencies_to_specs(scf.core_paragraph->depends, t)); + ret.build_edges.emplace("core", + filter_dependencies_to_specs(scfl.source_control_file->core_paragraph->depends, + t)); - for (const auto& feature : scf.feature_paragraphs) + for (const auto& feature : scfl.source_control_file->feature_paragraphs) ret.build_edges.emplace(feature->name, filter_dependencies_to_specs(feature->depends, t)); - ret.scf = &scf; + ret.scfl = &scfl; return ret; } @@ -151,12 +153,12 @@ namespace vcpkg::Dependencies } InstallPlanAction::InstallPlanAction(const PackageSpec& spec, - const SourceControlFile& scf, + const SourceControlFileLocation& scfl, const std::set<std::string>& features, const RequestType& request_type, std::vector<PackageSpec>&& dependencies) : spec(spec) - , source_control_file(scf) + , source_control_file_location(scfl) , plan_type(InstallPlanType::BUILD_AND_INSTALL) , request_type(request_type) , build_options{} @@ -268,37 +270,145 @@ namespace vcpkg::Dependencies return left->spec.name() < right->spec.name(); } - MapPortFileProvider::MapPortFileProvider(const std::unordered_map<std::string, SourceControlFile>& map) : ports(map) - { - } + MapPortFileProvider::MapPortFileProvider(const std::unordered_map<std::string, SourceControlFileLocation>& map) + : ports(map) + {} - Optional<const SourceControlFile&> MapPortFileProvider::get_control_file(const std::string& spec) const + Optional<const SourceControlFileLocation&> MapPortFileProvider::get_control_file(const std::string& spec) const { auto scf = ports.find(spec); if (scf == ports.end()) return nullopt; return scf->second; } - PathsPortFileProvider::PathsPortFileProvider(const VcpkgPaths& paths) : ports(paths) {} + std::vector<const SourceControlFileLocation*> MapPortFileProvider::load_all_control_files() const + { + return Util::fmap(ports, [](auto&& kvpair) -> const SourceControlFileLocation * { return &kvpair.second; }); + } - Optional<const SourceControlFile&> PathsPortFileProvider::get_control_file(const std::string& spec) const + PathsPortFileProvider::PathsPortFileProvider(const vcpkg::VcpkgPaths& paths, + const std::vector<std::string>* ports_dirs_paths) + : filesystem(paths.get_filesystem()) + { + if (ports_dirs_paths) + { + for (auto&& overlay_path : *ports_dirs_paths) + { + if (!overlay_path.empty()) + { + auto overlay = fs::stdfs::canonical(fs::u8path(overlay_path)); + + Checks::check_exit(VCPKG_LINE_INFO, + filesystem.exists(overlay), + "Error: Path \"%s\" does not exist", + overlay.string()); + + Checks::check_exit(VCPKG_LINE_INFO, + fs::stdfs::is_directory(overlay), + "Error: Path \"%s\" must be a directory", + overlay.string()); + + ports_dirs.emplace_back(overlay); + } + } + } + ports_dirs.emplace_back(paths.ports); + } + + Optional<const SourceControlFileLocation&> PathsPortFileProvider::get_control_file(const std::string& spec) const { auto cache_it = cache.find(spec); if (cache_it != cache.end()) { return cache_it->second; } - Parse::ParseExpected<SourceControlFile> source_control_file = - Paragraphs::try_load_port(ports.get_filesystem(), ports.port_dir(spec)); - if (auto scf = source_control_file.get()) + for (auto&& ports_dir : ports_dirs) { - auto it = cache.emplace(spec, std::move(*scf->get())); - return it.first->second; + // Try loading individual port + if (filesystem.exists(ports_dir / "CONTROL")) + { + auto maybe_scf = Paragraphs::try_load_port(filesystem, ports_dir); + if (auto scf = maybe_scf.get()) + { + if (scf->get()->core_paragraph->name == spec) + { + SourceControlFileLocation scfl{ std::move(*scf), ports_dir }; + auto it = cache.emplace(spec, std::move(scfl)); + return it.first->second; + } + } + else + { + vcpkg::print_error_message(maybe_scf.error()); + Checks::exit_with_message(VCPKG_LINE_INFO, + "Error: Failed to load port from %s", + spec, ports_dir.u8string()); + } + } + + auto found_scf = Paragraphs::try_load_port(filesystem, ports_dir / spec); + if (auto scf = found_scf.get()) + { + if (scf->get()->core_paragraph->name == spec) + { + SourceControlFileLocation scfl{ std::move(*scf), ports_dir / spec }; + auto it = cache.emplace(spec, std::move(scfl)); + return it.first->second; + } + } } + return nullopt; } + std::vector<const SourceControlFileLocation*> PathsPortFileProvider::load_all_control_files() const + { + // Reload cache with ports contained in all ports_dirs + cache.clear(); + std::vector<const SourceControlFileLocation*> ret; + for (auto&& ports_dir : ports_dirs) + { + // Try loading individual port + if (filesystem.exists(ports_dir / "CONTROL")) + { + auto maybe_scf = Paragraphs::try_load_port(filesystem, ports_dir); + if (auto scf = maybe_scf.get()) + { + auto port_name = scf->get()->core_paragraph->name; + if (cache.find(port_name) == cache.end()) + { + SourceControlFileLocation scfl{ std::move(*scf), ports_dir }; + auto it = cache.emplace(port_name, std::move(scfl)); + ret.emplace_back(&it.first->second); + } + } + else + { + vcpkg::print_error_message(maybe_scf.error()); + Checks::exit_with_message(VCPKG_LINE_INFO, + "Error: Failed to load port from %s", + ports_dir.u8string()); + } + continue; + } + + // Try loading all ports inside ports_dir + auto found_scf = Paragraphs::load_all_ports(filesystem, ports_dir); + for (auto&& scf : found_scf) + { + auto port_name = scf->core_paragraph->name; + if (cache.find(port_name) == cache.end()) + { + SourceControlFileLocation scfl{ std::move(scf), ports_dir / port_name }; + auto it = cache.emplace(port_name, std::move(scfl)); + ret.emplace_back(&it.first->second); + } + } + } + return ret; + } + std::vector<RemovePlanAction> create_remove_plan(const std::vector<PackageSpec>& specs, const StatusParagraphs& status_db) { @@ -495,10 +605,11 @@ namespace vcpkg::Dependencies VCPKG_LINE_INFO, "Error: Cannot find definition for package `%s`.", cluster.spec.name()); } + auto&& control_file = *p_source->scfl->source_control_file.get(); if (feature.empty()) { // Add default features for this package. This is an exact reference, so ignore prevent_default_features. - for (auto&& default_feature : p_source->scf->core_paragraph.get()->default_features) + for (auto&& default_feature : control_file.core_paragraph.get()->default_features) { auto res = mark_plus(default_feature, cluster, graph, graph_plan, prevent_default_features); if (res != MarkPlusResult::SUCCESS) @@ -513,7 +624,7 @@ namespace vcpkg::Dependencies if (feature == "*") { - for (auto&& fpgh : p_source->scf->feature_paragraphs) + for (auto&& fpgh : control_file.feature_paragraphs) { auto res = mark_plus(fpgh->name, cluster, graph, graph_plan, prevent_default_features); @@ -590,7 +701,8 @@ namespace vcpkg::Dependencies // Check if any default features have been added auto& previous_df = p_installed->ipv.core->package.default_features; - for (auto&& default_feature : p_source->scf->core_paragraph->default_features) + auto&& control_file = *p_source->scfl->source_control_file.get(); + for (auto&& default_feature : control_file.core_paragraph->default_features) { if (std::find(previous_df.begin(), previous_df.end(), default_feature) == previous_df.end()) { @@ -635,7 +747,7 @@ namespace vcpkg::Dependencies /// <param name="map">Map of all source control files in the current environment.</param> /// <param name="specs">Feature specifications to resolve dependencies for.</param> /// <param name="status_db">Status of installed packages in the current environment.</param> - std::vector<AnyAction> create_feature_install_plan(const std::unordered_map<std::string, SourceControlFile>& map, + std::vector<AnyAction> create_feature_install_plan(const std::unordered_map<std::string, SourceControlFileLocation>& map, const std::vector<FeatureSpec>& specs, const StatusParagraphs& status_db) { @@ -698,7 +810,11 @@ namespace vcpkg::Dependencies if (p_cluster->transient_uninstalled) { // If it will be transiently uninstalled, we need to issue a full installation command - auto pscf = p_cluster->source.value_or_exit(VCPKG_LINE_INFO).scf; + auto* pscfl = p_cluster->source.value_or_exit(VCPKG_LINE_INFO).scfl; + Checks::check_exit(VCPKG_LINE_INFO, + pscfl != nullptr, + "Error: Expected a SourceControlFileLocation to exist"); + auto&& scfl = *pscfl; auto dep_specs = Util::fmap(m_graph_plan->install_graph.adjacency_list(p_cluster), [](ClusterPtr const& p) { return p->spec; }); @@ -706,7 +822,7 @@ namespace vcpkg::Dependencies plan.emplace_back(InstallPlanAction{ p_cluster->spec, - *pscf, + scfl, p_cluster->to_install_features, p_cluster->request_type, std::move(dep_specs), diff --git a/toolsrc/src/vcpkg/export.cpp b/toolsrc/src/vcpkg/export.cpp index 23fc7441a..88c1526c5 100644 --- a/toolsrc/src/vcpkg/export.cpp +++ b/toolsrc/src/vcpkg/export.cpp @@ -488,7 +488,10 @@ With a project open, go to Tools->NuGet Package Manager->Package Manager Console // create the plan const StatusParagraphs status_db = database_load_check(paths); - Dependencies::PathsPortFileProvider provider(paths); + + // Load ports from ports dirs + Dependencies::PathsPortFileProvider provider(paths, args.overlay_ports.get()); + std::vector<ExportPlanAction> export_plan = Dependencies::create_export_plan(opts.specs, status_db); Checks::check_exit(VCPKG_LINE_INFO, !export_plan.empty(), "Export plan cannot be empty"); diff --git a/toolsrc/src/vcpkg/help.cpp b/toolsrc/src/vcpkg/help.cpp index b92e9ca59..6ee573dfc 100644 --- a/toolsrc/src/vcpkg/help.cpp +++ b/toolsrc/src/vcpkg/help.cpp @@ -111,10 +111,12 @@ namespace vcpkg::Help " vcpkg contact Display contact information to send feedback\n" "\n" "Options:\n" - " --triplet <t> Specify the target architecture triplet.\n" + " --triplet <t> Specify the target architecture triplet\n" " (default: " ENVVAR(VCPKG_DEFAULT_TRIPLET) // ", see 'vcpkg help triplet')\n" "\n" + " --overlay-ports=<path> Specify directories to be used when searching for ports\n" + "\n" " --vcpkg-root <path> Specify the vcpkg root " "directory\n" " (default: " ENVVAR(VCPKG_ROOT) // diff --git a/toolsrc/src/vcpkg/install.cpp b/toolsrc/src/vcpkg/install.cpp index 88b15fe7a..781629236 100644 --- a/toolsrc/src/vcpkg/install.cpp +++ b/toolsrc/src/vcpkg/install.cpp @@ -330,9 +330,10 @@ namespace vcpkg::Install System::printf("Building package %s...\n", display_name_with_features); auto result = [&]() -> Build::ExtendedBuildResult { - const Build::BuildPackageConfig build_config{action.source_control_file.value_or_exit(VCPKG_LINE_INFO), + const auto& scfl = action.source_control_file_location.value_or_exit(VCPKG_LINE_INFO); + const Build::BuildPackageConfig build_config{*scfl.source_control_file, action.spec.triplet(), - paths.port_dir(action.spec), + static_cast<fs::path>(scfl.source_location), action.build_options, action.feature_list}; return Build::build_package(paths, build_config, status_db); @@ -652,13 +653,10 @@ namespace vcpkg::Install Build::FailOnTombstone::NO, }; - auto all_ports = Paragraphs::load_all_ports(fs, paths.ports); - std::unordered_map<std::string, SourceControlFile> scf_map; - for (auto&& port : all_ports) - scf_map[port->core_paragraph->name] = std::move(*port); - MapPortFileProvider provider(scf_map); + //// Load ports from ports dirs + PathsPortFileProvider provider(paths, args.overlay_ports.get()); - // Note: action_plan will hold raw pointers to SourceControlFiles from this map + // Note: action_plan will hold raw pointers to SourceControlFileLocations from this map std::vector<AnyAction> action_plan = create_feature_install_plan(provider, FullPackageSpec::to_feature_specs(specs), status_db); diff --git a/toolsrc/src/vcpkg/postbuildlint.cpp b/toolsrc/src/vcpkg/postbuildlint.cpp index d6581c2b4..c760a034f 100644 --- a/toolsrc/src/vcpkg/postbuildlint.cpp +++ b/toolsrc/src/vcpkg/postbuildlint.cpp @@ -857,14 +857,15 @@ namespace vcpkg::PostBuildLint size_t perform_all_checks(const PackageSpec& spec, const VcpkgPaths& paths, const PreBuildInfo& pre_build_info, - const BuildInfo& build_info) + const BuildInfo& build_info, + const fs::path& port_dir) { System::print2("-- Performing post-build validation\n"); const size_t error_count = perform_all_checks_and_return_error_count(spec, paths, pre_build_info, build_info); if (error_count != 0) { - const fs::path portfile = paths.ports / spec.name() / "portfile.cmake"; + const fs::path portfile = port_dir / "portfile.cmake"; System::print2(System::Color::error, "Found ", error_count, diff --git a/toolsrc/src/vcpkg/remove.cpp b/toolsrc/src/vcpkg/remove.cpp index f997667ac..a40b27bd7 100644 --- a/toolsrc/src/vcpkg/remove.cpp +++ b/toolsrc/src/vcpkg/remove.cpp @@ -229,7 +229,8 @@ namespace vcpkg::Remove Checks::exit_fail(VCPKG_LINE_INFO); } - Dependencies::PathsPortFileProvider provider(paths); + // Load ports from ports dirs + Dependencies::PathsPortFileProvider provider(paths, args.overlay_ports.get()); specs = Util::fmap(Update::find_outdated_packages(provider, status_db), [](auto&& outdated) { return outdated.spec; }); diff --git a/toolsrc/src/vcpkg/update.cpp b/toolsrc/src/vcpkg/update.cpp index 2c52d5952..6320bae5b 100644 --- a/toolsrc/src/vcpkg/update.cpp +++ b/toolsrc/src/vcpkg/update.cpp @@ -23,10 +23,10 @@ namespace vcpkg::Update for (auto&& ipv : installed_packages) { const auto& pgh = ipv.core; - auto maybe_scf = provider.get_control_file(pgh->package.spec.name()); - if (auto p_scf = maybe_scf.get()) + auto maybe_scfl = provider.get_control_file(pgh->package.spec.name()); + if (auto p_scfl = maybe_scfl.get()) { - auto&& port_version = p_scf->core_paragraph->version; + auto&& port_version = p_scfl->source_control_file->core_paragraph->version; auto&& installed_version = pgh->package.version; if (installed_version != port_version) { @@ -57,7 +57,7 @@ namespace vcpkg::Update const StatusParagraphs status_db = database_load_check(paths); - Dependencies::PathsPortFileProvider provider(paths); + Dependencies::PathsPortFileProvider provider(paths, args.overlay_ports.get()); const auto outdated_packages = SortedVector<OutdatedPackage>(find_outdated_packages(provider, status_db), &OutdatedPackage::compare_by_name); diff --git a/toolsrc/src/vcpkg/vcpkgcmdarguments.cpp b/toolsrc/src/vcpkg/vcpkgcmdarguments.cpp index 8565c28f9..21bf4d028 100644 --- a/toolsrc/src/vcpkg/vcpkgcmdarguments.cpp +++ b/toolsrc/src/vcpkg/vcpkgcmdarguments.cpp @@ -45,6 +45,45 @@ namespace vcpkg option_field = new_setting; } + static void parse_multivalue(const std::string* arg_begin, + const std::string* arg_end, + const std::string& option_name, + std::unique_ptr<std::vector<std::string>>& option_field) + { + if (arg_begin == arg_end) + { + System::print2(System::Color::error, "Error: expected value after ", option_name, '\n'); + Metrics::g_metrics.lock()->track_property("error", "error option name"); + Help::print_usage(); + Checks::exit_fail(VCPKG_LINE_INFO); + } + + if (!option_field) + { + option_field = std::make_unique<std::vector<std::string>>(); + } + option_field->emplace_back(*arg_begin); + } + + static void parse_cojoined_multivalue(std::string new_value, + const std::string& option_name, + std::unique_ptr<std::vector<std::string>>& option_field) + { + if (new_value.empty()) + { + System::print2(System::Color::error, "Error: expected value after ", option_name, '\n'); + Metrics::g_metrics.lock()->track_property("error", "error option name"); + Help::print_usage(); + Checks::exit_fail(VCPKG_LINE_INFO); + } + + if (!option_field) + { + option_field = std::make_unique<std::vector<std::string>>(); + } + option_field->emplace_back(std::move(new_value)); + } + VcpkgCmdArguments VcpkgCmdArguments::create_from_command_line(const int argc, const CommandLineCharType* const* const argv) { @@ -117,6 +156,13 @@ namespace vcpkg parse_value(arg_begin, arg_end, "--triplet", args.triplet); continue; } + if (Strings::starts_with(arg, "--overlay-ports=")) + { + parse_cojoined_multivalue(arg.substr(sizeof("--overlay-ports=") - 1), + "--overlay-ports", + args.overlay_ports); + continue; + } if (arg == "--debug") { parse_switch(true, "debug", args.debug); @@ -166,7 +212,21 @@ namespace vcpkg const auto eq_pos = arg.find('='); if (eq_pos != std::string::npos) { - args.optional_command_arguments.emplace(arg.substr(0, eq_pos), arg.substr(eq_pos + 1)); + const auto& key = arg.substr(0, eq_pos); + const auto& value = arg.substr(eq_pos + 1); + + auto it = args.optional_command_arguments.find(key); + if (args.optional_command_arguments.end() == it) + { + args.optional_command_arguments.emplace(key, std::vector<std::string> { value }); + } + else + { + if (auto* maybe_values = it->second.get()) + { + maybe_values->emplace_back(value); + } + } } else { @@ -264,7 +324,51 @@ namespace vcpkg } else { - output.settings.emplace(option.name, it->second.value_or_exit(VCPKG_LINE_INFO)); + const auto& value = it->second.value_or_exit(VCPKG_LINE_INFO); + if (value.front().empty()) + { + // Fail when not given a value, e.g.: "vcpkg install sqlite3 --additional-ports=" + System::printf( + System::Color::error, "Error: The option '%s' must be passed an argument.\n", option.name); + failed = true; + } + else + { + output.settings.emplace(option.name, value.front()); + options_copy.erase(it); + } + } + } + } + + for (auto&& option : command_structure.options.multisettings) + { + const auto it = options_copy.find(option.name); + if (it != options_copy.end()) + { + if (!it->second.has_value()) + { + // Not having a string value indicates it was passed like '--a' + System::printf( + System::Color::error, "Error: The option '%s' must be passed an argument.\n", option.name); + failed = true; + } + else + { + const auto& value = it->second.value_or_exit(VCPKG_LINE_INFO); + for (auto&& v : value) + { + if (v.empty()) + { + System::printf( + System::Color::error, "Error: The option '%s' must be passed an argument.\n", option.name); + failed = true; + } + else + { + output.multisettings[option.name].emplace_back(v); + } + } options_copy.erase(it); } } @@ -306,7 +410,14 @@ namespace vcpkg { System::printf(" %-40s %s\n", (option.name + "=..."), option.short_help_text); } + for (auto&& option : command_structure.options.multisettings) + { + System::printf(" %-40s %s\n", (option.name + "=..."), option.short_help_text); + } System::printf(" %-40s %s\n", "--triplet <t>", "Set the default triplet for unqualified packages"); + System::printf(" %-40s %s\n", + "--overlay-ports=<path>", + "Specify directories to be used when searching for ports"); System::printf(" %-40s %s\n", "--vcpkg-root <path>", "Specify the vcpkg directory to use instead of current directory or tool directory"); diff --git a/toolsrc/src/vcpkg/vcpkgpaths.cpp b/toolsrc/src/vcpkg/vcpkgpaths.cpp index 512bcfc35..562a18c9b 100644 --- a/toolsrc/src/vcpkg/vcpkgpaths.cpp +++ b/toolsrc/src/vcpkg/vcpkgpaths.cpp @@ -81,9 +81,6 @@ namespace vcpkg fs::path VcpkgPaths::package_dir(const PackageSpec& spec) const { return this->packages / spec.dir(); } - fs::path VcpkgPaths::port_dir(const PackageSpec& spec) const { return this->ports / spec.name(); } - fs::path VcpkgPaths::port_dir(const std::string& name) const { return this->ports / name; } - fs::path VcpkgPaths::build_info_file_path(const PackageSpec& spec) const { return this->package_dir(spec) / "BUILD_INFO"; diff --git a/toolsrc/vcpkg/vcpkg.vcxproj b/toolsrc/vcpkg/vcpkg.vcxproj index 8edea2244..06d849a74 100644 --- a/toolsrc/vcpkg/vcpkg.vcxproj +++ b/toolsrc/vcpkg/vcpkg.vcxproj @@ -21,8 +21,8 @@ <PropertyGroup Label="Globals">
<ProjectGuid>{34671B80-54F9-46F5-8310-AC429C11D4FB}</ProjectGuid>
<RootNamespace>vcpkg</RootNamespace>
- <WindowsTargetPlatformVersion>8.1</WindowsTargetPlatformVersion>
- <PlatformToolset>v140</PlatformToolset>
+ <WindowsTargetPlatformVersion>10.0</WindowsTargetPlatformVersion>
+ <PlatformToolset>v142</PlatformToolset>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
diff --git a/toolsrc/vcpkglib/vcpkglib.vcxproj b/toolsrc/vcpkglib/vcpkglib.vcxproj index a64eb42fd..2a6853b8a 100644 --- a/toolsrc/vcpkglib/vcpkglib.vcxproj +++ b/toolsrc/vcpkglib/vcpkglib.vcxproj @@ -21,8 +21,8 @@ <PropertyGroup Label="Globals">
<ProjectGuid>{B98C92B7-2874-4537-9D46-D14E5C237F04}</ProjectGuid>
<RootNamespace>vcpkglib</RootNamespace>
- <WindowsTargetPlatformVersion>8.1</WindowsTargetPlatformVersion>
- <PlatformToolset>v140</PlatformToolset>
+ <WindowsTargetPlatformVersion>10.0</WindowsTargetPlatformVersion>
+ <PlatformToolset>v142</PlatformToolset>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
diff --git a/toolsrc/vcpkgmetricsuploader/vcpkgmetricsuploader.vcxproj b/toolsrc/vcpkgmetricsuploader/vcpkgmetricsuploader.vcxproj index e533d0e15..2e689c7fc 100644 --- a/toolsrc/vcpkgmetricsuploader/vcpkgmetricsuploader.vcxproj +++ b/toolsrc/vcpkgmetricsuploader/vcpkgmetricsuploader.vcxproj @@ -21,8 +21,8 @@ <PropertyGroup Label="Globals">
<ProjectGuid>{7D6FDEEB-B299-4A23-85EE-F67C4DED47BE}</ProjectGuid>
<RootNamespace>vcpkgmetricsuploader</RootNamespace>
- <WindowsTargetPlatformVersion>8.1</WindowsTargetPlatformVersion>
- <PlatformToolset>v140</PlatformToolset>
+ <WindowsTargetPlatformVersion>10.0</WindowsTargetPlatformVersion>
+ <PlatformToolset>v142</PlatformToolset>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
diff --git a/toolsrc/vcpkgtest/vcpkgtest.vcxproj b/toolsrc/vcpkgtest/vcpkgtest.vcxproj index 4cda29461..a7517ec54 100644 --- a/toolsrc/vcpkgtest/vcpkgtest.vcxproj +++ b/toolsrc/vcpkgtest/vcpkgtest.vcxproj @@ -48,8 +48,8 @@ <ProjectGuid>{F27B8DB0-1279-4AF8-A2E3-1D49C4F0220D}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>vcpkgtest</RootNamespace>
- <WindowsTargetPlatformVersion>8.1</WindowsTargetPlatformVersion>
- <PlatformToolset>v140</PlatformToolset>
+ <WindowsTargetPlatformVersion>10.0</WindowsTargetPlatformVersion>
+ <PlatformToolset>v142</PlatformToolset>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
@@ -84,7 +84,7 @@ <ImportGroup Label="Shared">
</ImportGroup>
<PropertyGroup Label="UserMacros" />
- <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
+ <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<IntDir>$(SolutionDir)msbuild.x86.debug\$(ProjectName)\</IntDir>
<OutDir>$(SolutionDir)msbuild.x86.debug\</OutDir>
</PropertyGroup>
|