[vcpkg] Track hashes of cmake script helpers in consuming portfiles (#12254)

* [vcpkg] Track hashes of cmake script helpers in consuming portfiles

* [vcpkg] Format

* [vcpkg_find_acquire_program] Use cpkg_from_sourceforge for FLEX and BISON

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>

7 files changed, 128 insertions, 88 deletions

diff --git a/docs/maintainers/portfile-functions.md b/docs/maintainers/portfile-functions.md
index 591516466..aec901dc4 100644
--- a/docs/maintainers/portfile-functions.md
+++ b/docs/maintainers/portfile-functions.md
@@ -1,5 +1,5 @@
-<!-- Run regenerate.ps1 to extract documentation from scripts\cmake\*.cmake -->

-

+<!-- Run regenerate.ps1 to extract documentation from scripts\cmake\*.cmake -->
+
 # Portfile helper functions

 - [execute\_process](execute_process.md)

 - [vcpkg\_acquire\_msys](vcpkg_acquire_msys.md)

@@ -21,8 +21,8 @@
 - [vcpkg\_configure\_make](vcpkg_configure_make.md)

 - [vcpkg\_configure\_meson](vcpkg_configure_meson.md)

 - [vcpkg\_copy\_pdbs](vcpkg_copy_pdbs.md)

-- [vcpkg\_copy\_tool\_dependencies](vcpkg_copy_tool_dependencies.md)

 - [vcpkg\_copy\_tools](vcpkg_copy_tools.md)

+- [vcpkg\_copy\_tool\_dependencies](vcpkg_copy_tool_dependencies.md)

 - [vcpkg\_download\_distfile](vcpkg_download_distfile.md)

 - [vcpkg\_execute\_build\_process](vcpkg_execute_build_process.md)

 - [vcpkg\_execute\_required\_process](vcpkg_execute_required_process.md)

@@ -31,10 +31,12 @@
 - [vcpkg\_fail\_port\_install](vcpkg_fail_port_install.md)

 - [vcpkg\_find\_acquire\_program](vcpkg_find_acquire_program.md)

 - [vcpkg\_fixup\_cmake\_targets](vcpkg_fixup_cmake_targets.md)

+- [vcpkg\_fixup\_pkgconfig](vcpkg_fixup_pkgconfig.md)

 - [vcpkg\_from\_bitbucket](vcpkg_from_bitbucket.md)

 - [vcpkg\_from\_git](vcpkg_from_git.md)

 - [vcpkg\_from\_github](vcpkg_from_github.md)

 - [vcpkg\_from\_gitlab](vcpkg_from_gitlab.md)

+- [vcpkg\_from\_sourceforge](vcpkg_from_sourceforge.md)

 - [vcpkg\_install\_cmake](vcpkg_install_cmake.md)

 - [vcpkg\_install\_gn](vcpkg_install_gn.md)

 - [vcpkg\_install\_make](vcpkg_install_make.md)

diff --git a/docs/maintainers/vcpkg_build_make.md b/docs/maintainers/vcpkg_build_make.md
index 5161ba3ce..3332191d1 100644
--- a/docs/maintainers/vcpkg_build_make.md
+++ b/docs/maintainers/vcpkg_build_make.md
@@ -4,16 +4,26 @@ Build a linux makefile project.
 
 ## Usage:
 ```cmake
-vcpkg_build_make([TARGET <target>])
+vcpkg_build_make([BUILD_TARGET <target>]
+                 [ADD_BIN_TO_PATH]
+                 [ENABLE_INSTALL])
+                 [LOGFILE_ROOT <logfileroot>])
 ```
 
-### TARGET
-The target passed to the configure/make build command (`./configure/make/make install`). If not specified, no target will
+### BUILD_TARGET
+The target passed to the make build command (`./make <target>`). If not specified, the 'all' target will
 be passed.
 
 ### ADD_BIN_TO_PATH
 Adds the appropriate Release and Debug `bin\` directories to the path during the build such that executables can run against the in-tree DLLs.
 
+### ENABLE_INSTALL
+IF the port supports the install target use vcpkg_install_make() instead of vcpkg_build_make()
+
+### BUILD_TARGET
+The target passed to the make build command (`./make <target>`). If not specified, the 'all' target will
+be passed.
+
 ## Notes:
 This command should be preceeded by a call to [`vcpkg_configure_make()`](vcpkg_configure_make.md).
 You can use the alias [`vcpkg_install_make()`](vcpkg_configure_make.md) function if your CMake script supports the
diff --git a/docs/maintainers/vcpkg_common_definitions.md b/docs/maintainers/vcpkg_common_definitions.md
index 276e04190..21b05aa1d 100644
--- a/docs/maintainers/vcpkg_common_definitions.md
+++ b/docs/maintainers/vcpkg_common_definitions.md
@@ -16,6 +16,7 @@ VCPKG_TARGET_IMPORT_LIBRARY_PREFIX       import library prefix for target (same
 VCPKG_TARGET_IMPORT_LIBRARY_SUFFIX       import library suffix for target (same as CMAKE_IMPORT_LIBRARY_SUFFIX)
 VCPKG_FIND_LIBRARY_PREFIXES              target dependent prefixes used for find_library calls in portfiles
 VCPKG_FIND_LIBRARY_SUFFIXES              target dependent suffixes used for find_library calls in portfiles
+VCPKG_SYSTEM_LIBRARIES                   list of libraries are provide by the toolchain and are not managed by vcpkg
 ```
 
 CMAKE_STATIC_LIBRARY_(PREFIX|SUFFIX), CMAKE_SHARED_LIBRARY_(PREFIX|SUFFIX) and CMAKE_IMPORT_LIBRARY_(PREFIX|SUFFIX) are defined for the target
diff --git a/docs/maintainers/vcpkg_configure_make.md b/docs/maintainers/vcpkg_configure_make.md
index a18621794..43f7e45f1 100644
--- a/docs/maintainers/vcpkg_configure_make.md
+++ b/docs/maintainers/vcpkg_configure_make.md
@@ -7,9 +7,6 @@ Configure configure for Debug and Release builds of a project.
 vcpkg_configure_make(
     SOURCE_PATH <${SOURCE_PATH}>
     [AUTOCONFIG]
-    [DISABLE_AUTO_HOST]
-    [DISABLE_AUTO_DST]
-    [GENERATOR]
     [NO_DEBUG]
     [SKIP_CONFIGURE]
     [PROJECT_SUBPATH <${PROJ_SUBPATH}>]
@@ -28,10 +25,6 @@ By convention, this is usually set in the portfile as the variable `SOURCE_PATH`
 ### PROJECT_SUBPATH
 Specifies the directory containing the ``configure`/`configure.ac`.
 By convention, this is usually set in the portfile as the variable `SOURCE_PATH`.
-Should use `GENERATOR NMake` first.
-
-### NO_DEBUG
-This port doesn't support debug mode.
 
 ### SKIP_CONFIGURE
 Skip configure process
@@ -39,21 +32,8 @@ Skip configure process
 ### AUTOCONFIG
 Need to use autoconfig to generate configure file.
 
-### DISABLE_AUTO_HOST
-Don't set host automatically, the default value is `i686`.
-If use this option, you will need to set host manually.
-
-### DISABLE_AUTO_DST
-Don't set installation path automatically, the default value is `${CURRENT_PACKAGES_DIR}` and `${CURRENT_PACKAGES_DIR}/debug`
-If use this option, you will need to set dst path manually.
-
-### GENERATOR
-Specifies the precise generator to use.
-NMake: nmake(windows) make(unix)
-MAKE: make(windows) make(unix)
-
 ### PRERUN_SHELL
-Script that needs to be called before configuration
+Script that needs to be called before configuration (do not use for batch files which simply call autoconf or configure)
 
 ### OPTIONS
 Additional options passed to configure during the configuration.
diff --git a/docs/maintainers/vcpkg_find_acquire_program.md b/docs/maintainers/vcpkg_find_acquire_program.md
index 461d13822..a838bb974 100644
--- a/docs/maintainers/vcpkg_find_acquire_program.md
+++ b/docs/maintainers/vcpkg_find_acquire_program.md
@@ -44,4 +44,4 @@ Note that msys2 has a dedicated helper function: [`vcpkg_acquire_msys`](vcpkg_ac
 * [qt5](https://github.com/Microsoft/vcpkg/blob/master/ports/qt5/portfile.cmake)
 
 ## Source
-[scripts/cmake/vcpkg_find_acquire_program.cmake](https://github.com/Microsoft/vcpkg/blob/master/scripts/cmake/vcpkg_find_acquire_program.cmake)
+[scripts/cmake/vcpkg_find_acquire_program.cmake](https://github.com/Microsoft/vcpkg/blob/master/scripts/cmake/vcpkg_find_acquire_program.cmake)

diff --git a/docs/maintainers/vcpkg_fixup_pkgconfig.md b/docs/maintainers/vcpkg_fixup_pkgconfig.md
new file mode 100644
index 000000000..25c6aae5d
--- /dev/null
+++ b/docs/maintainers/vcpkg_fixup_pkgconfig.md
@@ -0,0 +1,42 @@
+# vcpkg_fixup_pkgconfig
+
+Fix common paths in *.pc files and make everything relativ to $(prefix)
+
+## Usage
+```cmake
+vcpkg_fixup_pkgconfig(
+    [RELEASE_FILES <PATHS>...]
+    [DEBUG_FILES <PATHS>...]
+    [SYSTEM_LIBRARIES <NAMES>...]
+)
+```
+
+## Parameters
+### RELEASE_FILES
+Specifies a list of files to apply the fixes for release paths.
+Defaults to every *.pc file in the folder ${CURRENT_PACKAGES_DIR} without ${CURRENT_PACKAGES_DIR}/debug/
+
+### DEBUG_FILES
+Specifies a list of files to apply the fixes for debug paths.
+Defaults to every *.pc file in the folder ${CURRENT_PACKAGES_DIR}/debug/
+
+### SYSTEM_PACKAGES
+If the *.pc file contains system packages outside vcpkg these need to be listed here.
+Since vcpkg checks the existence of all required packages within vcpkg.
+
+### SYSTEM_LIBRARIES
+If the *.pc file contains system libraries outside vcpkg these need to be listed here.
+VCPKG checks every -l flag for the existence of the required library within vcpkg.
+
+### IGNORE_FLAGS
+If the *.pc file contains flags in the lib field which are not libraries. These can be listed here
+
+## Notes
+Still work in progress. If there are more cases which can be handled here feel free to add them
+
+## Examples
+
+Just call vcpkg_fixup_pkgconfig() after any install step which installs *.pc files.
+
+## Source
+[scripts/cmake/vcpkg_fixup_pkgconfig.cmake](https://github.com/Microsoft/vcpkg/blob/master/scripts/cmake/vcpkg_fixup_pkgconfig.cmake)

diff --git a/docs/maintainers/vcpkg_from_sourceforge.md b/docs/maintainers/vcpkg_from_sourceforge.md
index faad06215..c5b5ae6e8 100644
--- a/docs/maintainers/vcpkg_from_sourceforge.md
+++ b/docs/maintainers/vcpkg_from_sourceforge.md
@@ -1,63 +1,68 @@
-# vcpkg_from_sourceforge

-

-Download and extract a project from sourceforge.

-

-## Usage:

-```cmake

+# vcpkg_from_sourceforge
+
+Download and extract a project from sourceforge.
+
+## Usage:
+```cmake
 vcpkg_from_sourceforge(
-    OUT_SOURCE_PATH SOURCE_PATH

-    REPO <cunit/CUnit>

+    OUT_SOURCE_PATH SOURCE_PATH
+    REPO <cunit/CUnit>
     [REF <2.1-3>]
-    SHA512 <547b417109332...>

-    FILENAME <CUnit-2.1-3.tar.bz2>

-    [DISABLE_SSL]

-    [NO_REMOVE_ONE_LEVEL]

-    [PATCHES <patch1.patch> <patch2.patch>...]

-)

-```

-

-## Parameters:

-### OUT_SOURCE_PATH

-Specifies the out-variable that will contain the extracted location.

-

-This should be set to `SOURCE_PATH` by convention.

-

-### REPO

-The organization or user and repository (optional) on sourceforge.

-

-### REF

-A stable version number that will not change contents.

-

-### FILENAME

-The local name for the file. Files are shared between ports, so the file may need to be renamed to make it clearly attributed to this port and avoid conflicts.

-

-For example, we can get the download link:

-https://sourceforge.net/settings/mirror_choices?projectname=mad&filename=libmad/0.15.1b/libmad-0.15.1b.tar.gz&selected=nchc

-So the REPO is `mad/libmad`, the REF is `0.15.1b`, and the FILENAME is `libmad-0.15.1b.tar.gz`

-

-For some special links:

-https://sourceforge.net/settings/mirror_choices?projectname=soxr&filename=soxr-0.1.3-Source.tar.xz&selected=nchc

-The REPO is `soxr`, REF is not exist, and the FILENAME is `soxr-0.1.3-Source.tar.xz`

-

-### SHA512

-The SHA512 hash that should match the archive.

-

-### PATCHES

-A list of patches to be applied to the extracted sources.

-

-Relative paths are based on the port directory.

-

-### DISABLE_SSL

-Disable ssl when downloading source.

-

-### NO_REMOVE_ONE_LEVEL

-Specifies that the default removal of the top level folder should not occur.

-

-## Examples:

-

-* [cunit](https://github.com/Microsoft/vcpkg/blob/master/ports/cunit/portfile.cmake)

-* [polyclipping](https://github.com/Microsoft/vcpkg/blob/master/ports/polyclipping/portfile.cmake)

-* [tinyfiledialogs](https://github.com/Microsoft/vcpkg/blob/master/ports/tinyfiledialogs/portfile.cmake)

-

-## Source

+    SHA512 <547b417109332...>
+    FILENAME <CUnit-2.1-3.tar.bz2>
+    [DISABLE_SSL]
+    [NO_REMOVE_ONE_LEVEL]
+    [PATCHES <patch1.patch> <patch2.patch>...]
+)
+```
+
+## Parameters:
+### OUT_SOURCE_PATH
+Specifies the out-variable that will contain the extracted location.
+
+This should be set to `SOURCE_PATH` by convention.
+
+### REPO
+The organization or user and repository (optional) on sourceforge.
+
+### REF
+A stable version number that will not change contents.
+
+### FILENAME
+The local name for the file. Files are shared between ports, so the file may need to be renamed to make it clearly attributed to this port and avoid conflicts.
+
+For example, we can get the download link:
+https://sourceforge.net/settings/mirror_choices?projectname=mad&filename=libmad/0.15.1b/libmad-0.15.1b.tar.gz&selected=nchc
+So the REPO is `mad/libmad`, the REF is `0.15.1b`, and the FILENAME is `libmad-0.15.1b.tar.gz`
+
+For some special links:
+https://sourceforge.net/settings/mirror_choices?projectname=soxr&filename=soxr-0.1.3-Source.tar.xz&selected=nchc
+The REPO is `soxr`, REF is not exist, and the FILENAME is `soxr-0.1.3-Source.tar.xz`
+
+### SHA512
+The SHA512 hash that should match the archive.
+
+### WORKING_DIRECTORY
+If specified, the archive will be extracted into the working directory instead of `${CURRENT_BUILDTREES_DIR}/src/`.
+
+Note that the archive will still be extracted into a subfolder underneath that directory (`${WORKING_DIRECTORY}/${REF}-${HASH}/`).
+
+### PATCHES
+A list of patches to be applied to the extracted sources.
+
+Relative paths are based on the port directory.
+
+### DISABLE_SSL
+Disable ssl when downloading source.
+
+### NO_REMOVE_ONE_LEVEL
+Specifies that the default removal of the top level folder should not occur.
+
+## Examples:
+
+* [cunit](https://github.com/Microsoft/vcpkg/blob/master/ports/cunit/portfile.cmake)
+* [polyclipping](https://github.com/Microsoft/vcpkg/blob/master/ports/polyclipping/portfile.cmake)
+* [tinyfiledialogs](https://github.com/Microsoft/vcpkg/blob/master/ports/tinyfiledialogs/portfile.cmake)
+
+## Source
 [scripts/cmake/vcpkg_from_sourceforge.cmake](https://github.com/Microsoft/vcpkg/blob/master/scripts/cmake/vcpkg_from_sourceforge.cmake)