diff options
| author | Joachim Gehweiler <44170764+jgehw@users.noreply.github.com> | 2020-11-12 19:15:48 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2020-11-12 10:15:48 -0800 |
| commit | 11b4a16bcadd252fd9721204262ab2bb6d09464e (patch) | |
| tree | c2289c696aef43d52b07be320949f96849e906cd /ports | |
| parent | 4b222f8d2653603b74fa5086cf0b02a1963d3911 (diff) | |
| download | vcpkg-11b4a16bcadd252fd9721204262ab2bb6d09464e.tar.gz vcpkg-11b4a16bcadd252fd9721204262ab2bb6d09464e.zip | |
[tensorflow-cc] More or less completely rewritten port, update to 2.3.1 (#13028)
* fix build issues occurring from default bazel config being used because customized bazel config is stored in wrong directory
* [tensorflow-cc] Update CONTROL and ci.baseline.txt
* fix also applies to windows static build
* fix debug and static builds as well as library naming for non-linux
- add patch to fix debug builds
- add patch to fix exports for static linking
- really build debug (instead of cloning release)
- override bazel build options for debug (work around bazel bug)
- bazel doesn't support static libraries: work around by building dynamic library and constructing static linkage commands from build log
- Windows .pdb file can't be >4GB even on x64: work around using reduced debug information
- Windows doesn't support .lib files >4GB even on x64, so split into multiple libs
- vcpkg requires equal amount of libs for debug and release: work around using handcrafted empty dummy libs
- fix naming of libs (.dll on Windows and .dylib on macOS)
* upgrade tensorflow from v1.14 to v2.3
- adapt patch files to tensorflow code changes
- update bazel from v0.25.2 to v3.1
- on Windows use python installed on the host instead of embedded python obtained via vcpkg because embedded python lacks pip, which we need to obtain numpy
- on Windows add MSYS2 to the PATH so that bazel tools can access MSYS2 GIT
- add support for custom CA certificates when using HTTPS_PROXY
* fix execute process macro
The existing implementation totally screwed up commands if the command's arguments contained semicolons (this is the case, e.g., in the FindPython modules of the cmake distribution).
* extend overriden execute_process to more than one COMMAND as there actually are use cases for this
* added another patch required for tensorflow v2.3, fixed path and working directory
* Revert "incorporate changes from microsoft:master"
* Revert "Revert "incorporate changes from microsoft:master""
* final fixes for static build + improving out messages
* enabling linux and osx in CI to see if it works now
* simplified code, fixed version numbers, fixed generated include cmake file
* fix failing postbuild check on handcrafted empty dummy library by spreading the last real libraries contents over the required number of libraries
* remove dead code commit by mistake again
* improvements from code review
* cleaner fix for debug code
* find pip3 in PATH (PYTHON3_DIR apparently not valid for pip3)
* fix error in python helper script
* fix wrong libname in postbuild script
* fix python detection + switch to python on msys2 (instead of embedded python) for Windows as we need numpy
* fix order of arguments
* fix command (it may contain spaces such as C:\Program Files\...)
* revert last commit (root cause for CI failures is something different: there are line breaks in path)
* fix regex comparision
(value needs to be escaped as it may contains regex special characters such as brackets, eg C:/Program Files (x86)/...)
* fix linebreaks in generated file
* fix CRT linkage
(macOS doesn't support static CRT linkage; it's set to dynamic even static target triplets for macOS and linux)
* refactor implemenation to avoid as much code duplication as possible -- algorithmically identical
* fix version numbers in helper scripts
* enable work-around for Windows until bazel fix is available
* install missing python3-pip on linux
* fix linux build by patching
* apply timeout feature now available via merged master branch
* correct linux build patch
* improve debug build patches
(no functional difference because LOG(FATAL, ...) macro internally anyway calls abort(), which the compiler doesn't detect in debug mode...
* improve linux patch
* temporarily add debug to inspect what's going on on macOS CI
* remove temporary debug code and fix static linking scripts for linux and macOS
* fix regex escaping
* fix ambiguous match while grepping for the framework link command
* extend fix of ambiguous match while grepping for the framework link command
* fix what merge of master broke
* fix more what got broken by merging master
(all packages and their dependencies are now maintained manually instead of using pacman...)
* remove "unofficial" from filename
* added switch do distinct classic and manifest mode when generating config.cmake file
* create symlinks for libraries without version number
* fix linux postbuild script
* temporarily disable code making problems
* add note for linking on Linux and macOS
* forget to add README file in previous commit
* add file forgotton in macro fixing patch
* fix python library path
* fix macOS static link command
* update linkage instructions in README
* Update ports/tensorflow-cc/CONTROL
* Update ports/tensorflow-cc/portfile.cmake
* Update scripts/ci.baseline.txt
* use vcpkg_execute_required_process
* pass C_FLAGS and CXX_FLAGS to bazel
* fix INTERFACE_INCLUDE_DIRECTORIES
* fix optional c/cxx arguments
* also add linker opts
* update README
* merge static libs into one
to support force_load (cannot force_load both due to duplicate symbols)
* update README
* quote python path (it might contain spaces that don't get escaped inside outer quotes of bash command)
* fix python path also for static build
* add arm(64) as currently unsupported arch
* bazel 3.7 is available -> remove workaround
* update README, remove necessary c-ares from deps
* update msys package
* add uwp specific options, and minor general improvements
* fix string replace
* fix control file and windows path separator
* revert backslashes-fix -- the root cause was missing .exe extension
* upgrade to tf 2.3.1
* fix hard-coded version
* remove uwp work-in-progress code so that PR can be merged
* update README and print out usage info in portfile
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-windows
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/portfile.cmake
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/portfile.cmake
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/portfile.cmake
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/portfile.cmake
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/portfile.cmake
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* improve usage hints as discussed in review
* add comment
* apply changes from review
* make additional compiler / linker args space-proof
* Update ports/tensorflow-cc/README-macos
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/tensorflow-cc-config-shared.cmake.in
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/tensorflow-cc-config-shared.cmake.in
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-macos
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* apply changes from code review
* maybe fix the config files
* rob.maynard CRs
* fix windows static lib naming for first part
* Update ports/tensorflow-cc/generate_static_link_cmd_windows.py
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* apply rob maynards 1st suggestion
* nicole CRs
* format/fix-compile
* fix missing string termination
* prefer IMPORTED_LOCATION over IMPORTED_LOCATION_RELEASE to have default fall-back
* hopefully fix the issue where no libraries are generated
* final stuff
Co-authored-by: Gehweiler <Joachim_Gehweiler@McAfee.com>
Co-authored-by: wangli28 <wangli28@beyondsoft.com>
Co-authored-by: Jack·Boos·Yu <47264268+JackBoosY@users.noreply.github.com>
Co-authored-by: Joachim Gehweiler <joachim@Joachims-iMac.local>
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
Diffstat (limited to 'ports')
20 files changed, 1263 insertions, 120 deletions
diff --git a/ports/tensorflow-cc/CONTROL b/ports/tensorflow-cc/CONTROL index 2d0ff5007..d422260e3 100644 --- a/ports/tensorflow-cc/CONTROL +++ b/ports/tensorflow-cc/CONTROL @@ -1,6 +1,4 @@ Source: tensorflow-cc -Version: 1.14 -Port-Version: 5 +Version: 2.3.1 Description: Library for computation using data flow graphs for scalable machine learning -Build-Depends: c-ares -Supports: !x86 +Supports: !(x86|arm|uwp) diff --git a/ports/tensorflow-cc/README-linux b/ports/tensorflow-cc/README-linux new file mode 100644 index 000000000..4e0cb514a --- /dev/null +++ b/ports/tensorflow-cc/README-linux @@ -0,0 +1,13 @@ +Warning: Static TensorFlow build contains several external dependencies that may cause linking conflicts
+ (for example, one cannot use both openssl and TensorFlow in the same project, since TensorFlow contains boringssl).
+
+Note: Besides TensorFlow itself, you'll also need to pass its dependencies to the linker:
+ * libm
+ * libdl
+ * libpthread
+for example, '-ltensorflow_cc -lm -ldl -lpthread'.
+
+Note: There is no separate libtensorflow_framework.a as it got merged into libtensorflow_cc.a to avoid linking conflicts.
+
+Note: For some TensorFlow features (e.g. OpRegistry), it might be necessary to tell the linker to include the whole library:
+ '-Wl,--whole-archive -ltensorflow_cc -Wl,--no-whole-archive [rest of linker arguments]'
diff --git a/ports/tensorflow-cc/README-macos b/ports/tensorflow-cc/README-macos new file mode 100644 index 000000000..6bbfd0781 --- /dev/null +++ b/ports/tensorflow-cc/README-macos @@ -0,0 +1,12 @@ +Warning: Static TensorFlow build contains several external dependencies that may cause linking conflicts
+ (for example, one cannot use both openssl and TensorFlow in the same project, since TensorFlow contains boringssl).
+
+Note: Besides TensorFlow itself, you'll also need to pass its dependencies to the linker:
+ * CoreFoundation
+ * Security (only when linking against the whole library, see below)
+for example, '-ltensorflow_cc -framework CoreFoundation -framework Security'.
+
+Note: There is no separate libtensorflow_framework.a as it got merged into libtensorflow_cc.a to avoid linking conflicts.
+
+Note: For some TensorFlow features (e.g. OpRegistry), it might be necessary to tell the linker to include the whole library:
+ '-Wl,-force_load,path/to/libtensorflow_cc.a [rest of linker arguments]'
diff --git a/ports/tensorflow-cc/README-windows b/ports/tensorflow-cc/README-windows new file mode 100644 index 000000000..5a1952976 --- /dev/null +++ b/ports/tensorflow-cc/README-windows @@ -0,0 +1,5 @@ +Warning: Static TensorFlow build contains several external dependencies that may cause linking conflicts
+ (for example, one cannot use both openssl and TensorFlow in the same project, since TensorFlow contains boringssl).
+
+Note: For some TensorFlow features (e.g. OpRegistry), it might be necessary to convince the linker to include the whole library, i.e., link using options
+ '/WHOLEARCHIVE:tensorflow_cc.lib /WHOLEARCHIVE:tensorflow_cc-part2.lib /WHOLEARCHIVE:tensorflow_cc-part3.lib ...'
diff --git a/ports/tensorflow-cc/TensorflowCCConfig.cmake b/ports/tensorflow-cc/TensorflowCCConfig.cmake deleted file mode 100644 index db12811d5..000000000 --- a/ports/tensorflow-cc/TensorflowCCConfig.cmake +++ /dev/null @@ -1,28 +0,0 @@ -set(tensorflow_cc_INCLUDE_DIR "${CMAKE_CURRENT_LIST_DIR}/../../include") - -message(WARNING "Tensorflow has vendored dependencies. You may need to manually include files from tensorflow-external") -set(tensorflow_cc_INCLUDE_DIRS - ${tensorflow_cc_INCLUDE_DIR} - ${tensorflow_cc_INCLUDE_DIR}/tensorflow-external/ - ${tensorflow_cc_INCLUDE_DIR}/tensorflow-external/tensorflow/ - ${tensorflow_cc_INCLUDE_DIR}/tensorflow-external/external/com_google_absl - ${tensorflow_cc_INCLUDE_DIR}/tensorflow-external/bazel-out/k8-opt/bin/ - ${tensorflow_cc_INCLUDE_DIR}/tensorflow-external/external/protobuf_archive/src/ -) - -add_library(tensorflow_cc::tensorflow_framework SHARED IMPORTED) -set_target_properties(tensorflow_cc::tensorflow_framework - PROPERTIES - IMPORTED_LOCATION ${CMAKE_CURRENT_LIST_DIR}/../../lib/libtensorflow_framework.so.1.14.0 - INTERFACE_INCLUDE_DIRECTORIES "${tensorflow_cc_INCLUDE_DIRS}" -) - -add_library(tensorflow_cc::tensorflow_cc SHARED IMPORTED) -set_target_properties(tensorflow_cc::tensorflow_cc - PROPERTIES - IMPORTED_LOCATION ${CMAKE_CURRENT_LIST_DIR}/../../lib/libtensorflow_cc.so.1.14.0 - INTERFACE_INCLUDE_DIRECTORIES "${tensorflow_cc_INCLUDE_DIRS}" -) - -set(tensorflow_cc_FOUND TRUE) -set(tensorflow_framework_FOUND TRUE)
\ No newline at end of file diff --git a/ports/tensorflow-cc/change-macros-for-static-lib.patch b/ports/tensorflow-cc/change-macros-for-static-lib.patch new file mode 100644 index 000000000..b000d5f0e --- /dev/null +++ b/ports/tensorflow-cc/change-macros-for-static-lib.patch @@ -0,0 +1,266 @@ +diff --git a/tensorflow/core/platform/macros.h b/tensorflow/core/platform/macros.h +--- a/tensorflow/core/platform/macros.h ++++ b/tensorflow/core/platform/macros.h +@@ -57,16 +57,7 @@ + #define TF_SCANF_ATTRIBUTE(string_index, first_to_check) + #endif + +-// Control visibility outside .so +-#if defined(_WIN32) +-#ifdef TF_COMPILE_LIBRARY +-#define TF_EXPORT __declspec(dllexport) +-#else +-#define TF_EXPORT __declspec(dllimport) +-#endif // TF_COMPILE_LIBRARY +-#else +-#define TF_EXPORT __attribute__((visibility("default"))) +-#endif // _WIN32 ++#define TF_EXPORT + + #ifdef __has_builtin + #define TF_HAS_BUILTIN(x) __has_builtin(x) + +diff --git a/tensorflow/c/c_api.h b/tensorflow/c/c_api.h +--- a/tensorflow/c/c_api.h ++++ b/tensorflow/c/c_api.h +@@ -70,24 +70,7 @@ + // and the API just provides high level controls over the number of + // devices of each type. + +-// Macro to control visibility of exported symbols in the shared library (.so, +-// .dylib, .dll). +-// This duplicates the TF_EXPORT macro definition in +-// tensorflow/core/platform/macros.h in order to keep this .h file independent +-// of any other includes. +-#ifdef SWIG + #define TF_CAPI_EXPORT +-#else +-#if defined(_WIN32) +-#ifdef TF_COMPILE_LIBRARY +-#define TF_CAPI_EXPORT __declspec(dllexport) +-#else +-#define TF_CAPI_EXPORT __declspec(dllimport) +-#endif // TF_COMPILE_LIBRARY +-#else +-#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +-#endif // _WIN32 +-#endif // SWIG + + #ifdef __cplusplus + extern "C" { + +diff --git a/tensorflow/c/c_api_experimental.h b/tensorflow/c/c_api_experimental.h +--- a/tensorflow/c/c_api_experimental.h ++++ b/tensorflow/c/c_api_experimental.h +@@ -28,24 +28,7 @@ + // The API here is subject to changes in the future. + // -------------------------------------------------------------------------- + +-// Macro to control visibility of exported symbols in the shared library (.so, +-// .dylib, .dll). +-// This duplicates the TF_EXPORT macro definition in +-// tensorflow/core/platform/macros.h in order to keep this .h file independent +-// of any other includes.$a +-#ifdef SWIG + #define TF_CAPI_EXPORT +-#else +-#if defined(_WIN32) +-#ifdef TF_COMPILE_LIBRARY +-#define TF_CAPI_EXPORT __declspec(dllexport) +-#else +-#define TF_CAPI_EXPORT __declspec(dllimport) +-#endif // TF_COMPILE_LIBRARY +-#else +-#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +-#endif // _WIN32 +-#endif // SWIG + + #ifdef __cplusplus + extern "C" { + +diff --git a/tensorflow/c/c_api_macros.h b/tensorflow/c/c_api_macros.h +--- a/tensorflow/c/c_api_macros.h ++++ b/tensorflow/c/c_api_macros.h +@@ -16,18 +16,6 @@ + #ifndef TENSORFLOW_C_C_API_MACROS_H_ + #define TENSORFLOW_C_C_API_MACROS_H_ + +-#ifdef SWIG + #define TF_CAPI_EXPORT +-#else +-#if defined(_WIN32) +-#ifdef TF_COMPILE_LIBRARY +-#define TF_CAPI_EXPORT __declspec(dllexport) +-#else +-#define TF_CAPI_EXPORT __declspec(dllimport) +-#endif // TF_COMPILE_LIBRARY +-#else +-#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +-#endif // _WIN32 +-#endif // SWIG + + #endif // TENSORFLOW_C_C_API_MACROS_H_ + +diff --git a/tensorflow/c/kernels.h b/tensorflow/c/kernels.h +--- a/tensorflow/c/kernels.h ++++ b/tensorflow/c/kernels.h +@@ -21,24 +21,7 @@ + #include "tensorflow/c/tf_datatype.h" + #include "tensorflow/c/tf_status.h" + +-// Macro to control visibility of exported symbols in the shared library (.so, +-// .dylib, .dll). +-// This duplicates the TF_EXPORT macro definition in +-// tensorflow/core/platform/macros.h in order to keep this .h file independent +-// of any other includes. +-#ifdef SWIG + #define TF_CAPI_EXPORT +-#else +-#if defined(_WIN32) +-#ifdef TF_COMPILE_LIBRARY +-#define TF_CAPI_EXPORT __declspec(dllexport) +-#else +-#define TF_CAPI_EXPORT __declspec(dllimport) +-#endif // TF_COMPILE_LIBRARY +-#else +-#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +-#endif // _WIN32 +-#endif // SWIG + + #ifdef __cplusplus + extern "C" { + +diff --git a/tensorflow/c/ops.h b/tensorflow/c/ops.h +--- a/tensorflow/c/ops.h ++++ b/tensorflow/c/ops.h +@@ -76,19 +76,7 @@ + #include "tensorflow/c/tf_datatype.h" + #include "tensorflow/c/tf_status.h" + +-#ifdef SWIG + #define TF_CAPI_EXPORT +-#else +-#if defined(_WIN32) +-#ifdef TF_COMPILE_LIBRARY +-#define TF_CAPI_EXPORT __declspec(dllexport) +-#else +-#define TF_CAPI_EXPORT __declspec(dllimport) +-#endif // TF_COMPILE_LIBRARY +-#else +-#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +-#endif // _WIN32 +-#endif // SWIG + + #ifdef __cplusplus + extern "C" { + +diff --git a/tensorflow/c/tf_datatype.h b/tensorflow/c/tf_datatype.h +--- a/tensorflow/c/tf_datatype.h ++++ b/tensorflow/c/tf_datatype.h +@@ -18,24 +18,7 @@ + + #include <stddef.h> + +-// Macro to control visibility of exported symbols in the shared library (.so, +-// .dylib, .dll). +-// This duplicates the TF_EXPORT macro definition in +-// tensorflow/core/platform/macros.h in order to keep this .h file independent +-// of any other includes. +-#ifdef SWIG + #define TF_CAPI_EXPORT +-#else +-#if defined(_WIN32) +-#ifdef TF_COMPILE_LIBRARY +-#define TF_CAPI_EXPORT __declspec(dllexport) +-#else +-#define TF_CAPI_EXPORT __declspec(dllimport) +-#endif // TF_COMPILE_LIBRARY +-#else +-#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +-#endif // _WIN32 +-#endif // SWIG + + #ifdef __cplusplus + extern "C" { + +diff --git a/tensorflow/c/tf_status.h b/tensorflow/c/tf_status.h +--- a/tensorflow/c/tf_status.h ++++ b/tensorflow/c/tf_status.h +@@ -16,19 +16,7 @@ + #ifndef TENSORFLOW_C_TF_STATUS_H_ + #define TENSORFLOW_C_TF_STATUS_H_ + +-#ifdef SWIG + #define TF_CAPI_EXPORT +-#else +-#if defined(_WIN32) +-#ifdef TF_COMPILE_LIBRARY +-#define TF_CAPI_EXPORT __declspec(dllexport) +-#else +-#define TF_CAPI_EXPORT __declspec(dllimport) +-#endif // TF_COMPILE_LIBRARY +-#else +-#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +-#endif // _WIN32 +-#endif // SWIG + + #ifdef __cplusplus + extern "C" { + +diff --git a/tensorflow/c/tf_tensor.h b/tensorflow/c/tf_tensor.h +--- a/tensorflow/c/tf_tensor.h ++++ b/tensorflow/c/tf_tensor.h +@@ -22,24 +22,7 @@ + #include "tensorflow/c/tf_datatype.h" + #include "tensorflow/c/tf_status.h" + +-// Macro to control visibility of exported symbols in the shared library (.so, +-// .dylib, .dll). +-// This duplicates the TF_EXPORT macro definition in +-// tensorflow/core/platform/macros.h in order to keep this .h file independent +-// of any other includes. +-#ifdef SWIG + #define TF_CAPI_EXPORT +-#else +-#if defined(_WIN32) +-#ifdef TF_COMPILE_LIBRARY +-#define TF_CAPI_EXPORT __declspec(dllexport) +-#else +-#define TF_CAPI_EXPORT __declspec(dllimport) +-#endif // TF_COMPILE_LIBRARY +-#else +-#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +-#endif // _WIN32 +-#endif // SWIG + + #ifdef __cplusplus + extern "C" { + +diff --git a/tensorflow/c/eager/c_api.h b/tensorflow/c/eager/c_api.h +--- a/tensorflow/c/eager/c_api.h ++++ b/tensorflow/c/eager/c_api.h +@@ -22,24 +22,7 @@ + + #include "tensorflow/c/c_api.h" + +-// Macro to control visibility of exported symbols in the shared library (.so, +-// .dylib, .dll). +-// This duplicates the TF_EXPORT macro definition in +-// tensorflow/core/platform/macros.h in order to keep this .h file independent +-// of any other includes.$a +-#ifdef SWIG + #define TF_CAPI_EXPORT +-#else +-#if defined(_WIN32) +-#ifdef TF_COMPILE_LIBRARY +-#define TF_CAPI_EXPORT __declspec(dllexport) +-#else +-#define TF_CAPI_EXPORT __declspec(dllimport) +-#endif // TF_COMPILE_LIBRARY +-#else +-#define TF_CAPI_EXPORT __attribute__((visibility("default"))) +-#endif // _WIN32 +-#endif // SWIG + + #ifdef __cplusplus + extern "C" { diff --git a/ports/tensorflow-cc/convert_lib_params_linux.py b/ports/tensorflow-cc/convert_lib_params_linux.py new file mode 100644 index 000000000..ba2c35270 --- /dev/null +++ b/ports/tensorflow-cc/convert_lib_params_linux.py @@ -0,0 +1,29 @@ +import sys
+
+version = sys.argv[1]
+
+with open(f"libtensorflow_cc.a.{version}-2.params", "w") as f_out:
+ parts = []
+ with open(f"libtensorflow_framework.so.{version}-2.params", "r") as f_in:
+ skip_next = False
+ for line in f_in:
+ if skip_next:
+ skip_next = False
+ continue
+ if line.startswith("-o"):
+ skip_next = True
+ elif line.startswith("bazel-out"):
+ f_out.write(line)
+ parts.append(line)
+ parts = set(parts)
+ with open(f"libtensorflow_cc.so.{version}-2.params", "r") as f_in:
+ skip_next = False
+ for line in f_in:
+ if skip_next:
+ skip_next = False
+ continue
+ if line.startswith("-o"):
+ skip_next = True
+ elif line.startswith("bazel-out"):
+ if line not in parts:
+ f_out.write(line)
diff --git a/ports/tensorflow-cc/convert_lib_params_windows.py b/ports/tensorflow-cc/convert_lib_params_windows.py new file mode 100644 index 000000000..5fc2fdd68 --- /dev/null +++ b/ports/tensorflow-cc/convert_lib_params_windows.py @@ -0,0 +1,27 @@ +import os
+import sys
+
+file_no = 1
+with open("tensorflow_cc.dll-2.params", "r") as f_in:
+ lib_name = None
+ acc_size = 0
+ f_out = open("tensorflow_cc.lib-2.params-part1", "w")
+ for line in f_in:
+ if line.startswith("/OUT:"):
+ lib_name = line
+ line = line.replace(".dll", "-part1.lib")
+ f_out.write(line)
+ elif line.startswith("/WHOLEARCHIVE:"):
+ line = line[len("/WHOLEARCHIVE:"):]
+ size = os.stat(f"../../{line.strip()}").st_size
+ if acc_size + size > 0xFFFFFFFF:
+ # we need to split the library if it is >4GB, because it's not supported even on x64 Windows
+ f_out.close()
+ file_no += 1
+ f_out = open(f"tensorflow_cc.lib-2.params-part{file_no}", "w")
+ acc_size = 0
+ f_out.write(lib_name.replace(".dll", f"-part{file_no}.lib"))
+ acc_size += size
+ f_out.write(line)
+ f_out.close()
+
diff --git a/ports/tensorflow-cc/file-exists.patch b/ports/tensorflow-cc/file-exists.patch deleted file mode 100644 index 8cefb4828..000000000 --- a/ports/tensorflow-cc/file-exists.patch +++ /dev/null @@ -1,13 +0,0 @@ -diff --git a/configure.py b/configure.py
-index 43af22d..7989b1f 100644
---- a/configure.py
-+++ b/configure.py
-@@ -183,7 +183,7 @@ def get_python_path(environ_cp, python_bin_path):
-
- paths = []
- for path in all_paths:
-- if os.path.isdir(path):
-+ if os.path.isdir(path) or True:
- paths.append(path)
- return paths
-
diff --git a/ports/tensorflow-cc/fix-dbg-build-errors.patch b/ports/tensorflow-cc/fix-dbg-build-errors.patch new file mode 100644 index 000000000..63e95dbb8 --- /dev/null +++ b/ports/tensorflow-cc/fix-dbg-build-errors.patch @@ -0,0 +1,111 @@ +diff --git a/tensorflow/core/platform/path.cc b/tensorflow/core/platform/path.cc +--- a/tensorflow/core/platform/path.cc ++++ b/tensorflow/core/platform/path.cc +@@ -329,6 +329,7 @@ + } + LOG(FATAL) << "No temp directory found."; + #endif ++ std::abort(); + } + + bool GetTestUndeclaredOutputsDir(string* dir) { + +diff --git a/tensorflow/core/framework/device_base.cc b/tensorflow/core/framework/device_base.cc +--- a/tensorflow/core/framework/device_base.cc ++++ b/tensorflow/core/framework/device_base.cc +@@ -65,10 +65,12 @@ + + const DeviceAttributes& DeviceBase::attributes() const { + LOG(FATAL) << "Device does not implement attributes()"; ++ std::abort(); + } + + const string& DeviceBase::name() const { + LOG(FATAL) << "Device does not implement name()"; ++ std::abort(); + } + + void DeviceBase::set_eigen_cpu_device(Eigen::ThreadPoolDevice* d) { + +diff --git a/tensorflow/core/kernels/depthtospace_op.cc b/tensorflow/core/kernels/depthtospace_op.cc +--- a/tensorflow/core/kernels/depthtospace_op.cc ++++ b/tensorflow/core/kernels/depthtospace_op.cc +@@ -117,13 +117,13 @@ + // NCHW_VECT_C with 4 x qint8 can be treated as NCHW int32. + auto Tinput_v = input.template reinterpret_last_dimension<int32, 4>(); + auto Toutput_v = outputs_tensor->reinterpret_last_dimension<int32, 4>(); +- functor::DepthToSpaceOpFunctor<GPUDevice, int32, FORMAT_NCHW> functor; +- functor(context->eigen_device<GPUDevice>(), Tinput_v, block_size_, ++ functor::DepthToSpaceOpFunctor<Device, int32, FORMAT_NCHW> functor; ++ functor(context->eigen_device<Device>(), Tinput_v, block_size_, + Toutput_v); + return; + } else if (data_format_ == FORMAT_NCHW) { +- functor::DepthToSpaceOpFunctor<GPUDevice, T, FORMAT_NCHW> functor; +- functor(context->eigen_device<GPUDevice>(), Tinput, block_size_, ++ functor::DepthToSpaceOpFunctor<Device, T, FORMAT_NCHW> functor; ++ functor(context->eigen_device<Device>(), Tinput, block_size_, + Toutput); + return; + } +@@ -173,6 +173,15 @@ + } + } + }; ++#ifdef WIN32 ++template <typename T> ++struct DepthToSpaceOpFunctor<CPUDevice, T, FORMAT_NCHW> { ++ void operator()(const CPUDevice& d, typename TTypes<T, 4>::ConstTensor input, ++ int block_size, typename TTypes<T, 4>::Tensor output) { ++ LOG(FATAL) << "dummy implementation to make debug build compile"; ++ } ++}; ++#endif + } // namespace functor + + #define REGISTER(type) \ + +diff --git a/tensorflow/core/kernels/spacetodepth_op.cc b/tensorflow/core/kernels/spacetodepth_op.cc +--- a/tensorflow/core/kernels/spacetodepth_op.cc ++++ b/tensorflow/core/kernels/spacetodepth_op.cc +@@ -132,18 +132,18 @@ + // NCHW_VECT_C with 4 x qint8 can be treated as NCHW int32. + auto Tinput_v = input.template reinterpret_last_dimension<int32, 4>(); + auto Toutput_v = outputs_tensor->reinterpret_last_dimension<int32, 4>(); +- functor::SpaceToDepthOpFunctor<GPUDevice, int32, FORMAT_NCHW> functor; +- functor(context->eigen_device<GPUDevice>(), Tinput_v, block_size_, ++ functor::SpaceToDepthOpFunctor<Device, int32, FORMAT_NCHW> functor; ++ functor(context->eigen_device<Device>(), Tinput_v, block_size_, + Toutput_v); + } else if (data_format_ == FORMAT_NCHW) { + CHECK((std::is_same<T, RT>::value)); +- functor::SpaceToDepthOpFunctor<GPUDevice, RT, FORMAT_NCHW> functor; +- functor(context->eigen_device<GPUDevice>(), input.tensor<RT, 4>(), ++ functor::SpaceToDepthOpFunctor<Device, RT, FORMAT_NCHW> functor; ++ functor(context->eigen_device<Device>(), input.tensor<RT, 4>(), + block_size_, outputs_tensor->tensor<RT, 4>()); + } else { + CHECK((std::is_same<T, RT>::value)); +- functor::SpaceToDepthOpFunctor<GPUDevice, RT, FORMAT_NHWC> functor; +- functor(context->eigen_device<GPUDevice>(), input.tensor<RT, 4>(), ++ functor::SpaceToDepthOpFunctor<Device, RT, FORMAT_NHWC> functor; ++ functor(context->eigen_device<Device>(), input.tensor<RT, 4>(), + block_size_, outputs_tensor->tensor<RT, 4>()); + } + } else { +@@ -188,6 +188,15 @@ + } + } + }; ++#ifdef WIN32 ++template <typename T> ++struct SpaceToDepthOpFunctor<CPUDevice, T, FORMAT_NCHW> { ++ void operator()(const CPUDevice& d, typename TTypes<T, 4>::ConstTensor input, ++ int block_size, typename TTypes<T, 4>::Tensor output) { ++ LOG(FATAL) << "dummy implementation to make debug build compile"; ++ } ++}; ++#endif + } // namespace functor + + #define REGISTER(type) \ diff --git a/ports/tensorflow-cc/fix-linux-build.patch b/ports/tensorflow-cc/fix-linux-build.patch new file mode 100644 index 000000000..81a6a3be8 --- /dev/null +++ b/ports/tensorflow-cc/fix-linux-build.patch @@ -0,0 +1,64 @@ +diff --git a/third_party/aws/aws-checksums.bazel b/third_party/aws/aws-checksums.bazel +--- a/third_party/aws/aws-checksums.bazel ++++ b/third_party/aws/aws-checksums.bazel +@@ -7,6 +7,13 @@ + + exports_files(["LICENSE"]) + ++load("@bazel_skylib//lib:selects.bzl", "selects") ++ ++selects.config_setting_group( ++ name = "is_linux_debug", ++ match_all = ["@org_tensorflow//tensorflow:linux_x86_64", "@org_tensorflow//tensorflow:debug"], ++) ++ + cc_library( + name = "aws-checksums", + srcs = select({ +@@ -27,4 +34,10 @@ + deps = [ + "@aws-c-common", + ], ++ defines = select({ ++ ":is_linux_debug": [ ++ "DEBUG_BUILD" ++ ], ++ "//conditions:default": [], ++ }), + ) + +diff --git a/tensorflow/core/kernels/data/experimental/io_ops.cc b/tensorflow/core/kernels/data/experimental/io_ops.cc +--- a/tensorflow/core/kernels/data/experimental/io_ops.cc ++++ b/tensorflow/core/kernels/data/experimental/io_ops.cc +@@ -28,6 +28,12 @@ + namespace data { + namespace experimental { + ++/*static*/ constexpr const char* const LoadDatasetOp::kCompression; ++/*static*/ constexpr const char* const LoadDatasetOp::kReaderFunc; ++/*static*/ constexpr const char* const LoadDatasetOp::kReaderFuncTarguments; ++ ++/*static*/ constexpr const int SaveDatasetOp::kFileFormatVersion; ++ + SaveDatasetOp::SaveDatasetOp(OpKernelConstruction* ctx) + : HybridAsyncOpKernel(ctx, "tf_data_save_dataset") { + OP_REQUIRES_OK(ctx, ctx->GetAttr(kCompression, &compression_)); + +diff --git a/tensorflow/core/kernels/data/experimental/snapshot_dataset_op.cc b/tensorflow/core/kernels/data/experimental/snapshot_dataset_op.cc +--- a/tensorflow/core/kernels/data/experimental/snapshot_dataset_op.cc ++++ b/tensorflow/core/kernels/data/experimental/snapshot_dataset_op.cc +@@ -63,6 +63,14 @@ + namespace data { + namespace experimental { + ++/*static*/ constexpr const char* const SnapshotDatasetV2Op::kCompression; ++/*static*/ constexpr const char* const SnapshotDatasetV2Op::kReaderFunc; ++/*static*/ constexpr const char* const SnapshotDatasetV2Op::kShardFunc; ++/*static*/ constexpr const char* const SnapshotDatasetV2Op::kReaderFuncTarguments; ++/*static*/ constexpr const char* const SnapshotDatasetV2Op::kShardFuncTarguments; ++ ++/*static*/ constexpr const int SnapshotDatasetV2Op::kFileFormatVersion; ++ + // ==== Snapshot Implementation ==== + + /* The current snapshot on-disk layout is as follows: diff --git a/ports/tensorflow-cc/fix-more-build-errors.patch b/ports/tensorflow-cc/fix-more-build-errors.patch new file mode 100644 index 000000000..c7d6a9b7f --- /dev/null +++ b/ports/tensorflow-cc/fix-more-build-errors.patch @@ -0,0 +1,11 @@ +diff --git a/tensorflow/core/distributed_runtime/remote_device.cc b/tensorflow/core/distributed_runtime/remote_device.cc +--- a/tensorflow/core/distributed_runtime/remote_device.cc ++++ b/tensorflow/core/distributed_runtime/remote_device.cc +@@ -45,6 +45,7 @@ + ResourceMgr* resource_manager() override { + LOG(FATAL) << "Accessing the resource manager of a remote device is not " + << "supported."; ++ std::abort(); + } + + bool IsLocal() const override { return false; } diff --git a/ports/tensorflow-cc/generate_static_link_cmd_linux.py b/ports/tensorflow-cc/generate_static_link_cmd_linux.py new file mode 100644 index 000000000..3b7c04f49 --- /dev/null +++ b/ports/tensorflow-cc/generate_static_link_cmd_linux.py @@ -0,0 +1,22 @@ +import re
+import sys
+
+with open(sys.argv[1], "r") as f_in:
+ with open("static_link.sh", "w") as f_out:
+ p_cd = re.compile("^\\((cd .*) && \\\\$")
+ p_linker = re.compile("^\\s*(.+)gcc.+(@bazel-out\\S+libtensorflow_cc\\.so\\.\\d\\.\\d\\.\\d-2\\.params).*")
+ f_out.write("#!/bin/bash\n# note: ar/binutils version 2.27 required to support output files > 4GB\n")
+ env = []
+ for line in f_in:
+ if line.startswith("(cd"):
+ # new command, reset
+ env = [line]
+ else:
+ m1 = p_linker.match(line)
+ if m1:
+ m2 = p_cd.match(env[0])
+ f_out.write(m2.group(1) + "\n")
+ line = f'"{m1.group(1)}ar" rcs {m1.group(2)[1:-9].replace(".so", ".a")} {m1.group(2).replace(".so", ".a")}\n'
+ f_out.write(line)
+ else:
+ env.append(line)
diff --git a/ports/tensorflow-cc/generate_static_link_cmd_macos.py b/ports/tensorflow-cc/generate_static_link_cmd_macos.py new file mode 100644 index 000000000..f1dfbbc46 --- /dev/null +++ b/ports/tensorflow-cc/generate_static_link_cmd_macos.py @@ -0,0 +1,46 @@ +import re
+import sys
+
+with open(sys.argv[1], "r") as f_in:
+ with open("static_link.sh", "w") as f_out:
+ p_cd = re.compile("^\\((cd .*) && \\\\$")
+ p_linker1 = re.compile("^.*cc_wrapper.sh.+-shared.+-o (bazel-out\\S+libtensorflow_cc\\.\\d\\.\\d\\.\\d\\.dylib)")
+ p_linker2 = re.compile("^.*cc_wrapper.sh.+-shared.+-o (bazel-out\\S+libtensorflow_framework\\.\\d\\.\\d\\.\\d\\.dylib)")
+ f_out.write("#!/bin/bash\n# note: ar/binutils version 2.27 required to support output files > 4GB\n")
+ env = []
+ parts = None
+ for line in f_in:
+ if line.startswith("(cd"):
+ # new command, reset
+ env = [line]
+ else:
+ m1 = p_linker1.match(line)
+ m2 = p_linker2.match(line)
+ if m1:
+ tokens = line.split()
+ if parts is None:
+ parts = [t[16:] for t in tokens if t.startswith("-Wl,-force_load,")]
+ else:
+ m = p_cd.match(env[0])
+ f_out.write(m.group(1) + "\n")
+ tmp = [t[16:] for t in tokens if t.startswith("-Wl,-force_load,")]
+ old = set(parts)
+ parts += [t for t in tmp if t not in old]
+ line = f"libtool -static -o {m1.group(1).replace('.dylib', '.a')} {' '.join(parts)}\n"
+ f_out.write(line)
+ break
+ elif m2 and len(env) > 6:
+ tokens = line.split()
+ if parts is None:
+ parts = [t[16:] for t in tokens if t.startswith("-Wl,-force_load,")]
+ else:
+ m = p_cd.match(env[0])
+ f_out.write(m.group(1) + "\n")
+ tmp = [t[16:] for t in tokens if t.startswith("-Wl,-force_load,")]
+ old = set(parts)
+ parts += [t for t in tmp if t not in old]
+ line = f"libtool -static -o {m2.group(1).replace('_framework', '_cc').replace('.dylib', '.a')} {' '.join(parts)}\n"
+ f_out.write(line)
+ break
+ else:
+ env.append(line)
diff --git a/ports/tensorflow-cc/generate_static_link_cmd_windows.py b/ports/tensorflow-cc/generate_static_link_cmd_windows.py new file mode 100644 index 000000000..363b3ad88 --- /dev/null +++ b/ports/tensorflow-cc/generate_static_link_cmd_windows.py @@ -0,0 +1,41 @@ +import os.path
+import re
+import sys
+
+with open(sys.argv[1], "r") as f_in:
+ with open("static_link.bat", "w") as f_out:
+ p_setenv = re.compile("^\s*(SET .+=.*)$")
+ p_linker = re.compile(".+link\\.exe.+tensorflow_cc\\.dll-2\\.params.*")
+ env = []
+ for line in f_in:
+ if line.startswith("cd"):
+ # new command, reset
+ env = []
+ else:
+ m = p_setenv.match(line)
+ if m:
+ env.append(m.group(1))
+ else:
+ m = p_linker.match(line)
+ if m:
+ for e in env:
+ f_out.write(e + "\n")
+ tokens = line.split()
+ line = "\""
+ params_file = None
+ for t in tokens:
+ if t.endswith("link.exe"):
+ t = t[:-len("link.exe")] + "lib.exe\""
+ elif t == "/DLL" or t.lower()[1:].startswith("defaultlib:") or t.lower()[1:].startswith("ignore") or t.startswith("/OPT:") or t.startswith("/DEF:") or t.startswith("/DEBUG:") or t.startswith("/INCREMENTAL:"):
+ continue
+ elif t[0] == '@' and t.endswith("tensorflow_cc.dll-2.params"):
+ t = t[:-len("dll-2.params")] + "lib-2.params-part1"
+ params_file = t[1:-len("-part1")]
+ line += t + " "
+ f_out.write(line + "\n")
+ # check for more parts if library needs to be split
+ file_no = 2
+ while os.path.isfile(f"{params_file}-part{file_no}"):
+ f_out.write(line.replace("lib-2.params-part1", f"lib-2.params-part{file_no}") + "\n")
+ file_no += 1
+ break
diff --git a/ports/tensorflow-cc/portfile.cmake b/ports/tensorflow-cc/portfile.cmake index 0aa53e7ae..35dbb5d1b 100644 --- a/ports/tensorflow-cc/portfile.cmake +++ b/ports/tensorflow-cc/portfile.cmake @@ -1,61 +1,53 @@ -message(WARNING "This tensorflow port currently is experimental on Windows and Linux platforms.") +vcpkg_fail_port_install(ON_ARCH "x86" "arm" ON_TARGET "UWP") -if (VCPKG_TARGET_ARCHITECTURE STREQUAL x86) - message(FATAL_ERROR "TensorFlow does not support 32bit systems.") -endif() - -vcpkg_from_github( - OUT_SOURCE_PATH SOURCE_PATH - REPO tensorflow/tensorflow - REF v1.14.0 - SHA512 ac9ea5a2d1c761aaafbdc335259e29c128127b8d069ec5b206067935180490aa95e93c7e13de57f7f54ce4ba4f34a822face22b4a028f60185edb380e5cd4787 - HEAD_REF master - PATCHES - file-exists.patch # required or otherwise it cant find python lib path on windows - fix-build-error.patch # Fix namespace error -) +set(TF_VERSION 2.3.1) +set(TF_VERSION_SHORT 2.3) -# due to https://github.com/bazelbuild/bazel/issues/8028, bazel must be version 25.0 or higher vcpkg_find_acquire_program(BAZEL) get_filename_component(BAZEL_DIR "${BAZEL}" DIRECTORY) vcpkg_add_to_path(PREPEND ${BAZEL_DIR}) set(ENV{BAZEL_BIN_PATH} "${BAZEL}") -vcpkg_find_acquire_program(PYTHON3) -get_filename_component(PYTHON3_DIR "${PYTHON3}" DIRECTORY) -vcpkg_add_to_path(PREPEND ${PYTHON3_DIR}) -set(ENV{PYTHON_BIN_PATH} "${PYTHON3}") +function(tensorflow_try_remove_recurse_wait PATH_TO_REMOVE) + file(REMOVE_RECURSE ${PATH_TO_REMOVE}) + if(EXISTS "${PATH_TO_REMOVE}") + vcpkg_execute_required_process(COMMAND ${CMAKE_COMMAND} -E sleep 5 WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR} LOGNAME prerequesits-sleep-${TARGET_TRIPLET}) + file(REMOVE_RECURSE ${PATH_TO_REMOVE}) + endif() +endfunction() vcpkg_find_acquire_program(GIT) get_filename_component(GIT_DIR "${GIT}" DIRECTORY) vcpkg_add_to_path(PREPEND ${GIT_DIR}) -function(tensorflow_try_remove_recurse_wait PATH_TO_REMOVE) - file(REMOVE_RECURSE ${PATH_TO_REMOVE}) - if (EXISTS "${PATH_TO_REMOVE}") - _execute_process(COMMAND ${CMAKE_COMMAND} -E sleep 5) - file(REMOVE_RECURSE ${PATH_TO_REMOVE}) - endif() -endfunction() - if(CMAKE_HOST_WIN32) - vcpkg_acquire_msys(MSYS_ROOT PACKAGES bash unzip patch diffutils libintl gzip coreutils) - vcpkg_add_to_path(${MSYS_ROOT}/usr/bin) - set(BASH ${MSYS_ROOT}/usr/bin/bash.exe) - set(ENV{BAZEL_SH} ${MSYS_ROOT}/usr/bin/bash.exe) - - set(ENV{BAZEL_VS} $ENV{VSInstallDir}) - set(ENV{BAZEL_VC} $ENV{VCInstallDir}) - set(ENV{BAZEL_VC_FULL_VERSION} $ENV{VCToolsVersion}) + vcpkg_acquire_msys(MSYS_ROOT PACKAGES bash unzip patch diffutils libintl gzip coreutils mingw-w64-x86_64-python-numpy) + vcpkg_add_to_path(${MSYS_ROOT}/usr/bin) + vcpkg_add_to_path(${MSYS_ROOT}/mingw64/bin) + set(BASH ${MSYS_ROOT}/usr/bin/bash.exe) + + set(ENV{BAZEL_SH} ${MSYS_ROOT}/usr/bin/bash.exe) + set(ENV{BAZEL_VC} $ENV{VCInstallDir}) + set(ENV{BAZEL_VC_FULL_VERSION} $ENV{VCToolsVersion}) + + set(PYTHON3 "${MSYS_ROOT}/mingw64/bin/python3.exe") + vcpkg_execute_required_process(COMMAND ${PYTHON3} -c "import site; print(site.getsitepackages()[0])" WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR} LOGNAME prerequesits-pypath-${TARGET_TRIPLET} OUTPUT_VARIABLE PYTHON_LIB_PATH) +else() + vcpkg_find_acquire_program(PYTHON3) + get_filename_component(PYTHON3_DIR "${PYTHON3}" DIRECTORY) + vcpkg_add_to_path(PREPEND ${PYTHON3_DIR}) + + vcpkg_execute_required_process(COMMAND ${PYTHON3} -m pip install --user -U numpy WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR} LOGNAME prerequesits-pip-${TARGET_TRIPLET}) + vcpkg_execute_required_process(COMMAND ${PYTHON3} -c "import site; print(site.getusersitepackages())" WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR} LOGNAME prerequesits-pypath-${TARGET_TRIPLET} OUTPUT_VARIABLE PYTHON_LIB_PATH) endif() +set(ENV{PYTHON_BIN_PATH} "${PYTHON3}") +set(ENV{PYTHON_LIB_PATH} "${PYTHON_LIB_PATH}") -# we currently only support the release version -tensorflow_try_remove_recurse_wait(${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel) -file(RENAME ${SOURCE_PATH} ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel) -set(SOURCE_PATH "${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel") +# check if numpy can be loaded +vcpkg_execute_required_process(COMMAND ${PYTHON3} -c "import numpy" WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR} LOGNAME prerequesits-numpy-${TARGET_TRIPLET}) # tensorflow has long file names, which will not work on windows -set(ENV{TEST_TMPDIR} ${CURRENT_BUILDTREES_DIR}/../.bzl) +set(ENV{TEST_TMPDIR} ${BUILDTREES_DIR}/.bzl) set(ENV{USE_DEFAULT_PYTHON_LIB_PATH} 1) set(ENV{TF_NEED_KAFKA} 0) @@ -75,51 +67,379 @@ set(ENV{TF_NEED_IGNITE} 0) set(ENV{TF_NEED_ROCM} 0) set(ENV{TF_SET_ANDROID_WORKSPACE} 0) set(ENV{TF_DOWNLOAD_CLANG} 0) -set(ENV{TF_NCCL_VERSION} 2.3) +set(ENV{TF_NCCL_VERSION} ${TF_VERSION_SHORT}) set(ENV{NCCL_INSTALL_PATH} "") set(ENV{CC_OPT_FLAGS} "/arch:AVX") set(ENV{TF_NEED_CUDA} 0) set(ENV{TF_CONFIGURE_IOS} 0) -message(STATUS "Configuring TensorFlow") - -vcpkg_execute_required_process( - COMMAND ${PYTHON3} ${SOURCE_PATH}/configure.py --workspace "${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel" - WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel - LOGNAME config-${TARGET_TRIPLET}-rel -) -message(STATUS "Warning: Building TensorFlow can take an hour or more.") - -if(CMAKE_HOST_WIN32) - vcpkg_execute_build_process( - COMMAND ${BASH} --noprofile --norc -c "${BAZEL} build --verbose_failures -c opt --python_path=${PYTHON3} --incompatible_disable_deprecated_attr_params=false --define=no_tensorflow_py_deps=true ///tensorflow:libtensorflow_cc.so ///tensorflow:install_headers" - WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel - LOGNAME build-${TARGET_TRIPLET}-rel - ) +if(VCPKG_TARGET_IS_WINDOWS) + set(BAZEL_LIB_NAME tensorflow_cc.dll) + set(PLATFORM_SUFFIX windows) + set(STATIC_LINK_CMD static_link.bat) +elseif(VCPKG_TARGET_IS_OSX) + set(BAZEL_LIB_NAME libtensorflow_cc.dylib) + set(PLATFORM_SUFFIX macos) + set(STATIC_LINK_CMD sh static_link.sh) + if(VCPKG_LIBRARY_LINKAGE STREQUAL dynamic) + set(TF_LIB_NAME "libtensorflow_cc.dylib") + set(TF_LIB_NAME_SHORT "libtensorflow_cc.2.dylib") + set(TF_LIB_NAME_FULL "libtensorflow_cc.${TF_VERSION}.dylib") + set(TF_FRAMEWORK_NAME "libtensorflow_framework.dylib") + set(TF_FRAMEWORK_NAME_SHORT "libtensorflow_framework.2.dylib") + set(TF_FRAMEWORK_NAME_FULL "libtensorflow_framework.${TF_VERSION}.dylib") + else() + set(TF_LIB_NAME "libtensorflow_cc.a") + set(TF_LIB_NAME_SHORT "libtensorflow_cc.2.a") + set(TF_LIB_NAME_FULL "libtensorflow_cc.${TF_VERSION}.a") + endif() else() - vcpkg_execute_build_process( - COMMAND ${BAZEL} build --verbose_failures -c opt --python_path=${PYTHON3} --incompatible_disable_deprecated_attr_params=false --define=no_tensorflow_py_deps=true //tensorflow:libtensorflow_cc.so //tensorflow:install_headers - WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel - LOGNAME build-${TARGET_TRIPLET}-rel - ) + set(BAZEL_LIB_NAME libtensorflow_cc.so) + set(PLATFORM_SUFFIX linux) + set(STATIC_LINK_CMD sh static_link.sh) + if(VCPKG_LIBRARY_LINKAGE STREQUAL dynamic) + set(TF_LIB_NAME "libtensorflow_cc.so") + set(TF_LIB_NAME_SHORT "libtensorflow_cc.so.2") + set(TF_LIB_NAME_FULL "libtensorflow_cc.so.${TF_VERSION}") + set(TF_FRAMEWORK_NAME "libtensorflow_framework.so") + set(TF_FRAMEWORK_NAME_SHORT "libtensorflow_framework.so.2") + set(TF_FRAMEWORK_NAME_FULL "libtensorflow_framework.so.${TF_VERSION}") + else() + set(TF_LIB_NAME "libtensorflow_cc.a") + set(TF_LIB_NAME_SHORT "libtensorflow_cc.a.2") + set(TF_LIB_NAME_FULL "libtensorflow_cc.a.${TF_VERSION}") + endif() endif() -file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/bazel-genfiles/tensorflow/include/ DESTINATION ${CURRENT_PACKAGES_DIR}/include/tensorflow-external) +foreach(BUILD_TYPE dbg rel) + # prefer repeated source extraction here for each build type over extracting once above the loop and copying because users reported issues with copying symlinks + set(STATIC_ONLY_PATCHES) + set(LINUX_ONLY_PATCHES) + if(VCPKG_LIBRARY_LINKAGE STREQUAL static) + set(STATIC_ONLY_PATCHES change-macros-for-static-lib.patch) # there is no static build option - change macros via patch and link library manually at the end + endif() + if(VCPKG_TARGET_IS_LINUX) + set(LINUX_ONLY_PATCHES fix-linux-build.patch) + endif() + vcpkg_from_github( + OUT_SOURCE_PATH SOURCE_PATH + REPO tensorflow/tensorflow + REF "v${TF_VERSION}" + SHA512 e497ef4564f50abf9f918be4522cf702f4cf945cb1ebf83af1386ac4ddc7373b3ba70c7f803f8ca06faf2c6b5396e60b1e0e9b97bfbd667e733b08b6e6d70ef0 + HEAD_REF master + PATCHES + fix-build-error.patch # Fix namespace error + fix-dbg-build-errors.patch # Fix no return statement + fix-more-build-errors.patch # Fix no return statement + ${STATIC_ONLY_PATCHES} + ${LINUX_ONLY_PATCHES} + ) -if(CMAKE_HOST_WIN32) - file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/bazel-bin/tensorflow/libtensorflow_cc.so.1.14.0 DESTINATION ${CURRENT_PACKAGES_DIR}/lib) - file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/bazel-bin/tensorflow/libtensorflow_cc.so.1.14.0.if.lib DESTINATION ${CURRENT_PACKAGES_DIR}/lib) - file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/bazel-bin/tensorflow/libtensorflow_cc.so.1.14.0 DESTINATION ${CURRENT_PACKAGES_DIR}/debug/lib) - file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/bazel-bin/tensorflow/libtensorflow_cc.so.1.14.0.if.lib DESTINATION ${CURRENT_PACKAGES_DIR}/debug/lib) -else() - file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/bazel-bin/tensorflow/libtensorflow_cc.so.1.14.0 DESTINATION ${CURRENT_PACKAGES_DIR}/lib) - file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/bazel-bin/tensorflow/libtensorflow_framework.so.1.14.0 DESTINATION ${CURRENT_PACKAGES_DIR}/lib) - file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/bazel-bin/tensorflow/libtensorflow_cc.so.1.14.0 DESTINATION ${CURRENT_PACKAGES_DIR}/debug/lib) - file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/bazel-bin/tensorflow/libtensorflow_framework.so.1.14.0 DESTINATION ${CURRENT_PACKAGES_DIR}/debug/lib) + message(STATUS "Configuring TensorFlow (${BUILD_TYPE})") + tensorflow_try_remove_recurse_wait(${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}) + file(RENAME ${SOURCE_PATH} ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}) + set(SOURCE_PATH "${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}") + + vcpkg_execute_required_process( + COMMAND ${PYTHON3} ${SOURCE_PATH}/configure.py --workspace "${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}" + WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE} + LOGNAME config-${TARGET_TRIPLET}-${BUILD_TYPE} + ) + + if(DEFINED ENV{BAZEL_CUSTOM_CACERTS}) + file(APPEND ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/.bazelrc "startup --host_jvm_args=-Djavax.net.ssl.trustStore='$ENV{BAZEL_CUSTOM_CACERTS}'\n") + message(STATUS "Using custom CA certificate store at: $ENV{BAZEL_CUSTOM_CACERTS}") + if(DEFINED ENV{BAZEL_CUSTOM_CACERTS_PASSWORD}) + file(APPEND ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/.bazelrc "startup --host_jvm_args=-Djavax.net.ssl.trustStorePassword='$ENV{BAZEL_CUSTOM_CACERTS_PASSWORD}'\n") + message(STATUS "Using supplied custom CA certificate store password.") + endif() + else() + if(DEFINED ENV{HTTPS_PROXY}) + message(STATUS "You are using HTTPS_PROXY. In case you encounter bazel certificate errors, you might want to set: BAZEL_CUSTOM_CACERTS=/path/to/trust.store (and optionally BAZEL_CUSTOM_CACERTS_PASSWORD), and to enable vcpkg to actually use it: VCPKG_KEEP_ENV_VARS=BAZEL_CUSTOM_CACERTS;BAZEL_CUSTOM_CACERTS_PASSWORD") + if(CMAKE_HOST_WIN32) + message(STATUS "(For BAZEL_CUSTOM_CACERTS please use forward slashes instead of backslashes on Windows systems.") + endif() + endif() + endif() + + message(STATUS "Warning: Building TensorFlow can take an hour or more.") + set(COPTS) + set(CXXOPTS) + set(LINKOPTS) + if(VCPKG_TARGET_IS_WINDOWS) + set(PLATFORM_COMMAND WINDOWS_COMMAND) + else() + set(PLATFORM_COMMAND UNIX_COMMAND) + endif() + if(BUILD_TYPE STREQUAL dbg) + if(VCPKG_TARGET_IS_WINDOWS) + set(BUILD_OPTS "--compilation_mode=dbg --features=fastbuild") # link with /DEBUG:FASTLINK instead of /DEBUG:FULL to avoid .pdb >4GB error + else() + set(BUILD_OPTS "--compilation_mode=dbg") + endif() + + separate_arguments(VCPKG_C_FLAGS ${PLATFORM_COMMAND} ${VCPKG_C_FLAGS}) + separate_arguments(VCPKG_C_FLAGS_DEBUG ${PLATFORM_COMMAND} ${VCPKG_C_FLAGS_DEBUG}) + foreach(OPT IN LISTS VCPKG_C_FLAGS VCPKG_C_FLAGS_DEBUG) + list(APPEND COPTS "--copt='${OPT}'") + endforeach() + separate_arguments(VCPKG_CXX_FLAGS ${PLATFORM_COMMAND} ${VCPKG_CXX_FLAGS}) + separate_arguments(VCPKG_CXX_FLAGS_DEBUG ${PLATFORM_COMMAND} ${VCPKG_CXX_FLAGS_DEBUG}) + foreach(OPT IN LISTS VCPKG_CXX_FLAGS VCPKG_CXX_FLAGS_DEBUG) + list(APPEND CXXOPTS "--cxxopt='${OPT}'") + endforeach() + separate_arguments(VCPKG_LINKER_FLAGS ${PLATFORM_COMMAND} ${VCPKG_LINKER_FLAGS}) + separate_arguments(VCPKG_LINKER_FLAGS_DEBUG ${PLATFORM_COMMAND} ${VCPKG_LINKER_FLAGS_DEBUG}) + foreach(OPT IN LISTS VCPKG_LINKER_FLAGS VCPKG_LINKER_FLAGS_DEBUG) + list(APPEND LINKOPTS "--linkopt='${OPT}'") + endforeach() + else() + set(BUILD_OPTS "--compilation_mode=opt") + + separate_arguments(VCPKG_C_FLAGS ${PLATFORM_COMMAND} ${VCPKG_C_FLAGS}) + separate_arguments(VCPKG_C_FLAGS_RELEASE ${PLATFORM_COMMAND} ${VCPKG_C_FLAGS_RELEASE}) + foreach(OPT IN LISTS VCPKG_C_FLAGS VCPKG_C_FLAGS_RELEASE) + list(APPEND COPTS "--copt='${OPT}'") + endforeach() + separate_arguments(VCPKG_CXX_FLAGS ${PLATFORM_COMMAND} ${VCPKG_CXX_FLAGS}) + separate_arguments(VCPKG_CXX_FLAGS_RELEASE ${PLATFORM_COMMAND} ${VCPKG_CXX_FLAGS_RELEASE}) + foreach(OPT IN LISTS VCPKG_CXX_FLAGS VCPKG_CXX_FLAGS_RELEASE) + list(APPEND CXXOPTS "--cxxopt='${OPT}'") + endforeach() + separate_arguments(VCPKG_LINKER_FLAGS ${PLATFORM_COMMAND} ${VCPKG_LINKER_FLAGS}) + separate_arguments(VCPKG_LINKER_FLAGS_RELEASE ${PLATFORM_COMMAND} ${VCPKG_LINKER_FLAGS_RELEASE}) + foreach(OPT IN LISTS VCPKG_LINKER_FLAGS VCPKG_LINKER_FLAGS_RELEASE) + list(APPEND LINKOPTS "--linkopt='${OPT}'") + endforeach() + endif() + + if(VCPKG_LIBRARY_LINKAGE STREQUAL dynamic) + if(VCPKG_TARGET_IS_WINDOWS) + list(JOIN COPTS " " COPTS) + list(JOIN CXXOPTS " " CXXOPTS) + list(JOIN LINKOPTS " " LINKOPTS) + vcpkg_execute_build_process( + COMMAND ${BASH} --noprofile --norc -c "'${BAZEL}' build --verbose_failures ${BUILD_OPTS} ${COPTS} ${CXXOPTS} ${LINKOPTS} --python_path='${PYTHON3}' --define=no_tensorflow_py_deps=true ///tensorflow:tensorflow_cc.dll ///tensorflow:install_headers" + WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE} + LOGNAME build-${TARGET_TRIPLET}-${BUILD_TYPE} + ) + else() + vcpkg_execute_build_process( + COMMAND ${BAZEL} build --verbose_failures ${BUILD_OPTS} --python_path=${PYTHON3} ${COPTS} ${CXXOPTS} ${LINKOPTS} --define=no_tensorflow_py_deps=true //tensorflow:${BAZEL_LIB_NAME} //tensorflow:install_headers + WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE} + LOGNAME build-${TARGET_TRIPLET}-${BUILD_TYPE} + ) + endif() + else() + if(VCPKG_TARGET_IS_WINDOWS) + if(VCPKG_CRT_LINKAGE STREQUAL static) + if(BUILD_TYPE STREQUAL dbg) + list(APPEND COPTS "--copt=-MTd") + else() + list(APPEND COPTS "--copt=-MT") + endif() + endif() + list(JOIN COPTS " " COPTS) + list(JOIN CXXOPTS " " CXXOPTS) + list(JOIN LINKOPTS " " LINKOPTS) + vcpkg_execute_build_process( + COMMAND ${BASH} --noprofile --norc -c "${BAZEL} build -s --verbose_failures ${BUILD_OPTS} --features=fully_static_link ${COPTS} ${CXXOPTS} ${LINKOPTS} --python_path='${PYTHON3}' --define=no_tensorflow_py_deps=true ///tensorflow:tensorflow_cc.dll ///tensorflow:install_headers" + WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE} + LOGNAME build-${TARGET_TRIPLET}-${BUILD_TYPE} + ) + else() + vcpkg_execute_build_process( + COMMAND ${BAZEL} build -s --verbose_failures ${BUILD_OPTS} ${COPTS} ${CXXOPTS} ${LINKOPTS} --python_path=${PYTHON3} --define=no_tensorflow_py_deps=true //tensorflow:${BAZEL_LIB_NAME} //tensorflow:install_headers + WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE} + LOGNAME build-${TARGET_TRIPLET}-${BUILD_TYPE} + ) + endif() + if(NOT VCPKG_TARGET_IS_OSX) + if(VCPKG_TARGET_IS_WINDOWS) + vcpkg_execute_build_process( + COMMAND ${PYTHON3} "${CMAKE_CURRENT_LIST_DIR}/convert_lib_params_${PLATFORM_SUFFIX}.py" + WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-bin/tensorflow + LOGNAME postbuild1-${TARGET_TRIPLET}-${BUILD_TYPE} + ) + else() + vcpkg_execute_build_process( + COMMAND ${PYTHON3} "${CMAKE_CURRENT_LIST_DIR}/convert_lib_params_${PLATFORM_SUFFIX}.py" "${TF_VERSION}" + WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-bin/tensorflow + LOGNAME postbuild1-${TARGET_TRIPLET}-${BUILD_TYPE} + ) + endif() + endif() + vcpkg_execute_build_process( + COMMAND ${PYTHON3} "${CMAKE_CURRENT_LIST_DIR}/generate_static_link_cmd_${PLATFORM_SUFFIX}.py" "${CURRENT_BUILDTREES_DIR}/build-${TARGET_TRIPLET}-${BUILD_TYPE}-err.log" # for some reason stdout of bazel ends up in stderr + WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-${TARGET_TRIPLET}-${BUILD_TYPE} + LOGNAME postbuild2-${TARGET_TRIPLET}-${BUILD_TYPE} + ) + vcpkg_execute_build_process( + COMMAND ${STATIC_LINK_CMD} + WORKING_DIRECTORY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-${TARGET_TRIPLET}-${BUILD_TYPE} + LOGNAME postbuild3-${TARGET_TRIPLET}-${BUILD_TYPE} + ) + endif() + + if(BUILD_TYPE STREQUAL "dbg") + set(DIR_PREFIX "/debug") + else() + set(DIR_PREFIX "") + endif() + + if(VCPKG_TARGET_IS_WINDOWS) + if(VCPKG_LIBRARY_LINKAGE STREQUAL dynamic) + file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-bin/tensorflow/tensorflow_cc.dll DESTINATION ${CURRENT_PACKAGES_DIR}${DIR_PREFIX}/bin) + # rename before copy because after copy the file might be locked by anti-malware scanners for some time so that renaming fails + file(RENAME ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-bin/tensorflow/tensorflow_cc.dll.if.lib ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-bin/tensorflow/tensorflow_cc.lib) + file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-bin/tensorflow/tensorflow_cc.lib DESTINATION ${CURRENT_PACKAGES_DIR}${DIR_PREFIX}/lib) + if(BUILD_TYPE STREQUAL dbg) + file(COPY ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-bin/tensorflow/tensorflow_cc.pdb DESTINATION ${CURRENT_PACKAGES_DIR}${DIR_PREFIX}/bin) + message(STATUS "Warning: debug information tensorflow_cc.pdb will be of limited use because only a reduced set could be produced due to the 4GB internal PDB file limit even on x64.") + endif() + else() + if(BUILD_TYPE STREQUAL dbg) + set(library_parts_variable TF_LIB_PARTS_DEBUG) + else() + set(library_parts_variable TF_LIB_PARTS_RELEASE) + endif() + set(${library_parts_variable}) + + # library might have been split because no more than 4GB are supported even on x64 Windows + foreach(PART_NO RANGE 1 100) + set(source "${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-bin/tensorflow/tensorflow_cc-part${PART_NO}.lib") + if(EXISTS "${source}") + file(COPY "${source}" DESTINATION "${CURRENT_PACKAGES_DIR}${DIR_PREFIX}/lib") + list(APPEND ${library_parts_variable} "tensorflow_cc-part${PART_NO}.lib") + else() + break() + endif() + endforeach() + endif() + else() + file(COPY + ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-bin/tensorflow/${TF_LIB_NAME_FULL} + DESTINATION ${CURRENT_PACKAGES_DIR}${DIR_PREFIX}/lib + ) + + # Note: these use relative links + file(CREATE_LINK ${TF_LIB_NAME_FULL} + ${CURRENT_PACKAGES_DIR}${DIR_PREFIX}/lib/${TF_LIB_NAME_SHORT} + SYMBOLIC + ) + file(CREATE_LINK ${TF_LIB_NAME_FULL} + ${CURRENT_PACKAGES_DIR}${DIR_PREFIX}/lib/${TF_LIB_NAME} + SYMBOLIC + ) + if(VCPKG_LIBRARY_LINKAGE STREQUAL dynamic) + file(COPY + ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-${BUILD_TYPE}/bazel-bin/tensorflow/${TF_FRAMEWORK_NAME_FULL} + DESTINATION ${CURRENT_PACKAGES_DIR}${DIR_PREFIX}/lib + ) + file(CREATE_LINK + ${TF_FRAMEWORK_NAME_FULL} + ${CURRENT_PACKAGES_DIR}${DIR_PREFIX}/lib/${TF_FRAMEWORK_NAME_SHORT} + SYMBOLIC + ) + file(CREATE_LINK + ${TF_FRAMEWORK_NAME_FULL} + ${CURRENT_PACKAGES_DIR}${DIR_PREFIX}/lib/${TF_FRAMEWORK_NAME} + SYMBOLIC + ) + endif() + endif() +endforeach() + +file(COPY + ${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/bazel-bin/tensorflow/include/ + DESTINATION ${CURRENT_PACKAGES_DIR}/include/tensorflow-external +) + +if(VCPKG_LIBRARY_LINKAGE STREQUAL static) + message(STATUS "Warning: Static TensorFlow build contains several external dependencies that may cause linking conflicts (for example, one cannot use both openssl and TensorFlow in the same project, since TensorFlow contains boringssl).") + if(VCPKG_TARGET_IS_WINDOWS) + message(STATUS "Note: For some TensorFlow features (e.g. OpRegistry), it might be necessary to tell the linker to include the whole library, i.e., link using options '/WHOLEARCHIVE:tensorflow_cc-part1.lib /WHOLEARCHIVE:tensorflow_cc-part2.lib ...'") + else() + message(STATUS "Note: There is no separate libtensorflow_framework.a as it got merged into libtensorflow_cc.a to avoid linking conflicts.") + if(VCPKG_TARGET_IS_OSX) + message(STATUS "Note: Beside TensorFlow itself, you'll need to also pass its dependancies to the linker, for example '-ltensorflow_cc -framework CoreFoundation'") + message(STATUS "Note: For some TensorFlow features (e.g. OpRegistry), it might be necessary to tell the linker to include the whole library: '-Wl,-force_load,path/to/libtensorflow_cc.a -framework CoreFoundation -framework Security [rest of linker arguments]'") + else() + message(STATUS "Note: Beside TensorFlow itself, you'll need to also pass its dependancies to the linker, for example '-ltensorflow_cc -lm -ldl -lpthread'") + message(STATUS "Note: For some TensorFlow features (e.g. OpRegistry), it might be necessary to tell the linker to include the whole library: '-Wl,--whole-archive -ltensorflow_cc -Wl,--no-whole-archive [rest of linker arguments]'") + endif() + endif() + + configure_file( + ${CMAKE_CURRENT_LIST_DIR}/README-${PLATFORM_SUFFIX} + ${CURRENT_PACKAGES_DIR}/share/tensorflow-cc/README + COPYONLY) endif() file(COPY ${SOURCE_PATH}/LICENSE DESTINATION ${CURRENT_PACKAGES_DIR}/share/tensorflow-cc) file(RENAME ${CURRENT_PACKAGES_DIR}/share/tensorflow-cc/LICENSE ${CURRENT_PACKAGES_DIR}/share/tensorflow-cc/copyright) -file(COPY ${CMAKE_CURRENT_LIST_DIR}/TensorflowCCConfig.cmake DESTINATION ${CURRENT_PACKAGES_DIR}/share/unofficial-tensorflow-cc) -file(RENAME ${CURRENT_PACKAGES_DIR}/share/unofficial-tensorflow-cc/TensorflowCCConfig.cmake ${CURRENT_PACKAGES_DIR}/share/unofficial-tensorflow-cc/unofficial-tensorflow-cc-config.cmake) + +# NOTE: if this port ever supports VCPKG_BUILD_TYPE, use that to set these. +set(TENSORFLOW_HAS_RELEASE ON) +set(TENSORFLOW_HAS_DEBUG ON) + +if(VCPKG_TARGET_IS_WINDOWS) + if(VCPKG_LIBRARY_LINKAGE STREQUAL dynamic) + configure_file( + ${CMAKE_CURRENT_LIST_DIR}/tensorflow-cc-config-windows-dll.cmake.in + ${CURRENT_PACKAGES_DIR}/share/tensorflow-cc/tensorflow-cc-config.cmake + @ONLY) + else() + configure_file( + ${CMAKE_CURRENT_LIST_DIR}/tensorflow-cc-config-windows-lib.cmake.in + ${CURRENT_PACKAGES_DIR}/share/tensorflow-cc/tensorflow-cc-config.cmake + @ONLY) + + set(VCPKG_POLICY_MISMATCHED_NUMBER_OF_BINARIES enabled) + + set(prefix [[${TENSORFLOW_INSTALL_PREFIX}]]) + + set(libs_to_link) + foreach(lib IN LISTS TF_LIB_PARTS_RELEASE) + list(APPEND libs_to_link "$<$<CONFIG:Release>:${prefix}/lib/${lib}>") + endforeach() + foreach(lib IN LISTS TF_LIB_PARTS_DEBUG) + list(APPEND libs_to_link "$<$<CONFIG:Debug>:${prefix}/debug/lib/${lib}>") + endforeach() + if(TENSORFLOW_HAS_RELEASE) + set(TF_LIB_PARTS_DEFAULT ${TF_LIB_PARTS_RELEASE}) + set(prefix_DEFAULT "${prefix}") + elseif(TENSORFLOW_HAS_DEBUG) + set(TF_LIB_PARTS_DEFAULT ${TF_LIB_PARTS_DEBUG}) + set(prefix_DEFAULT "${prefix}/debug") + endif() + + foreach(lib IN LISTS TF_LIB_PARTS_DEFAULT) + list(APPEND libs_to_link + "$<$<NOT:$<OR:$<CONFIG:Release>,$<CONFIG:Debug>>>:${prefix}/lib/${lib}>") + endforeach() + + string(REPLACE ";" "\n\t\t" libs_to_link "${libs_to_link}") + file(APPEND ${CURRENT_PACKAGES_DIR}/share/tensorflow-cc/tensorflow-cc-config.cmake " +target_link_libraries(tensorflow_cc::tensorflow_cc + INTERFACE + ${libs_to_link} +)" + ) + endif() +else() + if(VCPKG_LIBRARY_LINKAGE STREQUAL dynamic) + configure_file( + ${CMAKE_CURRENT_LIST_DIR}/tensorflow-cc-config-shared.cmake.in + ${CURRENT_PACKAGES_DIR}/share/tensorflow-cc/tensorflow-cc-config.cmake + @ONLY) + else() + configure_file( + ${CMAKE_CURRENT_LIST_DIR}/tensorflow-cc-config-static.cmake.in + ${CURRENT_PACKAGES_DIR}/share/tensorflow-cc/tensorflow-cc-config.cmake + @ONLY) + endif() +endif() + +message(STATUS "You may want to delete ${CURRENT_BUILDTREES_DIR} and ${BUILDTREES_DIR}/.bzl to free diskspace.") diff --git a/ports/tensorflow-cc/tensorflow-cc-config-shared.cmake.in b/ports/tensorflow-cc/tensorflow-cc-config-shared.cmake.in new file mode 100644 index 000000000..98149e000 --- /dev/null +++ b/ports/tensorflow-cc/tensorflow-cc-config-shared.cmake.in @@ -0,0 +1,88 @@ +if(TARGET tensorflow_cc::tensorflow_cc OR TARGET tensorflow_cc::tensorflow_framework) + if(NOT (TARGET tensorflow_cc::tensorflow_cc AND TARGET tensorflow_cc::tensorflow_framework)) + message(FATAL_ERROR "Some (but not all) targets in this config.cmake were already defined.") + endif() + + return() +endif() + +add_library(tensorflow_cc::tensorflow_cc SHARED IMPORTED GLOBAL) +add_library(tensorflow_cc::tensorflow_framework SHARED IMPORTED GLOBAL) + +# Compute the installation prefix from this tensorflow-cc-config.cmake file location. +# CMAKE_CURRENT_LIST_DIR = [vcpkg_installed_dir]/[target_triplet]/share/tensorflow-cc +get_filename_component(TENSORFLOW_INSTALL_PREFIX "${CMAKE_CURRENT_LIST_DIR}" DIRECTORY) +# TENSORFLOW_INSTALL_PREFIX = [vcpkg_installed_dir]/[target_triplet]/share +get_filename_component(TENSORFLOW_INSTALL_PREFIX "${TENSORFLOW_INSTALL_PREFIX}" DIRECTORY) +# TENSORFLOW_INSTALL_PREFIX = [vcpkg_installed_dir]/[target_triplet] + +target_include_directories(tensorflow_cc::tensorflow_cc + INTERFACE + "${TENSORFLOW_INSTALL_PREFIX}/include" + "${TENSORFLOW_INSTALL_PREFIX}/include/tensorflow-external" + "${TENSORFLOW_INSTALL_PREFIX}/include/tensorflow-external/src" +) +target_include_directories(tensorflow_cc::tensorflow_framework + INTERFACE + "${TENSORFLOW_INSTALL_PREFIX}/include/tensorflow-external" + "${TENSORFLOW_INSTALL_PREFIX}/include/tensorflow-external/src" +) + +if(@TENSORFLOW_HAS_RELEASE@) + set_property(TARGET tensorflow_cc::tensorflow_cc + PROPERTY IMPORTED_LOCATION + "${TENSORFLOW_INSTALL_PREFIX}/lib/@TF_LIB_NAME@" + ) + set_property(TARGET tensorflow_cc::tensorflow_framework + PROPERTY IMPORTED_LOCATION + "${TENSORFLOW_INSTALL_PREFIX}/lib/@TF_FRAMEWORK_NAME@" + ) + + set_property(TARGET tensorflow_cc::tensorflow_cc + APPEND PROPERTY IMPORTED_CONFIGURATIONS RELEASE + ) + set_property(TARGET tensorflow_cc::tensorflow_framework + APPEND PROPERTY IMPORTED_CONFIGURATIONS RELEASE + ) + + set_property(TARGET tensorflow_cc::tensorflow_cc + PROPERTY IMPORTED_LOCATION_RELEASE + "${TENSORFLOW_INSTALL_PREFIX}/lib/@TF_LIB_NAME@" + ) + set_property(TARGET tensorflow_cc::tensorflow_framework + PROPERTY IMPORTED_LOCATION_RELEASE + "${TENSORFLOW_INSTALL_PREFIX}/lib/@TF_FRAMEWORK_NAME@" + ) +endif() + +if(@TENSORFLOW_HAS_DEBUG) + if(NOT @TENSORFLOW_HAS_RELEASE) + set_property(TARGET tensorflow_cc::tensorflow_cc + PROPERTY IMPORTED_LOCATION + "${TENSORFLOW_INSTALL_PREFIX}/debug/lib/@TF_LIB_NAME@" + ) + set_property(TARGET tensorflow_cc::tensorflow_framework + PROPERTY IMPORTED_LOCATION + "${TENSORFLOW_INSTALL_PREFIX}/debug/lib/@TF_FRAMEWORK_NAME@" + ) + endif() + + set_property(TARGET tensorflow_cc::tensorflow_cc + APPEND PROPERTY IMPORTED_CONFIGURATIONS DEBUG + ) + set_property(TARGET tensorflow_cc::tensorflow_framework + APPEND PROPERTY IMPORTED_CONFIGURATIONS DEBUG + ) + + set_property(TARGET tensorflow_cc::tensorflow_cc + PROPERTY IMPORTED_LOCATION_DEBUG + "${TENSORFLOW_INSTALL_PREFIX}/debug/lib/@TF_LIB_NAME@" + ) + set_property(TARGET tensorflow_cc::tensorflow_framework + PROPERTY IMPORTED_LOCATION_DEBUG + "${TENSORFLOW_INSTALL_PREFIX}/debug/lib/@TF_FRAMEWORK_NAME@" + ) +endif() + +set(tensorflow_cc_FOUND TRUE) +set(tensorflow_framework_FOUND TRUE) diff --git a/ports/tensorflow-cc/tensorflow-cc-config-static.cmake.in b/ports/tensorflow-cc/tensorflow-cc-config-static.cmake.in new file mode 100644 index 000000000..7f0171470 --- /dev/null +++ b/ports/tensorflow-cc/tensorflow-cc-config-static.cmake.in @@ -0,0 +1,56 @@ +if(TARGET tensorflow_cc::tensorflow_cc) + return() +endif() + +add_library(tensorflow_cc::tensorflow_cc STATIC IMPORTED GLOBAL) + +# Compute the installation prefix from this tensorflow-cc-config.cmake file location. +# CMAKE_CURRENT_LIST_DIR = [vcpkg_installed_dir]/[target_triplet]/share/tensorflow-cc +get_filename_component(TENSORFLOW_INSTALL_PREFIX "${CMAKE_CURRENT_LIST_DIR}" DIRECTORY) +# TENSORFLOW_INSTALL_PREFIX = [vcpkg_installed_dir]/[target_triplet]/share +get_filename_component(TENSORFLOW_INSTALL_PREFIX "${TENSORFLOW_INSTALL_PREFIX}" DIRECTORY) +# TENSORFLOW_INSTALL_PREFIX = [vcpkg_installed_dir]/[target_triplet] + +target_include_directories(tensorflow_cc::tensorflow_cc + INTERFACE + "${TENSORFLOW_INSTALL_PREFIX}/include/tensorflow-external" + "${TENSORFLOW_INSTALL_PREFIX}/include/tensorflow-external/src" +) + +if(@TENSORFLOW_HAS_RELEASE@) + set_property(TARGET tensorflow_cc::tensorflow_cc + PROPERTY IMPORTED_LOCATION + "${TENSORFLOW_INSTALL_PREFIX}/lib/@TF_LIB_NAME@" + ) + + set_property(TARGET tensorflow_cc::tensorflow_cc + APPEND PROPERTY IMPORTED_CONFIGURATIONS + RELEASE + ) + + set_property(TARGET tensorflow_cc::tensorflow_cc + PROPERTY IMPORTED_LOCATION_RELEASE + "${TENSORFLOW_INSTALL_PREFIX}/lib/@TF_LIB_NAME@" + ) +endif() + +if(@TENSORFLOW_HAS_DEBUG@) + if(NOT @TENSORFLOW_HAS_RELEASE@) + set_property(TARGET tensorflow_cc::tensorflow_cc + PROPERTY IMPORTED_LOCATION + "${TENSORFLOW_INSTALL_PREFIX}/debug/lib/@TF_LIB_NAME@" + ) + endif() + + set_property(TARGET tensorflow_cc::tensorflow_cc + APPEND PROPERTY IMPORTED_CONFIGURATIONS + DEBUG + ) + + set_property(TARGET tensorflow_cc::tensorflow_cc + PROPERTY IMPORTED_LOCATION_DEBUG + "${TENSORFLOW_INSTALL_PREFIX}/debug/lib/@TF_LIB_NAME@" + ) +endif() + +set(tensorflow_cc_FOUND TRUE) diff --git a/ports/tensorflow-cc/tensorflow-cc-config-windows-dll.cmake.in b/ports/tensorflow-cc/tensorflow-cc-config-windows-dll.cmake.in new file mode 100644 index 000000000..ed67e5ad4 --- /dev/null +++ b/ports/tensorflow-cc/tensorflow-cc-config-windows-dll.cmake.in @@ -0,0 +1,56 @@ +if(TARGET tensorflow_cc::tensorflow_cc) + return() +endif() + +add_library(tensorflow_cc::tensorflow_cc SHARED IMPORTED GLOBAL) + +# Compute the installation prefix from this tensorflow-cc-config.cmake file location. +# CMAKE_CURRENT_LIST_DIR = [vcpkg_installed_dir]/[target_triplet]/share/tensorflow-cc +get_filename_component(TENSORFLOW_INSTALL_PREFIX "${CMAKE_CURRENT_LIST_DIR}" DIRECTORY) +# TENSORFLOW_INSTALL_PREFIX = [vcpkg_installed_dir]/[target_triplet]/share +get_filename_component(TENSORFLOW_INSTALL_PREFIX "${TENSORFLOW_INSTALL_PREFIX}" DIRECTORY) +# TENSORFLOW_INSTALL_PREFIX = [vcpkg_installed_dir]/[target_triplet] + +target_include_directories(tensorflow_cc::tensorflow_cc + INTERFACE + "${TENSORFLOW_INSTALL_PREFIX}/include/tensorflow-external" + "${TENSORFLOW_INSTALL_PREFIX}/include/tensorflow-external/src" +) + +if(@TENSORFLOW_HAS_RELEASE@) + set_target_properties(tensorflow_cc::tensorflow_cc + PROPERTIES + IMPORTED_LOCATION "${TENSORFLOW_INSTALL_PREFIX}/bin/tensorflow_cc.dll" + IMPORTED_IMPLIB "${TENSORFLOW_INSTALL_PREFIX}/lib/tensorflow_cc.lib" + ) + + set_property(TARGET tensorflow_cc::tensorflow_cc + APPEND PROPERTY IMPORTED_CONFIGURATIONS RELEASE + ) + set_target_properties(tensorflow_cc::tensorflow_cc + PROPERTIES + IMPORTED_LOCATION_RELEASE "${TENSORFLOW_INSTALL_PREFIX}/bin/tensorflow_cc.dll" + IMPORTED_IMPLIB_RELEASE "${TENSORFLOW_INSTALL_PREFIX}/lib/tensorflow_cc.lib" + ) +endif() + +if(@TENSORFLOW_HAS_DEBUG@) + if(NOT @TENSORFLOW_HAS_RELEASE@) + set_target_properties(tensorflow_cc::tensorflow_cc + PROPERTIES + IMPORTED_LOCATION "${TENSORFLOW_INSTALL_PREFIX}/debug/bin/tensorflow_cc.dll" + IMPORTED_IMPLIB "${TENSORFLOW_INSTALL_PREFIX}/debug/lib/tensorflow_cc.lib" + ) + endif() + + set_property(TARGET tensorflow_cc::tensorflow_cc + APPEND PROPERTY IMPORTED_CONFIGURATIONS DEBUG + ) + set_target_properties(tensorflow_cc::tensorflow_cc + PROPERTIES + IMPORTED_LOCATION_DEBUG "${TENSORFLOW_INSTALL_PREFIX}/debug/bin/tensorflow_cc.dll" + IMPORTED_IMPLIB_DEBUG "${TENSORFLOW_INSTALL_PREFIX}/debug/lib/tensorflow_cc.lib" + ) +endif() + +set(tensorflow_cc_FOUND TRUE) diff --git a/ports/tensorflow-cc/tensorflow-cc-config-windows-lib.cmake.in b/ports/tensorflow-cc/tensorflow-cc-config-windows-lib.cmake.in new file mode 100644 index 000000000..0eab84c7f --- /dev/null +++ b/ports/tensorflow-cc/tensorflow-cc-config-windows-lib.cmake.in @@ -0,0 +1,19 @@ +if(TARGET tensorflow_cc::tensorflow_cc) + return() +endif() + +set(tensorflow_cc_FOUND TRUE) + +# Compute the installation prefix from this tensorflow-cc-config.cmake file location. +# CMAKE_CURRENT_LIST_DIR = [vcpkg_installed_dir]/[target_triplet]/share/tensorflow-cc +get_filename_component(TENSORFLOW_INSTALL_PREFIX "${CMAKE_CURRENT_LIST_DIR}" DIRECTORY) +# TENSORFLOW_INSTALL_PREFIX = [vcpkg_installed_dir]/[target_triplet]/share +get_filename_component(TENSORFLOW_INSTALL_PREFIX "${TENSORFLOW_INSTALL_PREFIX}" DIRECTORY) +# TENSORFLOW_INSTALL_PREFIX = [vcpkg_installed_dir]/[target_triplet] + +add_library(tensorflow_cc::tensorflow_cc INTERFACE IMPORTED GLOBAL) +target_include_directories(tensorflow_cc::tensorflow_cc + INTERFACE + "${TENSORFLOW_INSTALL_PREFIX}/include/tensorflow-external" + "${TENSORFLOW_INSTALL_PREFIX}/include/tensorflow-external/src" +) |
