diff options
| author | nicole mazzuca <mazzucan@outlook.com> | 2019-08-02 09:52:39 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2019-08-02 09:52:39 -0700 |
| commit | 165907550c8f6ce7506beef591f55cd3f8458d78 (patch) | |
| tree | a400e42fdb0391544aa8a829d54678c6e5690638 /docs | |
| parent | bacbcf3527390ae7d20cd9660d82a180f2c8e065 (diff) | |
| download | vcpkg-165907550c8f6ce7506beef591f55cd3f8458d78.tar.gz vcpkg-165907550c8f6ce7506beef591f55cd3f8458d78.zip | |
Update tests, and add documentation! (#7506)
This PR does the following:
* fix tests -- now, they're always built in the CMake scripts, and they work on VS2015
*add a new flag, BUILD_TESTING, which allows one to turn off testing builds
* Add documentation for running tests
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/index.md | 5 | ||||
| -rw-r--r-- | docs/tool-maintainers/testing.md | 162 |
2 files changed, 167 insertions, 0 deletions
diff --git a/docs/index.md b/docs/index.md index 5d83b5804..b2f0a53b2 100644 --- a/docs/index.md +++ b/docs/index.md @@ -21,6 +21,11 @@ Vcpkg helps you manage C and C++ libraries on Windows, Linux and MacOS. This too - [Portfile functions](maintainers/portfile-functions.md) - [Maintainer Guidelines](maintainers/maintainer-guide.md) +### Tool Maintainer Help + +- [Testing](tool-maintainers/testing.md) +- [Maintainer Guidelines](maintainers/maintainer-guide.md) + ### Specifications - [Export](specifications/export-command.md) diff --git a/docs/tool-maintainers/testing.md b/docs/tool-maintainers/testing.md new file mode 100644 index 000000000..28cc9e099 --- /dev/null +++ b/docs/tool-maintainers/testing.md @@ -0,0 +1,162 @@ +Testing +======= + +Testing vcpkg is important whenever one makes changes to the tool itself, and +writing new tests and keeping them up to date is also very important. If one's +code is subtly broken, we'd rather find it out right away than a few weeks down +the line when someone complains! + +Running Tests +------------- + +Before anything else, we should know whether you can actually run the tests! +All you should need is a way to build vcpkg -- anything will do! All you have to +do is follow the guide 😄 + +With `$VCPKG_DIRECTORY` being the directory where you have cloned vcpkg, create +a build directory in `$VCPKG_DIRECTORY/toolsrc` (commonly named `out`), and +`cd` into it. Make sure to clean it out if it already exists! + +```sh +$ cmake .. -DCMAKE_BUILD_TYPE=Debug -G Ninja +$ cmake --build . +$ ./vcpkg-test # ./vcpkg-test [$SPECIFIC_TEST] for a specific set of tests +$ # i.e., ./vcpkg-test [arguments] +``` + +If you make any modifications to `vcpkg`, you'll have to do the +`cmake --build .` step again. + +Writing Tests +------------- + +In your journey to write new tests, and to modify existing tests, reading the +[Catch2 documentation] will be very helpful! Come back after reading those 😀 + +You'll want to place your tests in one of the existing files, or, if it doesn't +belong in any of those, in a [new file](#adding-new-test-files). + +The layout of these tests is as follows: + +```cpp +// ... includes + +TEST_CASE("Name of test", "[filename without the .cpp]") { + // setup and the like + REQUIRE(some boolean expression); +} + +// etc. +``` + +You want to give these test cases good, descriptive, unique names, like +`SourceParagraph construct minimum` -- it doesn't need to be extremely clear +english, and shorthand is good, but make sure it's clear what the test is from +the name. For the latter parameter, known as "tags", you should at least put the +name of the file which the test case is in -- e.g., in `arguments.cpp`, you'd +tag all of the test cases with `[arguments]`. + +If you wish to add helper functions, make sure to place them in an anonymous +namespace -- this will ensure that they don't trample over anybody else's +space. Additionally, there are a few helper functions that live in +`<vcpkg-test/util.h>` and `src/vcpkg-test/util.cpp` -- make sure to look into +them so that you're not rewriting functionality. + +That should be all you need to know to start writing your own tests! +Remember to check out the [Catch2 documentation] +if you'd like to get more advanced with your tests, +and good luck on your testing journey! + +Adding New Test Files +--------------------- + +Adding new test files should be easy and straightforward. All it requires is +creating a new source file in `toolsrc/src/vcpkg-test`, and then rerunning +`CMake` in order to pick up the glob changes. + +### Example + +Let's try writing a new test file called `example` (very creative, I know). + +First, we should create a file, `example.cpp`, in `toolsrc/src/vcpkg-test`: + +```cpp +// vcpkg-test/example.cpp +#include <vcpkg-test/catch.h> +``` + +This is the minimum file needed for tests; let's rebuild our CMake directory. +You'll have to clean out the existing `out` directory for CMake to rerun +globbing. + +```sh +$ cmake .. -DCMAKE_BUILD_TYPE=Debug -G Ninja +# ... +-- Build files have been written to: $VCPKG_DIRECTORY/toolsrc/out +$ cmake --build . +[80/80] Linking CXX executable vcpkg.exe +``` + +Okay, now let's make sure this worked; add a test case to `example.cpp`: + +```cpp +TEST_CASE("Example 1 - fail", "[example]") { + REQUIRE(false); +} +``` + +Now build the tests again, and run them: + +```sh +$ cmake --build . +[2/2] Linking CXX executable vcpkg-test.exe +$ ./vcpkg-test + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +vcpkg-test.exe is a Catch v2.9.1 host application. +Run with -? for options + +------------------------------------------------------------------------------- +Example 1 - fail +------------------------------------------------------------------------------- +$VCPKG_DIRECTORY/toolsrc/src/vcpkg-test/example.cpp(3) +............................................................................... + +$VCPKG_DIRECTORY/toolsrc/src/vcpkg-test/example.cpp(14): FAILED: + REQUIRE( false ) + +=============================================================================== +test cases: 102 | 101 passed | 1 failed +assertions: 3611 | 3610 passed | 1 failed +``` + +Hopefully, that worked! It should compile correctly, and have one failing test. +Now let's try a more complex test, after deleting the old one; + +```cpp +// add #include <vcpkg/base/strings.h> to the top of the file +namespace Strings = vcpkg::Strings; + +TEST_CASE("Example 2 - success", "[example]") { + std::string hello = "Hello"; + REQUIRE(Strings::case_insensitive_ascii_equals(hello, "hELLo")); + REQUIRE_FALSE(Strings::case_insensitive_ascii_starts_with(hello, "E")); +} +``` + +Now compile and build the tests, and this time let's only run our example tests: + +```sh +$ cmake --build . +[2/2] Linking CXX executable vcpkg-test.exe +$ ./vcpkg-test [example] +Filters: [example] +=============================================================================== +All tests passed (2 assertions in 1 test case) +``` + +Hopefully you have one test running and succeeding! If you have that, you have +succeeded at adding a new file to vcpkg's tests. Congratulations! Have fun on +the rest of your journey 🐱👤😁 + +[Catch2 documentation]: https://github.com/catchorg/Catch2/blob/master/docs/tutorial.md#top |
