Name: gtest-mpi-listener
Owner: Lawrence Livermore National Laboratory
Description: Header-only plugin for the Google Test framework defining listener(s) emitting sensible output when testing MPI-based, distributed-memory parallel software.
Created: 2018-03-04 08:01:11.0
Updated: 2018-03-26 20:35:54.0
Pushed: 2018-03-04 08:03:49.0
Homepage: null
Size: 17
Language: C++
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
The goals of this project are:
to augment the googletest framework with listener(s) that yield sensible output for testing MPI-based, distributed-memory parallel software
to keep this software as low-maintenance as possible
ideally, to merge some version of this functionality into Google Test
This software is BSD-3 licensed, and uses some example code from Google Test, which has the following license that must be duplicated in its entirety, per its terms:
yright 2005, Google Inc. All rights reserved.
istribution and use in source and binary forms, with or without
ification, are permitted provided that the following conditions are
:
* Redistributions of source code must retain the above copyright
ice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
yright notice, this list of conditions and the following disclaimer
the documentation and/or other materials provided with the
tribution.
* Neither the name of Google Inc. nor the names of its
tributors may be used to endorse or promote products derived from
s software without specific prior written permission.
S SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
ITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
ARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
ER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
CIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
ITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
A, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
ORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
CLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
1) In the root directory of this repository, clone the googletest GitHub repo:
git clone https://github.com/google/googletest.git
2) Change to the new googletest directory created via cloning:
pushd googletest
3) Within this directory, make a build directory; this step is
standard for CMake-based buildsystems:
mkdir build
4) Change to this build directory:
cd build
5) Run CMake to generate a Makefile that will build googletest:
cmake ..
You should see some CMake output indicating compiler detection, etc.
6) Assuming step 5 completed successfully, call make to build the
googletest library:
make
7) Return to the root directory of the repository:
popd (if you followed this directions)
8) Build the example test runner:
make
9) Assuming the test runner build proceeds successfully, run the test runner example:
bin/example.exe
Please read the Google Test Primer and Google Test Advanced Guide for background before reading the remainder of this section.
Usage of this listener is illustrated by the example in
src/example.cpp, which includes some very simple-minded MPI unit
tests that I've used for basic testing. To use this listener, you will
need to write your own int main(int argc, char** argv) function –
you cannot use the stock gtest_main supplied with Google Test. Using
a custom main function enables us to:
Environment object that will finalize MPI when main
terminatesTestEventListener, which emits output on testsTestEventListener that will emit sensible output (that is,
rank-ordered output to stdout) for MPI-based unit testsall of which is necessary, and not possible using the stock gtest_main function.
Directions for writing tests can be found in Google Test Primer and Google Test Advanced Guide. My design assumption is that these tests will be executed in distributed fashion using the MPI distributed-memory parallel programming model (that is, a shared-nothing, process-local memory address space). This assumption implies that tests will be run by all MPI processes, or disabled on all processes. Conditionals can be used to execute parts (or all) of the body of a test on a given MPI process or set of processes.
After writing tests, your int main(int argc, char** argv) function should look like the example test runner:
main(int argc, char** argv) {
ilter out Google Test arguments
sting::InitGoogleTest(&argc, argv);
nitialize MPI
Init(&argc, &argv);
dd object that will finalize MPI on exit; Google Test owns this pointer
sting::AddGlobalTestEnvironment(new MPIEnvironment);
et the event listener list.
sting::TestEventListeners& listeners =
sting::UnitTest::GetInstance()->listeners();
emove default listener
te listeners.Release(listeners.default_result_printer());
dds MPI listener; Google Test owns this pointer
eners.Append(new MPIMinimalistPrinter);
un tests, then clean up and exit
ALL_TESTS();
rn 0;
Comments in this example describe what each line does. The ordering is also important, and should be preserved if you augment this example with additional code.
The most important design consideration was to write something portable, quickly. This meant using MPI-1.x calls (because supercomputers sometimes run older MPI implementations), and relying on the public API calls of a portable unit testing library that is currently being maintained. As a result of these decisions:
this software communicates unit test results to MPI process 0 and then writes them to standard out (because MPI I/O is defined in MPI-2)
consequently, this software will have an output bottleneck at extreme scale, because the output is, to borrow from Bill Gropp's MPI I/O slides (see slide 3), “worse than sequential”
this software doesn't have the awesome, fancy, color-coded output
that stock Google Test uses, because all of those functions are
implemented as static in googletest/googletest/src/gtest.cc, and
thus are inaccessible from outside that translation unit because
they are not part of the Google Test public API
this software is header-only, to make vendoring it into software packages no more painful than vendoring in Google Test
The current setup should be usable for small numbers of MPI processes
– I've tested it on 256 MPI processes and it seems to work fine. If
there is a need to write infrastructure for testing on 100,000 MPI
processes, then an MPI I/O-based TestEventListener makes more sense,
and would relax the output bottleneck. This new code should be written
in a second header to isolate MPI-2-conforming code from
MPI-1-conforming code. Although I haven't done it yet, I suspect this
modification could be done in days.
Adding fancy, color-coded output is also possible, but it means reimplementing private functions from Google Test that could change at any moment. I'd rather not reimplement that myself, because it will also mean that this software can no longer be header-only due to the way Google Test implements terminal color-coding.
If you want to hack on this code, the most important things to note are:
::testing::TestEventListener::OnTestPartResult is only called
after assertion failures or invocation of the SUCCESS() macro, so
it may not be called on some MPI processes, and there should be no
communication in this function
In TestEventListener::OnTestEnd, although ::testing::TestInfo
has methods to query ::testing::TestPartResult objects within it,
when I called those methods, the information I wanted (namely,
::testing::TestPartResult objects) was no longer present, which is
why I instead store that information within a class member instead
As stated above, I aim to keep this software low-maintenance, because this package was written in a couple days in order to make the parallel software development for my work easier. I'm happy to accept bug-fixes, and I'm willing to discuss or consider feature requests/additions, but I'd like any feature to be easy to maintain. I do not get paid to write testing frameworks, which is why any responses to bug-fixes, feature requests, documentation improvements, or questions may be delayed; it is also why I want this software to be low-maintenance. In general, even though I intend to respond to correspondence about this software (I would like it to improve), users should assume I may not get back to them in a timely fashion, or at all.