QAbstractTestLogger Class
Base class for test loggers. More...
Header: | #include <QAbstractTestLogger> |
Inherited By: | QAppleTestLogger, QCsvBenchmarkLogger, QJUnitTestLogger, QPlainTestLogger, QTapTestLogger, QTeamCityLogger, and QXmlTestLogger |
Public Types
enum | IncidentTypes { Pass, XFail, Fail, XPass, Skip, …, BlacklistedXPass } |
enum | MessageTypes { QInfo, QWarning, QDebug, QCritical, QFatal, …, Warn } |
Public Functions
QAbstractTestLogger(const char *filename) | |
virtual | ~QAbstractTestLogger() |
virtual void | addMessage(QtMsgType type, const QMessageLogContext &context, const QString &message) |
bool | isLoggingToStdout() const |
void | outputString(const char *msg) |
virtual void | startLogging() |
virtual void | stopLogging() |
Protected Functions
void | filterUnprintable(char *str) const |
Detailed Description
\internal
\inmodule
QtTest
Implementations of logging for QtTest should implement all pure virtual methods of this class and may implement the other virtual methods. This class's documentation of each virtual method sets out how those implementations are invoked by the QtTest code and offers guidance on how the logging class should use the data. Actual implementations may have different requirements - such as a file format with a defined schema, or a target audience to serve - that affect how it interprets that guidance.
Member Type Documentation
enum QAbstractTestLogger::IncidentTypes
Constant | Value | Description |
---|---|---|
QAbstractTestLogger::Pass | 1 | The test ran to completion successfully. |
QAbstractTestLogger::XFail | 2 | The test failed a check that is known to fail; this failure does not prevent successful completion of the test and may be followed by further checks. |
QAbstractTestLogger::Fail | 3 | The test fails. |
QAbstractTestLogger::XPass | 4 | A check which was expected to fail actually passed. This is counted as a failure, as it means whatever caused the known failure no longer does, so the test needs an update. |
QAbstractTestLogger::Skip | 0 | The current test ended prematurely, skipping some checks. |
QAbstractTestLogger::BlacklistedPass | 5 | As Pass but the test was blacklisted. |
QAbstractTestLogger::BlacklistedXFail | 8 | As XFail but the test was blacklisted. |
QAbstractTestLogger::BlacklistedFail | 6 | As Fail but the test was blacklisted. |
QAbstractTestLogger::BlacklistedXPass | 7 | As XPass but the test was blacklisted. |
A test may also skip (see MessageTypes). The first of skip, Fail, XPass or the blacklisted equivalents of the last two to arise is decisive for the outcome of the test: loggers which should only report one outcome should thus record that as the outcome and either ignore later incidents (or skips) in the same run of the test function or map them to some form of message.
Note: tests can be "blacklisted" when they are known to fail unreliably. When testing is used to decide whether a change to the code under test is acceptable, such failures are not automatic grounds for rejecting the change, if the unreliable failure was known before the change. QTest::qExec(), as a result, only returns a failing status code if some non-blacklisted test failed. Logging backends may reasonably report a blacklisted result just as they would report the non-blacklisted equivalent, optionally with some annotation to indicate that the result should not be taken as damning evidence against recent changes to the code under test.
See also QAbstractTestLogger::addIncident().
enum QAbstractTestLogger::MessageTypes
The members whose names begin with Q
describe messages that originate in calls, by the test or code under test, to Qt logging functions (implemented as macros) whose names are similar, with a q
in place of the leading Q
. The other members describe messages generated internally by QtTest.
Constant | Value | Description |
---|---|---|
QAbstractTestLogger::QInfo | 1 | An informational message from qInfo(). |
QAbstractTestLogger::QWarning | 2 | A warning from qWarning(). |
QAbstractTestLogger::QDebug | 0 | A debug message from qDebug(). |
QAbstractTestLogger::QCritical | 3 | A critical error from qCritical(). |
QAbstractTestLogger::QFatal | 4 | A fatal error from qFatal(), or an unrecognised message from the Qt logging functions. |
QAbstractTestLogger::Info | 5 | Messages QtTest generates as requested by the -v1 or -v2 command-line option being specified when running the test. |
QAbstractTestLogger::Warn | 6 | A warning generated internally by QtTest |
Note: For these purposes, some utilities provided by QtTestlib as helper functions to facilitate testing - such as QSignalSpy, QTestAccessibility, QTest::qExtractTestData(), and the facilities to deliver artificial mouse and keyboard events - are treated as test code, rather than internal to QtTest; they call qWarning() and friends rather than using the internal logging infrastructure, so that QTest::ignoreMessage() can be used to anticipate the messages.
See also QAbstractTestLogger::addMessage().
Member Function Documentation
QAbstractTestLogger::QAbstractTestLogger(const char *filename)
Constructs the base-class parts of the logger.
Derived classes should pass this base-constructor the filename of the file to which they shall log test results, or nullptr
to write to standard output. The protected member stream
is set to the open file descriptor.
[virtual noexcept]
QAbstractTestLogger::~QAbstractTestLogger()
Destroys the logger object.
If the protected stream
is not standard output, it is closed. In any case it is cleared.
[virtual]
void QAbstractTestLogger::addMessage(QtMsgType type, const QMessageLogContext &context, const QString &message)
This is an overloaded function.
This virtual method is called from the custom message handler QtTest installs in place of Qt's default message handler for the duration of testing, unless QTest::ignoreMessage() was used to ignore it, or too many messages have previously been processed. (The limiting number of messages is controlled by the -maxwarnings option to a test and defaults to 2002.)
Logging implementations should not normally need to override this method. The base implementation converts type to the matching MessageType, formats the given message suitably for the specified context, and forwards the converted type and formatted message to the overload that takes MessageType and QString.
See also QTest::ignoreMessage() and addIncident().
[protected]
void QAbstractTestLogger::filterUnprintable(char *str) const
Helper utility to blot out unprintable characters in str.
Takes a '\0'
-terminated mutable string and changes any characters of it that are not suitable for printing to '?'
characters.
bool QAbstractTestLogger::isLoggingToStdout() const
Returns true if the output
stream is standard output.
void QAbstractTestLogger::outputString(const char *msg)
Convenience method to write msg to the output stream.
The output msg must be a '\0'
-terminated string (and not nullptr
). A copy of it is passed to filterUnprintable() and the result written to the output stream
, which is then flushed.
[virtual]
void QAbstractTestLogger::startLogging()
Called before the start of a test run.
This virtual method is called before the first tests are run. A logging implementation might open a file, write some preamble, or prepare in other ways, such as setting up initial values of variables. It can use the usual Qt logging infrastucture, since it is also called before QtTest installs its own custom message handler.
See also stopLogging().
[virtual]
void QAbstractTestLogger::stopLogging()
Called after the end of a test run.
This virtual method is called after all tests have run. A logging implementation might collate information gathered from the run, write a summary, or close a file. It can use the usual Qt logging infrastucture, since it is also called after QtTest has restored the default message handler it replaced with its own custom message handler.
See also startLogging().