QTapTestLogger Class
Header: | #include <QTapTestLogger> |
Inherits: | QAbstractTestLogger |
Detailed Description
\internal
\inmodule
QtTest
QTapTestLogger implements the Test Anything Protocol v13.
The Test Anything Protocol (TAP) is a simple plain-text interface between testing code and systems for reporting and analyzing test results. Since QtTest doesn't build the table for a data-driven test until the test is about to be run, we don't typically know how many tests we'll run until we've run them, so we put The Plan at the end, rather than the beginning. We summarise the results in comments following The Plan.
YAMLish
The TAP protocol supports inclusion, after a Test Line, of a "diagnostic block" in YAML, containing supporting information. We use this to package other information from the test, in combination with comments to record information we're unable to deliver at the same time as a test line. By convention, TAP producers limit themselves to a restricted subset of YAML, known as YAMLish, for maximal compatibility with TAP consumers.
YAML (see yaml.org
for details) supports three data-types: mapping (hash or dictionary), sequence (list, array or set) and scalar (string or number), much as JSON does. It uses indentation to indicate levels of nesting of mappings and sequences within one another. A line starting with a dash (at the same level of indent as its context) introduces a list entry. A line starting with a key (more indented than its context, aside from any earlier keys at its level) followed by a colon introduces a mapping entry; the key may be either a simple word or a quoted string (within which numeric escapes may be used to indicate unicode characters). The value associated with a given key, or the entry in a list, can appar after the key-colon or dasy either on the same line or on a succession of subsequent lines at higher indent. Thus
- first list item - second: list item is a mapping with: - two keys - the second of which is a list - back in the outer list, a third item
In YAMLish, the top-level structure should be a hash. The keys supported for this hash, with the meanings for their values, are:
message
: free text supplying supporting detailsseverity
: how bad is it ?source
: source describing the test, as an URL (compare file, line)at
: identifies the function (with file and line) performing the testdatetime
: when the test was run (ISO 8601 or HTTP format)file
: source of the test as a local file-name, when appropriateline
: line number within the sourcename
: test nameextensions
: sub-hash in which to store random other stuffactual
: what happened (a.k.a. found; contrast expected)expected
: what was expected (a.k.a. wanted; contrast actual)display
: description of the result, suitable for displaydump
: a sub-hash of variable values when the result aroseerror
: describes the errorbacktrace
: describes the call-stack of the error
In practice, only at
, expected
and actual
appear to be generally supported.
We can also have several messages produced within a single test, so the simple message
/ severity
pair of top-level keys does not serve us well. We therefore use extensions
with a sub-tag messages
in which to package a list of messages.
See also QAbstractTestLogger.