205 lines
11 KiB
Markdown
205 lines
11 KiB
Markdown
# QtTest regression tests
|
||
|
||
For instrumented builds, LCOV HTML reports, and Qt Creator coverage configuration,
|
||
see [Measuring code coverage](../README.md#measuring-code-coverage).
|
||
For the complete IDE build/run configuration set, see
|
||
[Configure all build variants in Qt Creator](../README.md#configure-all-build-variants-in-qt-creator).
|
||
The Coverage configuration uses `coverage_report.sh` after its instrumented
|
||
build; the script takes the source and build directories as its two arguments.
|
||
|
||
Open the repository's root `dbc.pro` in Qt Creator. It contains `core` (a static
|
||
C++17 library) and `tests` (the `dbc_tests` QtTest executable). Core is built first;
|
||
tests link its archive rather than compiling its implementation again.
|
||
Qt Core and Qt Test development packages are required for tests only.
|
||
The optional model tests also link QtGui, which is provided by Qt base development
|
||
packages. The adapter library itself still links only QtCore.
|
||
|
||
Build from the repository root with the installed Qt 6.8.2:
|
||
|
||
```sh
|
||
mkdir build
|
||
cd build
|
||
qmake6 ../dbc.pro
|
||
make -j
|
||
make check
|
||
```
|
||
|
||
Or run `./tests/dbc_tests` directly. Parser, decoder, pipeline, lifetime, validation, and additional coverage
|
||
suites run, and the executable returns nonzero when any suite fails. Decoder cases appear as
|
||
individual QtTest data rows; parser checks are grouped into named test slots.
|
||
Checks also run in release builds.
|
||
|
||
To build tests separately against an existing core build:
|
||
|
||
```sh
|
||
mkdir /tmp/dbc-tests-build
|
||
cd /tmp/dbc-tests-build
|
||
qmake6 /absolute/path/to/dbc/tests/tests.pro \
|
||
DBC_CORE_BUILD_DIR=/absolute/path/to/build/core
|
||
make
|
||
make check
|
||
```
|
||
|
||
`parser_tests.pri` and `decoder_tests.pri` include the original test sources;
|
||
`tests.pro` also includes `dbc_coverage_test.cpp`, `dbc_lifetime_test.cpp`,
|
||
`dbc_validation_test.cpp`, and `dbc_pipeline_test.cpp`.
|
||
`../core/core.pri` supplies the library include path and linkage.
|
||
|
||
## Decoder coverage
|
||
|
||
Coverage includes Motorola bit numbering within and across bytes, unaligned
|
||
signals, signed values, scaling, 64-bit extraction, signed/unsigned boundary values
|
||
(including `INT64_MIN`, `INT64_MAX`, and `UINT64_MAX`), numeric output, invalid lengths, insufficient
|
||
payloads, and Intel regression cases. Frame validity means the definition was
|
||
found; frame status reports unknown frames, short payloads, and signal errors.
|
||
Tests exercise empty/short/exact/long payloads, ignored trailing bytes, signals
|
||
outside declared DLC, invalid enum metadata and overflowing start bits rejected
|
||
at construction, and preservation of successful signals on partial payloads.
|
||
Physical conversion checks cover NaN/Inf parameters rejected at construction,
|
||
multiplication/addition overflow, finite limits, signed/unsigned raw retention,
|
||
and raw selector activation after physical conversion failure.
|
||
|
||
## Parser coverage
|
||
|
||
Structured diagnostic tests cover severity, code, supplied path, and physical
|
||
line numbers for skipped syntax and immediate/deferred errors; warnings preceding
|
||
an error are retained, and failed parses return an empty database. Common headers
|
||
and namespace symbol lists stay quiet. Tests also check file-open errors, fresh
|
||
parse sessions, and compatibility with the legacy throwing API.
|
||
The diagnostic audit also covers 15 unsupported-statement variants, all namespace
|
||
termination forms, CRLF and missing final newlines, relative paths with spaces,
|
||
first-error stopping, deferred diagnostic ordering, duplicate definitions/values,
|
||
selector ranges, overlaps, and preservation of supported decoding and metadata.
|
||
On Linux, reading a directory exercises the distinct `FileReadError` path.
|
||
|
||
Parser tests cover standard and extended CAN IDs, whitespace, payload storage
|
||
boundaries, missing fields, malformed and overflowing numbers, reserved ID bits,
|
||
error paths and line numbers, and integration with signals and comments.
|
||
Comment regressions cover tabs, escaped quotes/backslashes, multiline text with
|
||
blank lines and indentation, CRLF, missing final newlines, comments preceding
|
||
frame/signal definitions, last-wins and empty overwrites, normalized ID boundaries,
|
||
standard/extended separation, malformed syntax/IDs, and missing targets. Errors
|
||
verify both structured diagnostics and the throwing API at the starting line.
|
||
Duplicate definitions are rejected at their source lines, including equivalent
|
||
extended ID encodings and repeated signal names within a frame. Standard and
|
||
extended frames may share a numeric ID, and separate frames may reuse signal names.
|
||
Signal tests cover complete syntax and numeric validation, scientific notation,
|
||
finite physical parameters and ordered ranges, spaces and tabs, receiver lists,
|
||
unsupported multiplexing, and Intel/Motorola declared-payload boundaries.
|
||
Physical-number tests apply a custom comma-decimal C++ locale and an available
|
||
named comma-decimal C numeric locale. They check all four physical fields,
|
||
leading signs, fractions, scientific notation, finite boundaries, zero, malformed
|
||
numbers, comma/grouped numbers, overflow/underflow, and reversed ranges. Locale
|
||
changes are restored after each test. The named C locale case skips if none of
|
||
the supported locale names are installed; the custom C++ locale case always runs.
|
||
On Linux with locale sources and `localedef`, run both cases without installing a
|
||
system locale:
|
||
|
||
```sh
|
||
mkdir -p /tmp/dbc-test-locales
|
||
localedef --no-archive -i de_DE -f UTF-8 /tmp/dbc-test-locales/de_DE.UTF-8
|
||
LOCPATH=/tmp/dbc-test-locales make check
|
||
```
|
||
|
||
Run `make check` from the build directory as usual.
|
||
Value-description tests cover full `VAL_` syntax, whitespace, escaped quotes,
|
||
empty text, exact integer boundaries, duplicate keys within and across statements,
|
||
deferred and normalized frame references, missing targets, error locations,
|
||
raw lookup with scaling, unknown/invalid values, and both database builders.
|
||
Temporary DBC fixtures are created in the system temporary directory and removed
|
||
automatically.
|
||
|
||
Dispatch tests also verify exact keyword matching, handler construction and reuse,
|
||
unknown statements, and independent parsing sessions, including recovery after errors.
|
||
|
||
## Tree and optional model tests
|
||
|
||
`tree_node_test.cpp` is a standalone C++17 executable without Qt; see the root
|
||
README for its compiler command. It checks parents, sibling positions, null
|
||
children, bounds, and unique_ptr transfer.
|
||
|
||
Enable `CONFIG+=dbc_qt` in the root qmake invocation to build `dbc_qt` and
|
||
`dbc_model_tests` alongside existing tests. Qt 6.8.2 is required.
|
||
`make check` runs all suites. Model tests parse a temporary DBC fixture and use
|
||
QAbstractItemModelTester, verify navigation across columns, invalid/foreign
|
||
indexes, metadata and exact VAL_ integers, and persistent-index invalidation
|
||
when reloading or clearing the tree. Their QGuiApplication registers graphical
|
||
metatypes used by QAbstractItemModelTester, avoiding the invalid QPixmap type
|
||
warning in Qt 6.8.2. The test executable defaults to `QT_QPA_PLATFORM=offscreen`
|
||
when the variable is unset or empty, so no display server is needed. An explicit
|
||
platform setting is preserved.
|
||
|
||
Basic multiplexing coverage includes selector declarations after branches,
|
||
Intel/Motorola branch overlap, signed active values and VAL_ lookup, unconditional
|
||
signals, inactive branches, missing selector/branch payloads, exact UINT64_MAX
|
||
selector matching, malformed runtime selectors, source locations, and rejection
|
||
of invalid markers, selector references/ranges, overlaps, and SG_MUL_VAL_.
|
||
|
||
## Additional regression coverage
|
||
|
||
`dbc_coverage_test.cpp` runs as part of `dbc_tests`. It covers signed boundaries
|
||
below 64 bits, deterministic extraction comparisons for both byte orders across
|
||
lengths 1–64, frame lookup failures, exact decode statuses, construction rejection
|
||
of malformed runtime multiplexing, PGN boundaries, comment handling, file
|
||
formatting, metadata copying
|
||
and source lifetime, physical conversion, and TypeFactory registration/creation.
|
||
Direct resolver tests construct ambiguous databases through the public metadata
|
||
API, verify source locations, and check that ambiguous targets receive no values.
|
||
|
||
All QtTest targets use Qt 6.8.2 and `QVERIFY_THROWS_EXCEPTION`. Exception tests
|
||
explicitly discard `nodiscard` results when only the exception is being checked.
|
||
The standalone tree test requires only a C++17 compiler.
|
||
|
||
## Definition lifetime and reload coverage
|
||
|
||
`dbc_lifetime_test.cpp` checks retained definitions and value-description strings
|
||
across reload, clear, and destruction; independently saved signal results;
|
||
copied/moved results and database copies; owning lookup handles; and release of
|
||
the old snapshot after its final owner is removed. Compile-time checks enforce
|
||
const access to runtime metadata. Reload fixtures change ordering, frame names,
|
||
scaling, units, and descriptions, remove definitions, and rename signals.
|
||
|
||
Key tests check equality and hashing, standard/extended separation, the same
|
||
signal name in different frames, and missing lookups. Constructor tests reject
|
||
duplicate runtime keys, including manually supplied parsed metadata. Failed reload
|
||
leaves the active database intact; empty reload clears it. Invalid/unknown results
|
||
and insufficient payloads retain their documented ownership and status behavior.
|
||
Inactive branches and unavailable selectors also retain definitions across reload.
|
||
|
||
## Database validation coverage
|
||
|
||
`dbc_validation_test.cpp` checks direct construction and manually assembled
|
||
`DbcDatabase` input to `DbcDecodeBuilder`: standard/extended CAN ID limits,
|
||
encoded/reserved ID bits, duplicate keys, lengths 0/1–64/65/UINT32_MAX,
|
||
Intel/Motorola payload boundaries and overflowing start bits, invalid enums,
|
||
NaN and both infinities in every numeric parameter, and reversed physical ranges.
|
||
Selector tests include missing, duplicate, and signed selectors, branch ranges,
|
||
UINT64_MAX selection, mixed-order overlaps, and exclusive branches sharing bits.
|
||
Valid definitions cover the full byte-count storage range, including zero-byte
|
||
frames without signals and 255-byte frames. Existing extraction tests cover
|
||
lengths 1–64. Snapshot tests mutate retained frame, signal, and description aliases
|
||
after rvalue construction, and verify unchanged lookups and metadata. Failed
|
||
constructor/builder replacement preserves the active snapshot; clear removes its
|
||
index while saved handles remain alive.
|
||
|
||
Layout validation is also compared with independent DBC bit traversal for every
|
||
start position through one byte beyond payload sizes 0, 1, 8, 64, and 255, and
|
||
all lengths 1–64, for both byte orders. Zero/65-bit lengths and uint32 maximum
|
||
inputs are rejected. Builder snapshots survive input mutation and destruction;
|
||
failed replacement with duplicate frame or signal keys preserves the existing
|
||
frame and signal lookups.
|
||
|
||
## End-to-end pipeline coverage
|
||
|
||
`dbc_pipeline_test.cpp` parses temporary DBC files and checks independent expected
|
||
metadata against the parser, runtime builder, and tree builder. It exercises
|
||
standard and extended frames with the same numeric ID and signal name, Intel
|
||
and signed Motorola decoding, scaling, raw value descriptions, basic multiplexing,
|
||
PGN metadata, receivers, units, ranges, and frame/signal comments. Tree checks
|
||
include node types, payloads, parents, rows, names, and child counts.
|
||
|
||
Empty and headers-only fixtures produce an empty runtime database and a valid
|
||
empty tree root. Replacement fixtures change definitions, metadata and decoded
|
||
values, remove old lookup keys, and then clear both outputs through an empty
|
||
fixture. Saved decoded results and trees retain their original metadata.
|