Additional tests

This commit is contained in:
deeaitch committed 2026-10-07 19:52:28 -04:00
1 parent 344beaf58c
commit 1d56a37090
3 files changed
+292

No files matched your search

+165
View File
@@ -1188,3 +1188,168 @@ signal syntax and payload bounds, `VAL_` syntax and reference validation,
raw description lookup and preservation through both builders, statement dispatch, signed and unsigned
64-bit boundaries, physical conversion, numeric output, and invalid signal
payloads.
## Measuring code coverage
Coverage instrumentation is separate from the QtTest suite named `CoverageTest`:
that suite runs extra regression checks, but coverage counters require a specially
instrumented build. Use a separate build directory for each coverage tool.
### GCC, gcov, and LCOV on Debian/Ubuntu
Install the build and report tools:
```sh
sudo apt install build-essential qt6-base-dev qt6-base-dev-tools qmake6 lcov
# Optional IDE:
sudo apt install qtcreator
```
GCC supplies `gcov`; the `lcov` package supplies `lcov` and `genhtml`. Use the
`gcov` version matching the GCC used by the selected Qt Creator kit. These
instructions use GCC and Qt 6.
From the repository root, create a fresh coverage build. `--coverage` must be
passed to both compilation and linking; Debug avoids optimization that can make
source-level results harder to interpret. `-fprofile-abs-path` makes source paths
in the coverage data independent of the compilation working directory.
```sh
DBC_SOURCE_DIR="$(pwd)"
mkdir -p build-coverage
cd build-coverage
qmake6 "$DBC_SOURCE_DIR/dbc.pro" \
CONFIG+=dbc_qt CONFIG+=debug CONFIG-=release \
'QMAKE_CXXFLAGS+=--coverage -fprofile-abs-path' \
'QMAKE_LFLAGS+=--coverage'
make -j4
# The standalone tree test is not a qmake target; instrument it separately.
mkdir -p standalone
c++ -std=c++17 -O0 -g --coverage -fprofile-abs-path \
-I"$DBC_SOURCE_DIR/core" \
"$DBC_SOURCE_DIR/tests/tree_node_test.cpp" \
"$DBC_SOURCE_DIR/core/tree_node.cpp" \
"$DBC_SOURCE_DIR/core/dbc_tree_builder.cpp" \
-o standalone/tree_node_tests
# Capture a zero-count baseline so unexecuted compiled code remains in the report.
lcov --capture --initial --directory . --branch-coverage \
--include "$DBC_SOURCE_DIR/core/*" --include "$DBC_SOURCE_DIR/qt/*" \
--output-file baseline.info
# Reset counts before each new measurement; otherwise executions accumulate.
lcov --zerocounters --directory .
make check
./standalone/tree_node_tests
lcov --capture --directory . --branch-coverage \
--include "$DBC_SOURCE_DIR/core/*" --include "$DBC_SOURCE_DIR/qt/*" \
--output-file executed.info
lcov --add-tracefile baseline.info --add-tracefile executed.info \
--branch-coverage --output-file coverage.info
genhtml coverage.info --branch-coverage --demangle-cpp --output-directory html
lcov --summary coverage.info --branch-coverage
```
Open `build-coverage/html/index.html` in a browser. The report includes the
library's `core/` and `qt/` sources and instantiated header code; test sources,
generated moc files, and Qt/STL headers are excluded. Click a filename to inspect
execution counts, uncovered lines, and branch outcomes. Branch coverage is a
separate metric: executing a line does not necessarily exercise every outcome.
C++ branch data can also include compiler-generated exception paths.
For another run with unchanged sources, repeat the reset, test execution,
capture, merge, and HTML generation steps. After changing sources or compiler
flags, rebuild and regenerate `baseline.info` before measuring again. A fresh
build directory avoids mixing old profiling data with a different build.
An initial measurement with GCC 14.2 and Qt 6.8.2 on 2026-10-07, including both
QtTest executables and the standalone tree test, yielded 97.8% line coverage
(724/740) and 70.7% branch coverage (833/1178). These are a snapshot, not fixed
expected values for later changes or other toolchains.
See [GCC instrumentation options](https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html)
and the [LCOV project documentation](https://github.com/linux-test-project/lcov).
### Configure the GCC coverage build in Qt Creator
1. Open `dbc.pro` and select a GCC kit with Qt 6.
2. In **Projects > Build Settings**, clone **Debug** as **Coverage**.
3. Choose a separate build directory, such as `build-coverage`.
4. Expand the **qmake** build step and add these **Additional arguments**:
```text
CONFIG+=dbc_qt CONFIG+=debug CONFIG-=release "QMAKE_CXXFLAGS+=--coverage -fprofile-abs-path" "QMAKE_LFLAGS+=--coverage"
```
5. Run qmake and build the project. Both `dbc_tests` and `dbc_model_tests` must
run to include coverage from the core and optional model adapter.
6. In a terminal in the coverage build directory, compile the standalone test
and run the LCOV/report commands above. Set `DBC_SOURCE_DIR` to the absolute
repository path if using a new terminal. `make check` runs both QtTest
executables; the standalone tree executable must be run separately.
7. View `html/index.html` in a browser. This workflow produces HTML reports;
the Coco plugin described below uses a different instrumentation toolchain.
### Coco integration in Qt Creator
Coco is a separate licensed toolchain. The Qt Creator plugin alone does not
install CoverageScanner, CoverageBrowser, or activate a Coco license. Obtain a
Coco license or evaluation license and download the Linux installer from
[Qt Account downloads](https://account.qt.io/s/downloads).
For this project's Debian/Ubuntu build, install the compiler, Qt development
tools, and IDE before installing Coco:
```sh
sudo apt install build-essential qt6-base-dev qt6-base-dev-tools qmake6 qtcreator
# Coco's installer uses locate to discover compilers on Unix:
sudo apt install plocate
sudo updatedb
```
Install Coco using the actual downloaded filename, for example:
```sh
bash /path/to/SquishCocoSetup_VERSION_Linux_x86_64.run
```
The exact filename and supported Linux package depend on the downloaded release.
Coco's own tools come from this installer; `lcov` is not required for Coco.
Installing or configuring a new compiler later requires rerunning the installer
or using `cocosetup` to register the compiler.
For Coco 7.4 or later, set up and check the Qt License Service using the installed
`cocolic` executable (use its full path if it is not on `PATH`):
```sh
cocolic --setup-qls
cocolic --check
```
Then configure Qt Creator:
1. Enable **Coco** in the plugin manager (**Help > About Plugins** on Linux) and
restart Qt Creator if requested. The Debian Qt Creator 16.0.1 package used for
this project's coverage check already includes `libCoco.so`.
2. In **Preferences > Coco**, set the Coco installation directory.
3. Clone **Debug** as **DebugCoco**, with its own build directory. Use an ordinary
Debug configuration as the starting point, without GCC `--coverage` flags.
4. In **Projects > Project Settings > Coco Code Coverage**, enable coverage for
this configuration and save/reconfigure. Depending on the Qt Creator version,
the **Coco code coverage** build step also offers **Enable Coverage**.
5. Exclude test sources and Qt/STL headers as appropriate for a library-only
report. Enable `CONFIG+=dbc_qt` to build the optional model tests.
6. Build using Coco's compiler wrappers, then run both test executables. Use the
Coco plugin's report view or CoverageBrowser to inspect the collected results.
For qmake, the plugin generates `cocoplugin.prf` in the source directory. Coco
must instrument the library subprojects as well as the test executables.
The GCC/LCOV commands above do not collect Coco data.
See the official [Coco installation and license setup](https://doc.qt.io/coco/installation.html),
[Coco license requirements](https://doc.qt.io/coco/coco-license-utilization.html),
and [Qt Creator Coco configuration](https://doc.qt.io/qtcreator/creator-coco.html).