# DBC Parser and Runtime Decode Module Documentation ## Overview This module provides a minimal but extensible DBC parser and runtime decode foundation for the **FrameTap** project. Its purpose is to: - parse DBC files - store parsed frame and signal metadata - build a tree for future UI integration - build a runtime-ready decode database - decode live CAN frames - decode CAN trace records using the same decoder The implementation follows a simple and practical design: - parser logic is separated from UI logic - runtime decode structures are separated from tree structures - Qt is not required at the parser or decoder level - the same decode engine can be reused for live traffic and trace replay The core library uses C++17 and has no Qt dependency. Qt Core and Qt Test are required only for the regression tests. Live CAN acquisition, trace-file readers, and a Qt model adapter are left to the consuming application. See [changelog.md](changelog.md) for the release and development history, [tests/README.md](tests/README.md) for test instructions, and [generated API documentation](docs/html/index.html) for the API reference. --- ## High-Level Architecture The module is divided into two main paths. ### 1. Parse and UI path Used for: - browsing frames and signals - later integration with Qt `Model/View` - displaying metadata Pipeline: ```text DBC file ↓ DbcParser ↓ DbcDatabase ↓ DbcTreeBuilder ↓ TreeNode hierarchy ↓ future Qt UI ``` ### 2. Runtime decode path Used for: - live CAN decoding - CAN trace decoding - fast lookup by CAN ID Pipeline: ```text DBC file ↓ DbcParser ↓ DbcDatabase ↓ DbcDecodeBuilder ↓ DecodeDatabase ↓ DbcDecoder ↓ Decoded values ``` This separation is intentional. The tree is useful for UI, but it is **not** the main data structure for runtime decoding. --- ## Why the Runtime Decode Layer Exists A tree structure is convenient for browsing, but a runtime decoder needs something different: - fast lookup by CAN ID - direct access to signal decode definitions - minimal overhead during repeated decoding - the same logic for live frames and trace frames Because of that, the design uses a dedicated runtime-ready structure: - `DecodeDatabase` - `DecodeFrame` - `DecodeSignal` This avoids forcing UI-oriented structures into a decode role they were not meant for. --- ## Module Layout ```text core/ C++17 parser, statement handlers, metadata, tree, and decoder examples/ Command-line demo (built separately) tests/ QtTest parser and decoder regression suites docs/ Generated Doxygen documentation dbc.pro Root qmake project: core + tests Doxyfile API documentation configuration ``` | Component | Source files | | --- | --- | | Parsed metadata | `core/dbc_database.h`, `core/frame_info.h`, `core/signal_info.h` | | Parser and dispatch | `core/dbc_parser.*`, `core/dbc_statement_handler.*`, `core/typefactory.h` | | UI tree | `core/tree_node.*`, `core/dbc_tree_builder.*` | | Runtime database | `core/decode_database.h`, `core/dbc_decode_builder.*` | | Decoder | `core/dbc_decoder.*` | | Demo | `examples/main.cpp`, `examples/demo.pro` | | Build integration | `core/core.pro`, `core/core_files.pri`, `core/core.pri` | --- ## Parsed Data Structures ## `SignalInfo` Represents one signal as parsed from the DBC file. Fields: - `name` - `startBit` - `length` - `isLittleEndian` - `isSigned` - `factor` - `offset` - `minimum` - `maximum` - `unit` - `receivers` - `comment` Notes: - `receivers` is a list because a signal may have more than one receiver ECU - `factor` and `offset` define physical conversion - this structure is close to DBC content and easy to inspect Physical value rule: ```text physical = raw * factor + offset ``` --- ## `FrameInfo` Represents one frame as parsed from the DBC file. Fields: - `name` - `canId` - `isExtended` - `pgn` - `hasPgn` - `dlc` - `transmitter` - `comment` - `signals` Notes: - `signals` is a list of `SignalInfo` - `isExtended` is determined during CAN ID normalization - `pgn` is derived using simplified J1939 logic when applicable --- ## `DbcDatabase` Top-level parsed DBC container. Conceptually: ```text DbcDatabase └── vector ``` This is the central structure produced by `DbcParser`. --- ## UI Tree Layer ## `TreeNode` The UI tree contains three node types: - `Root` - `Frame` - `Signal` Example hierarchy: ```text dbc ├── EngineData │ ├── EngineSpeed │ ├── OilTemp │ └── CoolantTemp └── VehicleData ├── VehicleSpeed └── Odometer ``` Frame and signal nodes store copies of the corresponding metadata; the root has no metadata payload. The payload types are: - `FrameInfo` - `SignalInfo` The tree is intended for browsing and later Qt model integration. It is **not** the primary runtime decode structure. --- ## Runtime Decode Layer ## Purpose The decode layer exists so that decoding can be fast and independent from UI concerns. Instead of searching a tree, the decoder uses a prepared database with direct lookup. --- ## `ByteOrder` Runtime byte order enum: - `Intel` - `Motorola` This is better for decode code than passing around raw DBC characters. --- ## `ValueType` Numeric type enum: - `Unsigned` - `Signed` This is clearer than combining multiple boolean flags during runtime logic. --- ## `DecodeSignal` Represents one runtime-ready signal definition. Fields: - `name` - `startBit` - `length` - `byteOrder` - `valueType` - `factor` - `offset` - `minimum` - `maximum` - `unit` - `receivers` - `comment` This structure contains all information required for extracting and converting a signal value from raw frame data. --- ## `DecodeFrame` Represents one runtime-ready frame definition. Fields: - `name` - `canId` - `isExtended` - `dlc` - `pgn` - `hasPgn` - `transmitter` - `comment` - `signals` This structure is used directly by the decoder. --- ## `FrameKey` Fast lookup key for runtime frame matching. Fields: - `canId` - `isExtended` This matters because the same numeric identifier must not be confused between standard and extended frames. --- ## `DecodeDatabase` Top-level runtime decode container. Fields: - `frames` - `frameIndexByKey` Conceptually: ```text DecodeDatabase ├── vector └── unordered_map ``` This gives the decoder fast access to a frame definition using CAN ID and frame type. --- ## Decoder Layer ## `RawCanFrame` Represents a raw CAN frame to decode. Fields: - `canId` - `isExtended` - `data` This same structure can be used for: - live CAN bus input - replayed trace records - unit tests --- ## `DecodedSignalValue` Represents one decoded signal result. Fields: - `definition` - `rawValue` - `physicalValue` - `valid` --- ### Raw value representation and precision `DecodedSignalValue::rawValue` is a C++17 `std::variant`. For a valid signal, signed definitions produce the `int64_t` alternative and unsigned definitions produce the `uint64_t` alternative. Raw values remain exact across their full 64-bit ranges. Check `valid` before using the value; an invalid result's default value is not decoded data. Use `std::get` when the numeric type is known, or `std::get_if` to handle both: ```cpp if (const auto *value = std::get_if(&decodedSignal.rawValue)) { std::cout << *value; } else { std::cout << std::get(decodedSignal.rawValue); } ``` This replaces the previous `int64_t` field: consumers that read or print `rawValue` directly must handle the variant. Physical conversion uses the active numeric type: `physicalValue = static_cast(raw) * factor + offset`. `physicalValue` remains a `double` and is approximate for large integers. On usual IEEE 754 binary64 implementations, integers are exactly representable through 2^53; above that, some integers round to the same floating-point value. For example, unsigned `UINT64_MAX` remains exact in `rawValue`, but converts to approximately 18446744073709551616 in binary64 before scaling. Use `rawValue` whenever exact integer values are required. --- ## `DecodedFrameValue` Represents one decoded frame result. Fields: - `definition` - `signals` - `valid` `valid` means a matching frame definition was found. Check each signal's `valid` separately: insufficient payload data can leave individual signals invalid even when the frame is valid. An unknown frame returns `valid == false`. The frame and signal `definition` pointers refer into `DecodeDatabase`. Keep that database alive and avoid changes that invalidate its frame or signal storage while using decoded results. The decoder applies factor and offset; it does not clamp physical values to the declared minimum and maximum. --- ## `DbcDecoder` Main runtime decoder class. Responsibilities: - find a frame definition by CAN ID - decode all signals in a frame - extract raw values - sign-extend signed values - convert raw values into physical values Main methods: - `FindFrame(...)` - `Decode(...)` --- ## Parser Support The current parser supports the following DBC elements: - `BO_` - `SG_` - `CM_ BO_` - `CM_ SG_` - `VAL_` signal value descriptions --- ## Supported DBC Syntax ## Frame definition Example: ```text BO_ 256 EngineData: 8 EEC1 ``` Parsed fields: - frame CAN ID - frame name - DLC - transmitter ECU --- ### Frame validation and parse errors `BO_` definitions require a decimal CAN ID, a nonempty single-token frame name, a colon, a decimal payload length, and a transmitter token. Spaces or tabs may separate fields, and whitespace before the colon is accepted. Additional trailing fields are rejected. Numeric fields must contain decimal digits only; signs, hexadecimal notation, fractional values, suffixes, and overflow are rejected. CAN IDs may contain the 29-bit identifier plus the optional DBC extended flag (bit 31). Bits 29 and 30 are rejected rather than silently discarded. For compatibility, unflagged IDs above `0x7FF` are still treated as extended. The payload length must fit `uint8_t` (`0..255`). This is a storage-range check, not CAN or CAN FD protocol validation; no new 8-byte or 64-byte limit is imposed. Parsing stops at the first detected error. Exceptions raised while parsing a line are reported as `std::runtime_error` with the supplied file path and the one-based physical line number, including blank lines: ```text example.dbc:3: Invalid CAN ID: invalid ``` Frame and signal definitions are validated as described here; comprehensive DBC grammar coverage is still planned. File-open errors contain the path but have no line number. --- ## Signal definition Example: ```text SG_ EngineSpeed : 0|16@1+ (0.125,0) [0|8000] "rpm" ECU1,ECU2 ``` Parsed fields: - signal name - start bit - signal length - byte order - signedness - factor - offset - minimum - maximum - unit - receivers --- ### Signal validation and parse errors `SG_` statements are matched completely. Signal names must be nonempty and contain no whitespace. Spaces and tabs are accepted between syntax elements; byte order and signedness remain adjacent (`@0+`, `@0-`, `@1+`, or `@1-`). A quoted unit (which may be empty) and a nonempty comma-separated receiver list are required. Unexpected trailing fields are rejected. Start bits must be unsigned decimal integers fitting `uint32_t`; lengths must be within `1..64` bits. Signs, fractional values, numeric suffixes, and overflow are rejected for these fields. Every signal bit must fit within the frame's declared payload length, using Intel increasing bit positions or Motorola sawtooth traversal. This uses the stored payload length without introducing an additional CAN or CAN FD payload limit. Factor, offset, minimum, and maximum accept signed decimal numbers, fractions, and scientific notation. The entire numeric token must be valid; hexadecimal numbers, NaN, infinity, conversion overflow, and conversion underflow are rejected. Minimum must not exceed maximum. Zero and negative factors are allowed. Multiplexing markers (`M`, `mN`, and combinations) are currently rejected rather than interpreted as part of a signal name. Multiplexing support and duplicate frame/signal detection remain separate work. Invalid signal definitions stop parsing with `path:line: description`, including when the signal exceeds the declared payload length. --- ## Comments Frame comment example: ```text CM_ BO_ 256 "Engine data frame"; ``` Signal comment example: ```text CM_ SG_ 256 EngineSpeed "Actual engine speed"; ``` Stored in: - `FrameInfo::comment` - `SignalInfo::comment` --- ## CAN ID Normalization The parser normalizes frame identifiers. Common DBC behavior: - extended identifiers may be stored with bit 31 set - the actual 29-bit identifier must be extracted from that value The parser therefore stores: - normalized `canId` - separate `isExtended` flag This is important both for correct lookup and for future interoperability with live CAN APIs. --- ## PGN Extraction PGN is derived only when the frame is treated as extended. The current logic is simplified J1939 extraction: - `pf` - `ps` - `dp` This is enough for a practical start but should not be treated as full J1939 validation. --- ## Decode Flow Typical runtime decode flow: ```text RawCanFrame ↓ Find frame in DecodeDatabase ↓ For each signal: extract raw bits apply sign extension if needed convert to physical value ↓ DecodedFrameValue ``` --- ## Intel and Motorola Extraction The decoder currently has separate extraction paths: - `ExtractIntel(...)` - `ExtractMotorola(...)` This is important because byte order is not just metadata once decoding starts. Intel and Motorola require different bit extraction logic. This is one of the main reasons why the runtime decode layer should be explicit and prepared in advance. --- ## Example Usage Include the public headers from the `core/` include directory supplied by `core/core.pri`: ```cpp #include "dbc_parser.h" #include "dbc_tree_builder.h" #include "dbc_decode_builder.h" #include "dbc_decoder.h" ``` ### Parse DBC ```cpp DbcParser parser; DbcDatabase database = parser.ParseFile("example.dbc"); ``` ### Build UI tree ```cpp DbcTreeBuilder treeBuilder; std::unique_ptr root = treeBuilder.Build(database); ``` ### Build runtime decode database ```cpp DbcDecodeBuilder decodeBuilder; DecodeDatabase decodeDatabase = decodeBuilder.Build(database); ``` ### Decode a raw frame ```cpp RawCanFrame rawFrame; rawFrame.canId = 256; // EngineData from the DBC example above rawFrame.isExtended = false; rawFrame.data = {0x01, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}; DbcDecoder decoder; DecodedFrameValue decoded = decoder.Decode(decodeDatabase, rawFrame); ``` --- ## Unified Decode Strategy A key design goal is that the same decoder should work for both: - live CAN frames - trace replay frames That means this architecture supports: ### live path ```text live CAN input ↓ RawCanFrame ↓ DbcDecoder ↓ decoded signal values ``` ### trace path ```text trace reader ↓ RawCanFrame ↓ DbcDecoder ↓ decoded signal values ``` This avoids duplicating decode logic in two separate parts of the application. --- ## Intended Use in FrameTap This module is meant to support at least the following FrameTap workflows: - load a DBC file - browse frames and signals - search signals - drag a signal into a plot later - decode live CAN traffic - decode recorded traces - convert raw values into physical values - show metadata like units, comments, transmitter, receivers, and PGN Example combined workflow: ```text Load DBC ↓ Parse into DbcDatabase ↓ Build UI tree ↓ Build DecodeDatabase ↓ Use same decode engine for: - live frames - trace replay ``` --- ## Why the Tree Is Not Enough The tree exists for browsing. However, runtime decode should not rely on tree traversal because that would introduce unnecessary coupling and inefficiency. A runtime decoder needs: - fast key-based access - minimal interpretation at decode time - direct signal definitions already prepared That is why `DecodeDatabase` is a separate layer. --- ## Statement Handler Factory `DbcParser` owns file reading, trimming, line numbers, and error reporting. Each `ParseFile()` call creates a fresh `DbcParseContext` and a `DbcStatementDispatcher`. The context contains the parsed database and current frame; handlers receive it by reference and retain no parsing state. The dispatcher uses `TypeFactory` to create registered handlers once per parsing session, then reuses them for all matching lines: - `BO_` → `DbcFrameHandler` - `SG_` → `DbcSignalHandler` - `CM_` → `DbcCommentHandler` - `VAL_` → `DbcValueDescriptionHandler` Grammar implementations live in `core/dbc_statement_handler.cpp`; their interface, context, and dispatcher live in `core/dbc_statement_handler.h`. `core/typefactory.h` provides the generic registry and object creation. To add a construct, implement a default-constructible handler deriving from `DbcStatementHandler` with: ```cpp void Handle(const std::string &line, DbcParseContext &context) const override; ``` Register it in the dispatcher constructor: ```cpp RegisterType("CUSTOM_"); // after implementing MyCustomHandler ``` Registration creates one handler; replacing a registration replaces that instance. Register during initialization before dispatching. A line is routed using its full first whitespace-delimited keyword, so `BO_TX_BU_` cannot match `BO_`. Unregistered keywords remain ignored. `SG_` accepts spaces and tabs after its keyword; `CM_` retains its existing policy requiring a literal space. This factory handles statements inside DBC files. A separate format-level factory can be introduced when another file format is implemented. The decoder and database builders remain independent of statement dispatch. --- ## Signal value descriptions (`VAL_`) Signal descriptions attach text to exact raw integer values: ```dbc BO_ 256 Status: 8 ECU SG_ State : 0|8@1+ (1,0) [0|255] "" ECU VAL_ 256 State 0 "Off" 1 "On" 2 "Error"; ``` Each statement must occupy one physical line, contain at least one integer/text pair, and end with a semicolon. Spaces and tabs are accepted between fields; descriptions may contain whitespace, semicolons, and empty text. Inside quoted text, `\"` and `\\` represent a quote and a backslash; other escapes are rejected. Keys are decimal integers in `INT64_MIN..UINT64_MAX`, with an optional minus sign; plus signs, fractions, hexadecimal numbers, exponents, and overflow are rejected. Keys never pass through `double`. `VAL_` may appear before or after its frame and signal declarations. References are resolved after reading the file using the same CAN ID normalization as `BO_`, including bit 31 for extended identifiers. Standard and extended frames with the same normalized ID remain distinct. Missing or ambiguous references, malformed statements, and duplicate integer keys produce `path:line: description` errors at the originating `VAL_` line. Distinct keys from multiple statements are merged; `0`, `00`, and `-0` are the same key. Keys need not fit the referenced signal's bit width or signedness; unreachable keys simply cannot match a decode. Mappings are available as `SignalInfo::valueDescriptions` and `DecodeSignal::valueDescriptions`, and survive both database builders, including `TreeNode::GetSignal()`. Consumers can look up a decoded result directly: ```cpp const DecodedSignalValue &value = decodedFrame.signals.at(0); if (const std::string *description = value.GetValueDescription()) { std::cout << *description; } ``` `GetValueDescription()` uses the exact raw integer before factor/offset scaling. It returns `nullptr` for an invalid decode or an unknown value; numeric decoding and `physicalValue` are unchanged. A mapped empty string returns a non-null pointer. The pointer belongs to the signal definition and stays valid while that definition and mapping remain alive and unchanged. For metadata or standalone raw values, use `FindValueDescription(signal.valueDescriptions, rawValue)`, where `rawValue` is `RawSignalValue` (`std::variant`). The helper matches nonnegative signed and unsigned values identically and keeps negative values distinct from large unsigned values. Stored keys use negative `int64_t` and nonnegative `uint64_t`; when constructing mappings manually, normalize keys with `NormalizeValueKey()` before insertion. Named `VAL_TABLE_` tables, environment-variable mappings, multiline statements, and Qt presentation are outside this implementation. --- ## Current Limitations This is still a minimal implementation. Not supported yet: - multiplexed signals - named `VAL_TABLE_` tables and environment-variable value descriptions - `BA_` attributes - `BA_DEF_` definitions - advanced comment handling - full DBC grammar coverage - full J1939 validation - extensive edge-case handling for unusual DBC formatting Additional behavior to account for: - unregistered DBC keywords are ignored - an `SG_` statement before the first frame is rejected - duplicate frame and signal definitions are not rejected globally; ambiguous `VAL_` references to them are rejected - payload lengths are checked against storage bounds (`0..255`), rather than CAN or CAN FD protocol limits - comment handling is limited to the supported forms and already parsed targets Motorola extraction uses DBC sawtooth numbering: a start bit of 7 traverses `7..0, 15..8, 23..16`. Regression tests cover aligned and unaligned signals, byte boundaries, and 64-bit values. --- ## Recommended Next Steps A practical development order would be: ### Stage 1 - already implemented - `BO_` - `SG_` - `CM_` - `VAL_` signal value descriptions - normalized CAN ID - `isExtended` - transmitter - receivers - comments - tree representation - runtime decode database - runtime decoder ### Stage 2 Recommended additions: - parent pointer in `TreeNode` - Qt model adapter - better display strings for UI - selective decoding of only chosen signals ### Stage 3 Recommended additions: - multiplexing support - attribute parsing - richer J1939 support - CSV export of decoded traces - optimized filtering and signal selection ### Stage 4 Advanced functionality: - live plot integration - signal subscriptions - per-signal trace decode pipelines - decoder-assisted export formats --- ## Build Integration The repository supplies qmake projects. A C++17 compiler is required; the root build also requires Qt Core and Qt Test development packages. Other build systems can compile the sources listed in `core/core_files.pri`; no CMake configuration is currently provided. Open `dbc.pro` in Qt Creator to see two subprojects: `core` and `tests`. `core/core.pro` builds the C++17 static library `dbc_core` without a Qt dependency. `tests/tests.pro` builds the QtTest executable `dbc_tests` and links that library. The root project declares `tests.depends = core` for parallel builds. Build out of source (Qt 6; use `qmake` for Qt 5): ```sh mkdir build cd build qmake6 ../dbc.pro make -j make check ``` `core/core_files.pri` lists the library sources and headers. Consumers include `core/core.pri` for include paths and library linkage; the test source lists live in `tests/parser_tests.pri` and `tests/decoder_tests.pri`. When building a consumer separately, pass `DBC_CORE_BUILD_DIR=/absolute/path/to/build/core` to qmake. ### Build core without Qt libraries From the repository root: ```sh mkdir -p build/core cd build/core qmake6 ../../core/core.pro make -j ``` qmake is still the build tool; this target does not link Qt libraries. ### Build and run the demo After building core, run these commands from the repository root: ```sh mkdir -p build/examples cd build/examples qmake6 ../../examples/demo.pro DBC_CORE_BUILD_DIR=../core make -j ./dbc_demo /absolute/path/to/example.dbc ``` The demo prints the parsed tree and decodes eight zero bytes using the first frame definition. Replace that sample payload in `examples/main.cpp` to feed live CAN data or trace records. ### Generate API documentation With Doxygen installed, run from the repository root: ```sh doxygen Doxyfile ``` Open `docs/html/index.html` to view the generated reference. --- ## Regression tests See [tests/README.md](tests/README.md) for QtTest build and execution commands. Tests cover frame validation and error locations, Intel and Motorola extraction, signal syntax and payload bounds, `VAL_` syntax and reference validation, raw description lookup and preservation through both builders, statement dispatch, signed and unsigned 64-bit boundaries, physical conversion, numeric output, and invalid signal payloads.