# DBC Parser and Runtime Decode Module Documentation ## Overview This module provides a minimal but extensible DBC parser and runtime decode foundation for the **FrameTap** project. Its purpose is to: - parse DBC files - store parsed frame and signal metadata - build a tree for future UI integration - build a runtime-ready decode database - decode live CAN frames - decode CAN trace records using the same decoder The implementation follows a simple and practical design: - parser logic is separated from UI logic - runtime decode structures are separated from tree structures - Qt is not required at the parser or decoder level - the same decode engine can be reused for live traffic and trace replay This is **not a full production-grade DBC implementation yet**, but it is a strong architectural base. --- ## High-Level Architecture The module is divided into two main paths. ### 1. Parse and UI path Used for: - browsing frames and signals - later integration with Qt `Model/View` - displaying metadata Pipeline: ```text DBC file ↓ DbcParser ↓ DbcDatabase ↓ DbcTreeBuilder ↓ TreeNode hierarchy ↓ future Qt UI ``` ### 2. Runtime decode path Used for: - live CAN decoding - CAN trace decoding - fast lookup by CAN ID Pipeline: ```text DBC file ↓ DbcParser ↓ DbcDatabase ↓ DbcDecodeBuilder ↓ DecodeDatabase ↓ DbcDecoder ↓ Decoded values ``` This separation is intentional. The tree is useful for UI, but it is **not** the main data structure for runtime decoding. --- ## Why the Runtime Decode Layer Exists A tree structure is convenient for browsing, but a runtime decoder needs something different: - fast lookup by CAN ID - direct access to signal decode definitions - minimal overhead during repeated decoding - the same logic for live frames and trace frames Because of that, the design uses a dedicated runtime-ready structure: - `DecodeDatabase` - `DecodeFrame` - `DecodeSignal` This avoids forcing UI-oriented structures into a decode role they were not meant for. --- ## Module Layout ### Parsed DBC structures Files: - `signal_info.h` - `frame_info.h` - `dbc_database.h` These store a readable representation of the parsed DBC file. ### UI tree structures Files: - `tree_node.h` - `tree_node.cpp` - `dbc_tree_builder.h` - `dbc_tree_builder.cpp` These convert parsed DBC content into a tree hierarchy suitable for UI and model/view usage later. ### Runtime decode structures Files: - `decode_database.h` - `dbc_decode_builder.h` - `dbc_decode_builder.cpp` These convert parsed DBC content into a structure optimized for decoding. ### Runtime decoder Files: - `dbc_decoder.h` - `dbc_decoder.cpp` These perform actual decoding of raw CAN frames using `DecodeDatabase`. ### Parser Files: - `dbc_parser.h` - `dbc_parser.cpp` - `dbc_statement_handler.h` - `dbc_statement_handler.cpp` - `typefactory.h` The parser reads files and reports errors; registered statement handlers parse DBC constructs. ### Demo File: - `main.cpp` Used as a small integration example. --- ## Parsed Data Structures ## `SignalInfo` Represents one signal as parsed from the DBC file. Fields: - `name` - `startBit` - `length` - `isLittleEndian` - `isSigned` - `factor` - `offset` - `minimum` - `maximum` - `unit` - `receivers` - `comment` Notes: - `receivers` is a list because a signal may have more than one receiver ECU - `factor` and `offset` define physical conversion - this structure is close to DBC content and easy to inspect Physical value rule: ```text physical = raw * factor + offset ``` --- ## `FrameInfo` Represents one frame as parsed from the DBC file. Fields: - `name` - `canId` - `isExtended` - `pgn` - `hasPgn` - `dlc` - `transmitter` - `comment` - `signals` Notes: - `signals` is a list of `SignalInfo` - `isExtended` is determined during CAN ID normalization - `pgn` is derived using simplified J1939 logic when applicable --- ## `DbcDatabase` Top-level parsed DBC container. Conceptually: ```text DbcDatabase └── vector ``` This is the central structure produced by `DbcParser`. --- ## UI Tree Layer ## `TreeNode` The UI tree contains three node types: - `Root` - `Frame` - `Signal` Example hierarchy: ```text dbc ├── EngineData │ ├── EngineSpeed │ ├── OilTemp │ └── CoolantTemp └── VehicleData ├── VehicleSpeed └── Odometer ``` Each node stores either: - `FrameInfo` - `SignalInfo` The tree is intended for browsing and later Qt model integration. It is **not** the primary runtime decode structure. --- ## Runtime Decode Layer ## Purpose The decode layer exists so that decoding can be fast and independent from UI concerns. Instead of searching a tree, the decoder uses a prepared database with direct lookup. --- ## `ByteOrder` Runtime byte order enum: - `Intel` - `Motorola` This is better for decode code than passing around raw DBC characters. --- ## `ValueType` Numeric type enum: - `Unsigned` - `Signed` This is clearer than combining multiple boolean flags during runtime logic. --- ## `DecodeSignal` Represents one runtime-ready signal definition. Fields: - `name` - `startBit` - `length` - `byteOrder` - `valueType` - `factor` - `offset` - `minimum` - `maximum` - `unit` - `receivers` - `comment` This structure contains all information required for extracting and converting a signal value from raw frame data. --- ## `DecodeFrame` Represents one runtime-ready frame definition. Fields: - `name` - `canId` - `isExtended` - `dlc` - `pgn` - `hasPgn` - `transmitter` - `comment` - `signals` This structure is used directly by the decoder. --- ## `FrameKey` Fast lookup key for runtime frame matching. Fields: - `canId` - `isExtended` This matters because the same numeric identifier must not be confused between standard and extended frames. --- ## `DecodeDatabase` Top-level runtime decode container. Fields: - `frames` - `frameIndexByKey` Conceptually: ```text DecodeDatabase ├── vector └── unordered_map ``` This gives the decoder fast access to a frame definition using CAN ID and frame type. --- ## Decoder Layer ## `RawCanFrame` Represents a raw CAN frame to decode. Fields: - `canId` - `isExtended` - `data` This same structure can be used for: - live CAN bus input - replayed trace records - unit tests --- ## `DecodedSignalValue` Represents one decoded signal result. Fields: - `definition` - `rawValue` - `physicalValue` - `valid` --- ### Raw value representation and precision `DecodedSignalValue::rawValue` is a C++17 `std::variant`. For a valid signal, signed definitions produce the `int64_t` alternative and unsigned definitions produce the `uint64_t` alternative. Raw values remain exact across their full 64-bit ranges. Check `valid` before using the value; an invalid result's default value is not decoded data. Use `std::get` when the numeric type is known, or `std::visit` to handle both: ```cpp std::visit([] (auto raw) { std::cout << raw; }, decodedSignal.rawValue); ``` This replaces the previous `int64_t` field: consumers that read or print `rawValue` directly must handle the variant. Physical conversion uses the active numeric type: `physicalValue = static_cast(raw) * factor + offset`. `physicalValue` remains a `double` and is approximate for large integers. On usual IEEE 754 binary64 implementations, integers are exactly representable through 2^53; above that, some integers round to the same floating-point value. For example, unsigned `UINT64_MAX` remains exact in `rawValue`, but converts to approximately 18446744073709551616 in binary64 before scaling. Use `rawValue` whenever exact integer values are required. --- ## `DecodedFrameValue` Represents one decoded frame result. Fields: - `definition` - `signals` - `valid` This is the decoder output for one raw frame. --- ## `DbcDecoder` Main runtime decoder class. Responsibilities: - find a frame definition by CAN ID - decode all signals in a frame - extract raw values - sign-extend signed values - convert raw values into physical values Main methods: - `FindFrame(...)` - `Decode(...)` --- ## Parser Support The current parser supports the following DBC elements: - `BO_` - `SG_` - `CM_ BO_` - `CM_ SG_` --- ## Supported DBC Syntax ## Frame definition Example: ```text BO_ 256 EngineData: 8 EEC1 ``` Parsed fields: - frame CAN ID - frame name - DLC - transmitter ECU --- ### Frame validation and parse errors `BO_` definitions require a decimal CAN ID, a nonempty single-token frame name, a colon, a decimal payload length, and a transmitter token. Spaces or tabs may separate fields, and whitespace before the colon is accepted. Additional trailing fields are rejected. Numeric fields must contain decimal digits only; signs, hexadecimal notation, fractional values, suffixes, and overflow are rejected. CAN IDs may contain the 29-bit identifier plus the optional DBC extended flag (bit 31). Bits 29 and 30 are rejected rather than silently discarded. For compatibility, unflagged IDs above `0x7FF` are still treated as extended. The payload length must fit `uint8_t` (`0..255`). This is a storage-range check, not CAN or CAN FD protocol validation; no new 8-byte or 64-byte limit is imposed. Parsing stops at the first detected error. Exceptions raised while parsing a line are reported as `std::runtime_error` with the supplied file path and the one-based physical line number, including blank lines: ```text example.dbc:3: Invalid CAN ID: invalid ``` This change validates frame definitions. Signal syntax and unsupported DBC constructs retain their existing behavior; comprehensive validation is still planned. File-open errors contain the path but have no line number. --- ## Signal definition Example: ```text SG_ EngineSpeed : 0|16@1+ (0.125,0) [0|8000] "rpm" ECU1,ECU2 ``` Parsed fields: - signal name - start bit - signal length - byte order - signedness - factor - offset - minimum - maximum - unit - receivers --- ## Comments Frame comment example: ```text CM_ BO_ 256 "Engine data frame"; ``` Signal comment example: ```text CM_ SG_ 256 EngineSpeed "Actual engine speed"; ``` Stored in: - `FrameInfo::comment` - `SignalInfo::comment` --- ## CAN ID Normalization The parser normalizes frame identifiers. Common DBC behavior: - extended identifiers may be stored with bit 31 set - the actual 29-bit identifier must be extracted from that value The parser therefore stores: - normalized `canId` - separate `isExtended` flag This is important both for correct lookup and for future interoperability with live CAN APIs. --- ## PGN Extraction PGN is derived only when the frame is treated as extended. The current logic is simplified J1939 extraction: - `pf` - `ps` - `dp` This is enough for a practical start but should not be treated as full J1939 validation. --- ## Decode Flow Typical runtime decode flow: ```text RawCanFrame ↓ Find frame in DecodeDatabase ↓ For each signal: extract raw bits apply sign extension if needed convert to physical value ↓ DecodedFrameValue ``` --- ## Intel and Motorola Extraction The decoder currently has separate extraction paths: - `ExtractIntel(...)` - `ExtractMotorola(...)` This is important because byte order is not just metadata once decoding starts. Intel and Motorola require different bit extraction logic. This is one of the main reasons why the runtime decode layer should be explicit and prepared in advance. --- ## Example Usage ## Parse DBC ```cpp DbcParser parser; DbcDatabase database = parser.ParseFile("example.dbc"); ``` ## Build UI tree ```cpp DbcTreeBuilder treeBuilder; std::unique_ptr root = treeBuilder.Build(database); ``` ## Build runtime decode database ```cpp DbcDecodeBuilder decodeBuilder; DecodeDatabase decodeDatabase = decodeBuilder.Build(database); ``` ## Decode a raw frame ```cpp RawCanFrame rawFrame; rawFrame.canId = 0x123; rawFrame.isExtended = false; rawFrame.data = {0x01, 0x02, 0x03, 0x04}; DbcDecoder decoder; DecodedFrameValue decoded = decoder.Decode(decodeDatabase, rawFrame); ``` --- ## Unified Decode Strategy A key design goal is that the same decoder should work for both: - live CAN frames - trace replay frames That means this architecture supports: ### live path ```text live CAN input ↓ RawCanFrame ↓ DbcDecoder ↓ decoded signal values ``` ### trace path ```text trace reader ↓ RawCanFrame ↓ DbcDecoder ↓ decoded signal values ``` This avoids duplicating decode logic in two separate parts of the application. --- ## Intended Use in FrameTap This module is meant to support at least the following FrameTap workflows: - load a DBC file - browse frames and signals - search signals - drag a signal into a plot later - decode live CAN traffic - decode recorded traces - convert raw values into physical values - show metadata like units, comments, transmitter, receivers, and PGN Example combined workflow: ```text Load DBC ↓ Parse into DbcDatabase ↓ Build UI tree ↓ Build DecodeDatabase ↓ Use same decode engine for: - live frames - trace replay ``` --- ## Why the Tree Is Not Enough The tree exists for browsing. However, runtime decode should not rely on tree traversal because that would introduce unnecessary coupling and inefficiency. A runtime decoder needs: - fast key-based access - minimal interpretation at decode time - direct signal definitions already prepared That is why `DecodeDatabase` is a separate layer. --- ## Statement Handler Factory `DbcParser` owns file reading, trimming, line numbers, and error reporting. Each `ParseFile()` call creates a fresh `DbcParseContext` and a `DbcStatementDispatcher`. The context contains the parsed database and current frame; handlers receive it by reference and retain no parsing state. The dispatcher uses `TypeFactory` to create registered handlers once per parsing session, then reuses them for all matching lines: - `BO_` → `DbcFrameHandler` - `SG_` → `DbcSignalHandler` - `CM_` → `DbcCommentHandler` Grammar implementations live in `dbc_statement_handler.cpp`; their interface, context, and dispatcher live in `dbc_statement_handler.h`. `typefactory.h` provides the generic registry and object creation. To add a construct, implement a default-constructible handler deriving from `DbcStatementHandler` with: ```cpp void Handle(const std::string &line, DbcParseContext &context) const override; ``` Register it in the dispatcher constructor: ```cpp RegisterType("VAL_"); // after implementing DbcValueHandler ``` Registration creates one handler; replacing a registration replaces that instance. Register during initialization before dispatching. A line is routed using its full first whitespace-delimited keyword, so `BO_TX_BU_` cannot match `BO_`. Unregistered keywords remain ignored. Existing `SG_` and `CM_` syntax policies are preserved, including requiring a literal space after those keywords. This factory handles statements inside DBC files. A separate format-level factory can be introduced when another file format is implemented. The decoder and database builders remain independent of statement dispatch. --- ## Current Limitations This is still a minimal implementation. Not supported yet: - multiplexed signals - `VAL_` tables - `BA_` attributes - `BA_DEF_` definitions - advanced comment handling - full DBC grammar coverage - full J1939 validation - extensive edge-case handling for unusual DBC formatting Motorola extraction is implemented, but it should still be verified carefully against real-world DBC files and expected values. --- ## Recommended Next Steps A practical development order would be: ### Stage 1 - already implemented - `BO_` - `SG_` - `CM_` - normalized CAN ID - `isExtended` - transmitter - receivers - comments - tree representation - runtime decode database - runtime decoder ### Stage 2 Recommended additions: - parent pointer in `TreeNode` - Qt model adapter - `VAL_` support for enum-style signals - better display strings for UI - selective decoding of only chosen signals ### Stage 3 Recommended additions: - multiplexing support - attribute parsing - richer J1939 support - CSV export of decoded traces - optimized filtering and signal selection ### Stage 4 Advanced functionality: - live plot integration - signal subscriptions - per-signal trace decode pipelines - decoder-assisted export formats --- ## Build Integration The module does not depend on any specific build system. It can be integrated with: - CMake - qmake - Makefile Just add the source files to the project. --- ## Summary This module is now split into two intentionally separate layers: ### Parsed representation Used for: - storing parsed DBC content - browsing - UI tree generation ### Runtime decode representation Used for: - fast frame lookup - live CAN decode - trace decode - physical value conversion That separation is the main architectural improvement. In short, the system now looks like this: ```text DBC parser → DbcDatabase → DecodeDatabase → DbcDecoder ↘ TreeNode → future UI ``` This gives FrameTap a much better foundation for real use, because both browsing and decoding are supported without forcing one representation to do the other's job. ## Regression tests See [tests/README.md](tests/README.md) for compiler and qmake commands. Tests cover frame validation and error locations, Intel and Motorola extraction, signed and unsigned 64-bit boundaries, physical conversion, numeric output, and invalid signal payloads.