From f0e706542bb09d34d06ecad046eeb9204f7443aa Mon Sep 17 00:00:00 2001 From: deeaitch Date: Fri, 13 Mar 2026 13:34:07 -0400 Subject: [PATCH] Update documentation --- README.md | 1073 +++++++++++++++++++++++++++++++++++------------------ 1 file changed, 718 insertions(+), 355 deletions(-) diff --git a/README.md b/README.md index 30cdb2d..dde0824 100644 --- a/README.md +++ b/README.md @@ -1,255 +1,503 @@ - -# DBC Parser Module Documentation +# DBC Parser and Runtime Decode Module Documentation ## Overview -This module implements a minimal DBC parser intended for use in the **FrameTap** project. +This module provides a minimal but extensible DBC parser and runtime decode foundation for the **FrameTap** project. -Its purpose is to read a DBC file and convert it into an internal representation that can later be used for: +Its purpose is to: -- displaying CAN frames and signals -- building a hierarchical structure for UI -- integrating with Qt Model/View -- selecting signals for plotting -- preparing for runtime CAN/J1939 decoding +- parse DBC files +- store parsed frame and signal metadata +- build a tree for future UI integration +- build a runtime-ready decode database +- decode live CAN frames +- decode CAN trace records using the same decoder -The implementation intentionally follows the **KISS principle**: +The implementation follows a simple and practical design: -- simple -- readable -- easy to modify -- independent from Qt +- parser logic is separated from UI logic +- runtime decode structures are separated from tree structures +- Qt is not required at the parser or decoder level +- the same decode engine can be reused for live traffic and trace replay -This is **not a full DBC parser**, but a practical foundation that can be extended gradually. +This is **not a full production-grade DBC implementation yet**, but it is a strong architectural base. --- -# Architecture +## High-Level Architecture -The module uses a simple processing pipeline. +The module is divided into two main paths. -DBC file -↓ -DbcParser -↓ -DbcDatabase -↓ -DbcTreeBuilder -↓ -TreeNode hierarchy -↓ -Future Qt Model/View integration +### 1. Parse and UI path -Each component has a clear responsibility. +Used for: -| Component | Responsibility | -|-----------|---------------| -| DbcParser | Reads and parses the DBC file | -| DbcDatabase | Stores parsed frames and signals | -| DbcTreeBuilder | Converts the database into a tree | -| TreeNode | Represents hierarchical nodes | +- browsing frames and signals +- later integration with Qt `Model/View` +- displaying metadata -This separation allows the parser to remain independent from UI or visualization logic. +Pipeline: + +```text +DBC file + ↓ +DbcParser + ↓ +DbcDatabase + ↓ +DbcTreeBuilder + ↓ +TreeNode hierarchy + ↓ +future Qt UI +``` + +### 2. Runtime decode path + +Used for: + +- live CAN decoding +- CAN trace decoding +- fast lookup by CAN ID + +Pipeline: + +```text +DBC file + ↓ +DbcParser + ↓ +DbcDatabase + ↓ +DbcDecodeBuilder + ↓ +DecodeDatabase + ↓ +DbcDecoder + ↓ +Decoded values +``` + +This separation is intentional. + +The tree is useful for UI, but it is **not** the main data structure for runtime decoding. --- -# File Structure +## Why the Runtime Decode Layer Exists -The module consists of several logical parts. +A tree structure is convenient for browsing, but a runtime decoder needs something different: -## Data structures +- fast lookup by CAN ID +- direct access to signal decode definitions +- minimal overhead during repeated decoding +- the same logic for live frames and trace frames -Files: +Because of that, the design uses a dedicated runtime-ready structure: -signal_info.h -frame_info.h -dbc_database.h +- `DecodeDatabase` +- `DecodeFrame` +- `DecodeSignal` -These files define the internal representation of parsed data. - -## Tree representation - -Files: - -tree_node.h -tree_node.cpp - -Defines a simple hierarchical structure that can later be used by UI components. - -## Parser - -Files: - -dbc_parser.h -dbc_parser.cpp - -Responsible for reading the DBC file and extracting frame and signal information. - -## Tree builder - -Files: - -dbc_tree_builder.h -dbc_tree_builder.cpp - -Converts the parsed database into a tree representation. +This avoids forcing UI-oriented structures into a decode role they were not meant for. --- -# Data Structures +## Module Layout -## SignalInfo +### Parsed DBC structures -Represents a single signal defined in a DBC file. +Files: + +- `signal_info.h` +- `frame_info.h` +- `dbc_database.h` + +These store a readable representation of the parsed DBC file. + +### UI tree structures + +Files: + +- `tree_node.h` +- `tree_node.cpp` +- `dbc_tree_builder.h` +- `dbc_tree_builder.cpp` + +These convert parsed DBC content into a tree hierarchy suitable for UI and model/view usage later. + +### Runtime decode structures + +Files: + +- `decode_database.h` +- `dbc_decode_builder.h` +- `dbc_decode_builder.cpp` + +These convert parsed DBC content into a structure optimized for decoding. + +### Runtime decoder + +Files: + +- `dbc_decoder.h` +- `dbc_decoder.cpp` + +These perform actual decoding of raw CAN frames using `DecodeDatabase`. + +### Parser + +Files: + +- `dbc_parser.h` +- `dbc_parser.cpp` + +These parse the DBC file itself. + +### Demo + +File: + +- `main.cpp` + +Used as a small integration example. + +--- + +## Parsed Data Structures + +## `SignalInfo` + +Represents one signal as parsed from the DBC file. Fields: -- name -- startBit -- length -- isLittleEndian -- isSigned +- `name` +- `startBit` +- `length` +- `isLittleEndian` +- `isSigned` +- `factor` +- `offset` +- `minimum` +- `maximum` +- `unit` +- `receivers` +- `comment` + +Notes: + +- `receivers` is a list because a signal may have more than one receiver ECU +- `factor` and `offset` define physical conversion +- this structure is close to DBC content and easy to inspect + +Physical value rule: + +```text +physical = raw * factor + offset +``` + +--- + +## `FrameInfo` + +Represents one frame as parsed from the DBC file. + +Fields: + +- `name` +- `canId` +- `isExtended` +- `pgn` +- `hasPgn` +- `dlc` +- `transmitter` +- `comment` +- `signals` + +Notes: + +- `signals` is a list of `SignalInfo` +- `isExtended` is determined during CAN ID normalization +- `pgn` is derived using simplified J1939 logic when applicable + +--- + +## `DbcDatabase` + +Top-level parsed DBC container. + +Conceptually: + +```text +DbcDatabase + └── vector +``` + +This is the central structure produced by `DbcParser`. + +--- + +## UI Tree Layer + +## `TreeNode` + +The UI tree contains three node types: + +- `Root` +- `Frame` +- `Signal` + +Example hierarchy: + +```text +dbc + ├── EngineData + │ ├── EngineSpeed + │ ├── OilTemp + │ └── CoolantTemp + └── VehicleData + ├── VehicleSpeed + └── Odometer +``` + +Each node stores either: + +- `FrameInfo` +- `SignalInfo` + +The tree is intended for browsing and later Qt model integration. + +It is **not** the primary runtime decode structure. + +--- + +## Runtime Decode Layer + +## Purpose + +The decode layer exists so that decoding can be fast and independent from UI concerns. + +Instead of searching a tree, the decoder uses a prepared database with direct lookup. + +--- + +## `ByteOrder` + +Runtime byte order enum: + +- `Intel` +- `Motorola` + +This is better for decode code than passing around raw DBC characters. + +--- + +## `ValueType` + +Numeric type enum: + +- `Unsigned` +- `Signed` + +This is clearer than combining multiple boolean flags during runtime logic. + +--- + +## `DecodeSignal` + +Represents one runtime-ready signal definition. + +Fields: + +- `name` +- `startBit` +- `length` +- `byteOrder` +- `valueType` +- `factor` +- `offset` +- `minimum` +- `maximum` +- `unit` +- `receivers` +- `comment` + +This structure contains all information required for extracting and converting a signal value from raw frame data. + +--- + +## `DecodeFrame` + +Represents one runtime-ready frame definition. + +Fields: + +- `name` +- `canId` +- `isExtended` +- `dlc` +- `pgn` +- `hasPgn` +- `transmitter` +- `comment` +- `signals` + +This structure is used directly by the decoder. + +--- + +## `FrameKey` + +Fast lookup key for runtime frame matching. + +Fields: + +- `canId` +- `isExtended` + +This matters because the same numeric identifier must not be confused between standard and extended frames. + +--- + +## `DecodeDatabase` + +Top-level runtime decode container. + +Fields: + +- `frames` +- `frameIndexByKey` + +Conceptually: + +```text +DecodeDatabase + ├── vector + └── unordered_map +``` + +This gives the decoder fast access to a frame definition using CAN ID and frame type. + +--- + +## Decoder Layer + +## `RawCanFrame` + +Represents a raw CAN frame to decode. + +Fields: + +- `canId` +- `isExtended` +- `data` + +This same structure can be used for: + +- live CAN bus input +- replayed trace records +- unit tests + +--- + +## `DecodedSignalValue` + +Represents one decoded signal result. + +Fields: + +- `definition` +- `rawValue` +- `physicalValue` +- `valid` + +--- + +## `DecodedFrameValue` + +Represents one decoded frame result. + +Fields: + +- `definition` +- `signals` +- `valid` + +This is the decoder output for one raw frame. + +--- + +## `DbcDecoder` + +Main runtime decoder class. + +Responsibilities: + +- find a frame definition by CAN ID +- decode all signals in a frame +- extract raw values +- sign-extend signed values +- convert raw values into physical values + +Main methods: + +- `FindFrame(...)` +- `Decode(...)` + +--- + +## Parser Support + +The current parser supports the following DBC elements: + +- `BO_` +- `SG_` +- `CM_ BO_` +- `CM_ SG_` + +--- + +## Supported DBC Syntax + +## Frame definition + +Example: + +```text +BO_ 256 EngineData: 8 EEC1 +``` + +Parsed fields: + +- frame CAN ID +- frame name +- DLC +- transmitter ECU + +--- + +## Signal definition + +Example: + +```text +SG_ EngineSpeed : 0|16@1+ (0.125,0) [0|8000] "rpm" ECU1,ECU2 +``` + +Parsed fields: + +- signal name +- start bit +- signal length +- byte order +- signedness - factor - offset - minimum - maximum - unit - receivers -- comment - -Notes: - -- receivers may contain multiple ECUs -- comment is optional -- factor and offset define physical conversion - -Physical value is calculated as: - -physical = raw * factor + offset - ---- - -## FrameInfo - -Represents a CAN frame. - -Fields: - -- name -- canId -- dlc -- transmitter -- comment -- signals -- pgn -- hasPgn - -Notes: - -- signals is a vector of SignalInfo -- transmitter is the ECU that sends the frame -- pgn is derived using simplified J1939 logic - ---- - -## DbcDatabase - -Top level container for parsed data. - -DbcDatabase -└── vector - -This structure represents the entire DBC file content. - ---- - -# Tree Representation - -The module converts the parsed data into a tree. - -Node types: - -- Root -- Frame -- Signal - -Example structure: - -dbc -├── EngineData -│ ├── EngineSpeed -│ ├── OilTemp -│ └── CoolantTemp -└── VehicleData - ├── VehicleSpeed - └── Odometer - -Each node stores either: - -- FrameInfo -- SignalInfo - -This tree structure is designed to integrate easily with Qt Model/View. - ---- - -# Parser - -The parser is implemented in: - -dbc_parser.h -dbc_parser.cpp - -It performs line-based parsing of the DBC file. - -Supported elements: - -- BO_ -- SG_ -- CM_ - ---- - -# Supported DBC Syntax - -## Frame Definition - -Example: - -BO_ 256 EngineData: 8 EEC1 - -Parsed values: - -| Field | Meaning | -|------|---------| -| 256 | CAN identifier | -| EngineData | frame name | -| 8 | DLC | -| EEC1 | transmitter ECU | - ---- - -## Signal Definition - -Example: - -SG_ EngineSpeed : 0|16@1+ (0.125,0) [0|8000] "rpm" ECU1,ECU2 - -Parsed values: - -| Field | Meaning | -|------|---------| -| EngineSpeed | signal name | -| 0 | start bit | -| 16 | signal length | -| @1 | little endian | -| + | unsigned | -| factor | 0.125 | -| offset | 0 | -| minimum | 0 | -| maximum | 8000 | -| unit | rpm | -| receivers | ECU1, ECU2 | --- @@ -257,222 +505,337 @@ Parsed values: Frame comment example: +```text CM_ BO_ 256 "Engine data frame"; +``` Signal comment example: +```text CM_ SG_ 256 EngineSpeed "Actual engine speed"; +``` Stored in: -- FrameInfo::comment -- SignalInfo::comment +- `FrameInfo::comment` +- `SignalInfo::comment` --- -# PGN Extraction +## CAN ID Normalization -PGN is derived using simplified J1939 logic. +The parser normalizes frame identifiers. -Typical identifier layout: +Common DBC behavior: -Priority -Reserved -Data Page -PDU Format -PDU Specific -Source Address +- extended identifiers may be stored with bit 31 set +- the actual 29-bit identifier must be extracted from that value -The parser extracts PGN from the 29‑bit CAN identifier. +The parser therefore stores: -This is only an approximation and not full J1939 validation. +- normalized `canId` +- separate `isExtended` flag + +This is important both for correct lookup and for future interoperability with live CAN APIs. --- -# Example Usage +## PGN Extraction +PGN is derived only when the frame is treated as extended. + +The current logic is simplified J1939 extraction: + +- `pf` +- `ps` +- `dp` + +This is enough for a practical start but should not be treated as full J1939 validation. + +--- + +## Decode Flow + +Typical runtime decode flow: + +```text +RawCanFrame + ↓ +Find frame in DecodeDatabase + ↓ +For each signal: + extract raw bits + apply sign extension if needed + convert to physical value + ↓ +DecodedFrameValue +``` + +--- + +## Intel and Motorola Extraction + +The decoder currently has separate extraction paths: + +- `ExtractIntel(...)` +- `ExtractMotorola(...)` + +This is important because byte order is not just metadata once decoding starts. + +Intel and Motorola require different bit extraction logic. + +This is one of the main reasons why the runtime decode layer should be explicit and prepared in advance. + +--- + +## Example Usage + +## Parse DBC + +```cpp DbcParser parser; - DbcDatabase database = parser.ParseFile("example.dbc"); +``` -DbcTreeBuilder builder; +## Build UI tree -std::unique_ptr root = builder.Build(database); +```cpp +DbcTreeBuilder treeBuilder; +std::unique_ptr root = treeBuilder.Build(database); +``` -After this step, the tree can be used by a UI or visualization system. +## Build runtime decode database + +```cpp +DbcDecodeBuilder decodeBuilder; +DecodeDatabase decodeDatabase = decodeBuilder.Build(database); +``` + +## Decode a raw frame + +```cpp +RawCanFrame rawFrame; +rawFrame.canId = 0x123; +rawFrame.isExtended = false; +rawFrame.data = {0x01, 0x02, 0x03, 0x04}; + +DbcDecoder decoder; +DecodedFrameValue decoded = decoder.Decode(decodeDatabase, rawFrame); +``` --- -# Example Tree +## Unified Decode Strategy -dbc -├── EngineData -│ ├── EngineSpeed -│ ├── OilTemp -│ └── CoolantTemp -└── VehicleData - ├── VehicleSpeed - └── Odometer +A key design goal is that the same decoder should work for both: + +- live CAN frames +- trace replay frames + +That means this architecture supports: + +### live path + +```text +live CAN input + ↓ +RawCanFrame + ↓ +DbcDecoder + ↓ +decoded signal values +``` + +### trace path + +```text +trace reader + ↓ +RawCanFrame + ↓ +DbcDecoder + ↓ +decoded signal values +``` + +This avoids duplicating decode logic in two separate parts of the application. --- -# Limitations +## Intended Use in FrameTap -The current implementation intentionally does not support many advanced DBC features. +This module is meant to support at least the following FrameTap workflows: + +- load a DBC file +- browse frames and signals +- search signals +- drag a signal into a plot later +- decode live CAN traffic +- decode recorded traces +- convert raw values into physical values +- show metadata like units, comments, transmitter, receivers, and PGN + +Example combined workflow: + +```text +Load DBC + ↓ +Parse into DbcDatabase + ↓ +Build UI tree + ↓ +Build DecodeDatabase + ↓ +Use same decode engine for: + - live frames + - trace replay +``` + +--- + +## Why the Tree Is Not Enough + +The tree exists for browsing. + +However, runtime decode should not rely on tree traversal because that would introduce unnecessary coupling and inefficiency. + +A runtime decoder needs: + +- fast key-based access +- minimal interpretation at decode time +- direct signal definitions already prepared + +That is why `DecodeDatabase` is a separate layer. + +--- + +## Why No Abstract Factory Is Used + +At the current stage, abstract factory is intentionally avoided. + +The current design is already clean: + +```text +parser → parsed database → decode database + ↘ tree builder → UI tree +``` + +Introducing factory layers now would increase complexity without solving an immediate problem. + +If later the project requires multiple output representations or multiple build strategies, that can be added then. + +--- + +## Current Limitations + +This is still a minimal implementation. Not supported yet: - multiplexed signals -- BA_ attributes -- BA_DEF_ definitions -- VAL_ value tables -- signal groups -- complex syntax variations +- `VAL_` tables +- `BA_` attributes +- `BA_DEF_` definitions +- advanced comment handling +- full DBC grammar coverage +- full J1939 validation +- extensive edge-case handling for unusual DBC formatting -These can be added later if required. +Motorola extraction is implemented, but it should still be verified carefully against real-world DBC files and expected values. --- -# Design Principles +## Recommended Next Steps -## KISS +A practical development order would be: -The parser avoids complex grammar systems or tokenizers in order to keep the code easy to understand and maintain. +### Stage 1 - already implemented -## Separation of Concerns - -Parsing, storage, and tree building are separate components. - -This allows reuse for: - -- GUI display -- runtime decoding -- export -- filtering -- testing - -## Qt Independence - -The parser itself does not depend on Qt. - -Qt integration should be implemented through a separate adapter layer. - ---- - -# Future Development - -Recommended evolution path. - -## Stage 1 (current) - -- BO_ -- SG_ -- CM_ +- `BO_` +- `SG_` +- `CM_` +- normalized CAN ID +- `isExtended` - transmitter - receivers -- basic PGN extraction +- comments - tree representation +- runtime decode database +- runtime decoder -## Stage 2 +### Stage 2 -Add: +Recommended additions: -- extended CAN ID detection -- better PGN extraction -- parent pointer in TreeNode -- Qt Model adapter +- parent pointer in `TreeNode` +- Qt model adapter +- `VAL_` support for enum-style signals +- better display strings for UI +- selective decoding of only chosen signals -## Stage 3 +### Stage 3 -Add support for: +Recommended additions: -- VAL_ value tables -- BA_ attributes -- BA_DEF_ definitions -- extended comments +- multiplexing support +- attribute parsing +- richer J1939 support +- CSV export of decoded traces +- optimized filtering and signal selection -## Stage 4 +### Stage 4 Advanced functionality: -- multiplexing -- runtime signal decoding -- integration with CAN stream -- drag‑and‑drop plotting +- live plot integration +- signal subscriptions +- per-signal trace decode pipelines +- decoder-assisted export formats --- -# Intended Usage in FrameTap +## Build Integration -This module allows FrameTap to: +The module does not depend on any specific build system. -- load DBC files -- display frames and signals -- select signals -- search signals -- display metadata -- prepare runtime decoding - -Example workflow: - -Load DBC -↓ -Browse signals -↓ -Select signal -↓ -Decode CAN frames -↓ -Plot physical values - ---- - -# Why Abstract Factory Is Not Used - -An abstract factory is intentionally not used at this stage. - -The current structure already separates responsibilities clearly: - -parser → database → tree builder → tree - -Factories can be introduced later if multiple internal representations become necessary. - ---- - -# Build Integration - -The module is independent of any specific build system. - -It can be integrated using: +It can be integrated with: - CMake - qmake - Makefile -Simply include the source files in your project. +Just add the source files to the project. --- -# Summary +## Summary -This module provides a minimal but extensible DBC parser designed for **FrameTap**. +This module is now split into two intentionally separate layers: -Key properties: +### Parsed representation -- simple -- modular -- Qt‑independent -- extendable +Used for: -It converts a DBC file into an internal structure: +- storing parsed DBC content +- browsing +- UI tree generation -frame → signals +### Runtime decode representation -with metadata such as: +Used for: -- transmitter -- receivers -- comments -- basic PGN information +- fast frame lookup +- live CAN decode +- trace decode +- physical value conversion -This forms the foundation for future CAN signal visualization and decoding. +That separation is the main architectural improvement. + +In short, the system now looks like this: + +```text +DBC parser → DbcDatabase → DecodeDatabase → DbcDecoder + ↘ TreeNode → future UI +``` + +This gives FrameTap a much better foundation for real use, because both browsing and decoding are supported without forcing one representation to do the other's job.