Skip to content

Fix extraction of Motorola (big endian) signals - #27

Open
RyotaroIsoyama wants to merge 1 commit into
normaldotcom:masterfrom
RyotaroIsoyama:fix/motorola-signal-extraction
Open

Fix extraction of Motorola (big endian) signals#27
RyotaroIsoyama wants to merge 1 commit into
normaldotcom:masterfrom
RyotaroIsoyama:fix/motorola-signal-extraction

Conversation

@RyotaroIsoyama

Copy link
Copy Markdown

Fixes #2 (single-bit fields only work when set to Intel) and #3 (Motorola fields broken for messages shorter than 8 bytes).

The problem

DbcParser::parseSectionBoSg() converts the DBC start bit of a big endian signal into an MSB-first bit index counted from the first transmitted bit:

uint8_t normalized_position = (row_position * 8) + (7 - column_position);
signal->setStartBit(normalized_position);

That conversion is correct. But CanMessage::extractRawSignal() then uses the result as a little endian (LSB0) shift amount:

uint64_t data = le64toh(_u64[0]);
data >>= start_bit;      // start_bit is an MSB-first index, not an LSB0 index

The two are different coordinate systems, so the bit positions get mirrored inside the byte — bit 7-n is read where bit n was meant.

The length > 8 byte swap accidentally repairs this for signals that start at the MSB of a byte and are a whole number of bytes long (then the normalized start bit is a multiple of 8 and the shift lands on the right bytes). 8-bit signals need no swap and are also fine. So byte aligned signals decode correctly and only sub-byte fields are wrong, which makes this easy to miss: physical values look right while status bits and flags are silently mirrored.

Example

From a vehicle HVAC bus, AC_530h with payload C0 00 00 0B B8 1E 00 00:

Signal DBC Correct master
AC_Request 7|1@0+ 1 (bit 7 of byte 0) 0 (bit 0 of byte 0)
RearDefrost 6|1@0+ 1 0
Reserved_1 5|6@0+ 0 48 (= 0xC0 >> 2)
AC_speed_of_compressor 31|16@0+ 3000 3000

A reserved field returning 48 while both request flags read 0, on a bus where the compressor is visibly running, is the clearest symptom.

The change

Extract big endian signals directly from the payload read as one big endian value. No byte swap, no special case on the length:

uint64_t be = 0;
for (int i = 0; i < 8; i++) { be = (be << 8) | _u8[i]; }
return (be >> (64 - start_bit - length)) & mask;

Little endian extraction is unchanged apart from being folded into a single return.

The commented-out bounds check is restored (and clamped to the 8 bytes this function can actually reach), so signals outside the DLC return 0 instead of bits of adjacent or stale bytes — that is #3.

Note this keeps the normalization in DbcParser. It is also what CanDbSignal::isPresentInMessage() already assumes, so that function stays correct as-is.

Verification

Both implementations plus the proposed one were run against two logs recorded from a vehicle:

signal                      DBC     want    current      fixed
250h speed                 7|16     2597       2597       2597
250h power                 23|8       47         47         47
250h Reserved_byte3        31|6        0          1 x        0
250h status                25|2        1          0 x        1
250h error                 39|8        0          0          0
250h Reserved_byte5        47|8      129        129        129
250h current               55|8       95         95         95
250h voltage               63|8       20         20         20
530h AC_Request             7|1        1          0 x        1
530h RearDefrost            6|1        1          0 x        1
530h Reserved_1             5|6        0         48 x        0
530h AC_speed             31|16     3000       3000       3000
530h max_perm_power        47|8       30         30         30
[intel] 0|16               0|16     9482       9482       9482
[intel] 3|5                 3|5        1          1          1

mismatches:  current=5   fixed=0   (15 cases)

Expected values were cross-checked against cantools, and independently against the physical behaviour of the bus (the AC_Request bit pattern matches when the AC was switched on and off, and the compressor status follows it 110-460 ms later).

DLC < 8 was checked separately: a 16-bit big endian signal in bytes 0-1 of a 4-byte frame decodes correctly, and a signal pointing at byte 5 returns 0.

Relation to #11

PR #11 targets the same bug with a different approach (removing the normalization from the parser and walking bytes in extractRawSignal). It fixes the sub-byte cases but breaks 16-bit signals, because of an off-by-one when length - (start_bit % 8 + 1) is an exact multiple of 8:

int len = length - (start_bit%8 + 1);
if(len > 0)
    lastByte += (len/8)+1;      // reads one byte too many when len % 8 == 0

For the very common 7|16@0+ this yields 664879 instead of 2597. Worth knowing for anyone applying that patch, since the sub-byte signals do start working and it looks fixed.

DbcParser converts the DBC start bit of a big endian signal into an
MSB-first bit index counted from the first transmitted bit:

    normalized = (byte_index * 8) + (7 - bit_in_byte)

extractRawSignal() then used that value as a little endian (LSB0) shift
amount, which mirrors the bit positions inside the byte: bit 7-n is read
where bit n was meant. The `length > 8` byte swap happens to repair the
result for signals that start at the MSB of a byte and are a whole
number of bytes long, so byte aligned signals decode correctly and the
problem only shows up on sub-byte fields.

Concretely, for `SG_ AC_Request : 7|1@0+` with byte0 = 0xC0 the LSB of
byte0 was returned instead of the MSB, and for
`SG_ Reserved_1 : 5|6@0+` the reserved field returned 48 (= 0xC0 >> 2)
instead of 0. Status bits and flags therefore read as if they were
mirrored, while speeds, voltages and currents looked fine.

Extract big endian signals directly from the payload read as one big
endian value instead, which needs no byte swap and no special case for
the length. Little endian extraction is unchanged.

Also restore the bounds check that was commented out, which stops
signals that lie outside the DLC from returning bits of adjacent or
stale bytes.

Verified against two recorded logs from a vehicle HVAC bus (11 big
endian signals plus little endian regression cases); see PR description
for the value table.

Fixes normaldotcom#2
Fixes normaldotcom#3

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Single-bit fields only work correctly if set to "Intel"

1 participant