Add hardware-in-the-loop and fault injection tests

.github/workflows/pull_request.yml

@@ -22,6 +22,14 @@ jobs:
       - name: Analyze project source
         run: flutter analyze
 
+      - name: Install HIL dependencies
+        working-directory: hil
+        run: flutter pub get
+
+      - name: Analyze HIL source
+        working-directory: hil
+        run: flutter analyze
+
       - name: Run Flutter tests
         run: flutter test
 
@@ -31,9 +39,18 @@ jobs:
           distribution: 'temurin'
           java-version: '17'
 
+      - name: Install example dependencies
+        working-directory: example
+        run: flutter pub get
+
+      - name: Set up Gradle
+        uses: gradle/actions/setup-gradle@v6
+        with:
+          gradle-version: '8.14.3'
+
       - name: Run Android unit tests
         working-directory: example/android
-        run: ./gradlew :universal_ble:testDebugUnitTest
+        run: gradle :universal_ble:testDebugUnitTest
 
       - name: Install Chrome
         uses: browser-actions/setup-chrome@latest

analysis_options.yaml

@@ -1,4 +1,10 @@
 include: package:flutter_lints/flutter.yaml
 
+analyzer:
+  exclude:
+    # `hil` is a standalone Flutter application with its own package config.
+    # It is analyzed separately in CI.
+    - hil/**
+
 # Additional information about this file can be found at
 # https://dart.dev/guides/language/analysis-options

hil/.gitignore

@@ -0,0 +1,53 @@
+# Miscellaneous
+*.class
+*.log
+*.pyc
+*.swp
+.DS_Store
+.atom/
+.build/
+.buildlog/
+.history
+.svn/
+.swiftpm/
+migrate_working_dir/
+
+# IntelliJ related
+*.iml
+*.ipr
+*.iws
+.idea/
+
+# The .vscode folder contains launch configuration and tasks you configure in
+# VS Code which you may wish to be included in version control, so this line
+# is commented out by default.
+#.vscode/
+
+# Flutter/Dart/Pub related
+**/doc/api/
+**/ios/Flutter/.last_build_id
+.dart_tool/
+.packages
+.flutter-plugins
+.flutter-plugins-dependencies
+.pub-cache/
+.pub/
+/build/
+/coverage/
+/pubspec_overrides.yaml
+
+# Generated desktop build state
+/windows/build/
+/windows/out/
+/windows/flutter/ephemeral/
+
+# Symbolication related
+app.*.symbols
+
+# Obfuscation related
+app.*.map.json
+
+# Android Studio will place build artifacts here
+/android/app/debug
+/android/app/profile
+/android/app/release

hil/.metadata

@@ -0,0 +1,30 @@
+# This file tracks properties of this Flutter project.
+# Used by Flutter tool to assess capabilities and perform upgrades etc.
+#
+# This file should be version controlled and should not be manually edited.
+
+version:
+  revision: "e1fd963c6f6922bd32afde2e9698a363cd0406d2"
+  channel: "stable"
+
+project_type: app
+
+# Tracks metadata for the flutter migrate command
+migration:
+  platforms:
+    - platform: root
+      create_revision: e1fd963c6f6922bd32afde2e9698a363cd0406d2
+      base_revision: e1fd963c6f6922bd32afde2e9698a363cd0406d2
+    - platform: android
+      create_revision: e1fd963c6f6922bd32afde2e9698a363cd0406d2
+      base_revision: e1fd963c6f6922bd32afde2e9698a363cd0406d2
+
+  # User provided section
+
+  # List of Local paths (relative to this file) that should be
+  # ignored by the migrate tool.
+  #
+  # Files that are not part of the templates will be ignored by default.
+  unmanaged_files:
+    - 'lib/main.dart'
+    - 'ios/Runner.xcodeproj/project.pbxproj'

hil/COVERAGE.md

@@ -0,0 +1,102 @@
+# Hardware-in-the-loop coverage
+
+The Windows HIL suite runs 41 tests against a physical nRF52 peripheral:
+15 baseline tests and 26 fault injection tests (FIT).
+
+The calls start at the public Dart API and go through the operation queue,
+Pigeon channel, Windows C++ plugin, WinRT, Windows Bluetooth stack, radio link,
+and finally the Zephyr GATT server on the fixture.
+
+## 1. Baseline coverage
+
+### 1.1. Scan and discovery (2 tests)
+
+- Verifies filtered discovery, name, service UUID, RSSI, all ten
+  characteristics, and their properties.
+- This catches problems in advertisement parsing, UUID conversion, WinRT
+  discovery, and Dart model mapping.
+
+### 1.2. Reads and writes (3 tests)
+
+- Verifies exact reads up to 200 bytes, 200-byte writes with response, and
+  120-byte writes without response using firmware mirrors and counters.
+- This catches payload corruption, bad length handling, confusion between the
+  two write paths, and incorrect native completion results.
+
+### 1.3. Notifications and indications (4 tests)
+
+- Verifies exact binary delivery, CCC state, unsubscribe/resubscribe, and 40
+  ordered 64-byte notifications.
+- This catches broken event registration, CCC state, byte conversion, packet
+  ordering, loss, and duplication.
+
+### 1.4. Connection lifecycle (3 tests)
+
+- Verifies remote disconnect delivery, reconnect with a successful read, and
+  ten host disconnect/reconnect cycles.
+- This catches incomplete cleanup, bad replacement ownership, missing events,
+  and GATT state that no longer works after reconnecting.
+
+### 1.5. Concurrency and platform results (3 tests)
+
+- Verifies recovery from overlapping subscriptions, a valid MTU result, and
+  `notImplemented` for Windows RSSI without disconnecting.
+- This catches subscription races, invalid MTU completion, incorrect error
+  mapping, and operations that damage an otherwise usable connection.
+
+## 2. Fault injection coverage
+
+### 2.1. Connection lifecycle
+
+| Test           | What it does                                                              | What it catches                                       |
+| -------------- | ------------------------------------------------------------------------- | ----------------------------------------------------- |
+| `FIT-CONN-002` | Immediate peripheral disconnect produces exactly one disconnect event     | Duplicate disconnect callbacks                        |
+| `FIT-CONN-003` | Immediate disconnect is followed by reconnect and a successful read       | Incomplete cleanup of the old connection              |
+| `FIT-CONN-007` | Host and peripheral disconnect concurrently; reconnect remains usable     | Double cleanup when both sides disconnect             |
+| `FIT-CONN-010` | Five immediate peripheral disconnect/reconnect cycles all remain readable | State or resources leaking between connections        |
+| `FIT-CONN-013` | Five rapid cycles emit alternating disconnected/connected events in order | Missing, duplicate, stale, and out-of-order callbacks |
+
+For immediate disconnects, observation starts before the control write. A lost
+write acknowledgement is accepted only when the disconnect occurs; otherwise
+the original write error is rethrown.
+
+### 2.2. Reads
+
+| Test           | What it does                                                                  | What it catches                                             |
+| -------------- | ----------------------------------------------------------------------------- | ----------------------------------------------------------- |
+| `FIT-READ-001` | ATT error `0x0e`; read fails, connection survives, next read succeeds         | WinRT error handling without native abort or poisoned state |
+| `FIT-READ-005` | Zero-length value returns an empty byte array                                 | Empty-buffer conversion                                     |
+| `FIT-READ-006` | Exact reads of 1, 20, 22, 23, 64, 128, and 244 bytes                          | Boundary and large-value handling                           |
+| `FIT-READ-007` | 11-second callback delay exceeds the 10-second timeout; later read succeeds   | Timeout completion, late callbacks, and queue release       |
+| `FIT-READ-008` | Disconnect 50 ms into a delayed read; read fails and succeeds after reconnect | Native state being freed while a read still uses it         |
+| `FIT-READ-011` | Three characteristics are read concurrently with queueing disabled            | Independent concurrent completion routing                   |
+
+### 2.3. Writes with response
+
+| Test            | What it does                                                                                    | What it catches                                                |
+| --------------- | ----------------------------------------------------------------------------------------------- | -------------------------------------------------------------- |
+| `FIT-WRITE-001` | ATT error `0x0e`; write fails, connection survives, next mirrored write succeeds                | Error propagation without corrupting connection or write state |
+| `FIT-WRITE-006` | 11-second callback delay times out; later mirrored write succeeds                               | Timeout completion, late callbacks, and queue release          |
+| `FIT-WRITE-007` | Disconnect precedes acknowledgement of a delayed write; write fails and recovery write succeeds | Native state being freed while a write still uses it           |
+
+### 2.4. Subscriptions
+
+| Test          | What it does                                                                        | What it catches                                  |
+| ------------- | ----------------------------------------------------------------------------------- | ------------------------------------------------ |
+| `FIT-SUB-013` | Subscription is cleared by disconnect; resubscription after reconnect delivers once | Old event-token and cached CCC cleanup           |
+| `FIT-SUB-014` | Five subscribe/unsubscribe cycles followed by one notification produce one callback | Leaked handler and duplicate delivery prevention |
+| `FIT-SUB-015` | Ten rapid subscribe/unsubscribe cycles still allow exact delivery                   | CCC serialization and handler churn recovery     |
+
+### 2.5. Notifications
+
+| Test             | What it does                                                                     | What it catches                                             |
+| ---------------- | -------------------------------------------------------------------------------- | ----------------------------------------------------------- |
+| `FIT-NOTIFY-001` | Notification emitted immediately after CCC enable is received exactly            | First-value loss at the enable boundary                     |
+| `FIT-NOTIFY-002` | Emission after CCC disable fails and no value reaches the stream                 | Disable completion and handler removal                      |
+| `FIT-NOTIFY-003` | Fixture emits `0, 1, 3, 4`; host receives exactly that gap                       | Transparent delivery without synthesized data               |
+| `FIT-NOTIFY-004` | Fixture emits `0, 1, 1, 2`; host receives both duplicate-numbered packets        | No application-level deduplication                          |
+| `FIT-NOTIFY-005` | Fixture emits `0, 2, 1, 3`; host preserves that order                            | No host-side payload reordering                             |
+| `FIT-NOTIFY-006` | Alternating 1-byte and 244-byte values arrive intact and in order                | Buffer resizing, ownership, and binary integrity            |
+| `FIT-NOTIFY-008` | Disconnect interrupts a 100-packet burst; partial delivery and reconnect succeed | Callback teardown during active delivery                    |
+| `FIT-NOTIFY-010` | Reconnect and resubscribe produce a clean `0, 1, 2, 3, 4` sequence               | Stale callback/value rejection and new handler installation |
+| `FIT-NOTIFY-012` | Host disconnects during a 500-packet burst and reconnects successfully           | Crashes, hangs, or use-after-free with queued callbacks     |

hil/README.md

@@ -0,0 +1,107 @@
+# Hardware-in-the-loop tests
+
+This is a standalone Flutter application for testing `universal_ble` against
+a physical nRF52 DK. It lives outside the normal package tests because it
+needs the DK and exclusive access to a Bluetooth adapter.
+
+The firmware is in the [`universal_ble_hil_firmware`](https://github.com/usmanmehmood55/universal_ble_hil_firmware)
+repository. Test commands go over BLE. USB serial is only used for Zephyr logs.
+
+## 1. Current coverage
+
+The Windows suite contains 41 implemented hardware tests:
+
+- 15 baseline tests for ordinary BLE behavior;
+- 26 fault injection tests for hostile peripheral behavior, timing, and
+  lifecycle races.
+
+[`COVERAGE.md`](COVERAGE.md) lists every implemented test, what it does, and
+the bugs it is meant to catch.
+
+[`lib/main.dart`](lib/main.dart) also provides an interactive Web Bluetooth
+runner for the portable baseline path.
+
+## 2. How fault injection tests work
+
+The test first writes a fault plan to the fixture's control characteristic.
+The firmware keeps that plan and applies it when the next matching GATT
+operation arrives.
+
+For example, a disconnect-during-read test performs these steps:
+
+1. Write a one-shot read plan containing a callback delay and disconnect delay.
+2. Start a normal read through `universal_ble`.
+3. Let the firmware disconnect while its read callback is pending.
+4. Assert bounded failure, reconnect, and a successful recovery read.
+
+The timing happens on the nRF52, where it is predictable. A reset clears fault
+plans, scheduled disconnects, bursts, values, and counters. It leaves the
+active connection and CCC state alone.
+
+## 3. Execution boundary
+
+The automated suite currently runs on Windows. Calls start at the public Dart
+API and go through Pigeon, the Windows C++ plugin, WinRT, the Bluetooth stack,
+the radio link, and finally the Zephyr GATT server.
+
+Fixture control also uses BLE, so faults are armed while the device is
+connected. The firmware then runs them locally using valid GATT behavior.
+
+## 4. Running the tests
+
+Flash the matching firmware and confirm that `UniversalBLE-HIL` is
+advertising. From this directory, run:
+
+```powershell
+flutter pub get
+flutter test integration_test/baseline_hil_test.dart -d windows
+flutter test integration_test/fault_injection_hil_test.dart -d windows
+```
+
+If you already know the Windows device ID, you can skip the initial scan:
+
+```powershell
+flutter test integration_test/baseline_hil_test.dart -d windows `
+  --dart-define=HIL_DEVICE_ID=AA:BB:CC:DD:EE:FF
+```
+
+For a longer reconnect test:
+
+```powershell
+flutter test integration_test/baseline_hil_test.dart -d windows `
+  --dart-define=HIL_RECONNECT_CYCLES=100
+```
+
+Run the Web baseline with:
+
+```powershell
+flutter run -d chrome
+```
+
+Web Bluetooth requires a secure context, a real user gesture, and browser
+device permission. Click **Select device and run**, then select
+`UniversalBLE-HIL` in Chrome's chooser.
+
+## 5. Isolation and diagnosis
+
+Each Windows test connects, resets the fixture, runs one case, and disconnects.
+The fixture only accepts one connection, so run the tests serially.
+
+A completed Dart call does not prove that the peripheral saw the same thing.
+Writes are therefore checked through read-only mirrors, subscriptions through
+fixture state, and notification bursts through sequence numbers.
+
+When a test fails, collect both outputs:
+
+- Flutter integration-test output and `universal_ble` debug logs;
+- Zephyr serial logs from the nRF52 DK.
+
+If setup cannot find the fixture, confirm that it is advertising and that no
+other host holds its single connection slot.
+
+## 6. Execution policy
+
+These tests do not run in normal pull-request CI. Run them manually when a
+change touches native BLE lifecycle, GATT operations, subscriptions, queueing,
+or the platform channel. A native crash can kill the test runner. Use
+`--plain-name` to rerun the failing case on its own.

hil/analysis_options.yaml

@@ -0,0 +1 @@
+include: package:flutter_lints/flutter.yaml

hil/android/.gitignore

@@ -0,0 +1,14 @@
+gradle-wrapper.jar
+/.gradle
+/captures/
+/gradlew
+/gradlew.bat
+/local.properties
+GeneratedPluginRegistrant.java
+.cxx/
+
+# Remember to never publicly share your keystore.
+# See https://flutter.dev/to/reference-keystore
+key.properties
+**/*.keystore
+**/*.jks