Skip to content
Open
194 changes: 194 additions & 0 deletions docs/SRU/reference/exception-NVidia-doca-ofed.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,194 @@
.. _reference-exception-NVIDIADocaOfed:

NVIDIA DOCA-OFED Driver Updates
===============================

Introduction
------------

This document describes the policy, process and criteria for updating
NVIDIA proprietary DPU drivers (DOCA-OFED) in a stable supported distro,
including LTS.

NVIDIA DOCA-OFED drivers are used in conjunction with NVIDIA Bluefield cards,
DGX and IGX platforms and Connect-X cards. These platforms are used in a variety
of scenarios, included data-centers, AI/ML ...

Support for new networking hardware, platforms, bug fixes and general
improvements are released regularly by NVIDIA during the year.

For an improved experience and support new features/hardware without installing
drivers from an unofficial or an external source (PPAs/FTP...), it is
essential to release the latest version of the drivers regularly to the new
stable releases of Ubuntu.


.. _what_kind_of_updates:

What kind of updates
--------------------

There are 4 expected releases during the year, where YY stands for the year
of release (i.e. YY = 26):

- **Long term support releases**: The YY.10 version is the sole LTS release of
the year; this release is designed exclusively for current upstream
kernels and will receive kernel compatibility updates for a year only.
This release only receives bug fixes and security updates.
- **Short lived releases**: YY.01, YY.04, YY.07 are released in their respective
months by NVIDIA, and are considered ad-interim releases, to be substituted
with the newer one and finish their path into the LTS release.
These releases receive bug fixes and security updates, but also new features.

The release targeted by the SRU are:

- **LTS** and **Non-LTS**

- Long term support releases
- Short lived releases

As NVIDIA uses a 2 different versioning schemes, the version number of the
driver is not directly related to the version number of the release, but it's
a semver; as an example 26.01 is uploaded in the archive as DOCA 3.3.x and
26.04 as 3.4.x; Canonical will use only the first two digits of the semver
to create a package (e.g. doca-ofed-3.4-dkms); the third digit will be used
to upgrade an existing package (for bug-fixes etc...)

When a new short lived release is available and uploaded, meta-packages will
make the user update to the newer release. This happens until the new release
is an LTS one.

For example if the non-LTS DOCA version 3.3.0 is in the Ubuntu LTS release,
and 3.4.0 is the next DOCA LTS:

Comment thread
Alessio-Faina marked this conversation as resolved.
- Version 3.3.1 is uploaded to the Ubuntu LTS, users will be upgraded.
- Version 3.4.0 is uploaded to the Ubuntu LTS, a transitional metapackage is created to upgrade the user to the latest version.
- Version 4.0.0 (DOCA GA) is uploaded to the Ubuntu LTS, user will not be upgraded and no transitional metapackage will be created.
- Version 4.4.0 (next DOCA LTS) is uploaded to the Ubuntu LTS. Once the previous DOCA GA and LTS versions reach end of support upstream, a transitional metapackage is created to upgrade users from those previous GA versions and from the previous LTS (3.4.0) to the latest version.

The last statement could change in the future depending on discussions with
NVIDIA and the community, but the idea is to not force users to upgrade to a
new release that might drop support for old hardware automatically.

.. _doca-ofed_release_process:

DOCA-OFED Release Process
--------------------------

- New version of the driver released by NVIDIA
- Packaging in Canonical git repository
- Internal release in personal PPA
- QA (smoke test, installs, operational tests, upgrade path test)
- Package ready for release, upload to archive (-proposed pocket)
- Promotion to the -updates pocket
- Prebuilt DKMS uploaded to the archive for a set of selected kernels


.. _doca-ofed_sru_process:

DOCA-OFED SRU Process
---------------------

The SRU should be done with a single process bug and not have multiple bug
reports for each bug fix. The LP bug should contain the following:

- The template at the end of this document should be used and all 'TODO'
filled out.
- The deb changelog will contain a single reference to the SRU LP bug only.
- Major changes must be listed in the SRU template, especially compatibility
issues.
- Results of all the conducted tests should be listed, even as a comment,
in the SRU bug. Any manual test or additional steps taken to perform
testing should be documented.
- Any packaging changes need to be stated.


.. _doca_ofed__qa_tests:

QA tests
--------

The QA test consists of running the mentioned
Comment thread
Alessio-Faina marked this conversation as resolved.
`linux-modules-doca-ofed-<version>-<kernel_flavour>.deb` package in an IGX and
a DGX machine, containing an NVIDIA ConnectX card.

The test confirms that:

- The kernel modules can be correctly loaded
- Using ibstatus all the Infiniband interfaces are available
- Using ib_write tests a loopback Infiniband fully works
- Start the NVIDIA fabric manager and confirm it correctly boots

The sequence of tests are manually run by an operator, and are
described in more details
`here <https://canonical-kteam-docs.readthedocs-hosted.com/latest/reference/nvidia/mofed/#testing-doca-ofed>`__


.. _nvidia_doca_ofed_sru_template:

NVIDIA DOCA-OFED SRU Template
-----------------------------

::

[Impact]
This release provides both bug fixes and new features and we would like to
make sure all of our users have access to these improvements.
The notable ones are:

*** <TODO: Create list with LP: # included >

See the changelog entry below for a full list of changes and bugs.

[Test Case]
The following development and SRU process was followed:
https://documentation.ubuntu.com/sru/en/latest/reference/exception-NVIDIA-doca-ofed/

<TODO Document any QA done, automated and manual>

The QA team that executed the tests will be in charge of attaching the artifacts and console output of the appropriate run to the bug. NVIDIA maintainers team members will not mark ‘verification-done’ until this has happened.

[Regression Potential]
In order to mitigate the regression potential, the results of the
aforementioned system level tests are attached to this bug.

<TODO: attach NVIDIA-proposed test artifacts for every SRU release, not a link as links expire>


[Discussion]
<TODO: other background>


<TODO: Paste in change log entry from NVIDIA for this version of the driver>

Additional notes
----------------

NVIDIA guarantees perpetual and backward binary compatibility,
as stated in their
public document `DOCA dependency compatibility policy <https://docs.nvidia.com/doca/sdk/doca-dependency-compatibility-policy/index.html>`__:
the document states the following key objectives for new releases:

- Decoupled updates: Supports environments where distinct teams manage separate parts of the software stack (e.g., firmware vs. user-space applications).
- Targeted upgrades: Enables upgrades targeted at specific subsets of devices within a server.
- Reduced maintenance overhead: Minimizes downtime and complexity by limiting updates to necessary components only.
- Operational continuity: Guarantees that customer applications continue to operate successfully after updating to a newer software platform release.


NVIDIA guarantees in the provided documentation that hardware will be retired
only in new GA versions, as stated in their `Customer affecting changes <https://docs.nvidia.com/doca/sdk/customer-affecting-changes/index.html>`__
policy and
`DOCA compatibility policy <https://docs.nvidia.com/doca/sdk/doca-compatibility-policy/index.html>`__
documents; this means by staying on a LTS release, customers will not lose
compatibility with old hardware.

References
----------

- `NVIDIA DOCA-OFED official download page <https://developer.nvidia.com/doca-downloads>`__
- `NVIDIA DOCA-OFED official documentation <https://docs.nvidia.com/doca/sdk/index.html>`__
- `NVIDIA DOCA-OFED release notes <https://docs.nvidia.com/doca/sdk/doca-release-notes/index.html>`__
- `Repackaged test PPA <https://launchpad.net/~alessiofaina/+archive/ubuntu/mofed-autoupload-test>`__
- `NVIDIA Dependency compatibility policy <https://docs.nvidia.com/doca/sdk/doca-dependency-compatibility-policy/index.html>`__
- `NVIDIA DOCA compatibility policy <https://docs.nvidia.com/doca/sdk/doca-compatibility-policy/index.html>`__
- `NVIDIA Customer affecting changes <https://docs.nvidia.com/doca/sdk/customer-affecting-changes/index.html>`__
1 change: 1 addition & 0 deletions docs/SRU/reference/package-specific.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ the Technical Board.
exception-MariaDB-Galera-Updates
exception-multipath-tools-Updates
exception-NVidia-Updates
exception-NVidia-doca-ofed
exception-Netplan-Updates
exception-OEMMeta-Updates
exception-OpenJDK-Updates
Expand Down