-
Notifications
You must be signed in to change notification settings - Fork 105
WIP: [SRU Exception] Add nVidia DOCA-OFED DKMS packaging #564
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Alessio-Faina
wants to merge
12
commits into
ubuntu:main
Choose a base branch
from
Alessio-Faina:afaina_doca_ofed
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
6fc84e5
Initial draft
Alessio-Faina 1ed2fd7
Add more info about QA and packaging
Alessio-Faina b80048f
Add suggestions from Jose
Alessio-Faina bf4c0db
Add documentation regarding hardware support
Alessio-Faina 726744e
Add documentation about committment to maintain use cases
Alessio-Faina d050fee
Fix links
Alessio-Faina 9b066fe
Fix LTS length
Alessio-Faina 7884216
Add info about SRU exceptions and tests
Alessio-Faina 4af7aaa
Clarify SRU scope
Alessio-Faina 44bc9b3
Fix requested by MitchellAugustin
Alessio-Faina 4173bd3
Add LTS -> LTS transition explanation
Alessio-Faina 4e85a65
Remove the repackaging section as deemed not necessary
Alessio-Faina File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,194 @@ | ||
| .. _reference-exception-NVIDIADocaOfed: | ||
|
|
||
| NVIDIA DOCA-OFED Driver Updates | ||
| =============================== | ||
|
|
||
| Introduction | ||
| ------------ | ||
|
|
||
| This document describes the policy, process and criteria for updating | ||
| NVIDIA proprietary DPU drivers (DOCA-OFED) in a stable supported distro, | ||
| including LTS. | ||
|
|
||
| NVIDIA DOCA-OFED drivers are used in conjunction with NVIDIA Bluefield cards, | ||
| DGX and IGX platforms and Connect-X cards. These platforms are used in a variety | ||
| of scenarios, included data-centers, AI/ML ... | ||
|
|
||
| Support for new networking hardware, platforms, bug fixes and general | ||
| improvements are released regularly by NVIDIA during the year. | ||
|
|
||
| For an improved experience and support new features/hardware without installing | ||
| drivers from an unofficial or an external source (PPAs/FTP...), it is | ||
| essential to release the latest version of the drivers regularly to the new | ||
| stable releases of Ubuntu. | ||
|
|
||
|
|
||
| .. _what_kind_of_updates: | ||
|
|
||
| What kind of updates | ||
| -------------------- | ||
|
|
||
| There are 4 expected releases during the year, where YY stands for the year | ||
| of release (i.e. YY = 26): | ||
|
|
||
| - **Long term support releases**: The YY.10 version is the sole LTS release of | ||
| the year; this release is designed exclusively for current upstream | ||
| kernels and will receive kernel compatibility updates for a year only. | ||
| This release only receives bug fixes and security updates. | ||
| - **Short lived releases**: YY.01, YY.04, YY.07 are released in their respective | ||
| months by NVIDIA, and are considered ad-interim releases, to be substituted | ||
| with the newer one and finish their path into the LTS release. | ||
| These releases receive bug fixes and security updates, but also new features. | ||
|
|
||
| The release targeted by the SRU are: | ||
|
|
||
| - **LTS** and **Non-LTS** | ||
|
|
||
| - Long term support releases | ||
| - Short lived releases | ||
|
|
||
| As NVIDIA uses a 2 different versioning schemes, the version number of the | ||
| driver is not directly related to the version number of the release, but it's | ||
| a semver; as an example 26.01 is uploaded in the archive as DOCA 3.3.x and | ||
| 26.04 as 3.4.x; Canonical will use only the first two digits of the semver | ||
| to create a package (e.g. doca-ofed-3.4-dkms); the third digit will be used | ||
| to upgrade an existing package (for bug-fixes etc...) | ||
|
|
||
| When a new short lived release is available and uploaded, meta-packages will | ||
| make the user update to the newer release. This happens until the new release | ||
| is an LTS one. | ||
|
|
||
| For example if the non-LTS DOCA version 3.3.0 is in the Ubuntu LTS release, | ||
| and 3.4.0 is the next DOCA LTS: | ||
|
|
||
| - Version 3.3.1 is uploaded to the Ubuntu LTS, users will be upgraded. | ||
| - Version 3.4.0 is uploaded to the Ubuntu LTS, a transitional metapackage is created to upgrade the user to the latest version. | ||
| - Version 4.0.0 (DOCA GA) is uploaded to the Ubuntu LTS, user will not be upgraded and no transitional metapackage will be created. | ||
| - Version 4.4.0 (next DOCA LTS) is uploaded to the Ubuntu LTS. Once the previous DOCA GA and LTS versions reach end of support upstream, a transitional metapackage is created to upgrade users from those previous GA versions and from the previous LTS (3.4.0) to the latest version. | ||
|
|
||
| The last statement could change in the future depending on discussions with | ||
| NVIDIA and the community, but the idea is to not force users to upgrade to a | ||
| new release that might drop support for old hardware automatically. | ||
|
|
||
| .. _doca-ofed_release_process: | ||
|
|
||
| DOCA-OFED Release Process | ||
| -------------------------- | ||
|
|
||
| - New version of the driver released by NVIDIA | ||
| - Packaging in Canonical git repository | ||
| - Internal release in personal PPA | ||
| - QA (smoke test, installs, operational tests, upgrade path test) | ||
| - Package ready for release, upload to archive (-proposed pocket) | ||
| - Promotion to the -updates pocket | ||
| - Prebuilt DKMS uploaded to the archive for a set of selected kernels | ||
|
|
||
|
|
||
| .. _doca-ofed_sru_process: | ||
|
|
||
| DOCA-OFED SRU Process | ||
| --------------------- | ||
|
|
||
| The SRU should be done with a single process bug and not have multiple bug | ||
| reports for each bug fix. The LP bug should contain the following: | ||
|
|
||
| - The template at the end of this document should be used and all 'TODO' | ||
| filled out. | ||
| - The deb changelog will contain a single reference to the SRU LP bug only. | ||
| - Major changes must be listed in the SRU template, especially compatibility | ||
| issues. | ||
| - Results of all the conducted tests should be listed, even as a comment, | ||
| in the SRU bug. Any manual test or additional steps taken to perform | ||
| testing should be documented. | ||
| - Any packaging changes need to be stated. | ||
|
|
||
|
|
||
| .. _doca_ofed__qa_tests: | ||
|
|
||
| QA tests | ||
| -------- | ||
|
|
||
| The QA test consists of running the mentioned | ||
|
Alessio-Faina marked this conversation as resolved.
|
||
| `linux-modules-doca-ofed-<version>-<kernel_flavour>.deb` package in an IGX and | ||
| a DGX machine, containing an NVIDIA ConnectX card. | ||
|
|
||
| The test confirms that: | ||
|
|
||
| - The kernel modules can be correctly loaded | ||
| - Using ibstatus all the Infiniband interfaces are available | ||
| - Using ib_write tests a loopback Infiniband fully works | ||
| - Start the NVIDIA fabric manager and confirm it correctly boots | ||
|
|
||
| The sequence of tests are manually run by an operator, and are | ||
| described in more details | ||
| `here <https://canonical-kteam-docs.readthedocs-hosted.com/latest/reference/nvidia/mofed/#testing-doca-ofed>`__ | ||
|
|
||
|
|
||
| .. _nvidia_doca_ofed_sru_template: | ||
|
|
||
| NVIDIA DOCA-OFED SRU Template | ||
| ----------------------------- | ||
|
|
||
| :: | ||
|
|
||
| [Impact] | ||
| This release provides both bug fixes and new features and we would like to | ||
| make sure all of our users have access to these improvements. | ||
| The notable ones are: | ||
|
|
||
| *** <TODO: Create list with LP: # included > | ||
|
|
||
| See the changelog entry below for a full list of changes and bugs. | ||
|
|
||
| [Test Case] | ||
| The following development and SRU process was followed: | ||
| https://documentation.ubuntu.com/sru/en/latest/reference/exception-NVIDIA-doca-ofed/ | ||
|
|
||
| <TODO Document any QA done, automated and manual> | ||
|
|
||
| The QA team that executed the tests will be in charge of attaching the artifacts and console output of the appropriate run to the bug. NVIDIA maintainers team members will not mark ‘verification-done’ until this has happened. | ||
|
|
||
| [Regression Potential] | ||
| In order to mitigate the regression potential, the results of the | ||
| aforementioned system level tests are attached to this bug. | ||
|
|
||
| <TODO: attach NVIDIA-proposed test artifacts for every SRU release, not a link as links expire> | ||
|
|
||
|
|
||
| [Discussion] | ||
| <TODO: other background> | ||
|
|
||
|
|
||
| <TODO: Paste in change log entry from NVIDIA for this version of the driver> | ||
|
|
||
| Additional notes | ||
| ---------------- | ||
|
|
||
| NVIDIA guarantees perpetual and backward binary compatibility, | ||
| as stated in their | ||
| public document `DOCA dependency compatibility policy <https://docs.nvidia.com/doca/sdk/doca-dependency-compatibility-policy/index.html>`__: | ||
| the document states the following key objectives for new releases: | ||
|
|
||
| - Decoupled updates: Supports environments where distinct teams manage separate parts of the software stack (e.g., firmware vs. user-space applications). | ||
| - Targeted upgrades: Enables upgrades targeted at specific subsets of devices within a server. | ||
| - Reduced maintenance overhead: Minimizes downtime and complexity by limiting updates to necessary components only. | ||
| - Operational continuity: Guarantees that customer applications continue to operate successfully after updating to a newer software platform release. | ||
|
|
||
|
|
||
| NVIDIA guarantees in the provided documentation that hardware will be retired | ||
| only in new GA versions, as stated in their `Customer affecting changes <https://docs.nvidia.com/doca/sdk/customer-affecting-changes/index.html>`__ | ||
| policy and | ||
| `DOCA compatibility policy <https://docs.nvidia.com/doca/sdk/doca-compatibility-policy/index.html>`__ | ||
| documents; this means by staying on a LTS release, customers will not lose | ||
| compatibility with old hardware. | ||
|
|
||
| References | ||
| ---------- | ||
|
|
||
| - `NVIDIA DOCA-OFED official download page <https://developer.nvidia.com/doca-downloads>`__ | ||
| - `NVIDIA DOCA-OFED official documentation <https://docs.nvidia.com/doca/sdk/index.html>`__ | ||
| - `NVIDIA DOCA-OFED release notes <https://docs.nvidia.com/doca/sdk/doca-release-notes/index.html>`__ | ||
| - `Repackaged test PPA <https://launchpad.net/~alessiofaina/+archive/ubuntu/mofed-autoupload-test>`__ | ||
| - `NVIDIA Dependency compatibility policy <https://docs.nvidia.com/doca/sdk/doca-dependency-compatibility-policy/index.html>`__ | ||
| - `NVIDIA DOCA compatibility policy <https://docs.nvidia.com/doca/sdk/doca-compatibility-policy/index.html>`__ | ||
| - `NVIDIA Customer affecting changes <https://docs.nvidia.com/doca/sdk/customer-affecting-changes/index.html>`__ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.