Restructuring website navigation and content - #5023
Open
sarhiri wants to merge 65 commits into
Open
Conversation
|
Contributor
Greptile SummaryThis PR restructures the Armada website content and navigation.
Confidence Score: 5/5This looks safe to merge.
|
| Filename | Overview |
|---|---|
| website/content/docs/user-guide.mdx | Adds the new user guide landing page with cards pointing at the flattened docs routes. |
| website/content/contribute/contributor-guide.mdx | Updates contribution links to use public extensionless routes. |
| website/content/contribute/community.mdx | Adds the new community page under the contribute section. |
| website/content/contribute/index.mdx | Updates the Community card to the new contribute route. |
| website/content/getting-started.mdx | Updates the Community card to the new contribute route. |
| website/content/meta.json | Simplifies the top-level website navigation. |
| website/content/docs/meta.json | Adds ordering for the new docs section. |
| website/.remarkrc-mdx.mjs | Skips route-shaped absolute links during MDX link validation. |
| website/.remarkrc-md.mjs | Skips route-shaped absolute links during Markdown link validation. |
Reviews (22): Last reviewed commit: "Merge branch 'master' into website-nav-c..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
sarhiri
force-pushed
the
website-nav-cleanup
branch
from
July 17, 2026 20:12
0adc2ea to
1e16844
Compare
built on Fumadocs components and custom JSX, keeping the existing sidebar and on-this-page TOC. Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
…l on design and content Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
…w for PR. Readme also updated Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
- Rename developer-guide to Local Development Setup - Remove deprecated sections per Dejan review: VS Code debugging, Delve, extending Armada, UI development, mage localdev - Keep and expand Goreman setup as the recommended approach - Add mage dev:up, dev:full, dev:down commands - Add Steps components for Goreman and auth setup flows - Add fake executor section for Kubernetes-free testing - Add debug port mappings table - Add troubleshooting section for port 6443 and Arm/M1 Mac issues - Remove operator-guide.mdx deprecated per Dejan - Remove orphaned redirect shims pointing to operator-guide - Add CTA block matching site-wide style Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
`navigator.clipboard.writeText()` in `CopyIconButton` was called without awaiting or catching its returned promise. When the document isn't focused during the click, the API rejects with `NotAllowedError: Document is not focused`. This PR fixes it by wrapping the call in a try-catch and surfacing the error via a snackbar. --------- Signed-off-by: Maurice Yap <mauriceyap@hotmail.co.uk> Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
…r to a functional component (armadaproject#4987) Our monitoring picked up a runtime error on the Job Sets page: > UnhandledRejection: Non-Error promise rejection captured with value: TypeError: Failed to fetch `JobSetsContainer` was a class component that fetched data via an async `loadJobSets()` method. That method had no error handling and was wired into `setInterval` (auto-refresh), a refresh button, and a result callback. None of these handle a rejected promise. When a fetch failed at the network level, the rejection went unhandled. This PR migrates `JobSetsContainer` from a class component to a functional component using hooks and TanStack Query. TanStack Query now owns the fetch promise lifecycle, so an app-managed promise can no longer reject unhandled, and loading/error state is reactive. User-facing behaviour (queue selection, ordering, active-only filter, auto-refresh, cancel/reprioritize dialogs) is unchanged. --------- Signed-off-by: Maurice Yap <mauriceyap@hotmail.co.uk> Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
<!-- Thanks for sending a pull request! Here are some tips for you: --> #### What type of PR is this? Enhancement #### What this PR does / why we need it Introuce base implementation for opentelemetry integration in Armada Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
#### What type of PR is this? Feature #### What this PR does / why we need it Adds two new `PoolConfig` flags, `DisableFairshareScheduling` and `DisableUrgencyScheduling`, that let an operator turn off either strategy per pool. In `NodeDb.selectNodeForJobWithTxnAtPriority` each pass is now guarded by its flag, so a pool can run with one, both, or neither strategy enabled. As part of this, the per-setting `NodeDb` mutators (`DisableAwayScheduling`/`EnableAwayScheduling`, `DisableHomeScheduling`, `DisableGangAwayScheduling`, `SetDisallowedJobResources`, and their `Enable*` counterparts) are replaced by a single `ConfigureScheduling(SchedulingOptions)` call. The scheduling algo and submitcheck now configure the `NodeDb` in one shot, which removes the previous enable/disable toggling and the need to reset state between pools. #### Special notes for your reviewer - `ConfigureScheduling` sets every field from the options struct on each call, so it implicitly resets state between pools. This replaces the explicit `Enable*`/`Disable*` toggling that `submitcheck.go` did per synthetic pool - worth confirming that "set everything every time" is the intended semantics there. - Both flags default to `false`, so existing deployments keep both strategies enabled with no config change. --------- Signed-off-by: Trey Guckian <24757349+tgucks@users.noreply.github.com> Co-authored-by: JamesMurkin <jamesmurkin@hotmail.com> Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
#### What type of PR is this? Feature/refactor #### What this PR does / why we need it Extracts short-job-penalty tracking out of the scheduling hot path and the jobDb retention logic into a dedicated, self-contained ShortJobPenalty service. Previously the penalty was recomputed every scheduling cycle by scanning all terminal jobs held in the jobDb. To make those jobs available for the scan, terminal short jobs were deliberately kept in the jobDb while their penalty was active. This required an occasional full GC to clean them up periodically. This PR makes ShortJobPenalty own its own state: - Terminal jobs are reported to the service once each via `ReportFinishedJob` at each point where a job can go terminal. This records the job's resources keyed by (pool, queue). - Penalties are snapshotted once per scheduling cycle via `Snapshot()` and read back per-pool from that immutable `ShortJobPenaltySnapshot` via `GetPenaltiesForPool`, replacing the inline per-job `ShouldApplyPenalty` accumulation in `calculateJobSchedulingInfo.` - Entries expire automatically via a deadline-ordered min-heap (`runStart + cutoff[pool]`), with a derived per-(pool, queue) running total cache kept in sync as entries are added and expired. Access is guarded by a mutex. This removes the need for the periodic full GC of terminal jobs from the jobDb. They're deleted as they go terminal now. #### Special notes for your reviewer - `ShortJobPenalty` is now stateful and should be concurrency-safe (sync.Mutex); the entries are the source of truth and sums is a derived cache. A `Snapshot()` taken once per cycle gives every pool a consistent point-in-time view. - `syncState`'s signature changed (dropped the `fullJobGc` bool); call sites in cycle and initialize were updated accordingly. - Tests in short_job_penalty_test.go were substantially expanded to cover reporting, dedup, expiry, and per-pool reads. --------- Signed-off-by: Trey Guckian <24757349+tgucks@users.noreply.github.com> Co-authored-by: JamesMurkin <jamesmurkin@hotmail.com> Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
<!-- Thanks for sending a pull request! Here are some tips for you: --> #### What type of PR is this? Enhancement #### What this PR does / why we need it Wire the observability packages to each service #### Special notes for your reviewer Depends on armadaproject#4975 --------- Co-authored-by: JamesMurkin <jamesmurkin@hotmail.com> Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Main changes:
- Scheduling cycles are now triggered on a regular cadence where
possible
- Previously schedulePeriod would determine how long we'd wait between
cycles. Measured since last cycle finished
- The issue with this is for long cycles, ideally we'd trigger again
immediately, instead we leave a gap.
- This was slowing scheduling down when it already slow
- Now we trigger the next cycle based on when the last cycle started
- For quick cycles, we'll still get a gap between them
- For long cycles we'll trigger again immediately
- Make the main loop check for an async scheduling result every cycle
- This makes us more responsive to async scheduler results
- Make UpdateJobPrices be called every cycle
- Update job prices has now been made much more efficient (most calls
are a noop)
- The ensures we keep as up to date as we can, while also simplifying
logic of when to call it
---------
Signed-off-by: JamesMurkin <jamesmurkin@hotmail.com>
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
Adding local development configurations to run the Lookout hot/cold partitioned database in parallel with the existing Lookout database. This includes the VSCode configurations in the `.vscode` directory, the goreman procfiles and config files under `_local`, and the JetBrains configurations under `.run` --------- Signed-off-by: David Slear <david_slear@yahoo.com> Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
…tal hot/cold) (armadaproject#4996) In the experimental hot-cold schema, the `job` table is LIST-partitioned on state with a composite primary key `(job_id, state)`, so `job_id` uniqueness is only enforced within a single partition. `CreateJobs` used an untargeted `ON CONFLICT DO NOTHING`, which only checks the partition the insert routes to. When a row for the same `job_id` already existed elsewhere, the create was not suppressed and a second row was written. This was observed both across partitions (an active create alongside an existing terminal row) and within the active partition (a Queued create alongside an existing Leased row). A later update that moved a row into the terminated partition then collided with `job_terminated_pkey` (which the ingester classified as retryable and retried to exhaustion). This commit guards both create paths with `WHERE NOT EXISTS` against the parent job table so a duplicate `job_id` is never written regardless of which partition the existing row lives in. This leaves the high-frequency update path untouched and is a no-op on the non-partitioned schema, where `job_id` is already globally unique. Signed-off-by: Maurice Yap <mauriceyap@hotmail.co.uk> Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
…s, and fix localdev doc drift (armadaproject#4992) `mage dev:up` looks broken on older mage binaries. The target takes an optional `-dap` flag through a pointer argument (`dap *bool`), and optional arguments only exist since mage v1.16. An older mage (for example the 1.15.0 that Homebrew shipped through 2023) does not reject the signature. It silently skips the target during parsing, with a message that only appears under `mage -debug`. The result is that `dev:up` vanishes from `mage -l` and every documented command fails with `Unknown target specified: "dev:up"`, which reads like a bug in the magefile rather than a stale local tool. The magefile cannot intercept that failure, because an old mage never registers the target in the first place. So this PR adds three layers instead: - The mage pin is bumped from v1.15.0 to v1.17.2 everywhere it exists: in go.mod (the magefiles import the mage library) and in the 21 CI workflow invocations of `go run github.com/magefile/mage@<version>`. Anyone without a new enough mage binary can use that same form: `go run github.com/magefile/mage@v1.17.2 <target>`. - `mage checkDeps` now checks the mage binary on PATH against `>= 1.16.0`, using the same semver-constraint helper as the other tool checks. Verified that checkDeps itself still runs under mage 1.15 (it takes no optional arguments), so it can explain the problem on exactly the machines that have it. - The localdev docs say what the `Unknown target` failure means and point at both remedies. Two small fixes in dev.go: the `no-auth` token is now treated as the default profile instead of being forwarded to docker compose as a bogus `--profile no-auth` flag (the VS Code tasks pass `no-auth` explicitly), and the doc comment examples now match the real CLI (the stale `-debug` becomes `-dap`, and `mage dev:up no-auth` replaces `mage dev:up ""`). The localdev documentation is also restructured. The main README's Local Development section is now a short quick start (the four `mage dev:*` commands) plus links onward. The full reference (how `dev:up` works, profile tokens, the auth profile details, the fake executor, compose profiles, procfiles, and the service ports table) moved into the local dev stack section of `docs/developer_guide.md`, mostly verbatim; nothing was dropped. A new `_local/README.md` describes the directory layout: what each subdirectory and file in `_local` is for. Doc corrections made while cross-referencing against the magefile (now mostly living in `docs/developer_guide.md`): - All `dev:up` examples include the required profile argument (`mage dev:up no-auth`); the previously documented bare `mage dev:up` never worked on any mage version because mage has no optional positional arguments. - Ports: the Server Metrics row said 9000 but `_local/server/config.yaml` sets 9009. Added rows for Scheduler HTTP (8080), Lookout API (8089), Pulsar admin (8090), Keycloak (8180, `auth` profile) and Prometheus (9090, `prometheus` profile). - The compose profiles table lists the `prometheus` profile, the procfiles section documents the `-dap` variants, and the `dev:deps`, `dev:migrate`, `dev:full` and `dev:fullDown` targets are covered. - `docs/developer_guide.md`: the "Setting up the local dev stack" section described the old localdev flow (bootstrapping tools.yaml, creating a kind cluster), which `dev:up` does not do. It now describes the goreman and containerized flows accurately and links to `_local/README.md`. The UI section says which port belongs to which flow: 3000 for the Vite dev server under goreman, 8089 for lookout under `dev:full`. The debugging section documents the `-dap` flag, and the prerequisites note the mage version and the `go run github.com/magefile/mage@v1.17.2` alternative. - The default and auth profiles run a real executor, which needs a Kubernetes cluster, but no doc said so: only the VS Code tasks knew to run `mage kind` first, and without `KUBECONFIG` the executor falls back to the default kubeconfig and connects to whatever cluster that selects. The README quick start and the developer guide now say to run `mage kind` and set `KUBECONFIG=.kube/external/config`. - The guide's "Testing if the local dev stack is working" and "Running the UI" sections were nested under the "port 6443" troubleshooting section, where a newcomer would not look. They are now subsections of "Setting up the local dev stack", and the guide's table of contents is regenerated (it listed a section that no longer exists and missed the current ones). - `_local/compose/stack.yaml`: the header comment said container-mode profiles were "intentionally not in this PR" with a follow-up planned. That follow-up shipped as `full.yaml`, so the comment now points there, and the profile and image override lists include prometheus. Removed: - README ports table rows for OTEL Collector gRPC (4317), OTEL Collector HTTP (4318) and Jaeger UI (16686). No otel-collector or jaeger service exists anywhere in `_local`, so the rows described services you cannot reach. - The "Choosing components to run" section in `docs/developer_guide.md`. `ARMADA_COMPONENTS` has no remaining references in the codebase. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Dejan Zele Pejchev <pejcev.dejan@gmail.com> Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
sarhiri
marked this pull request as draft
July 30, 2026 16:41
sarhiri
marked this pull request as ready for review
July 30, 2026 16:51
sarhiri
marked this pull request as draft
July 30, 2026 17:14
Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
sarhiri
marked this pull request as ready for review
July 30, 2026 17:49
…rn and greptile were arguing about static vs absolute routing paths, and by adding remark-validate-links to stop checking routes at the base level, but they still run during the build script Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
sarhiri
commented
Jul 30, 2026
sarhiri
commented
Jul 30, 2026
sarhiri
commented
Jul 30, 2026
sarhiri
commented
Jul 30, 2026
sarhiri
marked this pull request as draft
July 30, 2026 20:54
sarhiri
marked this pull request as ready for review
July 30, 2026 20:54
dejanzele
reviewed
Aug 11, 2026
|
|
||
| - **No job queue** — Kubernetes has no concept of ordering. Jobs compete for resources with no fairness guarantees. Armada adds a proper queue with priority, fair-share, and rate limiting. | ||
| - **No multi-cluster coordination** — Each Kubernetes cluster is an island. Armada routes jobs across as many clusters as you need from a single API. | ||
| - **No gang scheduling** — Distributed jobs that need all workers to start simultaneously (MPI, PyTorch, Spark) have no atomic startup guarantee in vanilla Kubernetes. Armada either starts the whole group or holds it. |
Member
There was a problem hiding this comment.
We should probably reword this as k8s is adding support - https://kubernetes.io/docs/concepts/scheduling-eviction/gang-scheduling/
Contributor
Author
There was a problem hiding this comment.
Addressed in next commit
dejanzele
reviewed
Aug 11, 2026
|
|
||
| → **[Full quickstart guide](https://armadaproject.io/quickstart)** — get up and running in under 15 minutes. | ||
|
|
||
| ### armadactl |
Member
There was a problem hiding this comment.
Armada Operator and make kind-all already installs and wires up armadactl
Contributor
Author
There was a problem hiding this comment.
Addressed in next commit
…as Armada fine grained capability & Simplify armadactl section to note make kind-all handles installation Signed-off-by: sarhiri <m.sofiasarhiri@gmail.com>
nikola-jokic
approved these changes
Aug 18, 2026
Contributor
|
Queued — the merge queue status continues in this comment ↓. |
Contributor
Author
|
@Mergifyio queue |
Contributor
☑️ Command disallowed due to command restrictions in the Mergify configuration.Details
|
Contributor
Author
Contributor
Merge Queue Status
Waiting for any of
All conditions
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Documentation / Bug fixes
What this PR does / why we need it
Restructures the Armada website content and navigation to reduce complexity and remove deprecated pages.
Changes:
docs/section consolidating all documentation pages (previously split acrossunderstanding-armada/,user-guide/, and root-level)operator-guide.mdx, deprecated per engineering lead reviewdesign/anddeveloper/folders (no meta.json, invisible to nav)understanding-armada/meta.json. Upstream bug causing build failures on clean checkoutscontribute/meta.jsonreferencingCODE_OF_CONDUCTas.mdxwhen the file is.mdWhich issue(s) this PR fixes
Fixes #
Special notes for your reviewer
N/A
UPDATE
Website and Website Navigation
What this does
Restructures the Armada website content and navigation to reduce complexity and remove deprecated pages.
Changes:
docs/section consolidating all documentation pages (previously split acrossunderstanding-armada/,user-guide/, and root-level)operator-guide.mdx, deprecated per engineering lead reviewdesign/anddeveloper/folders (nometa.json, invisible to nav)understanding-armada/meta.json. Upstream bug causing build failures on clean checkoutscontribute/meta.jsonreferencingCODE_OF_CONDUCTas.mdxwhen the file is.mdTroubleshooting notes
When the build stopped working and Greptile flagged an error, I expected an easy route fix. I resolved it through the GitHub web UI, but since that commit wasn't signed off, I had to rebase to sign it. The rebase ended up re-signing all unsigned commits on the branch, which accounts for some of the commit history here.
From there I found a real conflict in the routing, and that's what took most of the time. Correcting the links for Greptile would fail the
yarn/CI build. Correcting the routing foryarn/CI would trigger Greptile to flag the links again. For context, both versions built successfully and routed correctly on the live page. The two checks simply expect different link formats:content:checkwants relative.mdxpaths, Greptile and the browser want absolute extensionless routes.I resolved this in
.remarkrc-mdx.mjsand.remarkrc-md.mjsby configuring the base-level link check to skip absolute route-shaped links, leaving route validation to the build itself. Only four internal links are affected. All pages resolve correctly.Also found:
content:checkonly validates markdown[text](target)syntax, not JSXhrefprops. This means the homepage cards and CTA blocks were never validated by this check, or by anything else. Flagging this as a known gap rather than something introduced by this PR.