Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ To help the community judge your pull request (PR), please include the following

Your PR will be tested using [GitHub Actions](https://github.com/eXist-db/exist/actions) against a number of operating systems and environments. The build status is visible in the PR.

To detect errors in your PR before submitting it, please run eXist's full test suite on your own system via `mvn -V clean verify`.
To detect errors in your PR before submitting it, please run eXist's full test suite on your own system via `./mvnw -V clean verify`.

------

Expand Down
2 changes: 1 addition & 1 deletion .github/actions/install-mvnd/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ inputs:
version:
description: 'The version of the Maven Daemon to install'
required: true
default: '1.0.3'
default: '2.0.0-rc-3'
file-version-suffix:
description: 'A suffix to append to the version of the download file of Maven Daemon to install'
required: false
Expand Down
19 changes: 18 additions & 1 deletion .github/actions/maven-github-settings/action.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,19 @@
# Creates Maven settings.xml with auth for github and github-xqts-runner.
# Required for resolving from maven.pkg.github.com/eXist-db/exist and exist-xqts-runner.
name: Maven GitHub Packages settings
description: Create settings.xml with github and github-xqts-runner servers
description: Create settings.xml with github, github-xqts-runner, and optional central servers
inputs:
token:
description: 'GitHub token for package authentication'
required: true
central-token-username:
description: 'Optional Sonatype Central token username'
required: false
default: ''
central-token-password:
description: 'Optional Sonatype Central token password'
required: false
default: ''
runs:
using: 'composite'
steps:
Expand All @@ -14,6 +22,14 @@ runs:
run: |
mkdir -p ~/.m2
OWNER=$(echo "${{ github.repository_owner }}" | tr '[:upper:]' '[:lower:]')
CENTRAL_SERVER=""
if [ -n "${{ inputs.central-token-username }}" ] && [ -n "${{ inputs.central-token-password }}" ]; then
CENTRAL_SERVER="<server>
<id>central</id>
<username>${{ inputs.central-token-username }}</username>
<password>${{ inputs.central-token-password }}</password>
</server>"
fi
cat > ~/.m2/settings.xml << EOF
<settings xmlns="http://maven.apache.org/SETTINGS/1.2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.2.0 https://maven.apache.org/xsd/settings-1.2.0.xsd">
<servers>
Expand All @@ -27,6 +43,7 @@ runs:
<username>${OWNER}</username>
<password>${{ inputs.token }}</password>
</server>
${CENTRAL_SERVER}
</servers>
</settings>
EOF
28 changes: 20 additions & 8 deletions .github/workflows/ci-container.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ jobs:
name: Test and Publish Container Images
runs-on: ubuntu-latest
timeout-minutes: 60
# NOTE (DP): Publish on develop and master, test on PRs against these
if: github.ref == 'refs/heads/develop' || github.ref == 'refs/heads/master' || github.base_ref == 'develop' || github.base_ref == 'master'
# Publish on develop/master/tags, test on PRs against develop/master
if: github.ref == 'refs/heads/develop' || github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/eXist-') || github.base_ref == 'develop' || github.base_ref == 'master'
steps:
- uses: actions/checkout@v6
with:
Expand Down Expand Up @@ -40,11 +40,11 @@ jobs:
timeout-minutes: 35
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: mvn -V -B --no-transfer-progress -q -Pdocker -DskipTests -Ddependency-check.skip=true -P !mac-dmg-on-unix,!installer,!concurrency-stress-tests,!micro-benchmarks,skip-build-dist-archives clean package
run: ./mvnw -V -B --no-transfer-progress -q -Pdocker -DskipTests -Ddependency-check.skip=true clean package
- name: Check local images
run: docker image ls
- name: Check license headers
run: mvn --no-transfer-progress license:check
run: ./mvnw --no-transfer-progress license:check
working-directory: exist-docker
- name: Start exist-ci container
run: |
Expand Down Expand Up @@ -78,15 +78,27 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
run: mvn --no-transfer-progress -q -Ddocker.tag=latest -Ddocker.username=$DOCKER_USERNAME -Ddocker.password=$DOCKER_PASSWORD docker:build docker:push
run: ./mvnw --no-transfer-progress -q -Ddocker.tag=latest -Ddocker.username=$DOCKER_USERNAME -Ddocker.password=$DOCKER_PASSWORD docker:build docker:push
working-directory: ./exist-docker
- name: Publish release images
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/eXist-')
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
run: mvn --no-transfer-progress -q -Ddocker.tag=release -Ddocker.username=$DOCKER_USERNAME -Ddocker.password=$DOCKER_PASSWORD docker:build docker:push
run: |
# eXist tags are formatted like `eXist-<version>` (e.g. eXist-7.0.0).
# Docker image names include Maven `%v` (see exist-docker/pom.xml), so ensure
# release builds use a non-SNAPSHOT `project.version`.
REVISION=""
if [[ "${GITHUB_REF_NAME}" == eXist-* ]]; then
REVISION="${GITHUB_REF_NAME#eXist-}"
fi
EXTRA_ARGS=""
if [[ -n "${REVISION}" ]]; then
EXTRA_ARGS="-Drevision=${REVISION}"
fi
./mvnw --no-transfer-progress -q -Ddocker.tag=release -Ddocker.username=$DOCKER_USERNAME -Ddocker.password=$DOCKER_PASSWORD ${EXTRA_ARGS} docker:build docker:push
working-directory: ./exist-docker
# NOTE (DP): This is for debugging, publishes an experimental image from inside PRs against develop
# - name: Publish experimental images
Expand All @@ -95,5 +107,5 @@ jobs:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
# DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
# run: mvn -q -Ddocker.tag=experimental -Ddocker.username=$DOCKER_USERNAME -Ddocker.password=$DOCKER_PASSWORD docker:build docker:push
# run: ./mvnw -q -Ddocker.tag=experimental -Ddocker.username=$DOCKER_USERNAME -Ddocker.password=$DOCKER_PASSWORD docker:build docker:push
# working-directory: ./exist-docker
79 changes: 79 additions & 0 deletions .github/workflows/ci-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
name: Release

on:
push:
tags:
- 'eXist-*'
workflow_dispatch:
inputs:
tag:
description: 'Tag to release (for manual runs)'
required: false
type: string

permissions:
contents: write

jobs:
release:
name: Build and Publish Release
runs-on: ubuntu-latest
timeout-minutes: 120
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CENTRAL_TOKEN_USERNAME: ${{ secrets.CENTRAL_TOKEN_USERNAME }}
CENTRAL_TOKEN_PASSWORD: ${{ secrets.CENTRAL_TOKEN_PASSWORD }}
EXISTDB_RELEASE_KEY: ${{ secrets.EXISTDB_RELEASE_KEY }}
EXISTDB_RELEASE_KEY_PASSPHRASE: ${{ secrets.EXISTDB_RELEASE_KEY_PASSPHRASE }}
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Set up JDK 21
uses: actions/setup-java@v5
with:
distribution: temurin
java-version: '21'

- uses: ./.github/actions/maven-cache

- uses: ./.github/actions/maven-github-settings
with:
token: ${{ secrets.GITHUB_TOKEN }}
central-token-username: ${{ secrets.CENTRAL_TOKEN_USERNAME }}
central-token-password: ${{ secrets.CENTRAL_TOKEN_PASSWORD }}

- name: Release preflight + build + publish to Central
run: |
TAG_NAME="${{ github.event.inputs.tag }}"
if [ -z "${TAG_NAME}" ]; then
TAG_NAME="${GITHUB_REF_NAME}"
fi
# GitHub tag format: eXist-<version> (e.g. eXist-7.0.0)
REVISION="${TAG_NAME#eXist-}"

./mvnw -V -B --no-transfer-progress \
-Prelease-build \
-Dmaven.consumer.pom.flatten=true \
-Drevision="${REVISION}" \
-Drelease.preflight=true \
-DskipTests \
-Ddependency-check.skip=true \
-Dexistdb.release.key="${EXISTDB_RELEASE_KEY}" \
-Dexistdb.release.key.passphrase="${EXISTDB_RELEASE_KEY_PASSPHRASE}" \
clean deploy

- name: Collect release assets
run: |
mkdir -p /tmp/release-assets
cp -f exist-distribution/target/*.zip /tmp/release-assets/ 2>/dev/null || true
cp -f exist-distribution/target/*.tar.bz2 /tmp/release-assets/ 2>/dev/null || true
cp -f exist-distribution/target/*.dmg /tmp/release-assets/ 2>/dev/null || true
cp -f exist-installer/target/*.jar /tmp/release-assets/ 2>/dev/null || true
cp -f exist-installer/target/*.exe /tmp/release-assets/ 2>/dev/null || true

- name: Publish GitHub release assets
uses: softprops/action-gh-release@v2
with:
files: /tmp/release-assets/*
4 changes: 1 addition & 3 deletions .github/workflows/ci-snapshots.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,4 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
mvn -V -B --no-transfer-progress -q -DskipTests -Ddependency-check.skip=true \
-P !mac-dmg-on-unix,!installer,!concurrency-stress-tests,!micro-benchmarks,skip-build-dist-archives \
clean deploy
./mvnw -V -B --no-transfer-progress -q -DskipTests -Ddependency-check.skip=true -Dmaven.consumer.pom.flatten=true clean deploy
10 changes: 4 additions & 6 deletions .github/workflows/ci-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
distribution: temurin
java-version: ${{ env.DEV_JDK }}
- uses: ./.github/actions/maven-cache
- run: mvn -V -B --no-transfer-progress --offline license:check
- run: ./mvnw -V -B --no-transfer-progress --offline license:check
timeout-minutes: 10
dependencies:
name: Dependency checks
Expand Down Expand Up @@ -46,7 +46,7 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NVD_API_KEY: ${{ secrets.NVD_API_KEY }}
run: mvn -V -B --no-transfer-progress dependency-check:check
run: ./mvnw -V -B --no-transfer-progress dependency-check:check
timeout-minutes: 90
- name: Save OWASP dependency-check cache
if: github.event_name == 'push'
Expand Down Expand Up @@ -83,15 +83,13 @@ jobs:
id: install-mvnd
uses: ./.github/actions/install-mvnd
with:
version: '1.0.3'
file-version-suffix: ''
cache: 'true'
- name: Maven Build
timeout-minutes: 30
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
MAVEN_OPTS: ${{ env.MAVEN_OPTS }} -D'aether.connector.basic.connectTimeout=10000' -D'aether.connector.basic.requestTimeout=30000'
run: ${{ steps.install-mvnd.outputs.mvnd-dir }}/mvnd -V -B --no-transfer-progress -T 1C compile test-compile -Ponnx-model -D'dependency-check.skip' -D'license.skip' --projects '!exist-installer'
run: ${{ steps.install-mvnd.outputs.mvnd-dir }}/mvnd -V -B --no-transfer-progress -T 1C compile test-compile -Ponnx-model -D'dependency-check.skip' -D'license.skip'
- name: Maven Unit Tests
id: unit-tests
if: matrix.test-type == 'unit'
Expand All @@ -109,7 +107,7 @@ jobs:
run: ${{ steps.install-mvnd.outputs.mvnd-dir }}/mvnd -V -B --no-transfer-progress verify -Ponnx-model -DskipUnitTests=true -D'dependency-check.skip' -D'license.skip' -D'mvnd.maxLostKeepAlive=6000'
- name: Javadoc (ubuntu unit only)
if: matrix.os == 'ubuntu-latest' && matrix.test-type == 'unit'
run: ${{ steps.install-mvnd.outputs.mvnd-dir }}/mvnd -V -B --no-transfer-progress -q -T 1C install javadoc:javadoc -DskipTests -D'dependency-check.skip' -D'license.skip' --projects '!exist-distribution,!exist-installer' --also-make
run: ${{ steps.install-mvnd.outputs.mvnd-dir }}/mvnd -V -B --no-transfer-progress -q -T 1C install javadoc:javadoc -DskipTests -D'dependency-check.skip' -D'license.skip'
- name: Maven Code Coverage (Develop branch, ubuntu unit only)
if: github.event_name != 'pull_request' && github.ref == 'refs/heads/develop' && matrix.os == 'ubuntu-latest' && matrix.test-type == 'unit'
env:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-xqts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
timeout-minutes: 25
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: mvn -V -B --no-transfer-progress clean package -DskipTests -Ddependency-check.skip=true --projects exist-xqts --also-make
run: ./mvnw -V -B --no-transfer-progress clean package -DskipTests -Ddependency-check.skip=true --projects exist-xqts --also-make
- name: Run XQTS
timeout-minutes: 60
env:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/sonarcloud.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,4 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
run: mvn -V -B --no-transfer-progress -Dsurefire.useFile=false -DtrimStackTrace=false -Ddependency-check.skip=true -Ddocker=false -P \!mac-dmg-on-mac,\!codesign-mac-dmg,\!mac-dmg-on-unix,\!installer,\!concurrency-stress-tests,\!micro-benchmarks,\!build-dist-archives verify site org.sonarsource.scanner.maven:sonar-maven-plugin:sonar
run: ./mvnw -V -B --no-transfer-progress -Dsurefire.useFile=false -DtrimStackTrace=false -Ddependency-check.skip=true -Ddocker=false -Plocal-build verify site org.sonarsource.scanner.maven:sonar-maven-plugin:sonar
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ dependency-reduced-pom.xml

# Maven generated
effective-pom.xml
.flattened-pom.xml
flattened-pom.xml

# XQTS test artefacts
work/
Expand Down
3 changes: 3 additions & 0 deletions .mvn/maven-user.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
maven.logger.log.org.apache.hc=error
maven.logger.log.org.apache.commons.jcs3=error
maven.logger.log.org.apache.lucene=error
Binary file added .mvn/wrapper/maven-wrapper.jar
Binary file not shown.
3 changes: 3 additions & 0 deletions .mvn/wrapper/maven-wrapper.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
wrapperVersion=3.3.4
distributionType=only-script
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/4.0.0-rc-5/apache-maven-4.0.0-rc-5-bin.zip
16 changes: 8 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,14 @@ eXist-db is an open-source native XML database with full XQuery support. The mai

```bash
JAVA_HOME=$(/usr/libexec/java_home -v 21) \
mvn -T1.5C clean install -DskipTests -Ddependency-check.skip=true -Ddocker=false \
-P 'skip-build-dist-archives,!build-dist-archives,!mac-dmg-on-mac,!codesign-mac-dmg,!mac-dmg-on-unix,!installer,!concurrency-stress-tests,!micro-benchmarks,!appassembler-booter'
./mvnw -T1.5C clean install -DskipTests -Ddependency-check.skip=true -Ddocker=false \
-Plocal-build
```

### Build a single module

```bash
mvn install -pl exist-core -am -DskipTests -Ddependency-check.skip=true -Ddocker=false
JAVA_HOME=$(/usr/libexec/java_home -v 21) ./mvnw install -pl exist-core -am -DskipTests -Ddependency-check.skip=true -Ddocker=false
```

The `-am` (also-make) flag is required — exist-core has cross-module dependencies (e.g., `EXistClassLoader` in exist-start).
Expand All @@ -32,21 +32,21 @@ The `-am` (also-make) flag is required — exist-core has cross-module dependenc

```bash
# XQSuite tests (XQuery test framework)
mvn test -pl exist-core -Dtest="xquery.xquery3.XQuery3Tests" -Ddependency-check.skip=true -Ddocker=false
JAVA_HOME=$(/usr/libexec/java_home -v 21) ./mvnw test -pl exist-core -Dtest="xquery.xquery3.XQuery3Tests" -Ddependency-check.skip=true -Ddocker=false

# Full unit test suite
mvn test -pl exist-core -Ddependency-check.skip=true -Ddocker=false
JAVA_HOME=$(/usr/libexec/java_home -v 21) ./mvnw test -pl exist-core -Ddependency-check.skip=true -Ddocker=false

# Specific JUnit test class
mvn test -pl exist-core -Dtest="org.exist.xquery.XPathQueryTest" -Ddependency-check.skip=true -Ddocker=false
JAVA_HOME=$(/usr/libexec/java_home -v 21) ./mvnw test -pl exist-core -Dtest="org.exist.xquery.XPathQueryTest" -Ddependency-check.skip=true -Ddocker=false
```

### Docker image

```bash
# Build the Docker image
mvn -T1.5C clean package -DskipTests -Ddependency-check.skip=true -Ddocker=true \
-P 'skip-build-dist-archives,!build-dist-archives,!mac-dmg-on-mac,!codesign-mac-dmg,!mac-dmg-on-unix,!installer,!concurrency-stress-tests,!micro-benchmarks,!appassembler-booter' \
JAVA_HOME=$(/usr/libexec/java_home -v 21) ./mvnw -T1.5C clean package -DskipTests -Ddependency-check.skip=true -Ddocker=true \
-Plocal-build \
-pl exist-docker -am

cp exist-docker/target/classes/Dockerfile exist-docker/target/exist-docker-*-docker-dir/Dockerfile
Expand Down
26 changes: 22 additions & 4 deletions BUILD.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ To build eXist-db:
$ git clone https://github.com/eXist-db/exist.git
$ cd exist
$ git checkout master
$ mvn -DskipTests package
$ ./mvnw -DskipTests package
```

From here, you now have a compiled version of eXist-db in the `exist-distribution/target` folder that you may use just as you would an installed version of eXist-db. An installer is also build and present in `exist-installer/target` for easy installation elsewhere.
Expand All @@ -23,6 +23,11 @@ Useful build switches:
- `-DskipUnitTests=true` : run only integration tests
- `-Ddependency-check.skip=true` : skips validating dependencies

### Maven version

This repo uses **Maven Wrapper** (`./mvnw`) so you can build with the project’s pinned Maven version without installing Maven via a system package manager.
Maven 4 requires **Java 17+** to run; eXist-db builds with **Java 21**.

### Maven repositories

Maven resolves dependencies from these repositories (defined in `exist-parent/pom.xml`):
Expand All @@ -34,7 +39,7 @@ Maven resolves dependencies from these repositories (defined in `exist-parent/po

When building from `develop` (or any SNAPSHOT version), Maven resolves `exist-xqts-runner` from `https://maven.pkg.github.com/eXist-db/exist-xqts-runner`. GitHub Packages requires authentication; without it you get **401 Unauthorized**.

**Option 1 – Exclude XQTS** (no auth needed): use `mvn -DskipTests package -pl '!exist-xqts'` to skip the XQTS module.
**Option 1 – Exclude XQTS** (no auth needed): use `./mvnw -DskipTests package -pl '!exist-xqts'` to skip the XQTS module.

**Option 2 – Configure GitHub auth** (if you need XQTS or a full build): add a GitHub PAT to `~/.m2/settings.xml` as server `github-xqts-runner` (and optionally `github` for eXist snapshots). See `.github/actions/maven-github-settings/action.yml` for the expected `<server>` format.

Expand All @@ -44,9 +49,22 @@ Further build options can be found at: [eXist-db Build Documentation](http://www

From the repo root:

- **All tests:** `mvn -V -B verify -Ddependency-check.skip -Dlicense.skip`
- **All tests:** `./mvnw -V -B verify -Ddependency-check.skip -Dlicense.skip`
- **exist-core only:** add `--projects exist-core --also-make` to the above
- **Single test class:** `mvn -Dtest=fully.qualified.TestClass test --projects exist-core --also-make`
- **Single test class:** `./mvnw -Dtest=fully.qualified.TestClass test --projects exist-core --also-make`

### Maven 4 rerun/resume guidance

Use Maven 4's reactor resume mode (`-r` / `--resume`) for reruns after module failures:

- Initial CI-like run:
- `./mvnw -V -B --no-transfer-progress -DskipTests -Ddependency-check.skip=true clean verify`
- Retry from the failed module onward:
- `./mvnw -V -B --no-transfer-progress -DskipTests -Ddependency-check.skip=true -r verify`

This avoids rebuilding already successful modules and keeps reruns deterministic.

For release/snapshot publish jobs that call `deploy`, treat `-r deploy` as a full publish rerun strategy only. Maven 4 deploy behavior is effectively all-or-nothing when using deploy-at-end semantics, so do not assume partial publish recovery from a previous failed deploy.

**NOTE:**
In the above example, we switched the current (checked-out) branch from `develop` to `master`. We use the [GitFlow for eXist-db](#contributing-to-exist) process:
Expand Down
Loading
Loading