feat(server): add project scope and collaboration foundations

CHANGELOG.md

@@ -21,6 +21,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 - Server:
+  - Project-scoped scientific collaboration with private/internal/public discovery,
+    capability-based memberships and invitations, immutable scope storage,
+    fresh-schema scoped task identity, and authorized manifest-backed
+    cross-task artifact snapshots with persisted provenance.
   - runner-owned result storyboards: GREMLIN now declares logical Expected File
     Tree outputs and a trusted local Storyboard; ResultContext exposes only
     approved logical files while generic FileViewers and Files & diagnostics
@@ -47,6 +51,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Documentation:
   - prune always-loaded Claude guidance and move rare PR/release procedures to linked `docs/agents/` references.
 - Server:
+  - Project Scope hardening: immutable task submitter IDs, read-only archived
+    scientific records with eligible artifact reuse, capability-scoped user
+    discovery, non-member attribution redaction, and fail-fast schema-epoch
+    validation with an explicit development-state reset contract.
   - deployment control: allow `build --server-only` to rebuild web/worker images without rebuilding runner images or invalidating prepared SIFs.
   - deployment control: allow `down --keep-gateway` to leave Nginx serving the maintenance page while application services remain stopped.
   - deployment control: add a standalone operator/developer guide for modes, preparation, SIF activation, safety, cache behavior, and recovery.

server/PROJECT_SCOPE_AND_ARTIFACTS.md

@@ -0,0 +1,112 @@
+# Project Scope, Storage, and Artifact References
+
+REvoCompute tasks belong to exactly one authoritative scope. Scope controls
+authorization, discovery, storage, and artifact reuse; it is not presentation
+metadata.
+
+```text
+User
+ |
+ +-- Personal Scope
+ |     `-- Task
+ |          `-- Artifact
+ |
+ `-- Project Membership
+        `-- Project Scope
+             `-- Task
+                  `-- manifest Artifact
+                         `-- ArtifactReference
+                                `-- downstream Task input snapshot
+```
+
+## Authorization
+
+Global account roles (`admin`, `user`, and `guest`) remain independent from
+Project roles. Project roles are `owner`, `maintainer`, `contributor`, and
+`viewer`; the collaboration service maps them to explicit capabilities.
+Routes ask capability questions rather than interpreting role names.
+
+Visibility controls non-member discovery and reading:
+
+- `private`: members only; unknown callers receive the same response as a
+  missing Project.
+- `internal`: authenticated non-members may discover the read-only Project
+  surface.
+- `public`: anonymous callers may discover the read-only Project surface.
+
+Visibility does not grant membership, submission, artifact reuse, diagnostic
+downloads, or future runner-policy eligibility. A viewer can read Project
+results but cannot reuse artifacts. Task mutation requires `cancel_own_tasks`
+or `cancel_project_tasks`, independently of read access.
+
+## Scoped storage
+
+Usernames and Project names are presentation metadata. Each user and Project
+receives a persistent storage key with a readable initial prefix and random
+opaque suffix. Renames never update that key or move result trees.
+
+New task paths are resolved only by `StorageResolver`:
+
+```text
+results/
+  users/<user-storage-key>/tasks/<task-id>/
+  projects/<project-storage-key>/tasks/<task-id>/
+```
+
+Input snapshots use the same scope hierarchy beneath the configured input
+root. The physical hierarchy is an implementation detail and not an API.
+Routes, workers, Docker/SLURM jobs, manifest finalization, archives, recovery,
+and cleanup resolve paths from the persisted task scope.
+
+Project Scope ships with a fresh schema contract. Deployments adopting this
+version rebuild the REvoCompute databases and storage roots; there is no old
+task-layout resolver or username-based fallback. Every task row is complete at
+creation and every path is derived from its immutable scope identity.
+
+This release is a new persistent-state epoch. Startup validates the user, task,
+and collaboration schemas before serving work. Reusing an older or partial
+schema fails with an operator-facing reset message; startup never alters,
+backfills, guesses, or deletes persistent state.
+
+For the one-time development upgrade, stop REvoCompute, deliberately remove the
+test-era user/task/collaboration databases and old workspace/results roots,
+then start the release and recreate users and Projects. Ordinary restarts do
+not perform this reset and must preserve the current databases and scoped
+storage roots.
+
+Tasks persist two independent immutable identities: `scope_type` plus
+`scope_id` determines where the task belongs, while `submitted_by_user_id`
+records who submitted it. The optional username value is a historical display
+snapshot only and is never an authorization primitive.
+
+## Artifact references
+
+An input may use `@<task-id>/<logical-artifact-path>`. This is submission syntax
+only. Before any physical lookup, the server loads the source task and checks
+that the caller may reuse artifacts in the same Personal or Project scope.
+Cross-user and cross-Project reuse is denied.
+
+When a Project is archived, eligible members may snapshot its finalized
+manifest artifacts into their Personal scope. This narrow frozen-record rule
+does not permit submission into the archived Project or Project-to-Project
+sharing.
+
+The storage resolver then requires a finalized source task and an exact entry
+in its authoritative result manifest. It rejects absolute paths, traversal,
+empty path segments, symlinks, missing files, and content that no longer
+matches the manifest hash or size. The caller never supplies or receives a
+host, container, SLURM, or storage-key path.
+
+An authorized artifact is copied into the downstream task's immutable input
+snapshot and appears to the runner as an ordinary local input. Provenance
+records the downstream input, source task and scope, logical artifact path,
+SHA-256, size, media type, and timestamp.
+
+```text
+provenance propagates
+permissions do not
+```
+
+Archiving or renaming the source scope therefore cannot mutate a submitted
+downstream task, and access to the downstream task does not grant access to the
+upstream task.

server/README.md

@@ -5,6 +5,9 @@ and recovery procedures, see the
 [REvoCompute Deployment Control Guide](DEPLOYMENT_CONTROL_GUIDE.md). For the
 new-task/runtime-family adapter contract, see the
 [REvoCompute Operations and Task Adapter Guide](OPERATIONS_AND_TASK_ADAPTER_GUIDE.md).
+For collaboration authorization, immutable scope identity, scoped storage, and
+cross-task provenance, see
+[Project Scope, Storage, and Artifact References](PROJECT_SCOPE_AND_ARTIFACTS.md).
 
 REvoCompute is a Flask + Celery service for multi-user protein computation.
 It supports Docker execution and production SLURM + Apptainer execution across
@@ -31,8 +34,8 @@ The server loads the registry at startup via `CONFIG_DIR`. `gremlin` is always
 enabled; additional runners are gated by `ENABLED_TASKRUNNERS` in `.env`.
 
 Each runner container follows a standard contract (protocol v2):
-- Sees one immutable task snapshot at `/mnt/revocompute/<username>/inputs/`
-  and task-owned results at `/mnt/revocompute/<username>/outputs/`. Concurrent
+- Sees one immutable task snapshot at `/mnt/revocompute/<scope-storage-key>/inputs/`
+  and task-owned results at `/mnt/revocompute/<scope-storage-key>/outputs/`. Concurrent
   tasks have isolated host snapshots even though their virtual paths match.
 - Emits `REVODESIGN_STAGE:<marker>` on stdout for progress tracking
 - Is invoked as `run.sh -i <inputs>/task.json -o <outputs>`; the snapshot's
@@ -671,6 +674,14 @@ Create a writable `AUTH_DIR` before the first start. The web process creates
 `${AUTH_DIR}/users.sqlite3` with the current schema. Existing databases must
 already match that schema; server setup does not migrate them.
 
+Project Scope introduces a destructive development-state epoch transition.
+For the one-time upgrade, stop REvoCompute, deliberately reset the test-era
+user, task, and collaboration databases plus the old workspace/results roots,
+then start the new release and recreate users and Projects. Startup validates
+all three schemas and fails with reset instructions when old state is found; it
+never migrates or deletes that state. An ordinary restart never resets current
+databases or scoped storage.
+
 ### Equivalent Docker Compose commands
 
 These commands are equivalent only after `users.sqlite3` contains an account.

server/revocompute/app.py

@@ -18,6 +18,7 @@
 from revocompute.auth import _SECRET_KEY as _TOKEN_SIGNING_KEY  # noqa: E402
 from revocompute.auth import UserDatabase  # noqa: E402
 from revocompute.auth import _env_bool  # noqa: E402
+from revocompute.collaboration import CollaborationDatabase  # noqa: E402
 from revocompute.config import ComputeConfig
 from revocompute.config import ensure_directories as _ensure_directories
 from revocompute.config import env_csv as _env_csv
@@ -27,6 +28,7 @@
 from revocompute.config import resolve_docker_user as _resolve_docker_user
 from revocompute.maintenance.tasks.result_cleanup import delete_task_artifacts as _delete_result_artifacts
 from revocompute.maintenance.tasks.result_cleanup import deleted_status_from_task as _result_deleted_status
+from revocompute.storage import StorageResolver  # noqa: E402
 from revocompute.task_types import list_types as _list_task_types
 from sqlalchemy.exc import IntegrityError
 from werkzeug.utils import secure_filename
@@ -57,6 +59,8 @@
     "input-workspace.js",
     "input-workspace-rfdiffusion.js",
     "create-task.js",
+    "project.js",
+    "projects.js",
     "task-results.js",
 }
 
@@ -115,6 +119,7 @@ def _add_security_headers(response):
 # ---------------------------------------------------------------------------
 _user_db = UserDatabase()
 app.config["user_db"] = _user_db
+app.config["collaboration"] = CollaborationDatabase(os.path.join(CONFIG.server_dir, "collaboration.sqlite3"))
 ENABLE_REGISTER = _env_bool("ENABLE_REGISTER", False)
 
 # Force the auth cookie's Secure flag regardless of request.is_secure.
@@ -188,6 +193,7 @@ def _add_security_headers(response):
 app.config["RESULT_DOWNLOAD_MODE"] = CONFIG.result_download_mode
 
 _ensure_directories(CONFIG.upload_folder, CONFIG.workspace_folder, CONFIG.results_folder)
+app.config["storage_resolver"] = StorageResolver(CONFIG.results_folder, CONFIG.workspace_folder)
 
 # The authoritative task type registry is loaded by task_runtime's module-level
 # code.  Startup fails if the configured registry is absent or invalid.
@@ -344,8 +350,54 @@ def _is_admin_user() -> bool:
 def _task_access_allowed(task: dict[str, Any]) -> bool:
     if _is_admin_user():
         return True
-    current_user = _current_username() or ""
-    return bool(current_user) and task.get("username") == current_user
+    user = g.get("current_user")
+    if task.get("scope_type") == "project":
+        return app.config["collaboration"].can_view_project(
+            int(task["scope_id"]),
+            int(user["id"]) if user else None,
+            authenticated=user is not None,
+        )
+    if not user:
+        return False
+    return task.get("scope_type") == "personal" and str(task["scope_id"]) == str(user["id"])
+
+
+def _task_mutation_allowed(task: dict[str, Any]) -> bool:
+    """Authorize cancellation/deletion independently from read visibility."""
+    if _is_admin_user():
+        return True
+    user = g.get("current_user")
+    if not user:
+        return False
+    if task.get("scope_type") == "project":
+        store = app.config["collaboration"]
+        project_id = int(task["scope_id"])
+        if str(task.get("submitted_by_user_id")) == str(user["id"]):
+            return store.can(project_id, int(user["id"]), "cancel_own_tasks")
+        return store.can(project_id, int(user["id"]), "cancel_project_tasks")
+    return task.get("scope_type") == "personal" and str(task["scope_id"]) == str(user["id"])
+
+
+def _task_full_results_allowed(task: dict[str, Any]) -> bool:
+    """Return whether the caller may read inputs, diagnostics, and archives."""
+    if _is_admin_user():
+        return True
+    user = g.get("current_user")
+    if not user:
+        return False
+    if task.get("scope_type") != "project" or not task.get("scope_id"):
+        return _task_access_allowed(task)
+    membership = app.config["collaboration"].get_membership(int(task["scope_id"]), int(user["id"]))
+    return bool(membership and app.config["collaboration"].can(int(task["scope_id"]), int(user["id"]), "view_results"))
+
+
+def _task_artifact_access_allowed(task: dict[str, Any], artifact: dict[str, Any]) -> bool:
+    """Keep diagnostic/provenance files out of visibility-only surfaces."""
+    if not _task_access_allowed(task):
+        return False
+    if _task_full_results_allowed(task):
+        return True
+    return artifact.get("role") not in {"diagnostic", "provenance"}
 
 
 def _task_access_denied(md5sum: str):
@@ -361,10 +413,9 @@ def _task_access_denied(md5sum: str):
     )
 
 
-def _task_id_for_upload(content_md5: str, username: str | None) -> str:
-    # Keep task IDs owner-scoped so two users uploading the same FASTA never collide.
-    owner = username or "anonymous"
-    scoped_key = f"{owner}:{content_md5}"
+def _task_id_for_upload(content_md5: str, scope_identity: str) -> str:
+    # Keep task IDs scope-specific so identical inputs in different scopes never collide.
+    scoped_key = f"{scope_identity}:{content_md5}"
     return hashlib.md5(scoped_key.encode("utf-8"), usedforsecurity=False).hexdigest()
 
 

server/revocompute/auth.py

@@ -16,6 +16,7 @@
 import html
 import logging
 import os
+import re
 import secrets
 import smtplib
 import time
@@ -33,6 +34,7 @@
 from revocompute.config import env_int as _env_int
 from revocompute.config import env_str as _env_str
 from revocompute.redis_util import get_redis
+from revocompute.schema_epoch import require_current_schema
 from werkzeug.security import generate_password_hash
 
 # Pre-computed dummy hash used for constant-time comparison when a login
@@ -93,9 +95,17 @@
     sa.Column("registration_country", sa.String(8), nullable=True),
     sa.Column("token_version", sa.Integer, nullable=False, default=0),
     sa.Column("allow_gpu_use", sa.Boolean, nullable=False, default=False),
+    sa.Column("storage_key", sa.String(128), nullable=False, unique=True),
 )
 
 
+def _new_user_storage_key(username: str) -> str:
+    """Generate a readable storage key whose random suffix is immutable."""
+    prefix = re.sub(r"[^A-Za-z0-9]+", "-", username).strip("-").lower()[:32] or "user"
+    suffix = secrets.token_urlsafe(6).lower().replace("_", "-").replace("=", "")
+    return f"{prefix}-{suffix}"
+
+
 def _get_user_db_path() -> str:
     """Resolve the user database path.
 
@@ -133,20 +143,12 @@ def _initialize(self) -> None:
             conn.exec_driver_sql("PRAGMA busy_timeout=30000;")
             conn.exec_driver_sql("PRAGMA journal_mode=WAL;")
             conn.exec_driver_sql("PRAGMA synchronous=NORMAL;")
+            require_current_schema(
+                conn,
+                {"users": {column.name for column in _users_table.columns}},
+                database_name="user database",
+            )
             _metadata.create_all(conn, checkfirst=True)
-            # Migration: api_key_hash -> api_key_digest.  The old werkzeug KDF
-            # hashes are one-way and NOT convertible to a digest — every
-            # existing API key becomes invalid by design and must be
-            # re-issued.  The physical api_key_hash column, if present in an
-            # old DB, is left in place (harmless — the model no longer
-            # selects it).
-            columns = {row[1] for row in conn.exec_driver_sql("PRAGMA table_info(users)")}
-            if "api_key_digest" not in columns:
-                conn.exec_driver_sql("ALTER TABLE users ADD COLUMN api_key_digest VARCHAR(64)")
-                # Index the migrated column so validation stays a single
-                # lookup — same index name SQLAlchemy creates for new DBs
-                # (index=True), so fresh and migrated DBs match.
-                conn.exec_driver_sql("CREATE INDEX IF NOT EXISTS ix_users_api_key_digest ON users(api_key_digest)")
         try:
             os.chmod(self.path, 0o600)
         except OSError:
@@ -191,6 +193,7 @@ def create_user(
             registration_ip=registration_ip,
             registration_country=registration_country,
             user_status=user_status,
+            storage_key=_new_user_storage_key(username),
         )
         with self.engine.begin() as conn:
             result = conn.execute(stmt)