Skip to content

feat: ComfyUI fallback + session fixes#1538

Open
roseonlineownz-lab wants to merge 31 commits intomoeru-ai:mainfrom
roseonlineownz-lab:main
Open

feat: ComfyUI fallback + session fixes#1538
roseonlineownz-lab wants to merge 31 commits intomoeru-ai:mainfrom
roseonlineownz-lab:main

Conversation

@roseonlineownz-lab
Copy link
Copy Markdown

Summary

  • ComfyUI Fallback Architecture: Added COMFYUI_FALLBACK_BASE_URL support for GPU/CPU fallback pattern
  • NSFW Image Consumer: Fixed workflow validation, error reconciliation, and automatic fallback on GPU kernel failures
  • Session Management: Fixed session ID overwrite on agent creation (Issue v0.5.0 #156)
  • Gitignore: Added .env to prevent accidental commits of secrets

Key Changes

  1. apps/server/src/services/nsfw-image-consumer-handler.ts

    • Added fallback workflow support when stale custom workflows fail
    • Fixed ComfyUI history status parsing (error vs success vs running)
    • Added automatic GPU→CPU fallback on kernel errors
  2. apps/server/src/libs/env.ts

    • Added COMFYUI_FALLBACK_BASE_URL environment variable
  3. paseo/packages/server/src/server/agent/providers/claude-agent.ts

    • Fixed session ID overwrite on agent creation (Issue v0.5.0 #156)

Test Plan

  • Windows GPU ComfyUI accessible from WSL2
  • Docker CPU ComfyUI running on port 8189
  • AIRI .env configured with fallback URLs
  • Git verified commits

🤖 Generated with Claude Code

roseonlineownz-lab and others added 20 commits March 30, 2026 15:31
Character Schema:
- Add visibility, nsfwEnabled, nsfwLevel, relationshipMode
- Add personaProfile with personality, scenario, speakingStyle
- Add starterMessages and boundaries fields
- Migration 0008_lazy_wallflower.sql

Frontend Updates:
- CharacterDialog: New persona tab with NSFW/relationship controls
- CharacterItem: Show relationship/NSFW badges
- v2/index.vue: Better explore page with filters
- v2/[id].vue: New character detail page with profile view

Types & Stores:
- Updated CharacterBaseSchema with all new fields
- CharacterPersonaProfile interface added
- Local-first character store compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Character schema: visibility, nsfwEnabled, nsfwLevel, relationshipMode, personaProfile
- CharacterDialog: persona tab, NSFW controls, relationship settings
- CharacterItem: badges for relationship/visibility/NSFW
- v2/index.vue: filters, hero, categorization
- v2/[id].vue: character detail page with scenario, starters, boundaries
- Migration: 0008_lazy_wallflower.sql

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c5ccd9346f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

},
})

const fallbackHistory = await fetchComfyHistory(promptId, env.COMFYUI_FALLBACK_BASE_URL ?? comfyBaseUrl)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep ComfyUI host consistent in fallback history checks

In the custom-workflow retry path, the prompt is resubmitted to env.COMFYUI_BASE_URL, but this history lookup uses env.COMFYUI_FALLBACK_BASE_URL ?? comfyBaseUrl. When a fallback URL is configured and differs from the primary host, the worker polls the wrong ComfyUI instance, so successful jobs stay pending and can eventually be marked failed by timeout.

Useful? React with 👍 / 👎.

Comment on lines +23 to +24
.post('/jobs', async (c) => {
const user = c.get('user')!
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Enforce NSFW gating before accepting NSFW job requests

This endpoint creates and queues NSFW jobs for any authenticated user without checking the new gating fields (adultVerified, allowSensitiveContent, contentTier). That means clients can bypass the UI restriction by calling the API directly and still generate NSFW jobs, which defeats the account-level gating added in this change.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a comprehensive NSFW companion system, featuring user gating (adult verification and content tiers), character metadata (visibility, NSFW levels, and persona profiles), and a server-backed image generation pipeline using ComfyUI. The implementation includes database migrations, new API routes, a dedicated image consumer worker, and extensive frontend UI components. Review feedback identifies critical bugs in the ComfyUI job submission and history reconciliation logic, specifically addressing URL mismatches when using fallback servers and the requirement to use the prompt_id returned by the API for reliable job tracking.

},
})

const fallbackHistory = await fetchComfyHistory(promptId, env.COMFYUI_FALLBACK_BASE_URL ?? comfyBaseUrl)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

There is a critical URL mismatch here. The prompt was submitted to env.COMFYUI_BASE_URL (line 417), but history is being looked up on env.COMFYUI_FALLBACK_BASE_URL ?? comfyBaseUrl. If a fallback URL is configured, this lookup will always fail because the job doesn't exist on the fallback server yet. Additionally, it should use the prompt_id returned by the submission.

          const actualPromptId = fallbackSubmission.prompt_id ?? promptId
          const fallbackHistory = await fetchComfyHistory(actualPromptId, comfyBaseUrl)

workflow: fallbackWorkflow,
comfy: {
...comfyMeta,
baseUrl: env.COMFYUI_FALLBACK_BASE_URL ?? comfyBaseUrl,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Setting baseUrl to env.COMFYUI_FALLBACK_BASE_URL ?? comfyBaseUrl here is inconsistent with the submission at line 417 (which uses env.COMFYUI_BASE_URL). If FALLBACK_BASE_URL is defined, the job metadata will point to the fallback server while the prompt was actually sent to the primary server. This will cause history reconciliation to fail in subsequent polls.

Suggested change
baseUrl: env.COMFYUI_FALLBACK_BASE_URL ?? comfyBaseUrl,
baseUrl: comfyBaseUrl,

logger.withFields({ jobId, promptId }).warn('Discarding invalid custom ComfyUI workflow override and retrying with default workflow')

const fallbackResponse = await fetch(new URL('/prompt', env.COMFYUI_BASE_URL), {
method: 'POST',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The fallback submission is hardcoded to use env.COMFYUI_BASE_URL. If the job was already being retried on a fallback server (e.g., due to a previous GPU kernel error), this will force it back to the primary server, which might still be failing. It should use comfyBaseUrl instead.

          const fallbackResponse = await fetch(new URL('/prompt', comfyBaseUrl), {

},
})

const history = await fetchComfyHistory(promptId, comfyBaseUrl)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The history lookup should use the prompt_id returned by the ComfyUI submission response to ensure it reconciles the correct job, especially if ComfyUI generated a new ID.

Suggested change
const history = await fetchComfyHistory(promptId, comfyBaseUrl)
const history = await fetchComfyHistory(submission.prompt_id ?? promptId, comfyBaseUrl)

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bacd366fcb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +309 to +311
})
logger.withFields({ jobId, promptId, fallbackBaseUrl: env.COMFYUI_FALLBACK_BASE_URL }).warn('Retrying image job on ComfyUI fallback host after GPU kernel failure')
return
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep fallback retries pending instead of returning success

In the GPU-kernel fallback branch, the handler sets the job back to queued and then returns normally; in createMqWorker, a successful return path is ACKed, so this event is removed from the stream and no new event is published for the queued retry. In environments where fallback is needed, affected jobs can get stuck in queued forever and never actually resubmit to the fallback ComfyUI host.

Useful? React with 👍 / 👎.

Comment on lines +375 to +379
await mediaService.updateImageJobStatus(jobId, 'submitting', {
errorMessage: null,
params: {
...job.params,
workflow,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Submit prompt before transitioning job to submitting

The job status is moved to submitting before the /prompt HTTP call is attempted. If that fetch throws (network blip, DNS, connection reset), the worker leaves the message pending, but later retries only poll history for this prompt ID and eventually time out instead of re-attempting submission. This turns transient submit failures into deterministic job failures.

Useful? React with 👍 / 👎.

roseonlineownz-lab and others added 4 commits April 3, 2026 03:09
The 'on' function is not a named export from 'node:process' in Node 22.
Changed to use process.on() directly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Merges latest upstream changes including lockfile updates,
instrumentation, and new features while preserving fork's
ComfyUI fallback and NSFW gating additions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds missing valibot@catalog: specifier to lockfile for
packages/server-schema, fixing CI frozen-lockfile failures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove duplicate db property in buildApp call
- Remove unused integer import from nsfw-media schema
- Add missing nsfwMediaService and nsfwImageMq to test deps

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 238373b43a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +30 to +34
const job = await mediaService.createImageJob({
...result.output,
userId: user.id,
params: result.output.params ?? {},
})
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Prevent orphaned jobs when event publish fails

This route writes the image_jobs row before publishing to Redis, so if nsfwImageMq.publish(...) throws, the request fails but the queued job is already persisted without any stream event to drive processing. The new consumer only handles stream messages (worker.run(...onMessage...)) and does not reconcile queued DB rows, so these jobs can remain stuck forever and retries create duplicate jobs for the same request.

Useful? React with 👍 / 👎.

Comment on lines +64 to +67
const item = await mediaService.createGalleryItem({
...result.output,
userId: user.id,
})
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate gallery imageJobId ownership before insert

The gallery endpoint accepts and stores any client-provided imageJobId without checking that the referenced job belongs to the caller. Because gallery status/result hydration and job-to-gallery updates are keyed by image_job_id, a user who obtains another user's job ID can bind that job to their own gallery row and observe its status/media result updates, creating cross-account data leakage.

Useful? React with 👍 / 👎.

- Auto-fix lint issues
- Sync with moeru-ai/airi upstream

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ae8e5e1c73

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/airi-web.sh

log "starting on http://$HOST:$PORT"
(
cd /home/faramix/airi
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Resolve project path dynamically before starting web server

The launcher always runs cd /home/faramix/airi before invoking Vite, so start fails on any machine where the repo is checked out elsewhere (including CI or other developers' environments) with a directory-not-found error. This makes the helper script effectively unusable outside one specific filesystem layout; derive the repo root from the script location instead of hardcoding a user path.

Useful? React with 👍 / 👎.

Security: exclude Android keystore files from version control

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@nekomeowww nekomeowww added scope/extension Scope related to extension api, or internally known as tentacle api, mod api, plugin api scope/ui Scope related to UI/UX, or interface improve, perf, and bugs pr-review/way-too-large Pull Request that way too large, not easy to review, be careful when reviewing labels Apr 7, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 52f29d696b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +487 to +488
env:
BUTLER_API_KEY: ${{ secrets.GAME_PUBLISHING_ITCHIO }}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Remove stray top-level env block from release workflow

The trailing env: mapping after the last upload step is indented outside the step/job structure, which makes .github/workflows/release-tamagotchi.yml invalid YAML. With this parse error, GitHub Actions cannot load the workflow, so release runs fail before any build job starts.

Useful? React with 👍 / 👎.

- Change AUTH_GOOGLE_CLIENT_ID/SECRET and AUTH_GITHUB_CLIENT_ID/SECRET
  from required to optional with empty string defaults
- Conditionally include social providers only when credentials are set
- Server starts cleanly without OAuth configured (social login disabled)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ef02d1dd8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +298 to +300
...job.params,
comfy: {
...comfyMeta,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Drop stored workflow when scheduling fallback retries

When a GPU-kernel failure triggers the fallback path, the queued update keeps ...job.params, which preserves the previously submitted workflow. On the next pass, handleMessage reads that value via extractWorkflow(job.params) and treats it as an override, so buildDefaultComfyWorkflow never rebuilds with fallback settings (checkpoint/steps/dimensions). In practice this can keep retrying with the original heavy workflow even after switching baseUrl, so fallback retries may fail for the same reason instead of recovering.

Useful? React with 👍 / 👎.

.use('*', authGuard)
.get('/jobs', async (c) => {
const user = c.get('user')!
const jobs = await mediaService.listImageJobs(user.id, 'nsfw')
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Align job listing filter with accepted route values

This list endpoint always filters to 'nsfw', but the create schema for the same route accepts both 'normal' and 'nsfw'. If a client submits a 'normal' job (which the API currently allows), it is created successfully but will never appear in /api/v1/nsfw/jobs, so clients cannot observe status/results through this API surface. Either reject 'normal' at create time here or make listing route-aware.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-review/way-too-large Pull Request that way too large, not easy to review, be careful when reviewing scope/extension Scope related to extension api, or internally known as tentacle api, mod api, plugin api scope/ui Scope related to UI/UX, or interface improve, perf, and bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants