Skip to content

refactor: improve transcription UX and DX #1685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
drHuangMHT wants to merge 14 commits into
base: main
Choose a base branch
from
Draft

refactor: improve transcription UX and DX #1685

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
36e163a
move transcription related code into standalone composable
drHuangMHT Apr 18, 2026
e663936
implement transcription on portrait mode
drHuangMHT Apr 18, 2026
0f5f594
apply suggestions, refactor for code readability
drHuangMHT Apr 19, 2026
cd7c0fc
move isListening to internal state
drHuangMHT Apr 19, 2026
ffa9441
tests
drHuangMHT Apr 19, 2026
c25c376
regenerate lockfile
drHuangMHT Apr 25, 2026
d1c0a17
[autofix.ci] apply automated fixes
autofix-ci[bot] Apr 25, 2026
f09a8e2
Merge branch 'main' into use-transcriptions
drHuangMHT Apr 25, 2026
0cc7a1f
prevent state drilling, component reuse, separate monitoring and tran…
drHuangMHT May 2, 2026
cef84e9
hide the button when model binding does not exist
drHuangMHT May 2, 2026
1b0fc17
early return when audio input is not enabled
drHuangMHT May 2, 2026
fe73400
stop transcription when audio input is disabled
drHuangMHT May 2, 2026
a355544
properly detect electron environment
drHuangMHT May 2, 2026
9915f88
rerun analyzer setup when stream changes
drHuangMHT May 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions .../src/renderer/components/stage-islands/controls-island/controls-island-hearing-config.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import { onMounted, onUnmounted, watch } from 'vue'
const show = defineModel('show', { type: Boolean, default: false })

const settingsAudioDeviceStore = useSettingsAudioDevice()
const { enabled, selectedAudioInput, stream, audioInputs } = storeToRefs(settingsAudioDeviceStore)
const { enabled, stream } = storeToRefs(settingsAudioDeviceStore)

const getMediaAccessStatus = useElectronEventaInvoke(electron.systemPreferences.getMediaAccessStatus)
const { state: mediaAccessStatus, execute: refreshMediaAccessStatus } = useAsyncState(() => getMediaAccessStatus(['microphone']), 'not-determined')
Expand Down Expand Up @@ -58,10 +58,7 @@ onUnmounted(async () => {
<template>
<HearingConfigDialog
v-model:show="show"
v-model:enabled="enabled"
v-model:selected-audio-input="selectedAudioInput"
:granted="mediaAccessStatus !== 'denied' && mediaAccessStatus !== 'restricted'"
:audio-inputs="audioInputs"
:volume-level="volumeLevel"
>
<slot />
Expand Down
3 changes: 3 additions & 0 deletions packages/stage-layouts/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,11 @@
"@proj-airi/stage-shared": "workspace:^",
"@types/audioworklet": "catalog:",
"@types/three": "^0.184.0",
"@vue/test-utils": "catalog:",
"@webgpu/types": "catalog:",
"jsdom": "catalog:",
"unplugin-info": "catalog:",
"vitest": "catalog:",
"vue-tsc": "^3.2.6"
}
}
14 changes: 12 additions & 2 deletions packages/stage-layouts/src/components/Layouts/MobileInteractiveArea.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import type { ChatHistoryItem } from '@proj-airi/stage-ui/types/chat'
import type { ChatProvider } from '@xsai-ext/providers/utils'

import { isStageTamagotchi } from '@proj-airi/stage-shared'
import { ChatHistory, HearingConfigDialog } from '@proj-airi/stage-ui/components'
import { useAudioAnalyzer } from '@proj-airi/stage-ui/composables'
import { useAudioContext } from '@proj-airi/stage-ui/stores/audio'
Expand All @@ -24,6 +25,7 @@ import ActionAbout from './InteractiveArea/Actions/About.vue'
import ActionViewControls from './InteractiveArea/Actions/ViewControls.vue'
import ViewControlInputs from './ViewControls/Inputs.vue'

import { useTranscriptions } from '../../composables/use-transcriptions'
import { BackgroundDialogPicker } from '../Backgrounds'

const { isDark, toggleDark } = useTheme()
Expand Down Expand Up @@ -66,6 +68,14 @@ function isMobileDevice() {
return /Mobi|Android|iPhone|iPad|iPod|BlackBerry|IEMobile|Opera Mini/i.test(navigator.userAgent)
}

const { isListening } = useTranscriptions(
{
messageInputRef: messageInput,
sendMessage: handleSend,
isStageTamagotchi,
},
)

async function handleSubmit() {
if (!isMobileDevice()) {
await handleSend()
Expand Down Expand Up @@ -121,7 +131,7 @@ async function setupAnalyzer() {
analyzerSource.connect(analyser)
}

watch([hearingDialogOpen, enabled, stream], () => {
watch([enabled], () => {
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot May 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Watch dialog and stream changes when setting up analyzer

setupAnalyzer() still gates on hearingDialogOpen and stream, but the watcher now only tracks enabled. If the mic is already enabled, opening the hearing dialog or receiving the stream later will no longer re-run setup, so the volume meter can remain inactive until enabled is toggled again. This is a regression from the previous dependency set and affects normal UI interaction on mobile.

Useful? React with 👍 / 👎.

setupAnalyzer()
}, { immediate: true })

Expand Down Expand Up @@ -191,7 +201,7 @@ onMounted(() => {
title="Hearing"
>
<Transition name="fade" mode="out-in">
<IndicatorMicVolume v-if="enabled" size-5 color-class="text-neutral-500 dark:text-neutral-400" />
<IndicatorMicVolume v-if="enabled" size-5 :color-class="isListening ? undefined : 'text-neutral-500 dark:text-neutral-400'" />
<div v-else i-solar:microphone-3-outline size-5 text="neutral-500 dark:neutral-400" />
</Transition>
</button>
Expand Down
Loading
Loading