vllm can not run successfully

[primerL]

I run CUDA_VISIBLE_DEVICES=0,1 python inference_vllm.py --cfg configs/example_config_visual_guidance.py got below:

INFO 12-07 14:22:38 [__init__.py:216] Automatically detected platform cuda.
Traceback (most recent call last):
  File "/auto/u/bl5256/Emu3.5/inference_vllm.py", line 155, in <module>
    main()
  File "/auto/u/bl5256/Emu3.5/inference_vllm.py", line 129, in main
    model, tokenizer, vq_model = build_emu3p5_vllm(
                                 ^^^^^^^^^^^^^^^^^^
  File "/auto/u/bl5256/Emu3.5/src/utils/model_utils.py", line 82, in build_emu3p5_vllm
    from vllm import LLM
  File "/n/fs/vision-mix/bl5652/conda_envs/Emu3p5/lib/python3.12/site-packages/vllm/__init__.py", line 68, in __getattr__
    module = import_module(module_name, __package__)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/fs/vision-mix/bl5652/conda_envs/Emu3p5/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/fs/vision-mix/bl5652/conda_envs/Emu3p5/lib/python3.12/site-packages/vllm/entrypoints/llm.py", line 17, in <module>
    from vllm.config import (CompilationConfig, ModelDType,
  File "/n/fs/vision-mix/bl5652/conda_envs/Emu3p5/lib/python3.12/site-packages/vllm/config/__init__.py", line 52, in <module>
    from vllm.multimodal import MULTIMODAL_REGISTRY
  File "/n/fs/vision-mix/bl5652/conda_envs/Emu3p5/lib/python3.12/site-packages/vllm/multimodal/__init__.py", line 8, in <module>
    from .registry import MultiModalRegistry
  File "/n/fs/vision-mix/bl5652/conda_envs/Emu3p5/lib/python3.12/site-packages/vllm/multimodal/registry.py", line 15, in <module>
    from .processing import (BaseMultiModalProcessor, BaseProcessingInfo,
  File "/n/fs/vision-mix/bl5652/conda_envs/Emu3p5/lib/python3.12/site-packages/vllm/multimodal/processing.py", line 19, in <module>
    from vllm.transformers_utils.processor import cached_processor_from_config
  File "/n/fs/vision-mix/bl5652/conda_envs/Emu3p5/lib/python3.12/site-packages/vllm/transformers_utils/processor.py", line 7, in <module>
    from transformers import (AutoFeatureExtractor, AutoImageProcessor,
ImportError: cannot import name 'AutoVideoProcessor' from 'transformers' (/n/fs/vision-mix/bl5652/conda_envs/Emu3p5/lib/python3.12/site-packages/transformers/__init__.py)

I'm sure that transformers==4.48.2

[aikx]

for vllm inference, plz:

# Requires Python 3.12 or higher.
# Recommended: CUDA 12.8.
pip install -r requirements/vllm.txt
pip install flash_attn==2.8.3 --no-build-isolation

cd Emu3.5
python src/patch/apply.py

and transformers will not be 4.48.2

[primerL]

hi I want to check that to generate interleave image and text, the prompt should have some special words? I mean, I use the prompt like:

I am preparing a figure for my paper to illustrate the following framework:
This figure illustrates the overall pipeline of the HuMoCon framework for concept discovery in human motion understanding. The pipeline consists of two stages: encoder pretraining and LLM finetuning.

(a) **Encoder Pretraining**: A VQ-VAE-based architecture jointly encodes video and motion sequences through dual encoders and decoders. The training process includes masked reconstruction to recover the input from codebooks, velocity reconstruction to emphasize high-frequency motion details, and feature projection for alignment between modalities.

(b) **LLM Finetuning**: This stage contains two sub-modules. The **Modality Translation** module learns to map encoded video and motion features into the LLM embedding space via tunable translation layers. The **Instruction Tuning** module fine-tunes the LLM to perform downstream reasoning tasks related to video and motion, leveraging the translated features. Please generate: 1. A visual diagram showing how these components interact. 2. A separate caption summarizing the key idea of the figure. 

I use howto to generate，but the result only text（I can generate image and text if other prompt)

[lakshya-skyfall]

for vllm inference, plz:

# Requires Python 3.12 or higher.
# Recommended: CUDA 12.8.
pip install -r requirements/vllm.txt
pip install flash_attn==2.8.3 --no-build-isolation

cd Emu3.5
python src/patch/apply.py

and transformers will not be 4.48.2

I did the same but get the following error:

File "/workspace/Emu3.5/src/utils/model_utils.py", line 9, in <module>                                                                                                                                  
      from ..emu3p5 import Emu3ForCausalLM, Emu3Config                                                                                                                                                      
    File "/workspace/Emu3.5/src/emu3p5/__init__.py", line 2, in <module>                                                                                                                                    
      from .modeling_emu3 import Emu3ForCausalLM                                                                                                                                                            
    File "/workspace/Emu3.5/src/emu3p5/modeling_emu3.py", line 55, in <module>                                                                                                                              
      from transformers.utils.import_utils import is_torch_fx_available                                                                                                                                     
  ImportError: cannot import name 'is_torch_fx_available' from 'transformers.utils.import_utils' (/workspace/emu_vllm/lib/python3.12/site-packages/transformers/utils/import_utils.py)

[BoggySyb]

for vllm inference, plz:

# Requires Python 3.12 or higher.
# Recommended: CUDA 12.8.
pip install -r requirements/vllm.txt
pip install flash_attn==2.8.3 --no-build-isolation

cd Emu3.5
python src/patch/apply.py

and transformers will not be 4.48.2

I did the same but get the following error:

File "/workspace/Emu3.5/src/utils/model_utils.py", line 9, in <module>                                                                                                                                  
      from ..emu3p5 import Emu3ForCausalLM, Emu3Config                                                                                                                                                      
    File "/workspace/Emu3.5/src/emu3p5/__init__.py", line 2, in <module>                                                                                                                                    
      from .modeling_emu3 import Emu3ForCausalLM                                                                                                                                                            
    File "/workspace/Emu3.5/src/emu3p5/modeling_emu3.py", line 55, in <module>                                                                                                                              
      from transformers.utils.import_utils import is_torch_fx_available                                                                                                                                     
  ImportError: cannot import name 'is_torch_fx_available' from 'transformers.utils.import_utils' (/workspace/emu_vllm/lib/python3.12/site-packages/transformers/utils/import_utils.py)

I encountered the same issue as you.
Have you managed to resolve this issue? If so, could you please share how you fixed it?