Error with findColumnWithAllEntries(ids, metadata) in summarizedExperiment

[nvribeiro]

Description of the bug

Hi!
I am getting an error caused by findColumnWithAllEntries(ids, metadata) during the process NFCORE_RNASEQ:RNASEQ:QUANTIFY_PSEUDO_ALIGNMENT:QUANT_TXIMPORT_SUMMARIZEDEXPERIMENT:SE_GENE_UNIFIED (all_samples).

I saw the #1445, but that doesn't seem to be the case for me, since my sample names don't start with number nor have hyphens. The problem seems to be with the gene names. I check my GTF and txome FASTA files and they seem to have the same transcript names (the .X in the end).

Command used and terminal output

Script:
#!/bin/bash
#SBATCH --job-name=nf-rnaseq
#SBATCH --output=%x.out
#SBATCH --error=%x.err
#SBATCH --time=02:59:59
#SBATCH --cpus-per-task=12
#SBATCH --mem=50gb
#SBATCH --nodes=1

module load Java/21.0.7

SAMPLESHEET=/groups/umcg-immunogenetics/tmp02/users/NathanRibeiro/nf-rna-test/samplesheet.csv
OUTDIR=/groups/umcg-immunogenetics/tmp02/users/NathanRibeiro/nf-rna-test/output
GTF=/groups/umcg-immunogenetics/tmp02/users/NathanRibeiro/ref_genomes/hg38/alias/hg38/gencode_gtf/default/hg38.gtf.gz
FASTA=/groups/umcg-immunogenetics/tmp02/users/NathanRibeiro/ref_genomes/hg38/alias/hg38/fasta/default/hg38.fa
TXFASTA=/groups/umcg-immunogenetics/tmp02/users/NathanRibeiro/ref_genomes/hg38/alias/hg38/fasta_txome/default/hg38.fa

nextflow run nf-core/rnaseq \
        --input $SAMPLESHEET \
        --outdir $OUTDIR \
        --gtf $GTF \
        --gencode \
        --fasta $FASTA \
        --transcript_fasta $TXFASTA \
        --skip_alignment \
        --pseudo_aligner salmon \
        --extra_salmon_quant_args '--seqBias --validateMappings' \
        -r 3.23.0 \
        -profile singularity

Output:
Command exit status:
  1

Command output:
  (empty)

Command error:
  33289  ENST00000417267.1  ENSG00000234793.1      AC114730.2
  33290  ENST00000192314.6 ENSG00000154252.11         GAL3ST2
  33291  ENST00000437438.1  ENSG00000215023.2      AC131097.1
  33292  ENST00000413820.1  ENSG00000224272.2      AC131097.3
  33293  ENST00000439270.1  ENSG00000224272.2      AC131097.3
  33294  ENST00000420272.2  ENSG00000224272.2      AC131097.3
  33295  ENST00000407683.5 ENSG00000204099.11            NEU4
  33296  ENST00000415936.5 ENSG00000204099.11            NEU4
  33297  ENST00000405370.5 ENSG00000204099.11            NEU4
  33298  ENST00000476542.5 ENSG00000204099.11            NEU4
  33299  ENST00000435855.5 ENSG00000204099.11            NEU4
  33300  ENST00000618597.1 ENSG00000204099.11            NEU4
  33301  ENST00000423583.5 ENSG00000204099.11            NEU4
  33302  ENST00000404257.5 ENSG00000204099.11            NEU4
  33303  ENST00000391969.6 ENSG00000204099.11            NEU4
  33304 ENST00000325935.10 ENSG00000204099.11            NEU4
  33305  ENST00000435934.1 ENSG00000204099.11            NEU4
  33306  ENST00000435894.5 ENSG00000204099.11            NEU4
  33307  ENST00000426032.5 ENSG00000204099.11            NEU4
  33308  ENST00000420288.1 ENSG00000204099.11            NEU4
  33309  ENST00000494678.1 ENSG00000204099.11            NEU4
  33310  ENST00000488997.1 ENSG00000204099.11            NEU4
  33311  ENST00000428592.1 ENSG00000204099.11            NEU4
  33312  ENST00000442867.1  ENSG00000235151.1      AC131097.4
  33313  ENST00000334409.9 ENSG00000188389.10           PDCD1
  33314  ENST00000418831.1 ENSG00000188389.10           PDCD1
  33315  ENST00000343705.3 ENSG00000188389.10           PDCD1
  33316  ENST00000419912.1  ENSG00000188011.5            RTP5
  33317  ENST00000343216.3  ENSG00000188011.5            RTP5
  33318  ENST00000430555.5  ENSG00000233806.7       LINC01237
  33319  ENST00000415434.5  ENSG00000233806.7       LINC01237
  33320  ENST00000457686.5  ENSG00000233806.7       LINC01237
  33321  ENST00000429947.1  ENSG00000233806.7       LINC01237
  33322  ENST00000439601.1  ENSG00000233806.7       LINC01237
  33323  ENST00000404031.5  ENSG00000216921.8      AC131097.2
  33324  ENST00000452112.5  ENSG00000216921.8      AC131097.2
  33325  ENST00000401641.2  ENSG00000216921.8      AC131097.2
  33326  ENST00000429456.1  ENSG00000237940.3       LINC01238
  33327  ENST00000416204.1  ENSG00000237940.3       LINC01238
  33328  ENST00000426962.1  ENSG00000237940.3       LINC01238
  33329  ENST00000451070.1  ENSG00000237940.3       LINC01238
  33330  ENST00000625132.1  ENSG00000280119.1      AC093642.2
  33331  ENST00000412193.1  ENSG00000226423.1      AC093642.1
  33332  ENST00000446593.1  ENSG00000232002.1       LINC01880
  33333  ENST00000567549.2  ENSG00000261186.2       LINC01238
   [ reached 'max' / getOption("max.print") -- omitted 173361 rows ]
  Error in findColumnWithAllEntries(ids, metadata) : 
    No column contains all vector entries  ENSG00000000003.14, ENSG00000000005.5, ENSG00000000419.12, ENSG00000000457.13, ENSG00000000460.16, ENSG00000000938.12, ENSG00000000971.15, ENSG00000001036.13, ENSG00000001084.12, ENSG00000001167.14, ENSG00000001460.17, ENSG00000001461.16, ENSG00000001497.16, ENSG00000001617.11, ENSG00000001626.15, ENSG00000001629.9, ENSG00000001630.16, ENSG00000001631.15, ENSG00000002016.17, ENSG00000002079.14, ENSG00000002330.13, ENSG00000002549.12, ENSG00000002586.19, ENSG00000002587.9, ENSG00000002726.20, ENSG00000002745.12, ENSG00000002746.14, ENSG00000002822.15, ENSG00000002834.17, ENSG00000002919.14, ENSG00000002933.8, ENSG00000003056.7, ENSG00000003096.14, ENSG00000003137.8, ENSG00000003147.18, ENSG00000003249.13, ENSG00000003393.14, ENSG00000003400.14, ENSG00000003402.19, ENSG00000003436.15, ENSG00000003509.15, ENSG00000003756.16, ENSG00000003987.13, ENSG00000003989.17, ENSG00000004059.10, ENSG00000004139.13, ENSG00000004142.11, ENSG00000004399.12
  Calls: parse_metadata -> findColumnWithAllEntries
  Execution halted

Relevant files

nextflow.log

System information

Nextflow version 25.10.4
Hardware: HPC
Executor: Slurm
Container: Singularity
OS: Linux Rocky 9.6
Version of nf-core/rnaseq: 3.23.0

[pinin4fjords]

Hi @nvribeiro, thanks for reporting this.

This error happens when there's a mismatch between your --transcript_fasta and --gtf. When the transcript FASTA contains transcripts that aren't in the GTF, the pipeline creates placeholder gene entries using the transcript ID as the gene ID. These "fake" gene entries then can't be matched against the tx2gene mapping during SummarizedExperiment construction, which produces the findColumnWithAllEntries error you're seeing.

This is most likely because your transcript FASTA (fasta_txome) and GTF were generated from different sources or GENCODE versions, even if the IDs look similar at a glance.

The fix is to drop --transcript_fasta and let the pipeline generate the transcriptome from your --fasta and --gtf, which guarantees they're consistent. From the usage docs:

We recommend not providing a transcriptome FASTA file and instead allowing the pipeline to create it from the provided genome and annotation.

So your command would become:

nextflow run nf-core/rnaseq \
    --input $SAMPLESHEET \
    --outdir $OUTDIR \
    --gtf $GTF \
    --gencode \
    --fasta $FASTA \
    --skip_alignment \
    --pseudo_aligner salmon \
    --extra_salmon_quant_args '--seqBias --validateMappings' \
    -r 3.23.0 \
    -profile singularity

You can pass --save_reference to save the generated transcriptome for future runs so it doesn't need to be rebuilt each time.

We've also opened a PR (nf-core/modules#11141) to add defensive handling for this case in a future release. It will filter the problematic entries and emit a clear warning pointing to the FASTA/GTF mismatch, rather than the current cryptic R error.

[nvribeiro]

Thank you so much! That worked =)