[Bug]: Charset comparison in IMAP Charset Converter may fail for ISO-2022-JP

[nobitakun1234]

⚠️ This issue respects the following points: ⚠️

• This is not a troubleshooting question, general support matter, or webserver/proxy problem, but likely a bug (if unsure, ask the Community Help Forum).
• This issue is not already reported on Github OR solved at the Community Help Forum (I've searched!).
• I'm using a maintained major version of Nextcloud Server and tested against the latest patch level. (Supported major versions and current patch levels).
• I agree to follow Nextcloud's Code of Conduct.
• I've tried my best to provide clear reproduction steps that someone unfamiliar with this bug could use to reproduce it.

Bug description

I found a possible charset handling issue in lib/IMAP/Charset/Converter.php.

When converting the charset of an IMAP message, the code currently checks whether the charset is included in mb_list_encodings() as follows:

if (in_array($charset, mb_list_encodings(), true)) {
$converted = mb_convert_encoding($data, 'UTF-8', $charset);
} else {
$converted = iconv($charset, 'UTF-8', $data);
}

In my environment, the charset value can be reported with different casing, while mb_list_encodings() returns charset names with a different casing.

Changing the comparison to:

if (in_array(strtoupper($charset), mb_list_encodings(), true)) {

allows the comparison to be case-insensitive and appears to select the mb_convert_encoding() path as expected.

I am investigating an issue where emails encoded as ISO-2022-JP are displayed incorrectly in Nextcloud Mail, although the same emails are displayed correctly in other mail clients.

During the investigation, I found that the charset conversion logic depends on the exact string comparison between $charset and the values returned by mb_list_encodings().

For example, my tests showed:

UTF-8: OK
ISO-2022-JP: NG
EUC-JP: NG
SJIS: NG

The relevant code is:

if (in_array($charset, mb_list_encodings(), true)) {
$converted = mb_convert_encoding($data, 'UTF-8', $charset);
} else {
$converted = iconv($charset, 'UTF-8', $data);
}

Using strtoupper() for the comparison changes the behavior:

if (in_array(strtoupper($charset), mb_list_encodings(), true)) {
$converted = mb_convert_encoding($data, 'UTF-8', $charset);
} else {
$converted = iconv($charset, 'UTF-8', $data);
}

This is not intended as a final fix yet; I am reporting it as a possible bug in the charset detection/selection logic.

Steps to reproduce

1. Set up Nextcloud with the versions listed below and configure an IMAP mail account in Nextcloud Mail.
2. Send or receive an email whose MIME body is encoded as ISO-2022-JP and whose Content-Type specifies charset=ISO-2022-JP (or another casing of the same charset name).
3. Open the message in Nextcloud Mail.
4. Observe that the Japanese message body is displayed with the ISO-2022-JP escape sequences visible instead of being converted to UTF-8.

During investigation, modifying Converter.php so that the charset comparison uses strtouppe r($charset) caused the affected message to be converted correctly. Reverting that change restored the original behavior.

Expected behavior

Readable Japanese characters are displayed.
For example, the same message is displayed correctly by other mail clients:

Original:
申し添え遅れましたが、メーリングリストの送付先はこちらのアドレスです。
これはNextcloud MailのISO-2022-JP文字コード試験です。

Corrupted Text:
^[$B?=$7E:$(CY$l$^$7$?$,!"%a!<%j%s%0%j%9%H$NAwIU@h$O^[(B
^[$B$3$A$i$N%"%I%l%9$G$9!#^[(B

^[$B$3$l$O^[(BNextcloud Mail^[$B$N^[(BISO-2022-JP^[$BJ8;z%3!<%I;n83$G$9!#^[(B

Nextcloud Server version

32

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.5

Web server

No response

Database engine version

No response

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

No response

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "upgrade.disable-web": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "xx:8080",
            "localhost",
            "xxxx",
            "xxxxx"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "32.0.7.1",
        "overwriteprotocol": "https",
        "overwrite.cli.url": ",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "app_install_overwrite": [
            "otpmanager"
        ],
        "twofactor_enforced": "true",
        "twofactor_enforced_groups": [
            "staff",
            "staff_2",
            "student"
        ],
        "twofactor_enforced_excluded_groups": [
            "staff_3"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "forwarded_for_headers": [
            "HTTP_X_FORWARDED_FOR"
        ],
        "loglevel": 0,
        "maintenance": false,
        "contactsinteraction.enabled": false,
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "mail_smtpport": "25"
    }
}

List of activated Apps

Enabled:
  - activity: 5.0.0
  - app_api: 32.0.0
  - bruteforcesettings: 5.0.0
  - calendar: 6.2.2
  - circles: 32.0.0
  - cloud_federation_api: 1.16.0
  - comments: 1.22.0
  - contacts: 8.3.7
  - contactsinteraction: 1.13.1
  - dashboard: 7.12.0
  - dav: 1.34.2
  - decidesk: 1.0.0
  - deck: 1.16.7
  - federatedfilesharing: 1.22.0
  - federation: 1.22.0
  - files: 2.4.0
  - files_downloadlimit: 5.0.0-dev.0
  - files_pdfviewer: 5.0.0
  - files_reminders: 1.5.0
  - files_sharing: 1.24.1
  - files_trashbin: 1.22.0
  - files_versions: 1.25.0
  - firstrunwizard: 5.0.0
  - logreader: 5.0.0
  - lookup_server_connector: 1.20.0
  - mail: 5.7.6
  - nextcloud_announcements: 4.0.0
  - notifications: 5.0.0
  - oauth2: 1.20.0
  - password_policy: 4.0.0
  - photos: 5.0.0
  - privacy: 4.0.0
  - profile: 1.1.0
  - provisioning_api: 1.22.0
  - recommendations: 5.0.0
  - related_resources: 3.0.0
  - serverinfo: 4.0.0
  - settings: 1.15.1
  - sharebymail: 1.22.0
  - spreed: 22.0.10
  - support: 4.0.0
  - survey_client: 4.0.0
  - systemtags: 1.22.0
  - text: 6.0.1
  - theming: 2.7.0
  - twofactor_backupcodes: 1.21.0
  - twofactor_totp: 14.0.0
  - twofactor_webauthn: 2.6.0
  - updatenotification: 1.22.0
  - user_status: 1.12.0
  - viewer: 5.0.0
  - weather_status: 1.12.0
  - webhook_listeners: 1.3.0
  - workflowengine: 2.14.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

Additional info

No response

[odzhychko]

Hi @nobitakun1234, thx for the report.

This issue is a duplicate of #11416 and worked on in #12654.