Python 3.15 Defaults to UTF-8: What Happens to Legacy CSV Files?

You upgrade to Python 3.15 on Windows, and a script that has read the same CSV for years now fails on it. What changed, when neither the file nor the code did?

Python 3.15 Defaults to UTF-8: What Happens to Legacy CSV Files?
TLDR
  • Python 3.15 enables UTF-8 Mode by default. Text operations that omit an encoding deserve attention during an upgrade.
  • A legacy file is not converted when Python is upgraded. Specify the encoding used by its producer.
  • Use EncodingWarning to find reliance on defaults, then exercise your input and output paths with UTF-8 Mode enabled.

This changes text I/O, not the encoding of Python source code

Python 3.15 arrived on October 9, followed by 3.15.1 on October 21. One consequential change for existing scripts is that UTF-8 Mode is now enabled by default.

PEP 686 explains the compatibility problem: code that relied on the platform's default encoding can encounter decoding errors, garbled text or silent corruption after the switch. Windows installations using a non-UTF-8 locale are particularly relevant. Python source files already defaulted to UTF-8.

An expression such as open("sales.csv") does not detect a file's encoding. The 3.15 I/O documentation now specifies UTF-8 as the default; disabling UTF-8 Mode restores locale-dependent behavior. Neither setting rewrites an existing file.

Find calls that rely on an implicit default

The opt-in warning facility reports use of the default encoding:

python -X warn_default_encoding your_script.py

Make sure python selects the interpreter you intend to examine. Exercise representative files and less common branches: a runtime warning can only tell you about code that runs. It is neither an encoding detector nor a static scan of every possible execution path.

Choose an encoding from the file contract

Use the producer's specification rather than the filename or the language of the text. These choices apply when you have established how the file was encoded:

Known input formatRead withResult
UTF-8 textencoding="utf-8"Decode as UTF-8 regardless of the locale.
A Korean export encoded as CP949encoding="cp949"Use the encoding that produced those bytes.
UTF-8 text that may start with a BOMencoding="utf-8-sig"Skip a leading UTF-8 BOM when decoding.

The standard codec documentation lists CP949 and describes UTF-8 signature handling. For a confirmed CP949 export, an explicit read looks like this:

with open("sales.csv", encoding="cp949") as source:
    text = source.read()

This specifies decoding. To migrate the file itself, decode its original bytes and write the resulting text using the agreed target encoding. The application receiving that output must use the same contract. CP949 is one legacy example, not a default to apply to every Korean-language file.

Use the compatibility switch to diagnose, then make the contract explicit

The documented UTF-8 options let you compare the two modes:

python -X utf8=0 -X warn_default_encoding your_script.py
python -X utf8 your_script.py

Disabling UTF-8 Mode restores the environment's locale encoding; it does not force every computer to use the same legacy codec. When following the operating system locale is intentional, encoding="locale" expresses that choice even in UTF-8 Mode, as PEP 686 specifies.

For exchanged business files, agree on the encoding with the producer and consumer. Replace accidental reliance on defaults, then test both reading and writing under UTF-8 Mode.

Frequently asked questions

Does upgrading Python convert existing CSV files?

No. The default used to decode or encode text changes. Converting a file requires reading its original encoding and writing the chosen target encoding.

Does a .csv extension identify the encoding?

No. Use the encoding specified by the system that produced the file; the filename alone does not identify it.

Start the migration review at file operations that omit an encoding. An explicit contract removes an assumption that previously depended on the machine.