UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte — how to fix

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

Quick fix: Pass the real encoding: open(path, encoding='cp1252') (Windows Excel CSVs), 'utf-16' (byte 0xff/0xfe at position 0), or 'latin-1'. For binary files, open with 'rb'.

What it means

UTF-8 has strict rules for which byte sequences are valid. The file contains a byte that can't appear at that point in UTF-8, so it wasn't written as UTF-8.

Reading the clue in the message

Fix: specify the encoding

with open("data.csv", encoding="cp1252") as f:
    text = f.read()

pandas:

df = pd.read_csv("data.csv", encoding="cp1252")     # or "utf-16", "latin-1"

latin-1 never raises this error (every byte is valid), which makes it a handy fallback, but accented characters may come out wrong if the real encoding was different.

Don't know the encoding?

file -i data.csv

or in Python:

pip install charset-normalizer

from charset_normalizer import from_path
print(from_path("data.csv").best().encoding)

Keep going and replace bad bytes

open("log.txt", encoding="utf-8", errors="replace")   # bad bytes become �

It's a binary file

Images, PDFs, Excel .xlsx and pickles aren't text. Open them in binary mode ("rb") or with the right library (openpyxl/pd.read_excel for .xlsx).

On Windows, writing files

The reverse problem (UnicodeEncodeError: 'charmap' codec can't encode) comes from Windows' default encoding. Always pass encoding="utf-8" when writing, or set PYTHONUTF8=1.

Last updated 2026-10-27.