Unexpected token in JSON from a byte-order mark

The file looks like perfect JSON and still fails at position 0. The first character is one you cannot see: U+FEFF, the byte-order mark, which some programs write at the start of a UTF-8 file to announce its encoding. Most editors hide it, so the text appears to begin with {, but a JSON parser reads it as a character that has no business there.

{"id": 7}

Removed a byte-order mark

{"id": 7}

Recognising it from the message

Chrome and Node.js report an unexpected token followed by quotes that seem to hold nothing, Unexpected token '', …, because the token between them is the invisible mark itself. The excerpt after it looks like valid JSON, and that combination is the giveaway. Firefox gives no more than an unexpected character at line 1, column 1, the same words it uses for an HTML page, so check the first character before going hunting on the server. Safari calls it an unrecognized token. Python is the only one that names it: Unexpected UTF-8 BOM (decode using utf-8-sig), which also tells you the fix.

To confirm it, look at the first character's code: text.charCodeAt(0) === 0xFEFF in JavaScript, text[:1] == '' in Python, or the first three bytes of the file, EF BB BF, in a hex viewer.

In UTF-8 the mark takes three bytes and carries no information. UTF-8 has only one byte order, so there is nothing for the mark to announce except that the file is UTF-8, which a reader can usually work out anyway. It is a leftover from UTF-16, where the order of the two bytes in each character does matter.

Where it comes from

Windows tools are the usual source. Windows PowerShell 5.1 writes one when you ask Out-File or Set-Content for UTF-8, and several Windows editors offer UTF-8 with BOM as a save option, sometimes as the default. Exports from spreadsheet programs and some database tools add one so that other Windows software detects the encoding. The JSON standard, RFC 8259, says a program must not add a byte-order mark to JSON it sends over a network, and that a parser may ignore one rather than treat it as an error. JSON.parse does not ignore it.

Who strips it for you

Whether you ever see this error depends on how the text was read, because decoding bytes as UTF-8 usually removes the mark:

• Browsers: fetch(…).json() decodes the body first, so a response that starts with a byte-order mark parses without complaint. Checked in Chrome and Firefox.
• Node.js: fs.readFileSync(path, 'utf8') keeps the mark in the string, and JSON.parse then fails. Strip it with text.replace(/^/, '') before parsing.
• Python: json.loads given the raw bytes detects the mark and skips it, but given a str that still holds one, it raises the error above. Open the file with encoding='utf-8-sig', which removes the mark if there is one and changes nothing if there is not. The check is in the json module's loader.

Removing it

The lasting fix is to save the file as UTF-8 without a byte-order mark, which every editor that offers one can also do, or to change the script that writes it. In PowerShell 7, UTF-8 without the mark is the default.

For the text in front of you, paste it into the JSON Validator: its parser names the problem outright, as the last row of the table shows, instead of pointing at a character you cannot see. Repair removes the mark and changes nothing else. Opening a file on this site decodes it the way a browser does, so a mark at the start of a file is already gone by the time it reaches the editor; this message appears when the mark came in with pasted text. For text that fails further along, start from the Unexpected token page, or read the guide to fixing invalid JSON.