Investigate the invisible marker before the first import header
A file can look correct in a spreadsheet and still fail the contract used by an importer. The first header may carry an encoding signature that the viewer hides but the importer treats as part of the field name. Guessing a product match from the remaining columns would turn a detectable format problem into a record-integrity risk.
This fictional investigation uses a documented UTF-8 input contract that permits an optional byte order mark at the beginning of the file. The correction applies only at that boundary. It is not a rule to delete invisible characters from arbitrary data.
Preserve the rejected file and the reported reason
A supplier exports a small catalogue update with the columns reference, description and finish. The import rejects the file because its required reference header is absent. A spreadsheet displays that heading normally, and the two other headers are recognised.
The investigator retains an authorised copy of the rejected bytes and examines the first decoded header. It is U+FEFF followed by reference, rather than the exact expected name. The initial UTF-8 bytes are EF BB BF, the byte order mark or signature described in Unicode guidance.
The supplier's documented exchange contract permits that initial signature. The importer has decoded the file as plain UTF-8 text and left the marker attached to the first header. The strict header check therefore rejects the file before any product update occurs.
| Evidence | Finding in the example |
|---|---|
| Displayed spreadsheet heading | Looks like reference |
| Decoded first header | Contains an initial U+FEFF before reference |
| Input contract | UTF-8, optional signature at file start, exact named headers |
| Import result | Rejected before record writes |
| Same records without the initial signature | Headers recognised in the controlled comparison |
This evidence supports a decoding-boundary diagnosis. It does not justify treating every unknown header as a harmless spelling variation.
Correct the decoder, not the product identity
In the imagined importer, a supported UTF-8-with-signature decoding option consumes the permitted signature only at the beginning of the input. The normal parser then receives the intended reference header and applies the existing exact-header and record-validation rules.
The change does not search the entire file and remove every U+FEFF character. An interior occurrence may be data that needs explicit handling under the field contract. Quietly deleting it could alter a reference or conceal a different invalid input.
The importer also retains its refusal to guess missing product identities. A descriptive name that resembles a catalogue entry is not a substitute for the approved reference field. If the required header or an item relationship remains invalid after decoding, the record must follow the existing rejection or review path.
Check the boundary with contrasting files
The fictional verification uses a fixed, safe record set and records these outcomes:
| Input | Outcome |
|---|---|
| Permitted signature at file start | Signature consumed; expected headers recognised |
| Same UTF-8 file without a signature | Same parsed references and values |
Unknown header product_guess instead of reference |
Still rejected; no automatic renaming |
Description containing caffè |
Accented value preserved through decoding and parsing |
| Interior U+FEFF in a value | Not globally removed; retained for the field's defined validation or review |
The first two files produce equivalent intended records after the format boundary is handled. The unknown-header control proves that the change has not weakened the exchange contract into guesswork. The accented and interior-character controls protect the content from an overly broad cleanup.
The owner accepts the decoder correction for this documented feed. The original rejected attempt remains part of the investigation record. Any real reprocessing would need an authorised operation and a check that no partial writes occurred earlier; the fictional case establishes that rejection happened before writes.
Agree the contract with the exporter
A durable fix includes the parties that produce and consume the file. Record the encoding, permitted signature, delimiter, quoting, expected header names and treatment of invalid rows. A filename ending in .csv does not specify all of those details.
If a different supplier's contract forbids the signature or uses another encoding, this repair cannot simply be applied by analogy. The correct behaviour may be rejection with a useful explanation. The boundary should follow the actual agreement rather than a developer's preferred cleanup routine.
Use the Unicode FAQ on byte order marks for the primary encoding explanation. For a Python implementation, the documented utf-8-sig codec is one example of start-of-stream signature handling; other implementations need their own supported equivalent and tests.
Close the operational question
The completed result is not “all imports now work.” It is that this permitted input signature no longer prevents the required header from being recognised, while exact record identities and other validation remain intact.
The integration service places file exchange alongside account capabilities and record ownership. The support page explains how to report an import failure without sending a broad private data export. A small representative file, the stated contract and the actual rejection reason are often the most useful starting evidence.