A filename can lie.
Rename a JPEG photograph from holiday.jpg to holiday.txt and the bytes inside the file do not suddenly become text. Remove the extension entirely and the image does not stop being a JPEG. Save an old word-processing document with a familiar extension and the extension may tell you only a family name, not the precise version or encoding that determines whether future software can open it correctly.
That small mismatch points to a large preservation problem. Before an archive can decide how to store, migrate, validate, render, quarantine or preserve a digital object, it needs to know what the object actually is.
File format identification is the discipline of turning an opaque sequence of bytes into a defensible technical statement: this object appears to conform to this format, and often this particular version, based on these observable characteristics.
It sounds like housekeeping. In a serious digital repository it is closer to diagnosis.
The short answer
File format identification combines several clues: filenames and extensions, MIME types, internal byte patterns, structural characteristics, container information and registry knowledge. The National Archives in the United Kingdom describes precise format identification as a fundamental requirement of a digital repository. Its DROID tool performs automated batch identification and links results to PRONOM, its technical registry of file formats.
PRONOM records more than two thousand individual file formats and assigns persistent PRONOM Unique Identifiers, usually called PUIDs, such as fmt/18 or fmt/199. DROID uses internal and external signatures derived from PRONOM to identify files at scale.
But identification is only one stage. It does not automatically prove that a file is valid, complete, authentic, safe, readable, preservable or correctly described. The preservation chain is stronger when we keep those claims separate.
1. Why the extension is useful — and why it is not enough
Most people first meet file formats through extensions: .pdf, .docx, .jpg, .csv, .mp3, .zip. Extensions are useful because operating systems and applications use them as quick labels. They help users choose programs and help software make routing decisions.
They are also weak evidence.
- An extension can be missing.
- It can be mistyped.
- It can be changed deliberately or accidentally.
- One extension can cover several versions or variants.
- Different formats can share similar naming conventions.
- A container can hide many embedded formats behind one outer extension.
- Legacy systems may use local or proprietary naming practices.
So a repository should treat the extension as a clue rather than an oracle.
This distinction is easy to miss because the extension is visible while the format is usually invisible. Yet preservation has to care about the invisible layer. Future readability depends on the byte-level encoding, structural rules and dependencies that software actually encounters.
2. A format is a set of rules for turning bytes into information
The Library of Congress describes a digital format as a byte-serialised encoding of an information model. In practical terms, a file format defines how sequences of bits and bytes are organised so that software can interpret them as text, image, sound, video, data, geometry, layout, executable instructions or some other meaningful object.
Consider the difference between:
- a plain UTF-8 text file;
- a JPEG image;
- a ZIP archive;
- a Microsoft Office Open XML document;
- a SQLite database;
- a TIFF image with embedded metadata;
- a proprietary CAD model.
Each can be stored as bytes. What makes the bytes useful is the format specification, convention or implementation knowledge that tells software how to parse them.
That means preservation is not merely about keeping bytes intact. It is also about preserving enough knowledge to interpret those bytes later. This is why format registries matter.
3. The diagnostic ladder: extension → signature → structure → validation → preservation meaning
A useful way to think about format work is as a ladder of increasingly strong claims.
NAME OR EXTENSION → EXTERNAL SIGNATURE → INTERNAL SIGNATURE → STRUCTURAL IDENTIFICATION → FORMAT + VERSION → VALIDATION → CHARACTERISATION → PRESERVATION RISK → PRESERVATION ACTION
Each step answers a different question.
| Stage | Question |
|---|---|
| Extension | What does the filename claim? |
| Signature | Do characteristic bytes match a known format? |
| Structure | Does the internal organisation resemble the expected format? |
| Version identification | Which particular variant or version is likely? |
| Validation | Does the object conform to the formal rules of that format? |
| Characterisation | What significant technical properties does the object have? |
| Risk analysis | What threatens future usability? |
| Preservation action | What should the repository do? |
This ladder prevents a common failure: taking a weak early signal and silently promoting it into a stronger conclusion.
4. External signatures are quick but shallow
An external signature is usually something visible outside the file contents, most commonly the extension. It can be useful for triage and can sometimes identify a format family with high probability.
But two files both named .doc may not be technically identical, and a file called .xml tells us that it is XML without necessarily telling us which XML-based application vocabulary it contains.
External signatures work best as one feature in a larger identification system. They can help narrow candidates, flag inconsistencies and detect obviously strange cases, but preservation-grade identification benefits from evidence inside the object itself.
5. Internal signatures look inside the file
Many formats contain recognisable byte sequences at predictable locations. These can be magic numbers, headers, markers, identifiers or combinations of byte patterns and offsets.
For example, a format may begin with a characteristic sequence at the beginning of file, often abbreviated BOF. Another may place identifying information near the end of file, or at a fixed offset. Still others require several fragments to distinguish them from related formats.
DROID uses signatures drawn from PRONOM. The PRONOM record for its own signature-file format explains that DROID signature files contain collections of internal and external signatures and that updated signature files are issued as new formats are added to the registry.
This gives identification an important property: it is not just an algorithm. It is an algorithm connected to a maintained body of technical knowledge.
6. Signatures are evidence, not magic
A byte signature can be highly discriminating, but it is still evidence.
False positives are possible if a pattern is too short or too common. False negatives are possible if a format variant has not yet been represented in the registry. Corrupt files may contain enough of a signature to identify a format but still fail to render. Some formats have weak signatures. Some cannot be distinguished reliably without deeper parsing. Some are containers whose outer structure says little about the precise contents.
A robust identification workflow therefore keeps the result and its basis together:
IDENTIFICATION_RESULT + METHOD + SIGNATURE_VERSION + TOOL_VERSION + REGISTRY_REFERENCE + DATE + WARNING OR AMBIGUITY STATE
That turns a label into reproducible evidence.
7. PRONOM gives formats durable identities
PRONOM is an online technical registry maintained by The National Archives. It contains information about formats, versions, extensions, signatures, relationships and technical background. Its public documentation says the database now covers more than 2,000 individual file formats.
The most useful feature for machine workflows is the PUID: the PRONOM Unique Identifier.
A PUID such as fmt/... gives a repository a stable way to refer to a format independently of local wording. That matters because names drift. “PDF”, “Portable Document Format”, “PDF 1.4” and a local database label may all refer at different levels of specificity. A registry identifier can reduce that ambiguity.
This has several operational advantages:
- inventory systems can store machine-readable format identities;
- risk rules can target precise versions;
- migration policies can map from one PUID to another;
- reports can aggregate holdings by format;
- repositories can exchange technical information without relying on local names;
- future staff can resolve an identifier back to registry knowledge.
The identifier does not itself preserve the file. It helps preserve the meaning of the repository’s technical statement about the file.
8. DROID turns registry knowledge into batch identification
DROID stands for Digital Record Object IDentification. It was developed by The National Archives to identify file formats automatically, particularly at repository scale.
DROID can scan files and folders, use current signature data derived from PRONOM, and report results in forms suitable for further analysis. The project’s current documentation supports both a graphical desktop interface and command-line use, which makes the same underlying identification capability useful to archivists working interactively and engineers integrating it into workflows.
The important design pattern is:
FILES → DROID → SIGNATURE MATCHING → PRONOM IDENTITY → REPORT → INVENTORY / RISK ANALYSIS / PRESERVATION ACTION
The tool does not have to decide the entire preservation policy. It supplies one controlled layer of evidence to the larger repository.
9. Signature versions matter
Identification knowledge evolves.
PRONOM gains new formats, new signatures and improved distinctions. DROID can download updated signature files. Therefore the same collection scanned years apart may produce more precise results later even though the underlying files have not changed.
This is not necessarily inconsistency. It can be improved knowledge.
A repository should retain enough information to answer:
- Which version of DROID performed the scan?
- Which signature file was used?
- When was the scan performed?
- Was the file itself unchanged since the previous scan?
- Did the identification become more specific?
- Did a previous identification become deprecated or superseded?
Preservation metadata should therefore record the identification event, not merely overwrite one label with another.
10. Identification is not validation
This is one of the most important boundaries in the entire subject.
The National Archives’ technical literature distinguishes identification from validation. Identification determines what format an object appears to use. Validation asks whether the object actually conforms to the rules of that format.
A file can therefore be:
- identified as PDF but structurally damaged;
- identified as TIFF but non-conformant to a required profile;
- identified as XML but invalid against a particular schema;
- identified as a ZIP-based Office document while containing missing or broken internal parts.
The distinction can be written simply:
IDENTIFICATION: “What is this?” VALIDATION: “Does this obey the rules for what it claims to be?”
Repositories need both questions, and they should never collapse them.
11. Validation is also not preservation suitability
Even a perfectly valid file may be a poor preservation choice.
The Library of Congress evaluates formats using sustainability factors including disclosure, adoption, transparency, self-documentation, external dependencies, patents and technical protection mechanisms. These factors ask whether a format is likely to remain manageable as technologies change.
So there are at least three distinct claims:
IDENTIFIED CORRECTLY ≠ VALID FORMAT ≠ GOOD LONG-TERM PRESERVATION FORMAT
A valid proprietary file dependent on discontinued software may be harder to preserve than a well-documented open format. Conversely, an old format may remain sustainable if specifications, software and migration pathways are strong.
12. Characterisation asks what the file contains and how it behaves
Format identity is only one property of a digital object.
For preservation planning, repositories may also need characteristics such as image dimensions, colour space, compression, audio codec, sample rate, video frame rate, encryption state, embedded fonts, metadata fields, number of pages, presence of scripts, checksums, container members or schema versions.
This is characterisation: extracting technical properties relevant to management and future use.
Two files with the same PUID can still have very different preservation needs because one may use unusual dependencies, embedded content or features that are difficult to migrate.
A useful repository record can therefore resemble:
OBJECT ID FORMAT ID FORMAT VERSION IDENTIFICATION METHOD VALIDATION STATE TECHNICAL PROPERTIES DEPENDENCIES FIXITY RIGHTS PROVENANCE RISK STATE PRESERVATION ACTION
13. Containers make identification more difficult
Many modern formats are containers.
A ZIP file may contain arbitrary files. A DOCX document is itself a package of XML and related resources. A multimedia container may hold video, audio, subtitles, timecodes and metadata. An email archive may contain messages with nested attachments. A disk image may contain an entire filesystem full of objects.
This creates multiple identification layers:
OUTER CONTAINER → INTERNAL PACKAGE TYPE → COMPONENT FILES → EMBEDDED BITSTREAMS → CODECS / DEPENDENCIES
The question “what format is this file?” may therefore have several correct answers at different levels.
Preservation systems should avoid forcing a complex object into one flat label when downstream action depends on the internal structure.
14. MIME types solve a different problem
MIME types are essential to web and messaging infrastructure. Values such as image/jpeg, application/pdf and text/plain help systems communicate broad media types.
But MIME identifiers often operate at a different granularity from preservation registries. A MIME type may describe an entire family while a PUID distinguishes versions or subtypes relevant to preservation.
Neither system makes the other useless. They serve different interoperability needs.
A mature repository can preserve both:
- MIME type for general system interoperability;
- PUID for precise preservation identity;
- human-readable format name for operators;
- local policy class for preservation action.
15. Format identification should occur early
Early identification gives the repository options.
If format analysis happens during or near ingest, the repository can detect surprises before those surprises become embedded in long-term storage.
Examples include:
- a transfer said to contain PDF documents actually contains executable files;
- a collection expected to contain open formats contains proprietary legacy formats;
- extensions do not match signatures;
- files cannot be identified at all;
- formats fall outside the repository’s supported preservation policy;
- unexpected encryption or password protection appears;
- a supposedly homogeneous dataset contains many versions requiring different treatment.
The result can route the object into acceptance, quarantine, manual review, characterisation, migration planning or return to the depositor.
16. Unknown files are not failures to be hidden
A repository will encounter objects that cannot be identified confidently.
The dangerous response is to force a label merely because a workflow expects one.
An honest state machine should allow:
IDENTIFIED AMBIGUOUS PARTIALLY_IDENTIFIED UNKNOWN CORRUPT_OR_UNREADABLE ENCRYPTED_OR_OBSCURED MANUAL_RESEARCH_REQUIRED
Unknown is information. It tells the repository where knowledge is missing.
That gap may trigger sample analysis, software archaeology, comparison with known files, consultation with the depositor, registry research or submission of a new signature to the preservation community.
17. Community registries make preservation cumulative
One institution can discover a format problem. A registry can prevent thousands of institutions from rediscovering it independently.
PRONOM accepts contributions of formats and signatures. That creates a valuable network effect: research by one archive can improve identification for others, while new samples and edge cases can refine earlier signatures.
The preservation ecosystem therefore works best when local discoveries can return to shared infrastructure.
This is one reason open registries matter. A repository’s technical memory should not depend entirely on one employee’s private spreadsheet or one vendor’s undocumented database.
18. A format registry is not a risk register
Knowing what a format is does not automatically tell you how dangerous it is.
Risk depends on context:
- Is the specification public?
- How widely is the format adopted?
- Are multiple independent tools available?
- Does the format depend on particular hardware or online services?
- Is the content encrypted or technically protected?
- Are there licensing or patent constraints?
- Can significant properties be extracted and compared?
- Can the organisation migrate or emulate it?
- How important is the material?
- How severe would loss of functionality be?
The Library of Congress sustainability framework is useful precisely because it moves from identity to preservation feasibility.
19. Preferred formats are policy decisions, not universal truths
Institutions often publish preferred or recommended formats for deposit. The Library of Congress Recommended Formats Statement, for example, considers documentation, standards status, adoption, validation and content-specific requirements when describing preferred characteristics.
But a preferred format for one institution is not automatically the best format for every archive, every purpose or every object.
A preservation policy should connect preference to mission:
REPOSITORY PURPOSE + DESIGNATED COMMUNITY + SIGNIFICANT PROPERTIES + TECHNICAL CAPABILITY + RIGHTS + COST + RISK → FORMAT POLICY
A museum preserving born-digital art, a government records archive, a research-data repository and a school library may make different choices for good reasons.
20. Extensions that disagree with signatures are especially valuable
A disagreement is not merely an error to clean up. It is a diagnostic event.
Suppose a file named report.pdf has an internal signature matching a ZIP-based Office document. Several explanations are possible:
- the file was renamed incorrectly;
- the extension was changed to bypass a system rule;
- the object was exported incorrectly;
- the file is malicious or deceptive;
- the signature match is ambiguous or wrong.
The correct response is not to trust either signal automatically. It is to escalate evidence.
EXTENSION ≠ SIGNATURE → HOLD ASSUMPTION → INSPECT STRUCTURE → VALIDATE → CHECK PROVENANCE → RECORD RESOLUTION
21. Identification helps cybersecurity, but it is not a malware scanner
Format identification can expose suspicious mismatches and unexpected executables. That makes it useful to secure ingest.
However, it should not be confused with malware detection. A correctly identified PDF can still contain malicious content. A valid Office document can contain macros. An archive can contain dangerous payloads. Format tools answer technical identity questions; security tools examine different threat properties.
A strong intake pipeline can combine both:
TRANSFER → INVENTORY → MALWARE / SECURITY CHECK → FORMAT IDENTIFICATION → VALIDATION → CHARACTERISATION → FIXITY → PRESERVATION DECISION
The order may vary, but the responsibilities should remain distinguishable.
22. Identification results belong in preservation metadata
If a repository uses format identification to make preservation decisions, the result should not disappear after the workflow finishes.
The identification event can become part of the object’s provenance and technical metadata:
- tool name and version;
- signature or registry version;
- format name and version;
- PUID;
- MIME type where relevant;
- identification basis;
- warning state;
- date;
- operator or automated agent;
- subsequent validation result.
This lets future staff reinterpret older decisions when registry knowledge improves.
23. Re-identification should be possible without rewriting history
Suppose a file was identified in 2026 as a generic TIFF. In 2030, a richer signature set identifies it as a specific TIFF variant with preservation implications.
The new result should not erase the old event. Both are historically meaningful:
2026 IDENTIFICATION TOOL A / SIGNATURE SET X RESULT: GENERIC TIFF 2030 IDENTIFICATION TOOL A / SIGNATURE SET Y RESULT: SPECIFIC TIFF VARIANT OBJECT FIXITY: UNCHANGED
This tells us the object did not change; our knowledge did.
24. Format risk can be turned into a queue
Once files have stable format identities, the repository can build actionable inventories.
For example:
- all objects in obsolete proprietary formats;
- all files with unknown identification;
- all formats dependent on unsupported software;
- all files using weak or ambiguous signatures;
- all objects needing validation;
- all formats scheduled for migration assessment;
- all encrypted objects without sustainable key management;
- all containers whose embedded content has not yet been characterised.
This is where identification begins to pay operational rent. A format label becomes a way to prioritise work.
25. Risk should not be reduced to “old format = bad”
Age is a signal, not a verdict.
Some old formats remain widely supported and well documented. Some newer formats depend heavily on proprietary cloud services, remote APIs or opaque software stacks. A preservation system should therefore evaluate actual dependencies rather than assuming chronological age equals risk.
The stronger question is:
What must remain available for this information to stay intelligible and usable?
That question connects file format identification to the broader architecture described in How Trusted Digital Repositories Work.
26. File identification belongs beside fixity, not inside it
Fixity and format identification are often performed during the same workflow, but they prove different things.
FIXITY: Are the bytes still what we expected? FORMAT IDENTIFICATION: What technical encoding do these bytes appear to use?
A file can keep the same checksum and still be unidentified. It can be correctly identified and later become corrupted. It can be corrupt while retaining a recognisable signature.
The repository therefore needs both identities:
- content identity, represented by fixity information;
- format identity, represented by technical identification.
27. File-format knowledge is part of representation information
The OAIS preservation model emphasises that preserved bits need representation information if a future designated community is to understand them. File-format specifications, registries, schemas, software documentation and technical identifiers all contribute to that interpretive layer.
A repository that stores one million files without preserving reliable information about their encodings may be storing one million future puzzles.
This is why format registries are not peripheral catalogues. They are part of the knowledge infrastructure that turns bit preservation into information preservation.
28. A practical ingest architecture
A defensible repository pipeline can treat format identification as one checkpoint among several:
1. RECEIVE TRANSFER 2. FREEZE ORIGINAL STATE 3. INVENTORY FILES 4. CALCULATE FIXITY 5. SECURITY SCREEN 6. IDENTIFY FORMAT 7. RECORD PUID / MIME / METHOD 8. COMPARE EXTENSION AND SIGNATURE 9. VALIDATE WHERE REQUIRED 10. CHARACTERISE SIGNIFICANT PROPERTIES 11. ASSESS FORMAT RISK 12. APPLY ACCEPTANCE OR PRESERVATION POLICY 13. PACKAGE WITH METADATA 14. WRITE EVENT RECEIPT 15. COMMIT TO PRESERVATION STORAGE 16. VERIFY COMMIT
The exact order may change by institution. The important principle is that every stronger claim requires the appropriate evidence.
29. Failure modes
File-format identification systems fail in recognisable ways.
Failure: trusting extensions
Result: hidden mismatches, inaccurate inventory and wrong preservation rules.
Failure: treating identification as validation
Result: damaged or non-conformant files are mistaken for healthy files.
Failure: retaining only a human-readable format name
Result: version ambiguity and poor interoperability between systems.
Failure: discarding tool and signature versions
Result: later staff cannot reproduce or interpret old identification results.
Failure: forcing unknown files into a known category
Result: false confidence and potentially destructive automated action.
Failure: identifying only outer containers
Result: embedded codecs, attachments or internal resources remain invisible.
Failure: turning every risky format into automatic migration
Result: unnecessary transformations, loss of significant properties and broken provenance.
30. Preservation action should be policy-driven
Identification is most valuable when connected to explicit policy.
A repository might define rules such as:
| Observed state | Possible action |
|---|---|
| Preferred, well-supported format | Preserve as received; monitor. |
| Supported but non-preferred format | Preserve original; assess normalisation or access derivative. |
| Legacy proprietary format | Characterise; test rendering; plan migration or emulation. |
| Unknown format | Hold for research; do not auto-transform. |
| Extension/signature mismatch | Investigate and record resolution. |
| Identified but invalid | Preserve original evidence; assess repair or replacement source. |
| Encrypted or technically protected | Assess rights, keys, dependencies and preservation feasibility. |
The table is not a universal standard. It demonstrates the essential connection: observation → policy → controlled action → receipt.
31. Search keywords are really different reader jobs
People arrive at this subject through many phrases: file format identification, identify file type, DROID digital preservation, PRONOM, PUID, magic numbers, file signatures, format validation, obsolete file formats, preservation formats, format registry.
Those terms should not become separate pages merely because they are separate queries. Most belong to one coherent reader job: understanding how a repository determines technical file identity and uses that information safely.
This article therefore keeps the cluster together while routing adjacent jobs to their proper owners:
- How Digital Archives Work — the broader archive lifecycle.
- How Trusted Digital Repositories Work — repository integrity, audit and long-term stewardship.
- How Preservation and Conservation Work — preservation across physical and digital collections.
- How Metadata Standards and Interoperability Work — machine-readable description and cross-system meaning.
32. The receiver test
A useful test is to imagine a digital archivist twenty years from now opening the preservation record for one file.
Can that person tell:
- what format the file was believed to be;
- which version or variant;
- which tool made the identification;
- which signature set or registry version was used;
- whether the extension agreed with the internal evidence;
- whether the file was validated;
- which technical properties mattered;
- whether the object was migrated;
- whether later identification knowledge changed;
- what preservation action followed and why?
If the answer to these questions depends on institutional folklore, the identification workflow has not yet become preservation infrastructure.
33. What the apex preservation sites teach
The strongest preservation organisations converge on several principles.
- The National Archives: precise format identification is a foundational repository capability; DROID links automated identification to maintained PRONOM technical knowledge.
- PRONOM: format identities, signatures, versions and technical details should be shared as structured registry knowledge rather than rediscovered locally.
- Library of Congress: knowing a format is not enough; long-term sustainability depends on disclosure, adoption, transparency, self-documentation, dependencies, patents and technical protection mechanisms, together with content-specific quality factors.
- Trusted repository practice: identification gains value when it connects to fixity, provenance, validation, risk, preservation planning and auditable action.
The combined lesson is that file identification is neither a desktop convenience nor a one-off metadata task. It is a control point in a long-running preservation system.
34. A final mental model
Think of a future archive receiving a sealed box with no label.
The extension is the handwritten note on the outside. The byte signature is the first physical clue inside. The registry is the reference library that tells you what those clues mean. Validation tests whether the object follows the expected construction rules. Characterisation records its important properties. Risk analysis asks whether future tools can still use it. Preservation action decides what to do before those tools disappear.
The complete chain is:
OBSERVE BYTES → IDENTIFY FORMAT → RESOLVE TO STABLE TECHNICAL IDENTITY → TEST CONFORMANCE WHEN NEEDED → CHARACTERISE IMPORTANT PROPERTIES → ASSESS DEPENDENCIES AND RISK → CHOOSE PRESERVATION ACTION → RECORD THE DECISION → REPEAT WHEN KNOWLEDGE OR TECHNOLOGY CHANGES
That is how a filename becomes preservation knowledge.