Data are recorded representations of observations, facts, measurements, categories or events that can be stored, transmitted, compared and analysed. If you are asking “what is data?”, the useful answer begins with examples—numbers in a spreadsheet, words in a survey, pixels in an image, sensor readings, transaction records, audio samples, locations, labels and timestamps—but it quickly becomes more precise. Data are not the world itself. They are representations produced through a method, format and set of assumptions, which means every dataset has a source, structure, context and set of limits.
A complete definition of data must also separate data from information, evidence and knowledge. A temperature value such as “31.4” is data, but without a unit, location, time and measurement method it is difficult to interpret. “31.4 °C measured at 2:00 p.m. at this weather station” is more informative because context has been attached. When many such observations are analysed to test a claim, the data can become evidence. When results are integrated with concepts, models and experience, they can contribute to knowledge.
This guide explains data from first principles: observations, variables, records, structured and unstructured data, qualitative and quantitative data, categorical and numerical data, metadata, provenance, data quality, measurement, sampling, databases, missing data, bias, privacy, security, statistics, machine learning, big data, data visualisation, open data, reproducibility and artificial intelligence. The goal is a durable mental model: know what was observed, how it was represented, what context is required, which transformations were applied and what conclusions the data can—and cannot—support.
What Is Data? A Short Definition
Data are representations of information in a form suitable for communication, interpretation or processing. In computing, they may be encoded as bits and bytes. In science, they may be measurements, classifications, images, spectra or recorded observations. In social research, they may be survey responses, interview transcripts or administrative records.
The same underlying phenomenon can produce different data depending on the measurement method. A person’s height can be recorded in metres or centimetres. A city’s traffic can be represented as vehicle counts, average speed, travel time or GPS traces. Each representation highlights some aspects and omits others.
This means data are never completely context-free. To use them responsibly, we need to know what was measured, who or what was included, when and where observations were made, which units and categories were used and what processing occurred after collection.
Why Data Matter
Data make comparison possible. Instead of saying “the room feels warm”, we can record temperature over time. Instead of saying “sales increased”, we can compare revenue, units sold and customer counts across periods.
Data also create memory for systems. A hospital record preserves observations beyond one consultation. A scientific dataset allows later researchers to reanalyse measurements. A database lets organisations coordinate activity across people and time.
The power of data comes from disciplined representation. The weakness of data comes from the same source: if the representation is wrong, incomplete or misleading, analysis can become precise without becoming true.
Data Versus Information
Data are often described as raw facts, while information is data placed into meaningful context. The distinction is useful but should not be treated as absolute because even “raw” data have already been shaped by instruments, categories and recording choices.
A row containing “A12, 84, 2026-09-24” is data. If we know that A12 is a student identifier, 84 is an examination score and the final value is the test date, the row becomes interpretable information.
Information therefore depends on relationships and meaning. The same data can support different information needs depending on the question.
Data Versus Evidence
Data become evidence when they are used to support or challenge a claim under an explicit reasoning process.
A table of test scores is data. Using those scores to evaluate whether a teaching intervention improved performance makes the data part of an evidential argument.
The distinction matters because data do not interpret themselves. The same dataset can be weak evidence for one claim and strong evidence for another depending on design, measurement and alternative explanations.
Data Versus Knowledge
Knowledge is more than stored data. It includes concepts, causal relationships, procedures and justified expectations built from information and experience.
A weather station produces data. A meteorologist uses models and domain knowledge to interpret those data and forecast conditions.
This relationship is why accumulating more data does not automatically produce more knowledge. Good models, questions and reasoning remain necessary.
Observations and Records
An observation is a detected or measured feature of the world. A record is a stored representation of that observation.
A sensor reading can become one row in a database. A spoken interview can become an audio file and transcript. A wildlife sighting can become a location, species label, time and observer note.
Separating observation from record helps diagnose problems. A record may be corrupted even when the original observation was correct, or an observation may be biased even when it is stored perfectly.
Variables
A variable is a characteristic that can take different values. Height, age, temperature, category, colour and response time are variables.
Variables require definitions. “Income” could mean gross annual income, monthly take-home pay or household income. Different definitions produce different data even when the same people are studied.
Good datasets document variable names, units, allowable values, missing-value codes and the meaning of each category.
Cases, Rows and Observational Units
A dataset usually contains observational units: the things being measured. They might be people, schools, countries, products, transactions, days or sensor events.
A row often corresponds to one unit or event, but not always. Time-series datasets may place repeated measurements of one unit across many rows.
Before analysing, identify the unit. Mixing people, households and observations as though they were equivalent can produce serious errors.
Qualitative Data
Qualitative data describe qualities, meanings, experiences or categories that may not be naturally represented as numerical magnitude.
Interview transcripts, field notes, photographs and open-ended responses are common examples.
Qualitative analysis can still be systematic. Researchers code themes, compare cases, examine language and preserve context rather than simply “reading impressions”.
Quantitative Data
Quantitative data are numerical representations of amounts, measurements or counts.
Examples include distance, mass, revenue, age, response time and number of errors.
Numbers do not guarantee objectivity. A numerical measure can still be poorly defined, biased or invalid. The quality of quantitative data depends on how the number was produced.
Categorical Data
Categorical data place observations into groups such as species, region, product type or response category.
Nominal categories have no inherent order. Ordinal categories have a meaningful ranking, such as low, medium and high.
Category design matters. Combining distinct groups can hide differences; creating too many categories can make patterns unstable or difficult to interpret.
Continuous and Discrete Data
Discrete data take separated values, often counts: number of students, defects or website visits.
Continuous data can in principle take any value within a range, such as height, temperature or time.
Digital measurement makes even continuous phenomena appear as discrete recorded values because instruments have finite resolution.
Structured Data
Structured data follow a defined schema, usually with named fields and consistent types. Relational database tables are classic examples.
Structure makes validation, querying and aggregation easier. A date field can be checked as a date, a price field as a number and an identifier as a unique key.
The cost is rigidity. If the world changes or the schema was poorly designed, forcing new information into old fields can create distortion.
Semi-Structured Data
Semi-structured data contain organisation without requiring a fixed table. JSON, XML and many event logs use tags, keys and nested structures.
This makes them flexible for web services and data exchange while preserving machine-readable relationships.
Flexibility still requires governance. Two teams can use the same field name for different meanings or different names for the same concept.
Unstructured Data
Unstructured data include free text, images, audio and video that do not fit neatly into traditional rows and columns.
They still contain structure internally: language has syntax, images contain spatial patterns and audio has temporal organisation. “Unstructured” usually means the data lack a simple predefined database schema for the analytic task.
Natural language processing, computer vision and multimodal AI are widely used to analyse these forms.
Metadata
Metadata describe data. They can record who created a file, when it was captured, which instrument was used, what units apply, how categories are defined and which access restrictions exist.
Without metadata, data can become unusable. A column of numbers loses meaning when the unit and variable definition disappear.
Metadata should therefore be treated as part of the dataset, not optional decoration.
Provenance and Lineage
Provenance records where data came from and how they changed. Lineage tracks transformations across systems.
A scientific result should ideally be traceable from final chart back to source measurements and processing steps. A business metric should be traceable to the transactions and rules used to calculate it.
Good provenance makes error correction possible because analysts can identify where a value entered the pipeline and which outputs depend on it.
Data Dictionaries
A data dictionary is a human- and machine-readable description of fields in a dataset. It can include names, definitions, types, units, allowed values, category labels, source systems and missing-value conventions.
Data dictionaries matter because column names are often too short to carry meaning safely. A field called “score” could represent a percentage, raw mark, standardised value or risk index. The dictionary turns a terse label into an explicit semantic contract.
Well-maintained dictionaries also make handover possible. A future analyst should not need to locate the original developer simply to discover what “status=3” meant.
Semantic Contracts
A semantic contract is an agreement about what a data element means, not merely how it is formatted. Two systems can both store an integer called “age” while disagreeing about whether it means age at enrolment, age today or age at the event date.
Shared semantics become especially important when datasets are combined across departments, organisations or countries. Technical interoperability without semantic interoperability can create confident but invalid comparisons.
Good data architecture therefore treats definitions, reference dates, units and population boundaries as part of the data model itself.
Data Quality
Data quality means fitness for an intended use. Important dimensions include accuracy, completeness, consistency, timeliness, validity, uniqueness and traceability.
A dataset can be high quality for one purpose and unsuitable for another. Monthly sales totals may be adequate for annual planning but useless for detecting second-by-second equipment failures.
Quality should therefore be judged against the decision or analysis the data are expected to support.
Accuracy
Accuracy asks how closely recorded values represent the target quantity or state.
An inaccurate sensor can generate beautifully consistent but wrong data. A mistyped date can make one record incorrect even if the database stores it flawlessly.
Accuracy often requires comparison with reference standards, cross-checks or independent sources.
Completeness
Completeness concerns whether required observations or fields are present.
Missing entries are not always random. People may skip sensitive survey questions, sensors may fail during extreme conditions and customers who leave may disappear from future datasets.
The pattern of missingness can therefore change conclusions, not merely reduce sample size.
Consistency
Consistency asks whether values that should agree actually agree across records, systems or time.
One database may use metres while another uses centimetres. One department may define an “active customer” differently from another.
Data integration requires semantic consistency as well as technical compatibility.
Timeliness
Timeliness concerns whether data are current enough for the decision.
A census can be extremely accurate for its reference date yet too old for a rapidly changing operational problem.
Real-time data are not automatically better. Faster data may have less validation and more noise, so timeliness trades off against other quality dimensions.
Validity
Validity asks whether values conform to defined rules and whether the measurement actually represents the intended construct.
A date such as 32 January is technically invalid. A survey question intended to measure confidence may be syntactically valid yet conceptually poor.
Good data systems therefore need both field validation and measurement validity.
Duplicates
Duplicate records occur when the same underlying entity or event is represented more than once.
Duplicates can inflate counts and distort averages. Removing them requires identity rules: two similar names may represent one person or two different people.
Entity resolution is therefore a reasoning problem, not just a “delete identical rows” operation.
Missing Data
Missing data can arise because information was not collected, not applicable, lost, refused or unavailable.
These reasons should not all be encoded as the same blank. A value that is not applicable carries different information from a sensor failure.
Statistical handling depends on the missingness mechanism. Simple deletion can bias results when missingness is related to the variable being studied.
Outliers
An outlier is an observation unusually far from other values according to some criterion.
Outliers can be mistakes, rare genuine events or evidence that the assumed model is incomplete.
Deleting them automatically is poor practice. Investigate the source, measurement conditions and analytic purpose first.
Data Collection
Data collection is the process of generating or acquiring observations. Methods include sensors, surveys, experiments, administrative systems, transactions, remote sensing and manual coding.
Collection design determines what can later be inferred. If a variable was never measured, sophisticated analysis cannot recover it reliably from nothing.
Good collection begins with the question, not the database. Decide what evidence is needed before deciding which fields to gather.
Measurement
Measurement connects a real-world property to a recorded value.
Every measurement has a method, unit, resolution and uncertainty. These features should travel with the data.
A value without knowledge of the measurement process can look exact while being scientifically weak.
Sampling
Sampling selects a subset of a population or process for observation.
Representative sampling supports generalisation. Biased sampling can produce a precise estimate of the wrong population.
Large sample size reduces random sampling error but does not automatically remove selection bias.
Experimental Data
Experimental data arise when researchers manipulate conditions and measure outcomes.
Randomisation and controls can strengthen causal interpretation because they reduce alternative explanations.
Experimental data still require quality checks: treatment implementation, missing outcomes, measurement error and protocol deviations can affect validity.
Observational Data
Observational data record what occurs without assigning the exposure or treatment.
They are essential in astronomy, economics, epidemiology and many other fields where manipulation is impossible or unethical.
Causal claims from observational data require stronger assumptions about confounding, selection and measurement.
Time-Series Data
Time-series data record values across ordered time points.
Temporal order enables analysis of trends, seasonality, lagged relationships and sudden changes.
Time series are rarely independent observations. Yesterday’s value may influence today’s, so ordinary statistical methods can fail if temporal dependence is ignored.
Cross-Sectional Data
Cross-sectional data compare many units at one period or reference time.
They are useful for describing differences across people, organisations or regions.
They are weaker for understanding change because they do not directly observe the same units over time.
Panel and Longitudinal Data
Panel data follow the same units repeatedly over time.
This allows researchers to distinguish differences between units from changes within a unit.
Longitudinal data support richer causal and developmental analysis but create challenges such as attrition and changing measurement conditions.
Databases
A database is an organised system for storing and retrieving data.
Relational databases use tables and keys; document databases store flexible records; graph databases represent entities and relationships; time-series databases optimise temporal observations.
The database technology should follow the data relationships and access patterns rather than fashion.
Primary Keys and Identifiers
Identifiers distinguish records or entities. A primary key uniquely identifies a row in a relational table.
Stable identifiers help link records across systems without depending on mutable labels such as names.
Identifier design has privacy implications because linking datasets can reveal more about a person than either dataset alone.
Schemas
A schema defines the structure of stored data: fields, types, relationships and constraints.
Schemas make assumptions explicit. A poorly designed schema can embed ambiguity into every later analysis.
Schema evolution is therefore part of data governance. Changes should preserve compatibility or include migration rules.
Data Cleaning
Data cleaning identifies and repairs problems such as invalid values, inconsistent formats, duplicates and impossible combinations.
Cleaning is not supposed to make data look nicer. It should apply documented rules tied to the meaning of the variables.
Aggressive cleaning can erase real variation, especially when unusual observations are treated as errors simply because they are inconvenient.
Data Transformation
Transformation changes representation: converting units, normalising values, aggregating records, extracting features or encoding categories.
Transformations can make data easier to analyse but may also remove information.
Averages compress variation; categorising age into bands loses exact ages. The transformation should match the purpose and remain documented.
Aggregation
Aggregation combines detailed records into summaries such as totals, averages or rates.
Summaries are useful for decision-making but can hide important subgroup differences.
The same average can arise from very different distributions, so aggregated data should not be treated as a complete representation of the underlying records.
The Ecological Fallacy
The ecological fallacy occurs when relationships observed for groups are incorrectly assumed to hold for individuals.
A region with high average income and high average test scores does not prove that every higher-income individual in the region has a higher score.
Analysis should match the level of the data to the level of the conclusion.
Data Bias
Bias is systematic distortion introduced through collection, measurement, selection, processing or interpretation.
Historical data can reflect past discrimination or unequal access. Sensor networks may be denser in wealthy areas. Online data may overrepresent frequent platform users.
Bias cannot always be removed by statistical adjustment. Sometimes the dataset is simply inadequate for the intended claim.
Data Privacy
Data privacy concerns appropriate collection, use, sharing and retention of information about people.
Privacy is not identical to security. A dataset can be strongly encrypted yet still collected for a purpose that people did not expect.
Good practice minimises unnecessary data, limits access, defines retention and makes uses transparent where required.
Data Security
Data security protects confidentiality, integrity and availability.
Controls include authentication, access management, encryption, backups, monitoring and incident response.
Security should follow risk. Sensitive personal or operational data require stronger safeguards than public reference data.
Anonymisation and Re-Identification
Removing names does not automatically make data anonymous. Combinations of attributes can sometimes identify individuals when linked with other sources.
True anonymisation reduces re-identification risk enough that individuals cannot reasonably be singled out under the relevant standard.
Pseudonymisation replaces direct identifiers with codes but still permits controlled relinking, so it remains different from full anonymisation.
Open Data
Open data are made available for reuse under defined licences and formats.
Open data can improve transparency, research and innovation when documentation and quality are adequate.
Not all data should be open. Privacy, security, intellectual property and ethical obligations can justify restrictions.
Data Retention and Deletion
Keeping data forever is not automatically responsible. Old data can become inaccurate, create privacy risk and increase the consequences of security incidents.
Retention rules should connect to purpose, legal requirements, scientific value and operational need. Some records deserve long-term preservation; others should be deleted when their legitimate use ends.
Deletion also needs evidence. In distributed systems, removing a visible record may not remove backups or downstream copies immediately, so lifecycle policies should define what deletion means operationally.
Data Visualisation
Visualisation converts data into graphical representations such as charts, maps and diagrams.
Good visualisation helps users see distribution, comparison, trend and uncertainty.
Bad visualisation can mislead through truncated axes, distorted areas, inappropriate scales or selective omission. Visual design is part of analysis, not merely decoration.
Statistics
Statistics provides methods for summarising data and drawing inferences from samples.
Descriptive statistics report what the dataset contains. Inferential statistics estimate population properties or test models under uncertainty.
Statistical significance does not automatically mean practical importance, causation or a large effect.
Data and Probability
Probability models describe uncertainty about possible data or outcomes.
Statistics uses observed data to estimate the parameters or plausibility of those models.
This relationship lets researchers quantify sampling uncertainty rather than treating a sample result as exact truth about a population.
Big Data
Big data refers to datasets whose volume, velocity, variety or complexity create challenges for traditional tools.
The phrase often includes large streams from sensors, transactions, logs, images and online activity.
Size does not guarantee quality. Billions of biased observations can still support a biased conclusion.
Machine Learning Data
Machine-learning systems learn patterns from examples. Training data influence what relationships the model can learn.
Labels, sampling, class balance and measurement quality therefore shape model behaviour.
A model can reproduce systematic errors present in the data, which is why evaluation on relevant independent data is essential.
Training, Validation and Test Data
Training data are used to fit model parameters. Validation data help select settings and compare models. Test data estimate performance after development decisions have been made.
If test data influence repeated model tuning, they stop functioning as an independent test.
Proper separation protects against overfitting the evaluation process rather than the real-world task.
Data Leakage
Data leakage occurs when information unavailable at real prediction time enters model training or evaluation.
For example, using a future outcome variable indirectly embedded in a feature can produce spectacular test performance that collapses in deployment.
Leakage is a pipeline problem. Preventing it requires understanding when each field becomes available and how records are split.
Label Quality
Supervised machine learning depends on labels that represent the target.
Labels may contain disagreement, ambiguity or historical decisions rather than objective truth.
High-quality modelling therefore studies label provenance and inter-rater agreement instead of assuming every label is correct.
Synthetic Data
Synthetic data are artificially generated records designed to resemble aspects of real data.
They can support testing, privacy protection or simulation when real data are limited or sensitive.
Synthetic data inherit the assumptions of the generation process and can miss rare cases or distort relationships, so they require validation against the intended use.
Data in Artificial Intelligence
Modern AI systems depend on large collections of text, images, audio, code and structured records.
The relationship is not “more data always means better AI”. Data diversity, quality, rights, coverage and relevance matter.
AI outputs can also become new data, creating feedback loops if generated content is later used for training without careful provenance.
Data Governance
Data governance defines ownership, responsibilities, standards, access rules and lifecycle decisions.
A governance system answers who can create, change, approve, access and delete data and who is responsible when definitions conflict.
Governance is necessary because technical storage alone cannot decide organisational meaning or accountability.
Data Stewardship
Data stewards maintain definitions, quality rules and appropriate use within a domain.
Stewardship connects technical systems with subject expertise. A finance steward understands accounting meaning; a clinical steward understands medical context.
Without stewardship, organisations often accumulate multiple incompatible versions of the same metric.
Data Lifecycle
The data lifecycle includes collection, validation, storage, processing, analysis, sharing, archiving and disposal.
Each stage creates risks. Collection can be biased; storage can be insecure; processing can introduce errors; old data can become misleading.
Lifecycle thinking makes responsibility continuous rather than stopping after data have been collected.
Data Reproducibility
Reproducibility means others can repeat an analysis using the same data and methods and obtain compatible results.
This requires preserved code, versions, dependencies, data definitions and processing decisions.
A final chart without the transformation history is not enough for reliable scientific or organisational memory.
Data Versioning
Datasets change. New records arrive, errors are corrected and definitions evolve.
Versioning records which dataset was used for a particular analysis.
Without version control, two analysts can run identical code on different data snapshots and obtain different results without understanding why.
Common Misconceptions About Data
- “Data are the same as facts.” Data are representations produced through methods and assumptions.
- “More data are always better.” More biased or irrelevant data can worsen analysis.
- “Numbers are objective.” Numerical values still depend on definitions and measurement.
- “Missing values are just blanks.” Different missingness mechanisms carry different implications.
- “Cleaning means deleting unusual values.” Outliers may be genuine and important.
- “A database makes data trustworthy.” Storage integrity does not guarantee measurement validity.
- “Anonymous means names were removed.” Re-identification may remain possible through linked attributes.
- “AI learns truth from data.” Models learn statistical patterns present in their training material.
A Practical Data Checklist
Ask what each row represents, what each variable means, which units and categories are used and how missing values are encoded.
Trace provenance. Who collected the data, when, where, using which instrument or process? What transformations, exclusions or imputations occurred?
Then match analysis to the question. Are the data representative, timely and valid enough for the conclusion? What uncertainty or bias remains?
How to Learn Data Properly
Begin by collecting a small dataset yourself. Measure something repeatedly, define variables and record metadata.
Then clean and visualise it. Notice how unit conversion, missing values and aggregation change interpretation.
Finally, analyse a larger public dataset and document every transformation. The goal is to understand the full chain from observation to conclusion, not merely to manipulate a spreadsheet.
Frequently Asked Questions About Data
What is data in simple terms?
Data are recorded representations of observations, facts, measurements or categories that can be stored and processed.
What is the difference between data and information?
Data are recorded values or symbols; information is data interpreted in context so that it answers a question or conveys meaning.
What are structured data?
Structured data follow a predefined schema with consistent fields and types, such as records in a relational database.
What are unstructured data?
Unstructured data include text, images, audio and video that do not fit neatly into a fixed table for the intended task.
What is metadata?
Metadata describe the characteristics, meaning, origin or management of other data.
What makes data high quality?
High-quality data are fit for purpose: sufficiently accurate, complete, consistent, timely, valid and traceable for the intended decision.
Can data be biased?
Yes. Bias can enter through sampling, measurement, labels, missingness, historical systems and processing choices.
Why are data important for AI?
AI systems learn patterns from examples, so the coverage, quality, labels and biases of training data strongly influence model behaviour.
Authoritative Starting Points
For formal terminology, see the NIST definition of data and the NIST glossary entry for metadata. NIST’s information quality standards provide useful distinctions around accuracy, reliability, objectivity and integrity. For practical data categories such as structured, semi-structured and unstructured data, major computing references such as IBM’s data overview provide applied examples.
What Is Data? The Complete Idea
Data are representations created so that observations and states can persist beyond the moment in which they occur. They can be numbers, categories, words, images, sounds or structured relationships, but their meaning depends on definitions, metadata and provenance.
The strongest understanding keeps several distinctions clear: data are not the world itself; data are not automatically evidence; a number is not automatically accurate; more data are not automatically better; storage is not validation; cleaning is not permission to remove inconvenient observations; and AI training data are not guaranteed truth.
Once those distinctions are stable, data become a disciplined bridge between reality and analysis. Define the observation, record it faithfully, preserve the context, track transformations, test quality and match conclusions to what the dataset can actually support. Good data work is not the art of producing more numbers. It is the practice of preserving meaning as reality becomes representation.