eduKate Singapore

How Research Methods and Source Evaluation Work | From Question to Evidence, Reliability, Bias and Judgement

Research begins before a search box is opened. It begins when somebody notices that a question matters enough to deserve disciplined attention. A learner asks why two sources disagree. A scientist asks whether an observed effect is real. A historian asks what can be inferred from a surviving record. A parent asks whether a claim about learning is supported by evidence or merely repeated often enough to sound true. An engineer asks whether a failure was caused by design, materials, use, environment or some interaction among them.

The visible output of research may be a paper, graph, report, essay, dataset, policy recommendation, experiment, archive entry or classroom explanation. The invisible work underneath is more important: defining the question, choosing a method that can actually answer it, deciding what counts as evidence, checking where evidence came from, testing alternative explanations, estimating uncertainty, recording limitations and making sure the conclusion does not outrun what the evidence can support.

This is why research methods and source evaluation belong together. A strong source used with a weak method can mislead. A strong method applied to weak or inappropriate evidence can also mislead. Good research requires both a reliable route to evidence and disciplined judgement about what that evidence means.

The core research loop

A useful way to understand research is as a loop rather than a straight line:

QUESTION
→ DEFINE
→ SEARCH
→ SELECT
→ VERIFY
→ METHOD
→ OBSERVE OR MEASURE
→ ANALYSE
→ TEST ALTERNATIVES
→ ESTIMATE UNCERTAINTY
→ INTERPRET
→ REPORT
→ CHALLENGE
→ CORRECT
→ NEW QUESTION

Every stage can change the others. A search may reveal that the original question is too vague. A measurement problem may force a redesign. A conflicting source may expose an assumption. New evidence may make the initial hypothesis less plausible. Replication may show that an apparently strong result depends on a hidden condition. Research is therefore disciplined correction, not the ceremonial confirmation of what the researcher hoped to find.

1. Start with a question that can be answered

“What is the best education system?” sounds like a research question, but it hides several unresolved choices. Best for what outcome? Academic attainment, equity, creativity, wellbeing, social mobility, economic productivity or something else? At what age? Over what time period? Measured by which indicator? Compared across which populations? Under which constraints?

A researchable question reduces ambiguity without pretending the world is simple. It identifies the object of study, the comparison or relationship of interest, the relevant time and place, and the type of claim that could reasonably be supported.

A major research error happens when a method designed for one kind of question is used to answer another. Correlation can describe association, but by itself it does not establish causation. A vivid interview can reveal experience and meaning, but it cannot automatically estimate prevalence in a population. A national average may show a broad pattern while hiding important differences among regions, schools, occupations or age groups.

2. Turn concepts into observable evidence

Many important concepts cannot be observed directly. Intelligence, trust, poverty, institutional quality, resilience, motivation, social cohesion and learning are not single physical objects that can be placed on a scale. Researchers therefore operationalise concepts: they decide which observations or measurements will stand as evidence for the thing being studied.

This translation from concept to measure is one of the most consequential parts of research. If the measure does not represent the concept well, a precise result may still answer the wrong question.

The United States Office of Research Integrity stresses that responsible data collection depends on appropriate, reliable methods and that bias or inappropriate procedures can compromise research data. The same underlying lesson applies far beyond laboratory research: a method is not good because it is sophisticated; it is good when it is fit for the question and transparent enough to be evaluated.

3. Know what kind of evidence you are looking at

Evidence takes many forms. Each form can be powerful when used for the job it can actually perform.

Evidence formTypical strengthsTypical limits
Direct observationClose to the phenomenon; can reveal behaviour and contextObserver effects, limited coverage, interpretation risk
ExperimentStrong control over variables; can support causal inference under suitable designMay simplify reality; ethical or practical limits
SurveyCan cover large populations and standardised questionsSampling, wording, recall and response biases
InterviewDepth, motivation, meaning, lived experienceNot automatically representative; interviewer and memory effects
Administrative dataLarge scale, operational relevance, longitudinal potentialCollected for administrative purposes, not always for the research question
Historical recordEvidence from the period or institution being studiedSurvival bias, authorship, motive, incomplete context
Material artefactPhysical evidence of manufacture, use, environment and custodyProvenance gaps, dating uncertainty, interpretation limits
DatasetAllows systematic analysis and reuseDefinitions, missingness, collection method and metadata may constrain meaning
Model or simulationTests relationships, scenarios and mechanismsOutput depends on assumptions, structure and input quality
Secondary synthesisBrings multiple studies or sources togetherDepends on search strategy, inclusion criteria and underlying evidence

The important question is not “Is this evidence?” but “Evidence for what claim, under what conditions, with what uncertainty?”

4. Source evaluation is claim evaluation

Students are often taught to judge a source by checking its author, date, publisher and domain. These checks are useful, but they are only the beginning. A source is not globally reliable or unreliable. It may be authoritative for one claim and weak for another.

A government statistical agency may be authoritative for an official population estimate but not for an independent moral judgement about public policy. A company may be the primary authority on the specifications of its own product but not a neutral authority on whether that product is the best choice. A newspaper may accurately report that an event occurred but may not provide enough methodological detail to evaluate a scientific claim mentioned in the story. A peer-reviewed article may contain serious limitations even though it passed review.

Source evaluation therefore works best when tied to the specific claim being supported.

CLAIM
→ WHAT EVIDENCE WOULD SUPPORT IT?
→ WHO COULD KNOW THIS?
→ HOW WOULD THEY KNOW?
→ WHAT METHOD DID THEY USE?
→ WHAT INCENTIVES OR CONSTRAINTS EXIST?
→ CAN THE EVIDENCE BE CHECKED?
→ DOES AN INDEPENDENT SOURCE AGREE?
→ WHAT REMAINS UNCERTAIN?

5. Primary and secondary sources do different jobs

A primary source is close to the event, experiment, institution, artefact or dataset being studied. A secondary source interprets, analyses or synthesises primary material. Neither category is automatically superior.

A primary document can reveal what an institution officially said, but not whether the statement was true. A laboratory paper can report an original experiment, but a later systematic review may provide a stronger estimate of the broader evidence. A diary can illuminate an individual’s experience while leaving the experience of an entire population unknown. A museum accession record can establish a custody event, while provenance research may be needed to reconstruct earlier ownership.

Good research often moves between levels: primary evidence for closeness and detail; secondary synthesis for context, comparison and cumulative judgement.

6. Authority matters, but method matters more

Institutional authority is useful because strong organisations often maintain expert review, documented procedures, quality control and accountability. Yet authority should not be used as a substitute for reading the evidence.

The National Institutes of Health describes scientific rigor as the strict application of the scientific method to support unbiased and well-controlled design, methodology, analysis, interpretation and reporting. That emphasis on the full chain matters: a conclusion is only as trustworthy as the path that produced it.

7. Reliability is not the same as truth

A measurement can be reliable but invalid. A scale that is consistently five kilograms wrong may produce repeatable readings without producing accurate ones. A questionnaire can reliably measure test-taking confidence while being mistakenly interpreted as a direct measure of mathematical ability. A ranking system can consistently produce the same order while embedding a questionable definition of what counts as “best”.

Reliability asks whether a process behaves consistently. Validity asks whether the inference drawn from it is justified. Research needs both.

8. Bias enters before, during and after data collection

Bias is not simply a researcher having an opinion. In research, bias is a systematic process that pushes observations, measurements, selections, analyses or interpretations away from a fair representation of the phenomenon.

Bias cannot always be eliminated, but it should be anticipated, reduced where possible, measured when possible and declared when it remains.

9. Correlation, causation and mechanism

Two variables can move together because one causes the other, because the direction of influence runs the other way, because a third factor affects both, because of selection effects, because of measurement choices or because the apparent relationship arose by chance.

Causal research therefore asks for more than association. It looks for timing, plausible mechanisms, counterfactual reasoning, controls, natural or designed comparisons, robustness tests and alternative explanations. Different disciplines use different tools, but the underlying discipline is the same: do not upgrade an association into a causal claim without enough evidence.

10. Sampling determines what a study can speak about

A sample is a bridge from observed cases to a larger population. The bridge works only when the relationship between sample and target population is understood.

Large samples can reduce random error, but size does not repair systematic selection problems. Ten thousand voluntary online responses may be less representative of a population than a carefully designed probability sample of far fewer people. The researcher must therefore ask who had a chance to be included, who did not, who refused, who dropped out and whether those patterns matter to the result.

11. Transparency allows research to be challenged

Good research is not defined by never being wrong. It is defined partly by making the route to the result visible enough that errors can be discovered and corrected.

The National Academies’ work on reproducibility and replicability distinguishes computational reproducibility from the broader question of whether independent research can obtain consistent results. The larger lesson is that methods, data, code, assumptions and analytical choices should be described clearly enough for meaningful checking wherever ethical, legal and practical constraints allow.

12. Replication is a feature, not an insult

Research becomes stronger when important claims can survive independent checking. Replication may confirm a result, narrow the conditions under which it holds, expose a hidden dependency or reveal that the original finding was unstable. A non-replication does not automatically prove misconduct or incompetence; complex phenomena can vary across populations, settings, instruments and time.

What matters is whether the research community can learn from the difference. A mature research culture treats correction as part of knowledge production rather than as an embarrassment to be hidden.

13. Triangulation asks whether different routes converge

No single method is ideal for every dimension of a difficult question. Researchers often strengthen inference by triangulating across methods, datasets, institutions or forms of evidence.

For example, a study of urban transport reliability might combine operational records, passenger surveys, vehicle telemetry, maintenance logs, observation and policy documents. Agreement among independent routes can increase confidence. Disagreement can be even more useful because it identifies where definitions, perspectives or mechanisms need closer examination.

14. Uncertainty is information

Research claims are rarely all-or-nothing. Measurements have error. Samples vary. Models simplify. Sources are incomplete. Historical records contain gaps. Human testimony can be sincere and still imperfect. Predictions depend on assumptions.

A responsible conclusion therefore separates what is strongly supported, what is plausible, what is contested and what is unknown. Statistical uncertainty may be expressed through confidence or credible intervals, standard errors or sensitivity analyses. Qualitative research may express uncertainty through competing interpretations, source limitations, negative cases and explicit boundaries of inference. Historical work may distinguish documented fact from reconstruction or conjecture.

Uncertainty is not weakness. Hidden uncertainty is weakness.

15. A practical source-evaluation ladder

When a reader encounters an unfamiliar claim, the following ladder is useful:

  1. Identify the exact claim. Do not evaluate an entire article when only one sentence needs checking.
  2. Find the closest source. Look for the original dataset, study, document, law, standard, speech, archive record or institutional release.
  3. Check authority. Ask whether the source is in a position to know the claim.
  4. Check method. Ask how the evidence was produced.
  5. Check date and version. Determine whether the information is current for the claim.
  6. Check definitions. Many apparent disagreements are definition disagreements.
  7. Check independent support. Look for another reliable route to the same conclusion.
  8. Check incentives and omissions. Ask what the source gains, controls or leaves unexplained.
  9. Check scope. Determine whether the conclusion has been generalised beyond the population, time or setting studied.
  10. Record confidence. Decide what remains uncertain rather than forcing a false yes/no verdict.

16. Research in the humanities and history

Not all rigorous research looks like an experiment. Historians, literary scholars, archaeologists and researchers in the humanities often work with texts, artefacts, archives, images, material traces, language, institutions and cultural context.

Their methods may include source criticism, textual analysis, provenance reconstruction, dating, comparison, contextualisation and interpretation. A key discipline is to ask who produced the source, for whom, under what conditions, for what purpose, what it could observe, what it could not observe, why it survived and what other evidence supports or challenges it.

This is especially important because the archive is never a perfect recording of the past. What survives is shaped by power, institutions, storage, destruction, chance and later collection decisions.

17. Research in the social sciences

Social systems are difficult because human beings respond to institutions, incentives, expectations and one another. The act of measuring can sometimes change behaviour. Definitions vary across countries and time. Social categories can be historically contingent. Policies are rarely assigned randomly. Outcomes may have multiple causes operating at different levels.

Strong social research therefore pays close attention to case selection, measurement equivalence, confounding, institutional context, causal identification and the difference between individual-level and system-level conclusions. This is the bridge to eduKateSingapore’s World Knowledge Research Library, where cross-system claims should preserve definitions, context and ownership rather than flattening the world into a single ranking.

18. Research in the age of AI

AI changes the speed of research but does not remove the need for research judgement. A language model can help generate search terms, compare explanations, summarise a paper, classify evidence or suggest alternative hypotheses. It can also fabricate references, blur primary and secondary material, compress important uncertainty, reproduce bias or state an outdated claim fluently.

The correct research posture is therefore not “AI or sources”. It is AI routed through sources, methods and verification. The human or institutional research process remains responsible for identifying the claim, finding authoritative evidence, checking versions, preserving provenance and deciding what the evidence supports.

For eduKate, this is why retrieval readiness is not the same as publication readiness. A searchable page is useful only when the Library can know what it owns, what evidence supports it, when it was current and where the reader should go next.

19. Common research failure modes

20. A research workflow for learners

A student does not need a laboratory or research institute to work rigorously. A school-level research workflow can be simple and powerful:

ONE QUESTION
→ THREE POSSIBLE EXPLANATIONS
→ FIVE GOOD SOURCES
→ ONE PRIMARY SOURCE WHERE POSSIBLE
→ DEFINITIONS WRITTEN DOWN
→ EVIDENCE TABLE
→ CONTRADICTORY EVIDENCE
→ LIMITATIONS
→ BEST CURRENT CONCLUSION
→ WHAT WOULD CHANGE MY MIND?

The final question is especially important. If no possible evidence could change a conclusion, the activity has stopped being research and become defence of a belief.

21. A research workflow for institutions

Institutions need additional layers because knowledge must outlast individuals. The process should preserve source identity, version, method, ownership, permissions, corrections, unresolved disputes and update rules.

This is why eduKate separates acquisition, publishing, archives and data management. The Collections Development, Curation & Acquisitions Wing asks whether a real gap exists. Wintour House decides whether a manuscript deserves publication and whether its evidence, structure and edition are ready. The Research Collections Directory provides public research routes. How Data Management Works explains how evidence remains usable through capture, structure, validation, governance and preservation.

22. The difference between information and evidence

Information becomes evidence only in relation to a claim. A temperature reading is information. It becomes evidence when used to support a claim about fever, climate, equipment performance or chemical change, and the meaning depends on the measurement conditions. A photograph is information. It becomes evidence for a historical claim only after date, location, authorship, manipulation, subject and context are considered.

This distinction is central to critical thinking. The world contains more information than any person can inspect. Research is the discipline of deciding which information can legitimately bear the weight of which conclusions.

23. The strongest conclusion is often narrower

Weak research often tries to sound universal. Strong research frequently becomes more precise as it improves. Instead of “this teaching method works”, a better conclusion may be “under these conditions, for this group, using this outcome measure, the intervention produced an average improvement of this size, with these limitations”.

Narrower is not smaller when it is more accurate. Precision gives future research something solid to test.

24. Research ends by returning to the world

The purpose of disciplined research is not simply to accumulate citations. It is to improve what can be known, decided, taught, designed, preserved or questioned. A good research product therefore returns more than an answer. It returns a visible method, a source trail, an uncertainty boundary and a route for correction.

That is how a library becomes more than a collection of pages. Each article becomes a tested position in a larger knowledge system: one question answered as well as current evidence allows, connected to the evidence behind it, the systems around it and the next question it makes possible.

Sources and further reading

eduKate route: Continue with How Scientific Research Works, Research Data Management and FAIR Principles, How Archives Work and the Research Collections Directory.

More articles in this collection

Research methods, evidence and inference

Explore the connected learning guides

Choose the question that brought you here. Open one useful guide, try a small task, and stop when you have what you need.

Take one question further

The same learning habit can travel across subjects, while each subject keeps its own methods. These routes help you notice a difficulty, understand one part of it, and return to something you can do.

A word is familiar, but using it is difficult.

Move from recognising a word to retrieving it in a new context. Understand vocabulary plateaus.

Try it without the guide: Choose one word you already know. Close the guide and use it in a new sentence. Explain why it fits; try another context tomorrow.

A piece of writing has ideas, but the reader loses the thread.

Make the order of events and the links between sentences clear. Explore composition writing.

Try it without the guide: Choose one short paragraph. Read the relevant explanation, close it, and revise the paragraph. Ask someone to tell you what happened and why.

The Mathematics seems familiar, but marks still disappear.

Find the first point where the working stops being reliable. Find Secondary 4 A-Math mark leakage.

Try it without the guide: For a Secondary 4 A-Math question you have attempted, locate the first uncertain line. Repair that step, then try a comparable question without the worked answer.

A Science fact is remembered, but the explanation is incomplete.

Connect the evidence to a scientific idea and the resulting change. Follow the Primary Science learning route.

Try it without the guide: Choose a familiar Primary Science example. Explain the evidence, the idea and the result without notes. Then change one condition and explain your prediction.

Two accounts of the world seem to disagree.

Check the question, source, date and evidence before combining claims. Explore the World Knowledge research library.

Try it without the guide: Take one claim. Find the source best placed to support it, note its date, and state what remains uncertain. Return to your original question.

There is plenty of help, but independence is hard to see.

Check what the learner can understand and do after support is removed. Understand how education works.

Try it without the guide: Choose one small task the child has practised. Agree on a calm, brief attempt without prompts. Use what happens to choose one next step, then stop.

For the structure behind these connections, read the eduKateSingapore runtime manifest and the eduKate ecosystem boot contract. The reader map describes public navigation; those manifests preserve the wider ownership and return rules.

eduKate Asia

Subscribe to
our newsletter

Discover more from eduKate Singapore

Subscribe now to keep reading and get access to the full archive.

Continue reading