A survey percentage is the end of a chain of decisions. Someone defined a population, found a way to contact people, asked a particular question, handled missing answers and chose a denominator. Reading “70% support the proposal” responsibly means reconstructing that chain before treating the number as a statement about everyone affected.

This guide follows one fictional reading-room survey from invitation to headline. Every person, count and operational detail in the case is invented for teaching. Nothing here reports a real survey of Singapore residents, eduKateSingapore readers or library users. The purpose is to practise calculations and judgements on data whose boundaries are fully visible.

You need fractions, percentages and a readiness to ask “percentage of whom?” By the end, you should be able to audit a short survey report, explain several different percentages from the same counts, improve a question and write a conclusion that preserves both the finding and its limitations.

This is a worked companion to How Surveys and Sampling Work, which provides the broader methods framework. It belongs to the Social Sciences and Civic Life Library. Work through the case before moving to the more specialised reading on weighting or uncertainty.

Start with the decision and the population

A community reading room wants to know whether its current members support keeping the room open for two additional hours on Saturday evenings during a four-week trial. The intended population is the 1,000 people on its current membership list at the survey’s stated opening date. Membership is the eligibility rule in this exercise; residence in the neighbourhood is not.

This definition immediately limits the question. Non-members, former members and future visitors may have important needs, but the proposed member survey does not automatically describe them. If the decision is supposed to serve the whole neighbourhood, the research plan needs another route to those people. A precise population can reveal that a survey answers only part of the decision.

The administrator has email addresses for 600 of the 1,000 members. From that email list, the administrator invites 400 members selected for convenience, with priority given to people who visited recently. All 400 invitees are known to be eligible. This is not a probability sample of all members, and its limitation exists before anyone chooses whether to reply.

Keep three objects separate: the intended population of 1,000, the contact frame of 600 and the invited group of 400. A later response count cannot repair a person’s absence from the contact frame. Increasing replies among reachable members does not by itself provide evidence about those never reached.

The full count trail

The fictional fieldwork ends with 200 returned questionnaires. Of those, 180 contain an answer to the Saturday-hours question. The other 20 skip that question. Among the 180 answers, 126 support the trial, 36 oppose it and 18 select “unsure.” Each questionnaire belongs to one distinct invited member; there are no duplicate returns in this teaching dataset.

Stage or response	Count	What the count describes
Current members	1,000	Intended population at the defined date
Members with an email address	600	Contact frame available to the organiser
Eligible members invited	400	Convenience-selected invited group
Questionnaires returned	200	Respondents to the survey
Hours question answered	180	Base for the three response categories
Support	126	Recorded support answers
Oppose	36	Recorded opposition answers
Unsure	18	Recorded unsure answers
Returned questionnaire, question skipped	20	Item nonresponse

First check that the response categories reconcile: 126 + 36 + 18 = 180. Then check that answered plus skipped questionnaires reconcile: 180 + 20 = 200. These simple checks catch mistakes before interpretation begins. They do not establish representativeness, but an inconsistent count trail makes even the descriptive result unreliable.

The 200 invitees who returned no questionnaire are different from the 20 respondents who skipped this item. The first group represents nonresponse to the survey as a whole. The second represents missing answers to this particular question. Treating the two groups as if they had supplied the same response would erase useful information.

Four correct calculations with different meanings

Support among those answering the question is 126 divided by 180, multiplied by 100: 70%. Support recorded among all survey respondents is 126 divided by 200, or 63%. Support recorded among all invitees is 126 divided by 400, or 31.5%. Recorded supporters as a fraction of all members are 126 divided by 1,000, or 12.6%.

All four divisions are arithmetically correct. They answer different questions. Only the first is the proportion supporting among the three recorded answer categories. The last three use larger groups that include people whose answer to the hours question is unknown. Their labels must make that distinction visible.

In particular, 31.5% does not mean that the other 68.5% of invitees oppose the trial. That remainder includes recorded opponents, unsure respondents, skipped answers and people who did not return the survey. Likewise, 12.6% is a count of known recorded supporters relative to membership, not an estimate that only 12.6% of members support the trial.

The arithmetic is therefore inseparable from language. Before choosing a headline, write the numerator as a noun phrase and the denominator as another noun phrase. “Recorded support answers divided by valid answers to this question” is much more informative than a bare division on a calculator.

What belongs in the denominator?

The published base depends on the claim. If the question asks for the distribution of support, opposition and uncertainty among people who answered, all three categories belong in the denominator. Removing “unsure” changes the estimand: the quantity you are trying to describe. That change may be legitimate for a separate analysis, but it must be explicit.

Excluding the 18 unsure answers gives 126 divided by 162, approximately 77.8%, among respondents who expressed support or opposition. This is not a more accurate version of 70%. It describes a narrower group. Reporting only 77.8% without naming the exclusion makes support appear stronger by hiding uncertainty.

Missing answers require their own treatment. A skipped question is not automatically opposition, uncertainty or lack of interest. The person might have missed the page, been interrupted or found the wording unclear. Unless the questionnaire and coding rules establish otherwise, preserve the missing category instead of inventing an answer.

A useful report can show both the answer distribution and the missing count. For example: “Of 180 members who answered the item, 70% supported, 20% opposed and 10% were unsure; a further 20 survey respondents skipped the item.” That sentence gives readers enough information to reconstruct the base.

Why a response fraction cannot settle bias

In this simplified case, 200 of 400 eligible invitees returned a questionnaire, giving a 50% return fraction. All invitees’ eligibility is known and there are no partial-return classification complications. Real survey response-rate definitions can require additional dispositions and rules, so a published professional rate needs its method as well as its number.

The return fraction does not reveal the preferences of nonrespondents. If people who care strongly about evening access reply more often, the observed support may overstate support in the invited group. If members who most need late access have less time to respond, the direction might be different. The count alone cannot choose between these explanations.

The AAPOR Standard Definitions resource provides formal survey outcome-rate guidance. For this exercise, the essential discipline is to retain the disposition counts and label the simple calculation. Do not attach an impressive technical name to a rate whose denominator has not been explained.

A low return fraction raises questions; it does not prove a particular amount of bias. A high return fraction is encouraging about participation among invitees, but it does not repair a contact frame that excludes 400 members or an invitation procedure favouring recent visitors. Different stages can introduce different distortions.

Put honest bounds around the unknowns

Among the 400 invitees, we have 126 recorded supporters and 54 recorded non-support answers: 36 oppose and 18 are unsure. The remaining 220 have no recorded answer to the item, combining 200 nonreturns with 20 skips. For a deliberately extreme sensitivity calculation, suppose all 220 unknown answers would be non-support. Support among invitees would then be 126/400, or 31.5%.

At the opposite extreme, suppose all 220 unknown answers would be support. Support would be 346/400, or 86.5%. These endpoints show how much the unobserved answers could matter under this simple binary support versus non-support framing. They are not a confidence interval and do not assign probabilities to either extreme.

The exercise also depends on keeping the known answers fixed and imagining an answer for each unknown person at the same relevant time. Preferences can change, and “unsure” is a genuine observed category rather than a permanent opposition position. The bounds are a teaching device for missing information, not a complete behavioural model.

Do not choose the midpoint and call it the best estimate without further justification. The interval of logically possible completions does not tell us which completion is most plausible. Better information about recruitment, follow-up and differences between respondents and nonrespondents is needed to narrow the interpretation responsibly.

Inspect the exact question

Suppose the questionnaire actually asked: “Would you support our much-needed longer Saturday hours and free family workshops to make the reading room more welcoming?” The headline claims to measure support for longer hours, but the item bundles several ideas. Someone could support free workshops while opposing later closing, or like the aim of welcome while questioning the timetable.

“Much-needed” also signals an expected judgement before the respondent has supplied one. The proposal lacks precise times and a duration. One person may imagine an extra half hour; another may imagine late-night opening throughout the year. Their identical tick marks could represent different decisions.

A revised item is: “The reading room is considering opening from 6 pm to 8 pm on Saturdays for a four-week trial, in addition to its current hours. Do you support or oppose this proposal?” Offer support, oppose and unsure, with a separate option to skip. Ask about workshops in a different question.

AAPOR’s Best Practices for Survey Research recommends specific, understandable questions addressing one concept at a time, with carefully chosen answer options. The revised wording applies that principle to our original case; it is not a validated questionnaire or a claim that wording alone removes every source of error.

The decision may require more than preference

Even a well-worded support question does not measure actual attendance. A member may support longer hours for other people while expecting never to attend. Another may oppose the expense yet use the room if it opens. Support, intention and behaviour are related questions, but they are not interchangeable measures.

For planning, a separate item could ask which of the four trial Saturdays the member currently expects to use. A later attendance record could then describe what happened. Neither should be presented as a perfect forecast: plans change, and a survey response is not a booking commitment unless the process explicitly makes it one.

The proposal may also have costs or trade-offs. If later hours would replace an existing session, that condition belongs in the description of the option. Asking about an apparently free addition and then using the result to justify a substitution changes the decision respondents were asked to consider.

Before adding ten more questions, identify which decision each answer could affect. A short, well-targeted questionnaire can be more useful than a long one full of interesting but unused details. Keep the burden proportionate and avoid collecting personal information merely because a form makes it easy.

A small pretest with a clear purpose

Before fielding the revised survey, ask a few appropriate volunteers to explain what they think the question means. This is a comprehension check, not a miniature poll from which to estimate population support. Look for different interpretations of the time period, the additional hours and the available response options.

For example, one volunteer might think that “Saturday evening” means the existing afternoon session will disappear. Another might think workshops are included because an earlier question described them. These responses reveal problems in wording or context. The appropriate outcome is a revised instrument, not an average of the volunteers’ preferences.

Record what changed and why. If you remove a confusing sentence, retain the old and new wording in your working notes. That record helps explain later differences and prevents someone from treating results from two materially different questions as if they came from an unchanged measure.

AAPOR’s best-practice guidance also discusses pretesting, fieldwork and the importance of maintaining comparable measures when assessing change. In our case, the immediate goal is modest: make sure the question respondents answer is recognisably the question the organiser intended to ask.

A separate weighting example

The following is a new, independent teaching dataset. Do not combine its counts with the 126 supporters above. Suppose a target membership consists of 40% frequent visitors and 60% occasional visitors. An achieved sample of 100 contains 80 frequent visitors and 20 occasional visitors. Within the sample, 64 frequent visitors and 10 occasional visitors support a proposal.

Support among frequent visitors is 64/80 = 80%. Support among occasional visitors is 10/20 = 50%. The unweighted sample percentage is 74/100 = 74%. The sample gives frequent visitors much more influence than their 40% share in the target membership.

Applying the target shares gives 0.40 × 80% + 0.60 × 50% = 62%. This calculation preserves the observed within-group rates while changing the groups’ contribution to the combined estimate. It explains how a weighted result can differ from a simple respondent percentage without any individual answer changing.

The result depends on assumptions. The population shares must be suitable and accurate, the group labels must align and the observed rates must be useful for representing members within each group. Weighting by visit frequency cannot automatically correct every difference in schedules, preferences or survey participation inside those groups.

What weighting cannot conjure into existence

Imagine that no occasional visitors answered at all. We would then lack an observed support rate for that group. Assigning larger weights to frequent visitors does not create the missing occasional-visitor information. A model might introduce assumptions to estimate it, but those assumptions would need to be stated and evaluated.

Even when both groups respond, a small subgroup can make its estimate unstable. Ten supporters among 20 occasional visitors provide a descriptive rate, but changing a few answers changes that rate substantially. A larger weight increases the subgroup’s contribution; it does not increase the amount of observed evidence about its members.

The worked calculation is therefore an explanation of a mechanism, not a certificate that 62% is the true population value. A responsible report identifies the weighting variables, target totals and relevant limitations. It also distinguishes the number of actual respondents from a weighted total that may be expressed on another scale.

For the full treatment, continue to How Survey Weighting, Calibration and Nonresponse Adjustment Work. Keep this page as the place to practise seeing how a denominator and group composition alter one reported percentage.

Why we do not attach a conventional margin of error

Our main reading-room case used convenience invitations from an incomplete email frame, followed by voluntary response. We have not supplied a probability selection design or a justified inferential model. A conventional simple-random-sample margin of error would therefore suggest a basis for inference that this case does not possess.

This does not make the 180 answers disappear. Their descriptive distribution remains useful for understanding those answers and developing questions for further investigation. It means that we should avoid translating that distribution into a population estimate with a familiar-looking uncertainty number whose assumptions are absent.

Increasing the number of convenience respondents may give a more stable description of the people who respond while leaving systematic selection differences in place. Precision about a selected group and accuracy about the intended population are different achievements. A large count deserves inspection of recruitment just as a small count does.

For probability-based inference, design effects and interval interpretation, use How Statistical Inference and Uncertainty Work. Here the important practical decision is to label the current evidence honestly and identify what additional design information would be needed before making stronger claims.

Write a report that a reader can audit

A defensible short report for the main case reads: “In this fictional member survey, 126 of the 180 respondents who answered the Saturday-hours question supported the proposal, giving 70%. A further 20 survey respondents skipped the item. The organiser received 200 questionnaires from 400 eligible invitees, selected for convenience from an email list covering 600 of 1,000 current members. The result describes the recorded answers and does not establish support among all members.”

That paragraph preserves the main finding. It also gives the reader the base, missingness, recruitment route and population boundary. A headline could be “Seven in ten answers support a Saturday-hours trial,” followed immediately by the base and method. “Members overwhelmingly demand longer hours” goes beyond the evidence and introduces a judgement about intensity.

The exact question should accompany the report, especially when its wording or conditions affect interpretation. State whether the result is weighted. Give the fieldwork period and the membership date when reporting a real survey, because both define what the numbers refer to. Do not invent those details to make a thin report look complete.

A useful next action is to learn about excluded and less-represented members, clarify the actual timetable trade-off and test demand through a bounded trial if appropriate. The report should support a decision process while keeping unresolved evidence visible.

Practice questions

Use the main dataset unless a question explicitly introduces a new one. Write both a calculation and a sentence explaining its denominator.

1. What percentage of people answering the hours item opposed the proposal?
2. What percentage of all returned questionnaires contain a support answer?
3. A headline reports 77.8% support. Which exclusion could produce this figure, and what wording is needed?
4. Someone says that all 200 nonresponding invitees oppose the trial. What is wrong with the claim?
5. If the 20 skipped items were all support, with the original 180 answers unchanged, what would support among the 200 respondents be?
6. In the separate weighting example, why does the weighted result fall below the unweighted result?
7. A second survey changes both the question and the recruitment channel. Can a higher support percentage be read directly as a change in member opinion?
8. What one additional piece of information would most improve your interpretation of a percentage with no stated base?

For Questions 4 and 7, an answer that says only “bias” is incomplete. Name the missing information or changed condition and explain the mistaken inference. For Question 8, justify your choice rather than listing every survey detail you can remember.

Explained answers

Question 1: Opposition is 36/180 = 20% among those answering the item. The unsure category remains in the denominator because the claim concerns all recorded answers to the question. If you use 162, you are answering a different question about those who expressed support or opposition.

Question 2: The calculation is 126/200 = 63%. This is the fraction of returned questionnaires containing a support answer. The remaining 37% includes opposition, uncertainty and skipped items; it should not be labelled collectively as opposition.

Question 3: Removing the 18 unsure answers leaves 162 support-or-oppose answers. Then 126/162 is approximately 77.8%. The report must say that this percentage excludes unsure and missing answers and concerns respondents who chose support or opposition. It should preferably show the full distribution nearby.

Question 4: Nonresponse is missing information, not a recorded preference. The nonrespondents might differ from respondents, but the direction and size of that difference are unknown here. Calling every nonrespondent an opponent replaces absence of evidence with an invented answer.

Question 5: Support would become 146/200 = 73% under the stated hypothetical completion. This is a sensitivity calculation, not a discovery about the skipped answers. The original observed count of support answers remains 126 until genuine additional evidence changes it.

Question 6: Frequent visitors have the higher observed support rate and are overrepresented in the achieved sample. Replacing their 80% sample share with their 40% population share reduces their contribution. The resulting 62% still depends on whether the within-group rates are suitable for the intended inference.

Question 7: The difference could reflect opinion change, measurement change, selection change or some combination. A direct trend claim needs evidence that the quantities are comparable. Preserve both questionnaires and recruitment descriptions before deciding what the difference means.

Question 8: The answer base is an essential first request: who was counted in the denominator, and how many were there? That establishes the descriptive meaning of the percentage. Recruitment, wording and missingness remain necessary for wider interpretation, but a percentage without a defined base is already ambiguous.

A changed-condition exercise

Now imagine that the organiser sends the revised, clearly worded question to every one of the 1,000 members through a combination of email and paper delivery. All members are reached, but only 500 return the survey. Of 450 who answer the hours item, 270 support, 135 oppose and 45 are unsure; 50 respondents skip it.

The support percentage among item answers is 270/450 = 60%. Among returned questionnaires, recorded support is 270/500 = 54%. The invitation coverage has improved substantially in this hypothetical design, but nonresponse remains. Inviting the entire population is not the same as obtaining an answer from every member.

Do not conclude that opinion fell by ten percentage points solely because the earlier item-answer result was 70%. The wording, reach and responding group changed. The new design may answer the intended question better, yet the two percentages are not automatically a clean time series.

Write a revised report that names the improved coverage and the remaining limitation. A strong answer gives credit to the design change without claiming it solves every problem. It also keeps the 50 missing item answers visible instead of treating the larger respondent count as the base for every percentage.

A delayed task and the next reading

Tomorrow, start with a new fictional example: 300 households are invited to a neighbourhood survey, 120 reply, 100 answer a transport question and 65 choose walking. Calculate the share choosing walking among item answers, among respondents and among invitees. Then explain why the last percentage does not measure opposition to walking among everyone else.

The answers are 65%, approximately 54.2%, and approximately 21.7%, respectively. If you can produce the accompanying denominator sentences without looking back, you have learned the central operation. If the numbers are right but the interpretation is wrong, return to the section on four correct calculations.

Take the skill into the wider library through What Is Statistics? and Guided Reading and Inquiry. Whenever a percentage appears in a claim, recover the population, the route into the data, the wording and the denominator before allowing the number to speak for a larger group.