SaaS Surveys: Why Yours Gets 3% Response, and the Questions That Fix It
Your survey isn't ignored because people are busy. It's ignored because every question can be answered without thinking, and they can tell.
Response rates on impersonal outreach have been collapsing for twenty years. Pew Research Center tracked telephone survey response falling from 36% in 1997 to 6% in 2018. Every low-effort channel has followed the same curve for the same reasons.
So the standard advice — shorten it, add an incentive, send a reminder — is treating a structural problem as a tactical one. Those tweaks move a 3% response rate to 4%.
The bigger lever is question design, and it works in the opposite direction from what most teams assume. Questions that are harder to answer get answered more often, because they signal that a person will read the reply.
Four failure modes, a rewrite for every question SaaS teams keep asking, and the single question worth sending if you only get one.
TL;DR
- Four failure modes: answerable without thinking, asks for a prediction, asks for a summary, leaks the answer you want.
- Multiple choice fails by construction — it lets a respondent satisfy the question at zero cognitive cost.
- Replace ratings with recall: "when did you last use X, and what were you doing?" beats "how valuable is X (1-5)?"
- Every question after the fourth costs completions. Twelve questions means you deferred prioritisation to your respondents.
- The one-question survey: "What almost stopped you from [signing up / upgrading / staying]?"
It Is Not the Length
36% → 6%
Pew telephone survey response rate, 1997 to 2018
4
Questions before completions start dropping meaningfully
48h
Window in which event recall stays specific
1
Questions in the survey that outperforms most long ones
Length matters, but it is a second-order effect. A three-question survey made entirely of rating scales performs worse than a single well-designed open question, because the problem is not effort — it is that the respondent can tell the answers will not be read.
People are remarkably good at detecting this. A dropdown of four pre-written reasons communicates that you have already decided what the answers are and are counting them. An open question that names a specific moment communicates that a person is going to read the reply. Response rates follow that signal more closely than they follow length.
Harder questions get answered more often than easy ones, provided they are specific. “Rate your satisfaction 1–5” is easy and ignored. “What was the last thing that annoyed you?” takes ten seconds of thought and gets answered.
The Four Failure Modes
Answerable without thinking
The respondent can satisfy the question with a plausible answer at zero cognitive cost. Multiple choice does this by construction: the cheapest click wins, which is why “too expensive” is the most over-reported churn reason in SaaS.
Asks for a prediction
“Would you use this?” “How likely are you to recommend us?” People are poor forecasters of their own behaviour, and these questions measure politeness rather than demand. This is the finding Nielsen Norman Group has been repeating for two decades.
Asks for a summary
“How satisfied are you overall?” compresses a hundred experiences into one number and discards everything you needed. The compression is the whole problem — you cannot decompress a 4/5.
Leaks the answer you want
“How much did our new dashboard improve your workflow?” contains its own conclusion. Respondents are cooperative; they will give you the answer the question implies, and you will ship more of what you already believed.
Name the failure before you rewrite. Otherwise you will just write a different broken question — usually a longer one with the same defect.
The Rewrites, By Survey Type
Cancellation
| Instead of | Ask | Why | |
|---|---|---|---|
| Cancellation reason | "Why are you cancelling?" (dropdown) | "What was the last thing that happened before you decided?" | Events cannot be faked cheaply; categories can |
| Retention | "How could we have kept you?" | "What would have had to be true for you to still be a customer?" | Asks for a condition, not a product design |
| Win-back | "Would you consider coming back?" | "If we fix [X], would you want to hear about it?" | A low-cost commitment beats a prediction |
Onboarding and activation
| Instead of | Ask | Why | |
|---|---|---|---|
| Setup | "How easy was setup? (1–5)" | "What was the first thing that confused you?" | A 4/5 has no location; confusion does |
| Gaps | "What features would you like to see?" | "What did you try to do this week that you couldn’t?" | Gets the problem, not a solution from someone who hasn’t seen your codebase |
| Value | "Are you finding the product useful?" | "What did you use it for most recently, and what happened?" | Behaviour, not opinion |
Value and price
| Instead of | Ask | Why | |
|---|---|---|---|
| Fairness | "Is our pricing fair?" | "What else did you consider spending this budget on?" | Reveals your real competitive set |
| Willingness to pay | "How much would you pay for X?" | "What are you paying for [adjacent thing] today?" | Current spend is a fact; WTP is fantasy |
| Feature value | "How valuable is [feature]? (1–5)" | "When did you last use [feature], and what were you doing?" | Rated value and used value diverge wildly |
“Pay attention to what users do, not what they say. Self-reported claims are unreliable, as are user speculations about future behavior.”
Every rewrite above applies the same move: convert an opinion into a memory. Opinions are constructed on the spot to satisfy you. Memories are retrieved, and retrieval is much harder to fake.
The NPS Question, Specifically
“How likely are you to recommend us, 0–10?” commits three of the four failure modes at once. It asks for a prediction, of a social act, compressed into a summary number.
The rewrite is straightforward: “Have you recommended us to anyone? Who, and what did you say?” This asks whether the act happened rather than whether it might, and the free-text half tells you the words your advocates use — which is the most directly usable marketing input you will get all quarter.
None of which means abandon NPS if your board expects the number. Keep the score as a trend line, and stop treating it as a diagnosis. The follow-up free-text box was always the part with diagnostic value, and it is the part most teams never read.
Five Design Rules
- Length. Every question after the fourth costs completions. Twelve questions means you have not prioritised — you deferred the decision to your respondents, and they will decide by abandoning.
- Order. Open question first, while attention is highest. Demographics and multiple choice last. Most surveys do exactly the reverse, spending the attention budget on the least informative questions.
- Timing. Ask within 48 hours of the event you are asking about. After that, the specific frustration has been replaced by a tidy narrative, and the narrative is not what happened.
- Channel. In-app surveys reach people still using the product — which excludes, by definition, everyone who left. If your survey is in-app only, you are sampling survivors. This is the same structural bias that affects public feedback boards.
- The “other” box. If more than a fifth of respondents pick “other” and write something, your options are wrong. That is a finding, not noise, and it is telling you to stop using multiple choice for this question.
The Survey Question Rewrite Sheet
Every rewrite above plus the design rules and the one-question survey, in a format you can paste into your own survey tool.
- The four failure modes, with how to spot each one
- Before-and-after rewrites for cancellation, onboarding, value, price and satisfaction
- The five design rules: length, order, timing, channel, the "other" box
- The one-question survey and why it outperforms
- When a survey is the wrong instrument entirely
The One-Question Survey
“What almost stopped you from [signing up / upgrading / staying]?”
Four properties make it work. It is answerable from memory. It is about a specific moment. It does not leak the answer you want. And it surfaces the objection your funnel is quietly losing people to — from people who didn't leave, which is the only way to hear an objection that was overcome rather than fatal.
We have not found a rating scale that outperforms it on usable-answers-per-send. Ask it after signup, after an upgrade, and at renewal, and you have three-quarters of a feedback programme for the price of one field.
When To Stop Surveying Entirely
A survey is the right instrument when you already know the shape of the answer and need to size it. It is the wrong instrument when you do not know what you are looking for — that is what conversations are for.
The practical test: if your open-text answers are still surprising you, stop writing surveys and go have ten conversations. Come back to the survey once you know which two things you are trying to count.
This ordering is the thing most teams get backwards. They survey to discover, get vague results, and conclude that customers do not know what they want. Customers know perfectly well; the instrument was wrong. The full comparison of interviews and surveys works through where each one belongs.
Frequently asked questions
- How many questions should a SaaS survey have?
Four or fewer for anything sent by email or shown in-app. Completions drop measurably after that, and the questions you add are usually the ones you were least sure you needed. If you have twelve candidate questions, the exercise is to cut eight — not to send twelve and hope.
- Why is our survey response rate so low?
Most likely because the questions signal that nobody will read the answers. Multiple choice, rating scales, and generic satisfaction questions all communicate “we are counting, not listening”. Secondary causes: sending too long after the event, sending in-app to people who stopped logging in, and asking people to predict rather than recall.
- Should exit surveys be on mobile?
They should be wherever the person is, which for a departing user is usually email on a phone rather than inside your app. The practical implication is formatting: one question per screen, a large text field, no matrix grids, and no required fields beyond the first. Matrix questions are close to unusable on a phone and are a common silent cause of abandonment.
- Is it worth offering an incentive to complete a survey?
It raises completion and lowers signal. An incentive recruits people motivated by the incentive, who give you the shortest answer that qualifies. If your problem is sample size for a quantitative question, it can be worth it. If your problem is understanding why people leave, it works against you.
Sources & further reading
- 1Response rates in telephone surveys have resumed their decline — Pew Research CenterResponse rates fell from 36% in 1997 to 6% in 2018.
- 2What Low Response Rates Mean for Telephone Surveys — Pew Research CenterOn what low response rates do and do not imply about bias.
- 3First Rule of Usability? Don’t Listen to Users — Nielsen Norman GroupThe prediction-versus-recall distinction underneath every rewrite above.
- 4Net Promoter Score (NPS): the Good, the Bad and the Ugly — IMD Business School
Keep reading
The best survey question still loses to a follow-up.
A form cannot ask 'what did that look like, specifically?' saasfeedback.ai puts a real human on the phone with your churned users and trial drop-offs. See what comes back.
Book a demo