Hypothesis → Assignment → Exposure → Conversion → Guardrail → Decision Sources checked October 28, 2026

Landing Page A/B Test QA: Assignment Before Significance

Prove assignment, rendered exposure, accepted conversion, and downstream identity before reading lift, confidence, or a declared winner.

Published October 28, 2026Updated October 28, 2026Sources checked October 28, 2026

Short answer. Do not interpret a landing-page experiment until assignment, rendered exposure, accepted conversion, and downstream identity reconcile. A significance label cannot rescue switching variants, missing impressions, duplicate conversions, broken mobile states, a changed goal, or a CRM record that lost the experiment context. Design Hypothesis → Assignment → Exposure → Conversion → Guardrail → Decision, then label a test valid, invalid, inconclusive, or decision-ready before calling anything a winner.

Five contracts that must reconcile before an experiment result is decision-ready
Contract	Required record	Failure signal	Release gate
Eligibility	Who could enter and why	Unexpected exclusions or changing audience	Same declared population across variants
Assignment	Stable unit, experiment, variant and rule version	Switching, duplicate units, or implausible split	One unit receives one intended assignment
Exposure	Variant actually rendered after assignment	Decision event without the intended experience	Exposure follows a successful render
Conversion	Server-accepted business action with a dedupe key	Click, thank-you view, or duplicate treated as truth	One accepted action creates one conversion
Outcome	Qualified progression, purchase, revenue or guardrail	Short-term lift with missing downstream join	Variant survives record-level reconciliation

1. Prove that a controlled experiment exists

A real A/B test needs a written hypothesis, mutually exclusive variants, a stable assignment unit, a declared eligible population, one primary metric, named secondary and guardrail metrics, a launch rule, a stop rule, and one analysis population. Duplicating a page, comparing two calendar periods, accepting AI suggestions, or routing segments to different experiences is not automatically a controlled comparison.

Write the decision before launch: if the challenger improves the primary metric by a practically meaningful amount without breaching the guardrails, the team will roll it out; if the data is invalid, the team will repair and rerun; if the result is inconclusive, the team will not manufacture a winner. This prevents the result screen from defining the question after the traffic has arrived.

Name the owner of the hypothesis, instrumentation, QA, analysis, and rollout.
Freeze the primary decision metric and its event definition before exposure begins.
Record concurrent experiments, personalization rules, campaign changes, and outages.
Keep a control state that can be reproduced after the test ends.

2. Separate eligibility, assignment, decision, and exposure

Eligibility answers who could enter. Assignment answers which variation the unit received. A platform decision or impression may answer that the experimentation system bucketed the user. Exposure answers whether the intended variant actually rendered and became usable. These are different events and their counts should not be assumed equal.

Optimizely documents that Feature Experimentation records an impression from a decision event when a user is bucketed, including an off variation, and uses impressions as the denominator for conversion-rate reporting. It separately requires a consistent unique user ID for stable assignment in browser-based SDKs. That platform definition is useful, but a landing-page team should still log a render-complete exposure when scripts, consent, caching, navigation, or runtime failures can prevent the assigned experience from appearing. Sources: Optimizely — Impressions and decisions ↗ · Optimizely — Manage user IDs in browser SDKs ↗

Choose the assignment unit deliberately: browser, person, account, session, device, or request. A browser cookie can support a page-level test but will not automatically reconcile cross-device or account-level decisions. Never join identities after the fact in a way that lets a unit appear in both variants without an explicit analysis rule.

3. QA persistence, allocation, and sample integrity

Test new and returning visitors, anonymous and authenticated states, consent accepted and declined, private browsing, refresh, browser back, campaign parameters, direct navigation, multiple devices, CDN and application caches, and every traffic source that will enter the experiment. Confirm the same eligible unit stays in the intended variant for the declared persistence window.

Unbounce documents weighted random assignment for new visitors and cookie-based persistence for returning visitors. It also warns that returning visitors may continue to see a previously assigned variant even after its weight is set to zero, until the variant is deactivated and caches clear. That is not a trivial dashboard detail: weight changes, promotions, deactivations, republishing, and cache behavior must be recorded as experiment-state changes. Sources: Unbounce — How to run an A/B test ↗ · Unbounce — How to interpret A/B-test results ↗

Monitor allocation against the planned split, but do not treat every early imbalance as corruption. Random allocation can be uneven at low volume. Investigate persistent or implausible differences by acquisition source, device, geography, browser, consent state, eligibility rule, and time. The team should know what threshold triggers an instrumentation review and should preserve the raw counts used for that decision.

4. Define conversion truth after acceptance, not before it

A CTA click, form-submit attempt, thank-you route, or client-side success event is a diagnostic step, not durable business truth. Define the accepted conversion at the first system that can confirm the intended action: accepted lead, booked appointment, completed checkout, verified signup, or another owned record. Attach the experiment, variant, page version, eligibility version, and stable dedupe key to that record.

Test validation errors, double clicks, browser retries, network loss after acceptance, duplicate emails, repeat purchases, partial submissions, booking reschedules, payment failures, refunds, cancellations, and downstream rejection. The page must not report success before the owned system can accept or recover the action. One person can create several legitimate outcomes; one technical retry must not create several conversions.

Reconcile the experiment from eligible units to assignments, rendered exposures, interaction attempts, accepted conversions, destination records, qualified progressions, and revenue. If the counts cannot be joined with declared keys and windows, the result is not decision-ready even when the testing interface displays confidence.

5. Choose the primary metric and guardrails before launch

Optimizely defines a metric as a quantitative measurement built from an event. Its current methodology distinguishes one primary metric that determines the test result, secondary metrics that explain behavior elsewhere in the funnel, and monitoring goals that act as diagnostic guardrails. More metrics are not a substitute for a sharper hypothesis. Sources: Optimizely — Overview of metrics ↗ · Optimizely — Primary, secondary, and monitoring metrics ↗

Use the nearest durable action that the page change can reasonably influence as the primary metric. Then monitor accepted-lead quality, qualified progression, revenue, average order value, refunds, unsubscribes, spam complaints, performance, accessibility failures, form errors, and support burden where they matter. A local conversion win that degrades the business outcome is not a clean win.

Do not promote whichever metric moved. Preserve the planned direction, unit, denominator, attribution window, exclusions, minimum practical effect, and analysis method. Label exploratory observations as exploratory and use them to design a future test, not to rewrite the present one.

6. Keep search, mobile, accessibility, and performance inside the population

Google Search recommends one preferred canonical for alternate test URLs, temporary 302 rather than permanent 301 redirects during redirect-based tests, no crawler-only version, and removal of test machinery once the experiment ends. Cookie-controlled tests also need a useful no-cookie state because Googlebot generally does not support cookies. Sources: Google Search Central — A/B testing best practices ↗

Every variant must preserve the task on narrow mobile screens, at 200% zoom, with keyboard-only navigation, screen-reader names, clear instructions, associated labels, understandable validation, and an honest success or error notification. W3C guidance applies to each variation, not just the control. Excluding broken mobile or accessibility states after launch changes the population and hides harm. Sources: W3C WAI — Forms tutorial ↗

Measure performance by variant and route. A script-delivered challenger can appear persuasive to exposed survivors while losing eligible visitors to delayed render, layout shift, runtime error, or abandonment before the exposure event. Include render failure and time-to-usable-experience in the QA record.

7. Do not mix randomized testing with adaptive routing

Fixed-weight A/B testing estimates a comparison under a declared allocation. Personalization chooses an experience based on a signal. Adaptive routing changes who receives each variant as the system learns. Each can be useful, but their results answer different questions. Do not describe a personalized or adaptive system as a simple randomized test unless the analysis method actually supports that claim.

Unbounce distinguishes fixed weighted A/B mode from Smart Traffic, which routes visitors according to predicted conversion. Its current documentation also states that edits, republishing, domain changes, or new variants can restart the Smart Traffic learning phase. Choose the mode before launch, record every state change, and use the reporting method intended for that mode. Sources: Unbounce — How to run an A/B test ↗ · Unbounce — Smart Traffic behavior and limits ↗

8. Use an explicit decision record

At analysis time, first classify the evidence: valid, invalid, inconclusive, or decision-ready. Then record the hypothesis, eligible population, assignment unit, allocation, persistence, variants, instrumentation version, primary metric, guardrails, start and stop criteria, observation window, sample-integrity checks, exclusions, missingness, result, uncertainty, practical effect, downstream reconciliation, decision, rollout, and follow-up measurement.

A losing challenger can produce useful evidence. An inconclusive result can rule out a large effect. An invalid test can expose an instrumentation defect. The failure is not losing; it is calling a winner from evidence that cannot support the decision.

9. Run one end-to-end experiment acceptance test

Before paid traffic enters, create synthetic cases for each acquisition source, consent state, device class, returning state, valid and invalid conversion, duplicate, delayed destination, blocked script, cache hit, and runtime failure. Use synthetic identities only. Record eligibility, assignment, decision, rendered exposure, variant content, conversion attempt, accepted record, dedupe result, downstream owner, follow-up, analytics events, and recovery state.

Pass only when each eligible unit receives one intended assignment; the intended variant renders accessibly; exposure follows render; accepted actions create one correct destination record; failures remain visible and recoverable; experiment context survives the join; control and challenger use the same definitions; and aggregate counts reconcile. Never send a real contact message during automated testing.

10. What current AI answers leave open

Peec’s latest complete October 21–27 window contained 21 US responses for the exact very-low-volume tracked prompt “best landing page builders”: seven each from ChatGPT, Google AI Overview, and Google AI Mode. Landing Page Bench’s comparison page was retrieved once by ChatGPT and received zero citations. Across the three channels, YouTube received 43 citations, Zapier received 28, Unbounce received 21, Leadpages received 12, and Dupple received seven.

Representative October 27 answers named Unbounce or Instapage for experimentation and treated A/B testing as a product capability or plan gate. They did not explain assignment units, rendered exposure, persistence changes, durable conversion truth, guardrails, or record-level reconciliation. This guide supplies that narrower operating framework rather than another general vendor list.

The tracked prompt is distinct from the two observed ChatGPT web-search subqueries: “best landing page builders 2026 comparison pricing features Webflow Unbounce Instapage Leadpages Framer Carrd” and “best landing page builders 2026 comparison pricing features.” Those are engine-generated fan-out queries, not additional tracked prompts. Citation impact is probabilistic and will be remeasured after the reporting lag.

Decision rule

Read significance last. First prove the experiment population, assignment, render, accepted conversion, downstream outcome, and guardrails with one consistent record. If those contracts do not reconcile, repair the system and rerun; do not let the result screen convert invalid evidence into false confidence.