- contact@insightclickiq.com
- Every click, every session, explained by AI
Flags & experiments
Ship behind a flag, and let the AI tell you when to call it
Flags, rollouts and A/B tests live next to the analytics they are judged by. Exposures and conversions are events in the same store, the statistics are computed by the engine, and the model only words a verdict the rules can check.
Flags
Roll out to a share of the people who match
A flag is boolean or multivariate (variants with weights and an optional JSON payload). It is switched on or off, narrowed by targeting rules on person traits (user.plan), country, device, platform, app version, browser, OS or language, and rolled out to a fraction of the people who match. Raising the rollout only adds people; nobody flips back.
- Deterministic: the bucket is a hash of the flag key and the person, so the same person gets the same answer on the web, in the app and on the server
- Every SDK: browser snippet and npm, React Native, iOS and Android read flags from a local cache, synchronous from the next launch
- Test a person: the flag page evaluates any id with traits and says why (disabled, targeting, rollout or match)
- Exposures are a
$flag_exposureevent the first time a flag is read in a session
Experiments
Results with intervals, and a verdict in words
An experiment is a flag with a goal event, up to five secondary metrics, a minimum sample per arm and an optional hypothesis. The results table gives each variant’s exposed people, conversions and rate with a 95% interval, the uplift with its interval, the p-value and the probability of beating control, plus a cumulative chart and a sample-ratio check.
The readout card turns those numbers into ship, keep running or stop, with the reasons and the risks. On the Northwind demo, the checkout-redesign test reads Ship new: 12.65% against 10.99% over 14 days and 8,001 people, +15.1% (interval +2.2% to +27.9%), p = 0.022, a 98.9% probability of beating control, and no sample-ratio mismatch (p = 0.584).
The statistics
What the engine computes before the model says anything
| Number | How |
|---|---|
| Rate interval | Wilson 95% interval per variant |
| Uplift and its interval | (pvariant − pcontrol) / pcontrol, with the unpooled difference interval scaled by the control rate |
| p-value | Two-sided, pooled two-proportion z-test |
| P(beat control) | Beta posteriors, 20,000 Monte-Carlo draws with a fixed seed, so the answer does not wobble between page loads |
| Sample-ratio mismatch | Chi-square of the exposed counts against the weights; a mismatch at p < 0.001 stops the test |
| Who counts | People (across devices), in the first variant they saw; converted means the goal after their first exposure, inside the window |
How it works
An experiment readout: from a flag read to a verdict the rules agree with
What you can ask
Questions this answers
Type them into Ask, give them to an agent as a goal, or send them through MCP from your own assistant.
Is checkout-redesign ready to call?The readout: ship, keep running or stop, with reasons
Which experiments are running?The flags list filtered to experiments, through the list_flags tool
Did the new checkout hurt Checkout started?The secondary metric with its uplift and p-value
Show me the conversion of each variant by dayThe cumulative chart, or a trend on $flag_exposure
Who is in the free-shipping-banner rollout?Test a person, or a segment on the exposure event
Compare the new variant on mobile and desktopA breakdown of the goal by device for exposed people
Outcomes
What teams do with it
Calls you can defend
A verdict with intervals, a ratio check and named risks, not a green arrow.
One place for the release
The flag, its rollout, its experiment and its impact on errors and journeys sit on one event stream.
The same answer everywhere
Web, app and server read the same deterministic bucket for a person.
Where it stops. Goals are conversions (did the person do the goal event after exposure); revenue per visitor is not an experiment metric. Flags are evaluated by the server and cached by the SDKs, so a change reaches a browser on its next fetch (start, identify, reset or reload). A test needs the minimum sample per arm before the readout will call it.