Analytics - Control Group
The Control Group Analytics Dashboard provides insights into the performance and health of a control group used in experimentation, A/B testing, or revenue impact analysis. It helps analysts monitor revenue uplift, compare average revenue metrics between groups, and validate the integrity of the control group over time.
Dashboard Sections
1. Revenue Uplift
This section measures the incremental revenue generated by the experiment group compared to the control group.
1.1 Powerpop Revenue Uplift (%)
Purpose
- Displays the overall percentage increase in revenue attributed to the experiment.
- Provides a quick summary KPI for business stakeholders.
Metric
Revenue Uplift (%) = ((Experiment Revenue - Control Revenue) / Control Revenue) × 100
Components
- Large KPI value showing current uplift percentage.
- Historical trend sparkline for monitoring uplift changes over time.
- Date indicator showing the reporting date.
Example
Date: 2026-05-31 Revenue Uplift: 89.15%
Interpretation
- Positive value → Experiment is generating more revenue than control.
- Negative value → Experiment underperforms control.
- Near-zero value → Minimal impact.
1.2 Average Revenue Per Visitor by Group (%)
Purpose
Tracks daily incremental revenue differences between experiment and control groups.
Visualization
- Line chart displaying incremental revenue percentages over time.
- Peaks indicate periods where the experiment significantly outperformed control.
- Negative values indicate control outperformed experiment.
Metrics Displayed
- Incremental Revenue (%)
- Daily trend analysis
- Historical comparison
Use Cases
- Identify performance spikes.
- Detect anomalies.
- Monitor experiment stability.
2. Average Revenue Per Group
Purpose
Compares average revenue performance between:
- Control Group
- Exposed/Experiment Group
Visualization
Dual-line chart:
| Color | Group |
|---|---|
| Blue | Control Group |
| Orange | Exposed Group |
Metric
ARPV = Total Revenue / Total Visitors
Where:
- ARPV = Average Revenue Per Visitor
Insights
When Experiment Line > Control Line
The experiment generates higher revenue per visitor.
When Control Line > Experiment Line
The experiment may be underperforming.
Parallel Movement
Indicates both groups are affected similarly by external factors.
Example Observations
- Revenue spikes around mid-May.
- Experiment group closely follows control behavior.
- Significant uplift periods can be investigated further.
3. Control Group Health Check
The health check section validates whether the control group remains statistically reliable.
3.1 Control Group Distribution
Purpose
Ensures traffic allocation remains stable between:
- Control Group
- Experiment Group
Visualization
Bar chart showing daily group population distribution.
Metrics
- Control Group Count
- Experiment Group Count
- Daily allocation volume
Expected Behavior
A healthy control group should:
✅ Maintain stable daily volumes
✅ Follow expected allocation percentages
✅ Avoid sudden traffic drops
Potential Issues
Traffic Imbalance
Example:
Control: 95% Experiment: 5%
Can lead to unreliable experiment results.
Allocation Drift
Unexpected changes in assignment logic.
Sampling Problems
Large fluctuations in participant counts may indicate:
- Tracking issues
- User assignment bugs
- Data ingestion delays
Dashboard Filters & Controls
Time Range Selector
Allows users to:
- View daily trends
- Analyze weekly performance
- Compare monthly performance
Chart Controls
Available on each widget:
| Action | Description |
|---|---|
| Filter | Apply custom filters |
| Refresh | Reload data |
| Export | Download chart data |
| Full Screen | Expand visualization |
| More Options | Additional chart settings |
Key Metrics Summary
| Metric | Description |
|---|---|
| Revenue Uplift (%) | Incremental revenue generated by experiment |
| Incremental Revenue (%) | Daily revenue difference between groups |
| ARPV | Average Revenue Per Visitor |
| Control Group Count | Number of users in control |
| Experiment Group Count | Number of users exposed to experiment |
| Group Distribution | Traffic allocation across groups |
Business Questions Answered
Experiment Impact
- Is the experiment increasing revenue?
- How much incremental value is generated?
Revenue Trends
- Are uplift gains consistent over time?
- Are there any unusual spikes or drops?
Group Comparison
- Does the experiment group outperform the control group?
Experiment Validity
- Is the control group healthy and unbiased?
- Is traffic distribution stable?
Recommended Monitoring Thresholds
| Metric | Healthy Range |
|---|---|
| Revenue Uplift | Positive and statistically significant |
| Control Group Allocation | Stable within expected percentage |
| Daily Traffic Variance | < 10–15% fluctuation |
| ARPV Difference | Consistent trend over reporting period |
Dashboard Workflow
1. Check Revenue Uplift KPI
↓
2. Analyze Incremental Revenue Trend
↓
3. Compare ARPV Between Groups
↓
4. Validate Control Group Distribution
↓
5. Investigate Any Anomalies
↓
6. Make Experiment Decisions
Typical Decision Outcomes
- Scale Experiment → Consistent positive uplift.
- Continue Monitoring → Mixed or unstable performance.
- Stop/Roll Back → Negative revenue impact.
- Investigate Data Quality → Control group health issues detected.
This dashboard serves as a comprehensive monitoring tool for evaluating experiment performance, revenue impact, and control group integrity.