Types
Core type definitions, TypeDicts, and Literals for the xpyrment library.
This module houses all static typing definitions, data schemas, and literal constraints shared across the xpyrment library. It provides strict interfaces for statistical outputs to ensure perfect alignment between metrics, validation engines, statistical routers, and report generation.
| CLASS | DESCRIPTION |
|---|---|
MetricResult |
The canonical data schema representing the output of a statistical metric analysis. |
| ATTRIBUTE | DESCRIPTION |
|---|---|
MetricType |
Literal representing the supported category of metrics.
|
module-attribute
Literal representing the supported category of metrics.
Supported Types
"mean": A continuous or discrete numeric metric where statistics are calculated on a per-unit basis (e.g., average sessions per user, average page views)."proportion": A binary rate metric representing yes/no outcomes on a per-unit basis, equivalent to a Bernoulli trial (e.g., conversion rate, click-through-rate where the unit of analysis is the user)."ratio": An aggregated metric computed as the sum of a numerator divided by the sum of a denominator across all units (e.g., global Click-Through-Rate = total clicks / total impressions). Requires Delta Method for proper variance approximation."revenue": A highly skewed continuous monetary metric (e.g., revenue per user, average order value). Often subject to log-transformations or specialized variance reduction.
Bases: TypedDict
The canonical data schema representing the output of a statistical metric analysis.
This TypedDict establishes a contract for all inference engines (frequentist, Bayesian, and sequential) and reporting utilities, ensuring that every calculated metric contains both descriptive statistics and rigorous statistical validation metrics.
| ATTRIBUTE | DESCRIPTION |
|---|---|
metric_name |
The unique identifier assigned to the analyzed metric.
TYPE:
|
metric_type |
The standardized type string (e.g., "Mean", "Proportion", "Ratio", "Revenue").
TYPE:
|
control_mean |
The sample mean (\(\bar{Y}_C\)) or proportion (\(p_C\)) calculated for the control group.
TYPE:
|
treatment_mean |
The sample mean (\(\bar{Y}_T\)) or proportion (\(p_T\)) calculated for the treatment group.
TYPE:
|
control_var |
The sample variance (\(s^2_C\)) calculated for the control group. For ratios, this represents the Delta-method approximated variance.
TYPE:
|
treatment_var |
The sample variance (\(s^2_T\)) calculated for the treatment group. For ratios, this represents the Delta-method approximated variance.
TYPE:
|
control_n |
The total count of unique units in the control group (\(N_C\)).
TYPE:
|
treatment_n |
The total count of unique units in the treatment group (\(N_T\)).
TYPE:
|
absolute_difference |
The point estimate of the absolute treatment effect: $$ \Delta = \bar{Y}_T - \bar{Y}_C $$
TYPE:
|
relative_lift |
The percentage increase or decrease of the treatment mean relative to the control mean: $$ \text{Lift} = \frac{\bar{Y}_T - \bar{Y}_C}{\bar{Y}_C} $$
TYPE:
|
cuped_applied |
True if Controlled-comparison Using Pre-Existing Data (CUPED) was applied to adjust the variance of this metric. False otherwise.
TYPE:
|
variance_reduction |
The percentage reduction in variance achieved by CUPED, bounded in \([0, 1)\): $$ \text{Reduction} = 1 - \frac{\text{Var}(Y_{\text{CUPED}})}{\text{Var}(Y_{\text{original}})} $$
TYPE:
|
p_value |
The statistical p-value associated with the hypothesis test. For frequentist, this represents the probability of observing a test statistic at least as extreme as the one computed, under the null hypothesis (\(H_0\)).
TYPE:
|
ci_lower |
The lower bound of the absolute confidence/credible interval at the \((1 - \alpha)\) confidence level.
TYPE:
|
ci_upper |
The upper bound of the absolute confidence/credible interval at the \((1 - \alpha)\) confidence level.
TYPE:
|
rel_ci_lower |
The lower bound of the relative confidence/credible interval, scaled relative to the control mean: $$ \text{Rel CI Lower} = \frac{\text{CI Lower}}{\bar{Y}_C} $$
TYPE:
|
rel_ci_upper |
The upper bound of the relative confidence/credible interval, scaled relative to the control mean: $$ \text{Rel CI Upper} = \frac{\text{CI Upper}}{\bar{Y}_C} $$
TYPE:
|
power |
The statistical power (\(1 - \beta\)) achieved by the sample size, denoting the probability of correctly rejecting the null hypothesis when the true treatment effect equals the observed difference.
TYPE:
|