Conformal Prediction

Conformal prediction provides distribution-free prediction intervals with guaranteed coverage probability. Unlike parametric methods, conformal prediction makes minimal assumptions about the underlying distribution and provides valid coverage even for finite samples.

All TypeScalarTable Macro

Function	Description	Type
ts_conformal_quantile	Compute conformity score from residuals	Scalar
ts_conformal_intervals	Apply conformity score to create intervals	Scalar
ts_conformal_predict	Full split conformal prediction	Scalar
ts_conformal_predict_asymmetric	Asymmetric conformal for skewed residuals	Scalar
ts_mean_interval_width	Compute mean width of prediction intervals	Scalar
ts_conformal_coverage	Empirical coverage evaluation	Scalar
ts_conformal_evaluate	Full evaluation (coverage, width, Winkler)	Scalar
ts_conformal_by	High-level grouped conformal prediction	Table Macro
ts_conformal_calibrate	Calibrate conformity score from backtest table	Table Macro
ts_conformal_apply_by	Apply pre-computed score to forecast table	Table Macro
ts_interval_width_by	Compute interval widths for grouped results	Table Macro

Showing 11 of 11

---

How It Works

The conformal prediction system operates in two phases:

1. Calibration: Compute a conformity score from calibration residuals (actual - predicted)
2. Prediction: Apply the conformity score to new forecasts to create prediction intervals

The resulting intervals will cover the true value with probability at least 1 - alpha, where alpha is the miscoverage rate.

---

Scalar Functions

ts_conformal_quantile

Computes the empirical quantile of absolute residuals for split conformal prediction.

ts_conformal_quantile(residuals DOUBLE[], alpha DOUBLE) → DOUBLE

Parameters:

• residuals: Array of residuals (actual - predicted) from calibration set
• alpha: Miscoverage rate (0 < alpha < 1). Use 0.1 for 90% coverage, 0.05 for 95% coverage.

Returns: DOUBLE - The conformity score (quantile of absolute residuals)

Example:

SELECT ts_conformal_quantile(
    [1.0, -0.5, 2.0, -1.5, 0.8],
    0.1
) AS conformity_score;

---

ts_conformal_intervals

Applies a pre-computed conformity score to create symmetric prediction intervals.

ts_conformal_intervals(forecasts DOUBLE[], conformity_score DOUBLE)
→ STRUCT(lower DOUBLE[], upper DOUBLE[])

Parameters:

• forecasts: Array of point forecasts
• conformity_score: Pre-computed quantile from ts_conformal_quantile

Returns: STRUCT containing:

• lower: Array of lower interval bounds
• upper: Array of upper interval bounds

Example:

SELECT
    (ts_conformal_intervals([100.0, 110.0, 120.0], 5.0)).lower AS lower,
    (ts_conformal_intervals([100.0, 110.0, 120.0], 5.0)).upper AS upper;
-- Returns: lower = [95.0, 105.0, 115.0], upper = [105.0, 115.0, 125.0]

---

ts_conformal_predict

Full split conformal prediction: computes conformity score from residuals and applies to forecasts.

ts_conformal_predict(residuals DOUBLE[], forecasts DOUBLE[], alpha DOUBLE)
→ STRUCT(
    point DOUBLE[],
    lower DOUBLE[],
    upper DOUBLE[],
    coverage DOUBLE,
    conformity_score DOUBLE,
    method VARCHAR
)

Parameters:

• residuals: Calibration residuals (actual - predicted)
• forecasts: Point forecasts to generate intervals for
• alpha: Miscoverage rate

Example:

WITH backtest_residuals AS (
    SELECT [1.2, -0.8, 1.5, -1.0, 0.5]::DOUBLE[] AS residuals
),
future_forecasts AS (
    SELECT [100.0, 102.0, 104.0]::DOUBLE[] AS forecasts
)
SELECT
    (ts_conformal_predict(residuals, forecasts, 0.1)).*
FROM backtest_residuals, future_forecasts;

---

ts_conformal_predict_asymmetric

Asymmetric conformal prediction that computes separate upper and lower quantiles. Useful when residuals are not symmetric (e.g., skewed demand forecasts).

ts_conformal_predict_asymmetric(residuals DOUBLE[], forecasts DOUBLE[], alpha DOUBLE)
→ STRUCT(
    point DOUBLE[],
    lower DOUBLE[],
    upper DOUBLE[],
    coverage DOUBLE,
    conformity_score DOUBLE,
    method VARCHAR
)

Example:

-- Asymmetric residuals (over-predictions more common)
SELECT
    (ts_conformal_predict_asymmetric(
        [-0.5, -0.3, 0.2, 1.0, 2.5],
        [100.0, 110.0],
        0.1
    )).*;

---

ts_mean_interval_width

Computes the mean width of prediction intervals. Useful for comparing interval sharpness across models.

ts_mean_interval_width(lower DOUBLE[], upper DOUBLE[]) → DOUBLE

Example:

SELECT
    ts_mean_interval_width([95.0, 105.0], [105.0, 115.0]) AS model_a_width,
    ts_mean_interval_width([90.0, 100.0], [110.0, 120.0]) AS model_b_width;
-- Model A: 10.0 (sharper), Model B: 20.0 (wider)

---

ts_conformal_coverage

Computes empirical coverage: the fraction of actual values that fall within the prediction intervals.

ts_conformal_coverage(actuals DOUBLE[], lower DOUBLE[], upper DOUBLE[]) → DOUBLE

Example:

SELECT ts_conformal_coverage(
    [100.0, 105.0, 110.0],
    [95.0, 100.0, 105.0],
    [105.0, 110.0, 115.0]
) AS empirical_coverage;
-- Returns: 1.0 (all actuals within intervals)

---

ts_conformal_evaluate

Full evaluation of prediction intervals: coverage, violation rate, mean width, and Winkler score.

ts_conformal_evaluate(actuals DOUBLE[], lower DOUBLE[], upper DOUBLE[], alpha DOUBLE)
→ STRUCT(
    coverage DOUBLE,
    violation_rate DOUBLE,
    mean_width DOUBLE,
    winkler_score DOUBLE,
    n_observations INTEGER
)

Example:

SELECT (ts_conformal_evaluate(
    [100.0, 105.0, 110.0, 95.0],
    [97.0, 102.0, 107.0, 92.0],
    [103.0, 108.0, 113.0, 98.0],
    0.1
)).*;

Return fields:

• coverage: Fraction of actuals within intervals (target: 1 - alpha)
• violation_rate: Fraction of actuals outside intervals
• mean_width: Average interval width (narrower = sharper)
• winkler_score: Combined width + penalty for violations (lower = better)
• n_observations: Number of observations evaluated

---

Table Macros

ts_conformal_by

High-level macro that performs conformal prediction on grouped backtest results.

ts_conformal_by(
    backtest_results VARCHAR,
    group_col COLUMN,
    actual_col COLUMN,
    forecast_col COLUMN,
    point_forecast_col COLUMN,
    params STRUCT
) → TABLE

Params Options:

Key	Type	Default	Description
alpha	DOUBLE	0.1	Miscoverage rate
method	VARCHAR	'split'	'split' or 'asymmetric'

Example:

SELECT * FROM ts_conformal_by(
    'backtest_results',
    product_id,
    actual,
    forecast,
    point_forecast,
    {'alpha': 0.1, 'method': 'split'}
);

---

ts_conformal_calibrate

Calibrates a conformity score from backtest residuals.

ts_conformal_calibrate(
    backtest_results VARCHAR,
    actual_col COLUMN,
    forecast_col COLUMN,
    params STRUCT
) → TABLE(conformity_score DOUBLE, coverage DOUBLE, n_residuals INTEGER)

Returns: Single row table with:

• conformity_score: Computed quantile for intervals
• coverage: Theoretical coverage probability
• n_residuals: Number of residuals used

Example:

SELECT * FROM ts_conformal_calibrate(
    'backtest_results',
    actual,
    forecast,
    {'alpha': 0.05}
);

---

ts_conformal_apply_by

Applies a pre-computed conformity score to grouped forecast results.

ts_conformal_apply_by(
    forecast_results VARCHAR,
    group_col COLUMN,
    forecast_col COLUMN,
    conformity_score DOUBLE
) → TABLE

Example:

-- Two-step workflow: calibrate then apply
CREATE TABLE calibration AS
SELECT * FROM ts_conformal_calibrate('backtest', actual, forecast, {'alpha': 0.1});

SELECT * FROM ts_conformal_apply_by(
    'future_forecasts',
    product_id,
    yhat,
    (SELECT conformity_score FROM calibration)
);

---

ts_interval_width_by

Computes mean interval width for grouped forecast results.

ts_interval_width_by(
    results VARCHAR,
    group_col COLUMN,
    lower_col COLUMN,
    upper_col COLUMN
) → TABLE(group_col, mean_width DOUBLE, n_intervals BIGINT)

Example:

SELECT * FROM ts_interval_width_by(
    'forecast_results',
    product_id,
    lower,
    upper
);

---

Complete Workflow

-- Step 1: Generate backtest forecasts
CREATE TABLE cv_folds AS
SELECT * FROM ts_cv_folds_by('sales_data', store_id, date, revenue, 5, 7, MAP{});

CREATE TABLE backtest_results AS
SELECT * FROM ts_cv_forecast_by('cv_folds', store_id, date, revenue, 'AutoETS',
    MAP{'seasonal_period': '7'});

-- Step 2: Calibrate conformity score (90% coverage)
CREATE TABLE calibration AS
SELECT * FROM ts_conformal_calibrate(
    'backtest_results',
    y,
    yhat,
    {'alpha': 0.1}
);

-- Step 3: Generate future forecasts
CREATE TABLE future_forecasts AS
SELECT * FROM ts_forecast_by(
    'sales_data', store_id, date, revenue,
    'AutoETS', 7, '1d', MAP{'seasonal_period': '7'}
);

-- Step 4: Apply conformal intervals
SELECT * FROM ts_conformal_apply_by(
    'future_forecasts',
    store_id,
    yhat,
    (SELECT conformity_score FROM calibration)
);

-- Step 5: Evaluate interval quality
SELECT (ts_conformal_evaluate(
    LIST(y ORDER BY ds),
    LIST(yhat_lower ORDER BY ds),
    LIST(yhat_upper ORDER BY ds),
    0.1
)).*
FROM backtest_results;

---

When to Use Conformal Prediction

Scenario	Recommendation
Need guaranteed coverage	Use conformal prediction
Non-normal residuals	Use asymmetric conformal
Comparing model uncertainty	Use ts_mean_interval_width
Production intervals	Calibrate on backtest, apply to future
Small calibration set	Use conformal (valid for any n)
Evaluating interval quality	Use ts_conformal_evaluate for Winkler scores

---

Comparison with Parametric Intervals

Aspect	Parametric	Conformal
Distribution assumption	Requires normality	Distribution-free
Coverage guarantee	Asymptotic	Finite-sample valid
Interval shape	Always symmetric	Can be asymmetric
Calibration data	Not required	Required
Computation	Analytical	Empirical quantiles