Conformal Prediction

Conformal prediction provides distribution-free prediction intervals with guaranteed coverage probability. Unlike parametric methods, conformal prediction makes minimal assumptions about the underlying distribution and provides valid coverage even for finite samples.

All TypeScalarTable Macro

Function	Description	Type
ts_conformal_quantile	Compute conformity score from residuals	Scalar
ts_conformal_intervals	Apply conformity score to create intervals	Scalar
ts_conformal_predict	Full split conformal prediction	Scalar
ts_conformal_predict_asymmetric	Asymmetric conformal for skewed residuals	Scalar
ts_mean_interval_width	Compute mean width of prediction intervals	Scalar
ts_conformal	High-level grouped conformal prediction	Table Macro
ts_conformal_calibrate	Calibrate conformity score from backtest table	Table Macro
ts_conformal_apply	Apply pre-computed score to forecast table	Table Macro
ts_interval_width	Compute interval widths for grouped results	Table Macro

Showing 9 of 9

---

How It Works

The conformal prediction system operates in two phases:

1. Calibration: Compute a conformity score from calibration residuals (actual - predicted)
2. Prediction: Apply the conformity score to new forecasts to create prediction intervals

The resulting intervals will cover the true value with probability at least 1 - alpha, where alpha is the miscoverage rate.

---

Scalar Functions

ts_conformal_quantile

Computes the empirical quantile of absolute residuals for split conformal prediction.

ts_conformal_quantile(residuals DOUBLE[], alpha DOUBLE) → DOUBLE

Parameters:

• residuals: Array of residuals (actual - predicted) from calibration set
• alpha: Miscoverage rate (0 < alpha < 1). Use 0.1 for 90% coverage, 0.05 for 95% coverage.

Returns: DOUBLE - The conformity score (quantile of absolute residuals)

Example:

SELECT ts_conformal_quantile(
    [1.0, -0.5, 2.0, -1.5, 0.8],
    0.1
) AS conformity_score;

---

ts_conformal_intervals

Applies a pre-computed conformity score to create symmetric prediction intervals.

ts_conformal_intervals(forecasts DOUBLE[], conformity_score DOUBLE)
→ STRUCT(lower DOUBLE[], upper DOUBLE[])

Parameters:

• forecasts: Array of point forecasts
• conformity_score: Pre-computed quantile from ts_conformal_quantile

Returns: STRUCT containing:

• lower: Array of lower interval bounds
• upper: Array of upper interval bounds

Example:

SELECT
    (ts_conformal_intervals([100.0, 110.0, 120.0], 5.0)).lower AS lower,
    (ts_conformal_intervals([100.0, 110.0, 120.0], 5.0)).upper AS upper;
-- Returns: lower = [95.0, 105.0, 115.0], upper = [105.0, 115.0, 125.0]

---

ts_conformal_predict

Full split conformal prediction: computes conformity score from residuals and applies to forecasts.

ts_conformal_predict(residuals DOUBLE[], forecasts DOUBLE[], alpha DOUBLE)
→ STRUCT(
    point DOUBLE[],
    lower DOUBLE[],
    upper DOUBLE[],
    coverage DOUBLE,
    conformity_score DOUBLE,
    method VARCHAR
)

Parameters:

• residuals: Calibration residuals (actual - predicted)
• forecasts: Point forecasts to generate intervals for
• alpha: Miscoverage rate

Returns: STRUCT with:

• point: Point forecasts
• lower: Lower interval bounds
• upper: Upper interval bounds
• coverage: Theoretical coverage probability
• conformity_score: Computed quantile value
• method: Method identifier ('split')

Example:

WITH backtest_residuals AS (
    SELECT [1.2, -0.8, 1.5, -1.0, 0.5]::DOUBLE[] AS residuals
),
future_forecasts AS (
    SELECT [100.0, 102.0, 104.0]::DOUBLE[] AS forecasts
)
SELECT
    (ts_conformal_predict(residuals, forecasts, 0.1)).*
FROM backtest_residuals, future_forecasts;

---

ts_conformal_predict_asymmetric

Asymmetric conformal prediction that computes separate upper and lower quantiles. Useful when residuals are not symmetric (e.g., skewed demand forecasts).

ts_conformal_predict_asymmetric(residuals DOUBLE[], forecasts DOUBLE[], alpha DOUBLE)
→ STRUCT(
    point DOUBLE[],
    lower DOUBLE[],
    upper DOUBLE[],
    coverage DOUBLE,
    conformity_score DOUBLE,
    method VARCHAR
)

Example:

-- Asymmetric residuals (over-predictions more common)
SELECT
    (ts_conformal_predict_asymmetric(
        [-0.5, -0.3, 0.2, 1.0, 2.5],
        [100.0, 110.0],
        0.1
    )).*;

---

ts_mean_interval_width

Computes the mean width of prediction intervals. Useful for comparing interval sharpness across models.

ts_mean_interval_width(lower DOUBLE[], upper DOUBLE[]) → DOUBLE

Example:

SELECT
    ts_mean_interval_width([95.0, 105.0], [105.0, 115.0]) AS model_a_width,
    ts_mean_interval_width([90.0, 100.0], [110.0, 120.0]) AS model_b_width;
-- Model A: 10.0 (sharper), Model B: 20.0 (wider)

---

Table Macros

ts_conformal

High-level macro that performs conformal prediction on grouped backtest results.

ts_conformal(
    backtest_results VARCHAR,
    group_col IDENTIFIER,
    actual_col IDENTIFIER,
    forecast_col IDENTIFIER,
    point_forecast_col IDENTIFIER,
    params MAP
) → TABLE

Parameters:

• backtest_results: Table name with backtest results (quoted string)
• group_col: Column for grouping (unquoted, e.g., product_id)
• actual_col: Column with actual values (unquoted)
• forecast_col: Column with in-sample forecasts (unquoted)
• point_forecast_col: Column with point forecasts for interval generation (unquoted)
• params: MAP with optional parameters

Params MAP Options:

Key	Type	Default	Description
alpha	VARCHAR	"0.1"	Miscoverage rate
method	VARCHAR	'split'	'split' or 'asymmetric'

Example:

SELECT * FROM ts_conformal(
    'backtest_results',
    product_id,
    actual,
    forecast,
    point_forecast,
    MAP{'alpha': '0.1', 'method': 'split'}
);

---

ts_conformal_calibrate

Calibrates a conformity score from backtest residuals.

ts_conformal_calibrate(
    backtest_results VARCHAR,
    actual_col IDENTIFIER,
    forecast_col IDENTIFIER,
    params MAP
) → TABLE(conformity_score DOUBLE, coverage DOUBLE, n_residuals BIGINT)

Returns: Single row table with:

• conformity_score: Computed quantile for intervals
• coverage: Theoretical coverage probability
• n_residuals: Number of residuals used

Example:

SELECT * FROM ts_conformal_calibrate(
    'backtest_results',
    actual,
    forecast,
    MAP{'alpha': '0.05'}
);

---

ts_conformal_apply

Applies a pre-computed conformity score to forecast results.

ts_conformal_apply(
    forecast_results VARCHAR,
    group_col IDENTIFIER,
    forecast_col IDENTIFIER,
    conformity_score DOUBLE
) → TABLE(group_col, lower DOUBLE, upper DOUBLE)

Example:

-- Two-step workflow: calibrate then apply
WITH score AS (
    SELECT conformity_score
    FROM ts_conformal_calibrate('backtest', actual, forecast, MAP{'alpha': '0.1'})
)
SELECT * FROM ts_conformal_apply(
    'future_forecasts',
    product_id,
    forecast,
    (SELECT conformity_score FROM score)
);

---

ts_interval_width

Computes mean interval width for grouped forecast results.

ts_interval_width(
    results VARCHAR,
    group_col IDENTIFIER,
    lower_col IDENTIFIER,
    upper_col IDENTIFIER
) → TABLE(group_col, mean_width DOUBLE, n_intervals BIGINT)

Example:

SELECT * FROM ts_interval_width(
    'forecast_results',
    product_id,
    lower,
    upper
);

---

Complete Workflow

Here's a complete conformal prediction workflow:

-- Step 1: Generate backtest forecasts
CREATE TABLE backtest_results AS
SELECT * FROM ts_backtest_auto(
    'sales_data', store_id, date, revenue,
    7, 5, '1d', MAP{'method': 'AutoETS'}
);

-- Step 2: Calibrate conformity score (90% coverage)
CREATE TABLE calibration AS
SELECT * FROM ts_conformal_calibrate(
    'backtest_results',
    actual,
    forecast,
    MAP{'alpha': '0.1'}
);

-- Step 3: Generate future forecasts
CREATE TABLE future_forecasts AS
SELECT * FROM anofox_fcst_ts_forecast_by(
    'sales_data', 'store_id', 'date', 'revenue',
    'AutoETS', 7, MAP{}
);

-- Step 4: Apply conformal intervals
SELECT
    f.store_id,
    f.date,
    f.point_forecast,
    f.point_forecast - c.conformity_score AS lower_90,
    f.point_forecast + c.conformity_score AS upper_90
FROM future_forecasts f
CROSS JOIN calibration c;

-- Step 5: Compare interval widths across models
SELECT
    model_name,
    ts_mean_interval_width(
        list(lower_90),
        list(upper_90)
    ) AS avg_width
FROM backtest_results
GROUP BY model_name;

---

When to Use Conformal Prediction

Scenario	Recommendation
Need guaranteed coverage	Use conformal prediction
Non-normal residuals	Use asymmetric conformal
Comparing model uncertainty	Use ts_mean_interval_width
Production intervals	Calibrate on backtest, apply to future
Small calibration set	Use conformal (valid for any n)

---

Comparison with Parametric Intervals

Aspect	Parametric	Conformal
Distribution assumption	Requires normality	Distribution-free
Coverage guarantee	Asymptotic	Finite-sample valid
Interval shape	Always symmetric	Can be asymmetric
Calibration data	Not required	Required
Computation	Analytical	Empirical quantiles