Which Tabular Foundation Model Should You Use? Three Models, Five Datasets, Measured

In the last post we said switching foundation models is one keyword. So we switched it fifteen times and wrote down what happened.

The short version: for classification, model choice barely moved the needle. For regression, it moved it by 30%. The model that won most often is the one you're least likely to be allowed to use. And when we ran scikit-learn on the same splits, the foundation models won all five — by 2–7%, for up to 550× the compute.

This is part 2 of a series. Part 1 covers what tabular foundation models are and how tabfm_classify / tabfm_regress work. Here we're only answering one question: given a real table, which model do you pick?

Every number below was measured on one 8-core CPU against public Hugging Face datasets, with deterministic splits. Nothing is quoted from a paper.

The Contenders

anofox_tabfm ships four models built in. Three of them work end to end today; the fourth has a problem we'll get to honestly at the end.

SELECT model, family, license, commercial, max_rows, max_features, max_classes
FROM tabfm_list_models();

model	size	licence	commercial	max rows	max features	max classes
tabicl-v2 (Inria)	110 MB	BSD-3-Clause	yes	100,000	512	10
mitra (AWS AutoGluon)	303 MB	Apache-2.0	yes	10,000	100	10
tabfm-v1 (Google)	6.56 GB	non-commercial, gated	no	—	—	—
tabpfn-v2 (Prior Labs)	29 MB	Apache-2.0 + attribution	yes	10,000	500	10

Note the 60× size range. That alone should make you suspicious of "bigger is better" — and the results below don't support it either.

The Harness

One SQL pattern, repeated. Hash-based splits so it's reproducible, labels withheld from the scored side, metric computed in SQL against the held-out truth.

CREATE TABLE t AS
SELECT *, hash(row_id) % 100 AS b
FROM 'hf://datasets/scikit-learn/adult-census-income/**/*.csv';

CREATE TABLE tr AS SELECT * EXCLUDE (b)
  FROM (FROM t WHERE b < 70) USING SAMPLE 600 ROWS (reservoir, 42);
CREATE TABLE te AS SELECT * EXCLUDE (b, income)     -- label withheld
  FROM (FROM t WHERE b >= 70) USING SAMPLE 200 ROWS (reservoir, 42);

SELECT row_id, yhat
FROM tabfm_classify('tr', 'income', test := 'te', model := 'mitra');

Changing model := is the entire experiment. That's the point — a bake-off that would normally mean three separate library integrations is a loop over a string.

Here it is for real — the two commercially usable models, scored against the same held-out iris rows, in a single UNION ALL:

Classification: Model Choice Barely Matters

Three datasets, from easy to hard: iris (3 classes, 4 clean numeric features), telco churn (binary, 20 mixed-type columns, imbalanced), adult census income (binary, 14 columns, heavily imbalanced).

dataset	context / scored	metric	mitra	tabicl-v2	tabfm-v1	baseline
iris	97 / 53	accuracy	0.962	0.962	0.943	—
churn	500 / 300	F1	0.595	0.578	0.597	0.000
churn	500 / 300	accuracy	0.773	0.767	0.770	0.657
income	600 / 200	F1	0.741	0.658	0.765	0.000
income	600 / 200	accuracy	0.895	0.870	0.905	0.770

Baseline is the majority-class predictor: always guess "no churn" / "≤50K". It gets decent accuracy and an F1 of exactly zero, which is why accuracy alone is a trap on imbalanced targets.

Now look at the spread between best and worst model:

• iris: 2%. All three are effectively tied.
• churn: 3% on F1. All three land at 0.58–0.60. Swapping models is noise.
• income: 16% on F1. Here it finally matters — TabICL drops to 0.658 while TabFM reaches 0.765.

Two of the three classification datasets don't care which model you use. That's a genuinely useful thing to know before you spend a week benchmarking: on churn, the ceiling is the data, not the model. No model in this lineup extracts more than ~0.60 F1 from 500 context rows, because the signal isn't there. If you're unhappy with churn results, the fix is better features or more context — not a different foundation model.

Regression: Model Choice Matters a Lot

Same harness, tabfm_regress, MSE against the held-out values (lower is better). Baseline is predicting the training mean.

dataset	context / scored	mitra	tabicl-v2	tabfm-v1	mean baseline
tips	186 / 58	1.187	1.085	0.986	1.271
wine	200 / 300	0.626	0.481	0.486	0.788

The picture inverts. Spread between best and worst is 20% on tips and 30% on wine — an order of magnitude more consequential than the classification spreads.

And the ranking is not stable. tabfm-v1 wins tips; tabicl-v2 wins wine, beating a model 60× its size. mitra, the strongest all-round classifier, is the weakest regressor on both — it only beats the mean baseline by 7% on tips where TabICL manages 15% and TabFM 22%.

The practical rule that falls out:

• -> Classification on a hard, noisy target? Pick on licence and speed. The models agree.
• -> Regression? Benchmark. The spread is real, the ranking flips per dataset, and the cheapest model might win.

But Is It Better Than scikit-learn?

This is the question that matters, and skipping it would make the whole post a comparison between three ways of doing something you might not need to do at all.

So we ran the classics on the identical splits — same rows, same withheld labels — exported straight out of DuckDB to Parquet: gradient boosting, random forest, and a plain linear model.

Classical ML needs the preprocessing that foundation models don't, and that work is part of the comparison. For churn: 19 features, of which 12 are categorical, expanding to 530 one-hot columns, plus TotalCharges arriving as text and needing a numeric coercion. Median imputation for numerics, most-frequent for categoricals. The foundation models were handed the raw table.

Classification — best foundation model against each sklearn estimator:

dataset	metric	best TabFM	HistGradientBoosting	RandomForest	LogisticRegression
iris	accuracy	0.962	0.943	0.943	0.943
churn	F1	0.597	0.515	0.488	0.570
income	F1	0.765	0.699	0.700	0.713

Regression — MSE, lower is better:

dataset	best TabFM	HistGradientBoosting	RandomForest	Ridge
tips	0.986	1.194	1.091	1.025
wine	0.481	0.566	0.490	0.544

The foundation models win all five — but read the margins before you celebrate. They're 2% to 7%, not a different league. On wine, RandomForest lands at 0.490 against TabICL's 0.481; that's a rounding error, not a revolution.

Three things worth sitting with:

Gradient boosting never wins. That inverts the reflex. HistGradientBoosting is the default answer to "tabular problem", and here it ties on iris and loses on the other four — to logistic regression on churn and income, to ridge on tips, and to random forest on wine. It comes last outright on income, tips and wine. Boosting needs data to earn its variance, and 200–600 rows is exactly the regime where it doesn't have it. If your instinct is "just throw XGBoost at it", at this size that instinct is wrong twice over.

Restrict to commercially usable models and one result flips. Excluding the non-commercial tabfm-v1, the best available regressor on tips is TabICL at 1.085 — and plain Ridge regression beats it at 1.025. A three-line linear model out-predicting a foundation model is a healthy reminder of where this technology does and doesn't earn its keep.

The speed gap is enormous, and it runs the other way. On churn, LogisticRegression trains and predicts in 0.035 s. TabICL takes 2.3 s to score alone; Mitra takes 19.3 s. That is 66× and 550×.

churn, 500 train / 300 scored	fit + predict	vs LogisticRegression
LogisticRegression	0.035 s	—
RandomForest	0.417 s	12×
tabicl-v2	2.3 s	66×
mitra	19.3 s	550×

So the honest trade is: a few points of quality, and no feature engineering, for one to three orders of magnitude more compute.

When that trade is good:

• -> Few labelled rows. 100–1000 is exactly where boosting underperforms and in-context learning shines.
• -> Many small tasks. One model, no artifact per column — the setup cost you're avoiding is per-task, and it compounds.
• -> Messy mixed types. The 530-column one-hot expansion is code someone writes, tests, and maintains in two places.
• -> Nobody owns an ML pipeline. The comparison isn't sklearn vs TabFM, it's TabFM vs the column staying empty.

When it isn't:

• -> Plenty of labelled data. Give boosting 50,000 rows and this table looks very different.
• -> Latency budgets in milliseconds. 0.035 s versus 19 s is not a close call.
• -> A pipeline that already exists. If the encoders are written and the model is trained, a 4% F1 gain is unlikely to justify replacing it.

The most useful way to read this: run both. The harness at the end of this post takes minutes, and on your data the answer may well be that a linear model is enough.

Speed, and the Cost of Loading 6.56 GB

Runtime splits into two very different things: the one-time cost of reading weights off disk, and the per-query inference cost. Reporting them together flatters small models and slanders big ones — so here they are separately, scoring 300 churn rows against 500 context rows.

model	size	cold (first call)	warm	weight-loading cost
tabicl-v2	110 MB	2.3 s	2.3 s	~0 s
mitra	303 MB	39.3 s	19.3 s	~20 s
tabfm-v1	6.56 GB	37.2 s	9.7 s	~27 s

Read that table twice, because it breaks the obvious intuition.

Loading scales with size, exactly as you'd expect. TabICL's 110 MB is essentially free. TabFM's 6.56 GB costs ~27 seconds on first use, once per session. That's what CALL tabfm_load() is for — pay it at startup instead of on a user's first query.

Inference does not scale with size at all. mitra is 20× smaller than tabfm-v1 and takes twice as long to score the same 300 rows — 19.3 s versus 9.7 s. And tabicl-v2, smaller again, is 8× faster than Mitra.

Parameter count tells you almost nothing about inference cost here; architecture does. Mitra's per-row cost grows steeply with context size — on the small iris split (150 rows total) it finishes in ~1.5 s, but at 800 rows it's the slowest model in the lineup. If you're scoring in a loop, benchmark at your context size, not on a toy split.

One reassurance for anyone putting this in a pipeline: repeated identical calls returned byte-identical predictions for all three models. Same input, same seed, same answer — no sampling, nothing to pin down.

The Licence Is Part of the Benchmark

Here's the uncomfortable summary of the quality tables: tabfm-v1 wins or ties on four of five datasets. It is also the one model you probably cannot ship.

model	licence	can you use it in a commercial product?
mitra	Apache-2.0	Yes
tabicl-v2	BSD-3-Clause	Yes
tabpfn-v2	Apache-2.0 + attribution	Yes, with attribution
tabfm-v1	tabfm-non-commercial-v1.0, gated	No

tabfm-v1 won't even download until you explicitly accept:

SET anofox_tabfm_accept_hf_license = true;

That gate is deliberate. tabfm_list_models() exposes commercial as a column precisely so this is a query, not a legal archaeology project:

SELECT model FROM tabfm_list_models() WHERE commercial;

Restrict to that list and the picture changes completely: on income, your best available F1 drops from 0.765 to 0.741 (Mitra). On wine, you actually gain — TabICL's 0.481 is the best result in the whole table. The commercial-only lineup costs you very little.

Ceilings You'll Hit Before Quality Does

Quality differences of a few percent are easy to over-think. These limits will decide your model long before accuracy does:

	tabicl-v2	mitra	tabpfn-v2
max context rows	100,000	10,000	10,000
max features	512	100	500
max classes	10	10	10

mitra's 100-feature ceiling is the one that bites in practice. A wide ERP table — a materials master, an order header with all its flags — blows through 100 columns easily. TabICL takes 512 and ten times the context rows.

Ten classes is a hard ceiling on all of them. Predicting a material group with 40 values is out of scope for every model here, and no amount of context fixes it.

The Fourth Model

tabpfn-v2 is in the registry, is the smallest model of the four at 29 MB, and is the one this whole field is named after — TabPFN is the Nature paper that started it. It is absent from the tables above because we could not get it to run, and we would rather show you that than quote someone else's numbers.

IO Error: anofox_tabfm: the checkpoint for 'classification' is corrupted or does not
match the bundled model graph. ORT said: Failed to find existing initializer with name
m.transformer_encoder.layers.4.self_attn_between_features._w_out.

The cause is a naming mismatch, not a bad download. The checkpoint Prior Labs currently publishes names its parameters m.transformer_encoder.layers.N...., while the ONNX graph shipped in the extension expects blocks.N.per_sample_attention_between_features..... The translation table between them is stale — upstream reorganised the module tree, and the extension's map hasn't caught up. Re-downloading and re-converting doesn't help; the map itself needs re-deriving.

We've reported it. When it lands, expect TabPFN to be competitive: it is the smallest model here by a factor of ten, and the published literature is strong.

Picking, In Practice

your situation	pick
Commercial product, wide tables (>100 columns)	tabicl-v2
Commercial product, ordinary tables	mitra
Lots of context rows (>10k)	tabicl-v2 — the only one that takes them
Regression, quality above all	Benchmark. The ranking flips per dataset
Research / internal only, quality above all	tabfm-v1, if you can afford 6.56 GB
Latency-sensitive, called often	tabicl-v2
Tens of thousands of labelled rows, or millisecond budgets	scikit-learn — see the baseline above

If you want one default: mitra for classification, and benchmark for regression. If you want one default that never hits a ceiling: tabicl-v2.

Run Your Own Bake-Off

The only honest benchmark is the one on your data. This is the whole harness — point it at your table, hide labels you already have, and count:

INSTALL anofox_tabfm FROM community;
LOAD anofox_tabfm;
INSTALL httpfs; LOAD httpfs;

CALL tabfm_download('classification', model := 'mitra');
CALL tabfm_download('classification', model := 'tabicl-v2');

-- your split
CREATE TABLE tr AS SELECT * FROM your_table WHERE hash(id) % 100 <  70;
CREATE TABLE te AS SELECT * EXCLUDE (target) FROM your_table WHERE hash(id) % 100 >= 70;
CREATE TABLE ac AS SELECT id, target FROM your_table WHERE hash(id) % 100 >= 70;

-- one row per model
SELECT 'mitra' AS model, avg((a.target = p.yhat)::INT) AS accuracy
FROM tabfm_classify('tr','target', test := 'te', model := 'mitra') p
JOIN ac a USING (id)
UNION ALL
SELECT 'tabicl-v2', avg((a.target = p.yhat)::INT)
FROM tabfm_classify('tr','target', test := 'te', model := 'tabicl-v2') p
JOIN ac a USING (id);

Two models, one query, no training. If the spread is under a few percent — as it was on two of our three classification sets — stop optimising the model and go improve the data.

---

In this series:

1. Zero-Shot Machine Learning in SQL — in-context learning, the two shapes, real numbers
2. Which Tabular Foundation Model Should You Use? — the bake-off (this post)
3. Foundation Models in Production — cost, memory, and air-gapped deployment

---

Reference

📖 anofox-tabfm on GitHub → — the SQL API, and the runnable benchmark scripts behind every number here.