Which Tabular Foundation Model Should You Use? Three Models, Five Datasets, Measured
· 14 min read
In the last post we said switching foundation models is one keyword. So we switched it fifteen times and wrote down what happened.
The short version: for classification, model choice barely moved the needle. For regression, it moved it by 30%. The model that won most often is the one you're least likely to be allowed to use. And when we ran scikit-learn on the same splits, the foundation models won all five — by 2–7%, for up to 550× the compute.