Foundation models for tabular data: where they beat boosted trees
Pre-trained models such as TabPFN, TabICL and Mitra now rival tuned XGBoost on small tables. How they differ, where they fall short and who can use them.
Most business data lives in tables: loan applications, card transactions, patient records, sales leads. For about a decade, the best way to predict from such tables has been gradient-boosted trees, with tools such as XGBoost, LightGBM and CatBoost. A new group of "tabular foundation models" now claims to match or beat them on small and mid-sized tables, with almost no tuning. This deep dive explains how they work, puts the current numbers in one place, compares the main models, and is clear about what they cannot yet do and what is due next.
Current situation
The field is young, so it is worth fixing the numbers first. Each figure below says who measured it, because that matters a great deal here.
The research base. A paper in Nature on 8 January 2025 (volume 637, by Noah Hollmann, Frank Hutter and colleagues) introduced TabPFN v2 and started the current wave. It tested 29 classification and 28 regression datasets, each with up to 10,000 rows, and repeated every run ten times on different splits. In that test, default TabPFN took about 2.8 seconds per classification task and beat tree models tuned for up to four hours. On the paper's normalised accuracy score, default TabPFN reached about 0.94 against about 0.75 for default CatBoost. Switch between the paper's three headline comparisons below.
The independent benchmark. TabArena, a "living" benchmark that is updated as models change, is the most-cited yardstick. Its first version used 51 datasets picked from 1,053 candidates drawn from 14 earlier benchmarks, and compared 16 models. Datasets had between 500 and 250,000 training rows. Running it took roughly 15 years of computer time, about 25 million model-training runs. Its authors conclude that tuned and ensembled tree models stay strong, that deep learning has "caught up" with enough time and ensembling, and that foundation models do best on small datasets. TabPFN v2 led on the datasets it could handle even without tuning. Note that TabArena's authors include researchers from the TabPFN team, so it is not fully independent of the model it ranks first.
Here is the main evidence side by side.
Study · What it tested · Main finding · CaveatNature, January 2025 · TabPFN v2 on 57 datasets of up to 10,000 rows · Beat tree models tuned for up to 4 hours, in 2.8 s per classification task · Authors include Prior Labs staff
TabArena, 2025 · 16 models on 51 datasets (500 to 250,000 rows) · Trees strong, deep learning caught up with ensembling, foundation models excel on small data · Authors include TabPFN team members, and only three foundation models were in it
Credit risk, May 2026 (revised October) · 33 classification methods on 14 default datasets, 27 regression methods on 7 loss datasets · TabPFN-3 and TabICLv2 had the strongest average results, most on small and imbalanced data · Preprint, credit risk only
BeyondArena, June 2026 · 11 models on 142 datasets, including shifting, large and wide data · Foundation models excel on small and medium data from one distribution, trees and deep learning dominate the rest · Authors include TabPFN team members
Distribution shift, July 2026 · 9 foundation models on 3 shifting datasets · All nine got worse, by 0.003 to 0.060 · Preprint, metric not named in the abstract
SAP researchers, June 2026 · TabPFN, TabICL and ConTextTab on enterprise and public data · Enterprise data differs markedly, so rankings may not carry over · The abstract gives no numbers
TabArena ranks models with Elo, so it helps to know what a gap in points means. Try the slider.
How fast the size limit is moving.
TabPFN v2 (January 2025): about 10,000 rows, 500 columns, 10 classes.
TabPFN-2.5 (paper dated November 2025): the paper's abstract says 50,000 rows and 2,000 columns, while the maker's current documentation lists 100,000 rows for the 2.5 and 2.6 (April 2026) versions. Its paper reports a 100% win rate against default XGBoost on classification tasks of up to 10,000 rows and 500 columns, and 87% on larger ones.
TabPFN-3 (technical report submitted to arXiv on 13 May 2026, 41 authors): scales to 1 million training rows and is described as up to 20 times faster than TabPFN-2.5. The report says it beats gradient-boosted trees tuned for eight hours on datasets of up to 1 million rows and 200 columns, and that explaining its predictions is up to 120 times faster.
TabPFN-3.5 (technical report submitted on 15 September 2026, 47 authors): now the default model in the project's code, listed at up to 1 million rows and 20,000 columns, with a recommended ceiling of 6,000 columns. A "Fast" variant is up to three times faster than TabPFN-3. A "Thinking" variant, which spends extra computing at prediction time, is up to 12 times faster than the earlier Thinking variant and is limited to 200,000 rows. Both newest variants are the makers' claims.
Version · Date · Rows stated · Columns stated · Weights licenceTabPFN v2 · January 2025 · 10,000 · 500 · Apache 2.0 with attribution
TabPFN-2.5 · November 2025 · 100,000 (the paper says 50,000) · 2,000 · Non-commercial
TabPFN-2.6 · April 2026 · 100,000 · 2,000 · Non-commercial
TabPFN-3 · May 2026 · 1,000,000 · 2,000 · Non-commercial
TabPFN-3.5 · September 2026 · 1,000,000 · 20,000 (6,000 advised) · Non-commercial
Source: Prior Labs documentation and technical reports. Enter your own table size below to see which versions state they cover it.
The people and money. Prior Labs, the company behind TabPFN, was founded in 2024 and raised €9 million in February 2025. On 17 July 2026, SAP announced it had completed the acquisition of Prior Labs, after an earlier agreement it described in a May 2026 press release. SAP says Prior Labs will keep operating as an independent entity, with SAP committing more than €1 billion over four years. The price was not disclosed.
The newer challengers. TabICL v2 (Inria and partners, accepted at ICML 2026) says it beats the tuned, fine-tuned RealTabPFN-2.5 on two benchmarks without any tuning of its own. The project says a table of 50,000 rows and 100 columns takes under ten seconds on an H100 graphics processor, about ten times faster than TabPFN-2.5, and that it was pre-trained on 300 to 48,000 rows and works well up to about 100,000. Amazon's Mitra shipped inside AutoGluon 1.4. TabDPT's version 1.3 (8 September 2026, Apache-2.0) has a "Turbo" release the project says is about 120 times faster on average on TabArena than its earlier version. A 2026 healthcare study counts 45 distinct tabular foundation models.
How the benchmark scores. TabArena uses an Elo scale, like chess rankings, in which a default random forest scores 1,000 and a gap of 400 points means a 10-to-1 expected win rate. Its authors found that ensembling several tuned versions is needed to see a model's best: the top three single models (TabM, LightGBM and RealMLP) would all rank below CatBoost without it. A cross-model ensemble beat every single model and AutoGluon. Only TabPFNv2, TabICL and TabDPT were foundation models in the original study. The makers' newer Elo claims for TabPFN-3 are left out here, because they are not independently replicated.
Licences, in one place. TabPFN's code is Apache-2.0, but the weights for 2.5, 2.6, 3 and 3.5 are non-commercial. The maker's documentation allows "preliminary internal assessments" on your own data, and requires a commercial licence or its paid API for production, client projects, business decisions or procurement benchmarks.
Most of the favourable numbers above come from the teams that built the models. Independent tests are fewer and less flattering, which is why this article carries a Medium confidence rating.
What is this research about?
A foundation model is trained once on a very broad range of data, then reused. For text, the data is the internet. For tables there is no equivalent pile of clean public tables, so several teams train on synthetic tables: millions of artificial datasets produced from random cause-and-effect rules, with noise, missing values and odd column types thrown in. The Nature paper describes roughly 100 million such datasets and about two weeks of training on eight consumer-grade graphics cards.
At use time, you hand the model your labelled rows and the rows you want predicted, and it answers in one pass. It does not retrain on your data in the usual sense. This is called in-context learning. A good analogy is a quick study who has practised on a million mock exams and is now shown your worked examples, then asked to answer the next question in the same style.
This is different from how a tree model works. A tree model fits itself to your table over many rounds of training and tuning. A foundation model skips that and spends its effort at prediction time instead, because it must read your whole training set again for each batch of predictions.
The main models compared
The models differ in what they learned from, how big a table they handle, and what you are allowed to do with them. Here is the comparison at a glance, followed by notes on each.
Model · Maker · Trained on · Stated size · Weights licenceTabPFN 2.5 to 3.5 · Prior Labs · Synthetic tables, with real data added in the 2.5 line · Up to 1 million rows (versions 3 and 3.5) · Non-commercial, paid licence or API for production
TabICLv2 · Inria and partners · Synthetic tables · Good to about 100,000 rows, about 500,000 with offloading, million-scale claimed · BSD-3-Clause
TabDPT v1.3 · Layer6 · Real tables · No fixed limit stated, because it retrieves similar rows · Apache-2.0
Mitra · Amazon · Mixed synthetic generators · Not stated in the abstract · Apache-2.0
TabFM · Google Research · Synthetic tables · Tested on 700 to 150,000 samples · Non-commercial default weights
SAP-RPT-1-OSS (ConTextTab) · SAP · Real-world tables · Not checked · Not checked
TabPFN (Prior Labs and University of Freiburg). The model in Nature. Trained on synthetic tables only for v2, with the 2.5 line adding continued training on real datasets ("RealTabPFN"). Size has grown from 10,000 rows to a stated 1 million (versions 3 and 3.5). Code is Apache 2.0, but the weights are not free for work use: v2 uses a licence that adds an attribution requirement, while 2.5, 2.6, 3 and 3.5 are non-commercial, and commercial production use needs the paid Enterprise Edition. An API-only variant, TabPFN-3-Plus "Thinking", spends extra computing at prediction time.
TabICL (Inria and partners). Also trained on synthetic tables, but designed for speed on larger data. Version 1 was reported as 3 to 10 times faster than TabPFN v2 on large tables. Version 2 says it handles million-row tables within 50 GB of graphics memory, and its code for inference, pre-training and data generation is released. The project page calls its code permissive without naming the licence, and says performance may drop beyond about 100 columns. Both the code and the weights on Hugging Face are under the BSD-3-Clause licence.
TabDPT (Layer6, Canada). Trained on real tables rather than synthetic ones, and it retrieves similar rows at prediction time. Its authors argue that real data carries signal that synthetic data misses, and report that adding data and model size improves results in a steady, predictable way. Code and weights are public under the Apache-2.0 licence, and the project says retrieval removes any fixed limit on dataset size.
ConTextTab, now SAP-RPT-1-OSS (SAP). Reads the words in column names and cells, which most of the others ignore, and was pre-trained on real-world tables. The Hugging Face page says the model was renamed with the checkpoint unchanged.
Mitra (Amazon). Trained on a mixture of synthetic data generators, including cause-and-effect models, gradient-boosting, random-forest and decision-tree generators. Published at NeurIPS 2025, it ships inside AutoGluon, with classifier and regressor weights on Hugging Face. Its paper says it beats TabPFN v2 and TabICL, and the weights are under Apache-2.0 on Hugging Face.
TabFM (Google Research, 30 June 2026). Trained entirely on hundreds of millions of synthetic datasets, in a hybrid design that borrows TabPFN's alternating row and column attention and TabICL's compressed-row approach. Google tested it on TabArena's 38 classification and 13 regression datasets, ranging from 700 to 150,000 samples, and reports its own results. It is on Hugging Face and GitHub and in Google Cloud BigQuery. The default pre-trained weights carry a separate non-commercial licence.
What actually separates them
1. Synthetic or real training data. The synthetic camp (TabPFN v2, TabICL, Mitra) avoids any risk of having seen the benchmark tables. The real-data camp (TabDPT, RealTabPFN) says real tables teach patterns generators cannot invent. Neither side has settled this with an independent, head-to-head test.
2. Size and speed. TabICL's pitch is speed on large tables. TabPFN's recent releases chase size. The fastest claims are the least checked.
3. Licence. For a business this is often the deciding row. The open options are the ones whose licence files you have read yourself.
4. Open or sold. The strongest TabPFN variants are API-only, so you cannot inspect or run them inside your own systems.
No source I found names a single winner. Each paper compares itself favourably with the others: Mitra says it beats TabICL, and TabICL v2 says it beats TabPFN-2.5. That is how research normally moves, but it means the ranking of one model against another is not settled.
How will this impact you?
Where your table sits decides most of the answer. Set the size, the speed you need and your licence position below to see which way the evidence leans, and why.
Small datasets are where it matters most. A lender testing a new product with a few thousand borrowers, a hospital with a modest patient file, or a startup without a data scientist can get a strong first model in minutes instead of weeks of tuning. The Nature paper reports that, with half the training rows, TabPFN still matched the next-best method that used all of them.
Check the licence before you start. For TabPFN-2.5 and later, Prior Labs' documentation treats even an internal evaluation that feeds a business decision as commercial use. Open options exist, but read each licence file.
Check where the data goes. A hosted service means sending customer rows to someone else. Running open weights in your own environment avoids that, at the cost of owning graphics hardware.
Setup speed is not scoring speed. A tree model scores a row in a fraction of a millisecond. The 2025 paper reports about 0.2 seconds per prediction (0.6 seconds without a graphics card) on a table of 10,000 rows and 10 columns, against 0.0002 seconds for default CatBoost. That is fine for a nightly score and a poor fit for scoring every payment as it happens.
Keep a tree model as your baseline. Run the foundation model next to a tuned tree model on your own data, and decide on your own numbers. A June 2026 paper by SAP researchers finds that enterprise tables "markedly differ" from public benchmarks, so a model that ranks well publicly may do poorly on your data, and the reverse can also happen. The abstract gives no numbers, but the warning is clear.
A practical test plan. Pick one table you already model. Run a tuned tree model, an ensemble such as AutoGluon, and one open foundation model on the same splits. Compare accuracy, calibration, scoring time and total cost.
Mind the data-protection clock. If you would send customer rows to a hosted model, India's Digital Personal Data Protection Rules matter. The government's press release says they were notified on 14 November 2025 with an eighteen-month phased compliance period, which law-firm summaries put at mid-May 2027 for most duties on organisations that handle personal data. Check the notification for the exact dates.
Current limitations
These are limits that exist today, not open research questions.
Size and hardware.
The project's own documentation says a graphics card is recommended. Running on an ordinary processor is only practical for moderate data, and the limit is stated as 5,000 samples for versions 3 and 3.5 (1,000 for older versions). Some large datasets need about 16 GB of graphics memory.
Each prediction call re-reads the whole training set, so batching matters. The documentation advises splitting test sets above 1,000 rows into chunks.
The tested range is the claimed range. The earlier versions' own paper expected tree models or tuned ensembles to be stronger on larger or less smooth data. The newer million-row claims come from the vendor's own technical report, not from outside tests.
Not built for real time. The 2025 paper says the method is slower than optimised tree models at prediction time and is not designed for real-time use.
Access and terms. First use of TabPFN-2.5 and later needs an account login and licence acceptance, and commercial production use requires a separate paid edition. The strongest variant is API-only.
Data that changes. A July 2026 preprint tested nine of these models on three real datasets from the TableShift benchmark, where the data changes between training and use (HELOC credit data, voting data and childhood-lead data). All nine got worse, by 0.003 to 0.060 on the paper's measure, and the best ones needed more memory and compute than ordinary servers provide. A separate benchmark, BeyondArena (submitted 29 June 2026, 11 models, 142 datasets, authors including TabPFN team members), found that existing tabular foundation models excel on tiny to medium-sized data drawn from one distribution, while tree-based and deep learning models still dominate on data that shifts, large data and wide data. The TabPFN-3.5 report claims the new version extends to shifting data, and that claim is not independently tested.
Enterprise tables are not benchmark tables. The SAP researchers' June 2026 paper compares enterprise data with public benchmarks for TabPFN, TabICL and ConTextTab. It concludes enterprise data differs markedly, that a model strong on benchmarks may be weak on enterprise data, and that more enterprise-style benchmarks are needed. The abstract reports no rank correlations or accuracy figures, so the size of the effect is unknown.
Wide tables. The TabICL project says it was pre-trained on tables with 2 to 100 columns, benchmarked up to 2,000, and that accuracy may fall beyond about 100. Handling about 500,000 rows needs offloading to processor memory and disk, with possible loss of accuracy. Prior Labs lists much higher column limits for TabPFN-3.5, and these are its own claims.
Finance evidence leans favourable, with caveats. The strongest lending evidence is an academic benchmark by Bart Baesens, Stefan Lessmann, Cristián Bravo, David Martens and colleagues (submitted May 2026, revised 23 October 2026). It compared 33 classification methods on 14 probability-of-default datasets and 27 regression methods on 7 loss-given-default datasets. TabPFN-3 and TabICLv2 achieved the strongest average results, with the largest gains on small samples and strongly imbalanced data, while used with fixed pre-trained settings and no tuning. That is a preprint, and the benchmark covers credit risk only. Prior Labs' own case page with the lender-software firm Taktile reports bigger gains as data gets sparser on an account-opening fraud dataset, but the vendor ran that test. I found no independent study of real-time fraud scoring.
Benchmark weaknesses the authors admit. The TabArena authors say leaderboards can be compromised by tuning on the benchmark datasets or pre-training on them, that no feature engineering was applied, and that the per-model time limit can make results depend on hardware. They also found that the holdout method of validation underestimates every model compared with cross-validation.
Limitations and open questions
Independence. Two authors of the Nature paper work at Prior Labs, which sells these models, and some are named on patent applications filed by Bosch. Several newer claims are preprints or vendor pages, and have not been replicated.
Does a public ranking predict your result? No independent study I found measures this with numbers.
Are synthetic or real tables better? The two camps argue opposite cases. A clean test would pre-train the same architecture both ways.
What does the SAP deal change? The report of the acquisition says SAP will run Prior Labs as an independent lab, but it does not say what happens to open weights or licence terms.
Is the benchmark contaminated? If benchmark tables leak into pre-training, scores look better than real performance. The TabArena authors list this risk. Models trained only on generated data have less exposure, but the generators were designed by people who know the benchmarks.
Can the predictions be explained? The TabPFN-3 report claims explanations (SHAP values, a standard way to show which inputs drove a decision) up to 120 times faster. Lenders and hospitals will need explanations, and I found no independent study of how well these hold up.
Which clinical or financial setting favours which model? A 2026 healthcare framework reports that rankings change with the clinical context and with what matters most, such as fairness or data efficiency. That is a reason to evaluate on your own use case.
What is coming next
Forecasts below are labelled as such, and none is a promise.
Already dated. TabPFN-3 was reported on arXiv on 13 May 2026 and TabPFN-3.5 on 15 September 2026, with a "Fast" variant still labelled alpha in the documentation. TabICL v2 is an ICML 2026 paper, with its latest revision on 16 September 2026. TabDPT version 1.3 appeared on 8 September 2026.
Mid-May 2027. The eighteen-month phase-in of India's Digital Personal Data Protection Rules ends, per the government's release and law-firm summaries. This affects anyone sending customer tables to a hosted model.
Benchmark updates. TabArena is meant to be updated as models change. A vendor overview cites version 0.1.8.3 from September 2026. Expect new entries and re-ranking.
Money behind the field. SAP's stated commitment of more than €1 billion over four years (a plan, not a result) points to heavier investment and, plausibly, tabular models appearing inside business software.
Forecast, not fact: based on the pace of the last 18 months, size limits and speed claims will keep rising, and independent tests on shifting and real-world business data are the main thing still missing. Whether open and non-commercial licences will converge is unknown.
Key takeaways
Tabular foundation models can match or beat tuned tree models on small tables with no tuning, according to a peer-reviewed Nature paper and an independent-style benchmark that agrees on the small-data point.
The main models differ in training data (synthetic or real), size limits, speed and licence, and no independent test names a single best one.
Today's limits are real: graphics hardware, slower scoring, non-commercial licences and weaker results on data that changes.
On large tables, shifting data or real-time scoring, tree models and tuned ensembles remain a safe default.
Test any of them on your own data against a tuned tree model before you commit.
Evidence assessment
The rating is Medium. The core claim, that foundation models beat tuned trees on small tables, rests on a peer-reviewed paper in a top journal and a large benchmark that agrees on the small-data point. It is held back by developer involvement in the main benchmarks, by many newer claims that are preprints or vendor pages, and by weaker results on shifting data. An independent credit-risk benchmark is favourable, but it is a preprint and covers one domain. The speed and rank claims for the newest versions are the weakest part, and this article does not repeat their leaderboard scores for that reason.
Suggested follow-up
Read the Nature paper and the TabPFN-3 technical report for the test setup and size limits.
Watch the TabArena leaderboard and any benchmark that uses data that changes over time.
Run an open model on one of your own tables next to XGBoost or CatBoost, and keep the comparison.
Look for independent tests on financial data from banks and lenders.
Read each licence file before using any model at work.
Sources
- [Hollmann et al.: Accurate predictions on small data with a tabular foundation model, Nature 2025](https://pmc.ncbi.nlm.nih.gov/articles/PMC11711098/)
- [Prior Labs: TabPFN-2.5, arXiv](https://arxiv.org/abs/2511.08667)
- [TabPFN-3: Technical Report, arXiv, May 2026](https://arxiv.org/abs/2605.13986)
- [TabPFN-3.5: Technical Report, arXiv, September 2026](https://arxiv.org/abs/2609.17895)
- [Prior Labs documentation: model versions, limits and licences](https://docs.priorlabs.ai/models)
- [PriorLabs/TabPFN on GitHub: licences, limits and usage notes](https://github.com/PriorLabs/TabPFN)
- [TabICL repository on GitHub](https://github.com/soda-inria/tabicl)
- [TabDPT inference repository on GitHub](https://github.com/layer6ai-labs/TabDPT-inference)
- [Zhang et al.: Mitra, mixed synthetic priors for tabular foundation models, arXiv](https://arxiv.org/abs/2510.21204)
- [Kim et al.: Exploring differences between tabular enterprise data and public benchmarks, arXiv](https://arxiv.org/abs/2606.30452)
- [AZB & Partners: India's DPDP phased rollout and compliance milestones](https://www.azbpartners.com/bank/indias-digital-personal-data-protection-act-phased-rollout-and-key-compliance-milestones/)
- [Erickson et al.: TabArena, a living benchmark for tabular data](https://arxiv.org/html/2506.16791)
- [Qu et al.: TabICLv2, arXiv](https://arxiv.org/abs/2602.11139)
- [TabICL: a tabular foundation model for in-context learning on large data](https://arxiv.org/pdf/2502.05564)
- [Ma et al.: TabDPT, arXiv](https://arxiv.org/abs/2410.18164)
- [Amazon Science: Mitra, mixed synthetic priors for tabular foundation models](https://www.amazon.science/blog/mitra-mixed-synthetic-priors-for-enhancing-tabular-foundation-models)
- [Loza et al.: out-of-distribution performance of tabular foundation models, arXiv](https://arxiv.org/abs/2607.26000)
- [A comparative framework for evaluating foundation models on tabular data: healthcare case study, arXiv](https://arxiv.org/abs/2609.22154)
- [SAP News Center: SAP completes acquisition of Prior Labs, 17 July 2026](https://news.sap.com/2026/07/sap-completes-prior-labs-acquisition/)
- [Google Research: Introducing TabFM, 30 June 2026](https://research.google/blog/introducing-tabfm-a-zero-shot-foundation-model-for-tabular-data/)
- [Baesens et al.: Tabular foundation models for credit risk prediction, arXiv](https://arxiv.org/abs/2605.18147)
- [TabICL weights on Hugging Face (BSD-3-Clause)](https://huggingface.co/jingang/TabICL)
- [Mitra classifier on Hugging Face (Apache-2.0)](https://huggingface.co/autogluon/mitra-classifier)
- [Press Information Bureau: DPDP Rules, 2025 notified, 17 November 2025](https://pib.gov.in/PressReleasePage.aspx?PRID=2190655)
- [Qlik: the shift to table-native foundation models](https://www.qlik.com/blog/the-shift-to-table-native-foundation-models)
Tags: Tabular Data Foundation Models TabPFN Machine Learning Gradient Boosting