PR #51 - goldener-data/goldener-research

Bibliography for the AutoML theme requested in issue #4 ("Make bibliography on AutoML papers"), built with the sync-theme-issues workflow.

Theme search: AutoML (issue #4)

Queries (Semantic Scholar bulk search, sorted by citation count): AutoML, "automated machine learning", "hyperparameter optimization", "neural architecture search", "AutoML" survey, "hyperparameter optimization" subset. A first run with unquoted multi-word queries returned mostly off-topic papers, because each word was matched separately. Those queries were rerun as exact phrases.

Screening: 261 unique candidates. The top 60 by citations were title-screened against model_design's scope, which keeps AutoML work only when it studies the role of the data in the search: data subsets or proxies, transfer across similar datasets, validation and resampling splits. Fabolas (rank 65, 636 citations) was added just past the cutoff because it is the most data-centric paper in the ranking. 5 papers were confirmed by abstract. The pool ran out before reaching the cap of 20.

`model_design/BIBLIOGRAPHY.md` — new `### AutoML` subsection under "No embeddings"

Paper	Citations	Origin	Why relevant
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization	2,996	theme search (#4)	the resource allocated to configurations includes data samples, i.e. search on data subsets
Efficient and Robust Automated Machine Learning (auto-sklearn)	2,018	theme search (#4)	warm-starts the search from past performance on similar datasets
AutoML: A Survey of the State-of-the-Art	1,909	theme search (#4)	AutoML survey covering the data-preparation side of the pipeline
Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges	1,153	theme search (#4)	resampling-based performance evaluation for HPO
Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets (Fabolas)	636	theme search (#4)	explores configurations on small data subsets and extrapolates to the full dataset
Initializing Bayesian Hyperparameter Optimization via Meta-Learning	499	co-author expansion of auto-sklearn	starts from configurations that worked on similar datasets
Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning	431	co-author expansion of auto-sklearn	meta-learning across datasets plus budget allocation for large datasets
Resampling Methods for Meta-Model Validation with Recommendations for Evolutionary Computation	166	co-author expansion of the HPO survey (Bischl)	cross-validation, subsampling, bootstrapping and nested resampling for model selection
Practical Transfer Learning for Bayesian Optimization	54	co-author expansion of auto-sklearn	transfers HPO knowledge from previous datasets to a new one
Speeding Up Multi-Objective HPO by Task Similarity-Based Meta-Learning for the TPE	21	co-author expansion of auto-sklearn / Fabolas (Hutter)	meta-learning weighted by task similarity
MASIF: Meta-learned Algorithm Selection using Implicit Fidelity Information	10	co-author expansion of the HPO survey (Lindauer)	algorithm selection for a new dataset from the algorithms' learning behaviour
Reshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimization	8	co-author expansion of the HPO survey / auto-sklearn	reshuffling validation splits per configuration improves generalization

`data_selection/BIBLIOGRAPHY.md`

Paper	Section	Citations	Origin	Why relevant
Batch Active Learning at Scale	No embedding / Active learning	221	co-author expansion of Hyperband	uncertainty + diversity batch active learning at 100K-1M batch sizes
Combining MixMatch and Active Learning for Better Accuracy with Fewer Labels	No embedding / Active learning	39	co-author expansion of Hyperband	value of labeled vs unlabeled data along the learning curve
Is margin all you need? An extensive empirical study of active learning on tabular data	No embedding / Active learning	16	co-author expansion of Hyperband	benchmark showing margin sampling matches the alternatives
Two-Step Active Learning for Instance Segmentation with Uncertainty and Diversity Sampling	No embedding / Active learning	2	co-author expansion of Hyperband	uncertainty + diversity sampling for segmentation labeling
Leveraging Importance Weights in Subset Selection	No embedding / Smarter selection	6	co-author expansion of Hyperband	importance-sampling subset selection
GIST: Greedy Independent Set Thresholding for Max-Min Diversification with Submodular Utility	No embedding / Smarter selection	3	co-author expansion of Batch AL at Scale (Citovsky)	diversity-based subset selection for data sampling
Towards Localization via Data Embedding for TabPFN	Leverage embeddings / Smarter selection	13	co-author expansion of auto-sklearn	nearest-neighbour selection of the training context in a learned data embedding

Researchers

Counts include the shared seed paper and papers already in the repo.

model_design/RESEARCHERS.md

Frank Hutter (ELLIS Institute Tübingen): already listed in data_selection and losses, so no threshold applies. Adds auto-sklearn, Fabolas, Initializing BO via Meta-Learning, Auto-Sklearn 2.0, Practical Transfer Learning for BO, multi-objective TPE, MASIF.
Matthias Feurer (TU Dortmund University), 6 papers (6 found in this run, 0 in repo): auto-sklearn, Initializing BO via Meta-Learning, Auto-Sklearn 2.0, Practical Transfer Learning for BO, Reshuffling Resampling Splits (+ TabPFN localization in data_selection).
Marius Lindauer (Leibniz University Hannover), 3 papers (3 / 0): HPO survey, Auto-Sklearn 2.0, MASIF.
Katharina Eggensperger (TU Dortmund University), 3 papers (3 / 0): auto-sklearn, Auto-Sklearn 2.0 (+ TabPFN localization in data_selection).
Bernd Bischl (LMU Munich), 3 papers (3 / 0): HPO survey, Resampling Methods for Meta-Model Validation, Reshuffling Resampling Splits.
Afshin Rostamizadeh and Giulia DeSalvo (Google Research): Hyperband, plus their data_selection papers below.

data_selection/RESEARCHERS.md

Afshin Rostamizadeh (Google Research), 6 papers (5 / 1 in repo): Hyperband, Batch AL at Scale, MixMatch + AL, Is margin all you need?, IWeS, Analyzing Similarity Metrics for Data Selection (already in data_selection).
Giulia DeSalvo (Google Research), 5 papers (5 / 0): Hyperband, Batch AL at Scale, Two-Step AL, IWeS, GIST.
Gui Citovsky (Google), 4 papers (3 / 1 in repo): Batch AL at Scale, IWeS, GIST, Analyzing Similarity Metrics for Data Selection.
Matthias Feurer, Katharina Eggensperger: Towards Localization via Data Embedding for TabPFN.

Not added

Duplicates (already in the repo): LabelBench (frameworks/BIBLIOGRAPHY.md), Analyzing Similarity Metrics for Data Selection for Language Model Pretraining (data_selection/BIBLIOGRAPHY.md).

Highly cited AutoML papers rejected as not relevant. model_design excludes generic optimizers and architecture-search papers that don't study the data:

Optuna (12,051 citations): generic HPO framework.
EfficientNet (26,781), EfficientNetV2 (4,972), NAS with RL (6,077), MnasNet (3,536), ENAS (3,102), ProxylessNAS (2,159), PNAS (2,179) and other NAS papers: architecture search.
BOHB (1,426): generic HPO method; its abstract involves no data.
Automated Machine Learning: Methods, Systems, Challenges (book, 1,545): generic overview.
Hyperparameter Optimization (encyclopedia entry, 1,385): no abstract, generic.
Auto-WEKA (1,748), AutoGluon-Tabular (1,098), TabPFN (1,036), Meta-Learning in Neural Networks survey (2,833): no study of the data's role in the search.

Co-authors checked but under the 3-paper threshold:

Aaron Klein: 2 (auto-sklearn, Fabolas).
Jost Tobias Springenberg: 2.
Stefan Falkner: 2.
Kevin Jamieson: 2 (Hyperband, LabelBench already in repo).
Anne-Laure Boulesteix: 2 (HPO survey, cross-study validation).
Michel Lang: 2 (HPO survey, mlr3spatiotempcv).
Marc Becker: 2.
Lisha Li, Ameet Talwalkar, Xin He, Xiaowen Chu, Kaiyong Zhao, Manuel Blum, Martin Binder, Tobias Pielok, Jakob Richter, Stefan Coors, Janek Thomas, Theresa Ullmann, Difan Deng, Simon Bartels, Philipp Hennig: 1 each.

Deferred (the budget of 30 author checks was used up: 29 seed authors + Gui Citovsky). These come from the co-author-expansion papers and can be checked later with sync-researcher-issues:

Thomas Nagler, Lennart Schneider (Reshuffling); Mykhailo Koshil (TabPFN localization).
Benjamin Letham, Eytan Bakshy (Practical TL for BO).
Shuhei Watanabe, Noor Awad, Masaki Onishi (multi-objective TPE).
Tim Ruhkopf, Aditya Mohan, Alexander Tornede (MASIF).
Olaf Mersmann, Heike Trautmann, Claus Weihs (Resampling Methods).
Sanjiv Kumar, Srikumar Ramalingam, Claudio Gentile, Lazaros Karydas, Anand Rajagopalan (Batch AL at Scale / IWeS / GIST); Yunjuan Wang (IWeS).
Dara Bahri, Heinrich Jiang, Tal Schuster (margin study); Shuang Song, David Berthelot (MixMatch + AL).
Ke Yu and co-authors (Two-Step AL).
Matthew Fahrbach, Morteza Zadimoghaddam, Sara Ahmadian (GIST).

Notes for the reviewer

No new topic folder is suggested: every paper fits model_design or data_selection.
Placement is debatable for Batch Active Learning at Scale. It went under "No embedding / Active learning" because its abstract does not mention embeddings.

🤖 Generated with Claude Code

Summary by cubic

Adds the AutoML bibliography for issue #4, bringing in AutoML and hyperparameter optimization papers to model_design plus related active learning and subset selection papers to data_selection.

What changed

Adds a new ### AutoML subsection to model_design/BIBLIOGRAPHY.md with 12 papers on data subsets, transfer across datasets, and resampling splits.
Adds 7 active learning and subset selection papers to data_selection/BIBLIOGRAPHY.md.
Adds researcher profiles for 7 authors in model_design/RESEARCHERS.md and 5 in data_selection/RESEARCHERS.md.

Screening notes

Of 261 unique candidates, the top 60 by citations were title-screened; 5 were confirmed by abstract, and the pool ran out before the 20-paper cap.
Scope keeps only AutoML work that studies the role of the data in the search, so generic HPO and NAS frameworks (Optuna, EfficientNet, BOHB) were excluded as off-topic.

#51 Add AutoML bibliography - 2026-10-12