A Brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning Without Human Intervention

作者: Lukasz Romaszko , Alexander R. Statnikov , Núria Macià , Hugo Jair Escalante , Evelyne Viegas

DOI:

关键词: SoftwareIntervention (counseling)Scale (social sciences)Machine learningTest (assessment)Artificial intelligenceVariety (cybernetics)Feature (machine learning)Computer science

摘要: The ChaLearn AutoML Challenge team conducted a large scale evaluation of fully auto- matic, black-box learning machines for feature-based classi cation and regression problems. test bed was composed 30 data sets from wide variety application domains ranged across di erent types complexity. Over six rounds, participants succeeded in delivering software capable being trained tested without human intervention. Although improvements can still be made to close the gap between human-tweaked models, this competition contributes development automated environments by challenging practitioners solve problems under speci c constraints sharing their approaches; platform will remain available post-challenge submissions at http://codalab.org/AutoML.

参考文章(10)
Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown, Sequential model-based optimization for general algorithm configuration learning and intelligent optimization. pp. 507- 523 ,(2011) , 10.1007/978-3-642-25566-3_40
Artus Krohn-Grimberghe, Abhishek Thakur, AutoCompete: A Framework for Machine Learning Competition arXiv: Machine Learning. ,(2015)
Kevin Swersky, Jasper Snoek, Ryan Prescott Adams, None, Freeze-Thaw Bayesian Optimization arXiv: Machine Learning. ,(2014)
Isabelle Guyon, Kristin Bennett, Gavin Cawley, Hugo Jair Escalante, Sergio Escalera, Tin Kam Ho, Nuria Macia, Bisakha Ray, Mehreen Saeed, Alexander Statnikov, Evelyne Viegas, Design of the 2015 ChaLearn AutoML challenge international joint conference on neural network. pp. 1- 8 ,(2015) , 10.1109/IJCNN.2015.7280767
Chris Thornton, Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown, Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms knowledge discovery and data mining. pp. 847- 855 ,(2013) , 10.1145/2487575.2487629
Brent Komer, James Bergstra, Chris Eliasmith, Hyperopt-Sklearn: Automatic Hyperparameter Configuration for Scikit-Learn Proceedings of the 13th Python in Science Conference. pp. 32- 37 ,(2014) , 10.25080/MAJORA-14BD3278-006
Daniel Yamins, David Cox, James Bergstra, Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures international conference on machine learning. pp. 115- 123 ,(2013)
Kari Torkkola, Eugene Tuv, George Runger, Alexander Borisov, Feature Selection with Ensembles, Artificial Variables, and Redundancy Elimination Journal of Machine Learning Research. ,vol. 10, pp. 1341- 1366 ,(2009)
Jost Tobias Springenberg, Manuel Blum, Frank Hutter, Katharina Eggensperger, Aaron Klein, Matthias Feurer, Efficient and robust automated machine learning neural information processing systems. ,vol. 28, pp. 2755- 2763 ,(2015)
Jost Tobias Springenberg, Frank Hutter, Matthias Feurer, Initializing bayesian hyperparameter optimization via meta-learning national conference on artificial intelligence. pp. 1128- 1135 ,(2015)