Out-of-the-Box Reproducibility: A Survey of Machine Learning Platforms
作者： Richard Isdahl , Odd Erik Gundersen
DOI: 10.1109/ESCIENCE.2019.00017
关键词:
摘要: Even machine learning experiments that are fully conducted on computers not necessarily reproducible. An increasing number of open source and commercial, closed platforms being developed help address this problem. However, there is no standard for assessing comparing which features required to support reproducibility. We propose a quantitative method alleviates Based the proposed we assess compare current state art how well they making empirical results Our show BEAT Floydhub have best reproducibility with Codalab Kaggle as close contenders. The most commonly used provided by big tech companies poor
sci-hub.se 参考文章(24)
Victoria C. Stodden, Trust your science? Open your data and code AMSTAT news: the membership magazine of the American Statistical Association. ,vol. 409, pp. 21- 22 ,(2011) , 10.7916/D8CJ8Q0P
Joaquin Vanschoren, Mikio L. Braun, Cheng Soon Ong, Open science in machine learning arXiv: Learning. ,(2014)
Jonathan B. Buckheit, David L. Donoho, WaveLab and Reproducible Research Wavelets and Statistics. pp. 55- 81 ,(1995) , 10.1007/978-1-4612-2544-7_5
Brian A Nosek, George Alter, George C Banks, Denny Borsboom, Sara D Bowman, Steven J Breckler, Stuart Buck, Christopher D Chambers, Gilbert Chin, Garret Christensen, M Contestabile, A Dafoe, Eric Eich, Jeremy Freese, R Glennerster, D Goroff, Donald P Green, B Hesse, M Humphreys, John Ishiyama, D Karlan, A Kraut, A Lupia, P Mabry, T Madon, N Malhotra, Evan Mayo-Wilson, M McNutt, Edward Miguel, E Levy Paluck, Uri Simonsohn, Courtney Soderberg, Barbara A Spellman, J Turitto, G VandenBos, Simine Vazire, Eric-Jan Wagenmakers, R Wilson, TSCIENTIFICSTANDARDS Yarkoni, None, Promoting an open research culture Science. ,vol. 348, pp. 1422- 1425 ,(2015) , 10.1126/SCIENCE.AAB2374
R. D. Peng, Reproducible Research in Computational Science Science. ,vol. 334, pp. 1226- 1227 ,(2011) , 10.1126/SCIENCE.1213847
Jon F. Claerbout, Martin Karrenbach, Electronic Documents Give Reproducible Research a New Meaning Seg Technical Program Expanded Abstracts. pp. 601- 604 ,(1992) , 10.1190/1.1822162
Geir Kjetil Sandve, Anton Nekrutenko, James Taylor, Eivind Hovig, None, Ten Simple Rules for Reproducible Computational Research PLoS Computational Biology. ,vol. 9, pp. e1003285- ,(2013) , 10.1371/JOURNAL.PCBI.1003285
Chris Drummond, Replicability is not Reproducibility:Nor is it Good Science ,(2009)
Song-You Hong, Myung-Seo Koo, Jihyeon Jang, Jung-Eun Esther Kim, Hoon Park, Min-Su Joh, Ji-Hoon Kang, Tae-Jin Oh, An Evaluation of the Software System Dependency of a Global Atmospheric Model Monthly Weather Review. ,vol. 141, pp. 4165- 4172 ,(2013) , 10.1175/MWR-D-12-00352.1
Joan Starr, Eleni Castro, Mercè Crosas, Michel Dumontier, Robert R. Downs, Ruth Duerr, Laurel L. Haak, Melissa Haendel, Ivan Herman, Simon Hodson, Joe Hourclé, John Ernest Kratz, Jennifer Lin, Lars Holm Nielsen, Amy Nurnberger, Stefan Proell, Andreas Rauber, Simone Sacchi, Arthur Smith, Mike Taylor, Tim Clark, Achieving human and machine accessibility of cited data in scholarly publications PeerJ. ,vol. 1, ,(2015) , 10.7717/PEERJ-CS.1