Welcome to AutoML Decathlon!


The AutoML Decathlon is a competition that will evaluate the performance of participants' AutoML methods on carefully curated sets of tasks with an appropriate level of difficulty and coverage in application domain, size, and input/output characteristics.

At the start of the competition we will release 10 public development tasks that vary in their domain (including image, finance timeseries, audio, and natural sciences), problem type (including regression, single-label, and multi-label classification), and scale (ranging from several thousands to hundreds of thousands of observations). These public tasks will be representative of (but distinct from) the final set of test tasks on which participants' AutoML methods will be ultimately evaluated on at the conclusion of the competition. For more background, see our blog post.

A prize of $15,000 will be awarded to the winning team. We will also offer smaller cash prizes for best performance on certain subsets of tasks, to be announced soon! See the competition description for details.

Current Leaderboard

Currently the submission by competitor 'euxhenh' has the best overall performance. We will continue to update this leaderboard as more baselines are added and more participant methods are received. Go to the CodaLab page to learn details, make your submission, and beat the current best methods!

A description of the AUP (Area Under Performance-profile curve) metric used to rank the methods.

Announcements

  • The competition submission deadline has been moved to Nov 10, 2022.
  • As of Sept 01, 2022, the set of development tasks has been updated. Please redownload it for the latest version.
  • The official start of the competition has been moved from Jul 13, 2022 to Jul 15, 2022.
  • We have created a new CodaLab competition page, in order to facilitate certain design/leaderboard elements. The other links on this website have been updated to reflect this as well. Please register for this new competition. We will also be reaching out to participants of the old competition to notify them. Thank you for your understanding!

Important Dates

  • Jul 15, 2022: The official start of AutoML Decathlon; the starter kit will be made available and the submission portal will be opened to participants
  • Nov 10, 2022 (11:59 pm Anywhere on Earth): The submission portal will close and we will begin evaluating final submissions
  • Dec 07, 2022 (tentative): The final results will be announced

Getting Started

Please read through the competition details on this website. Once you are ready to begin, you can download the starter kit here. Feel free to email the organizers with any questions you may have. Good luck!