Yesterday, Rohit Prasad, vice president and head scientist, Alexa AI, announced the winner of the 2018 Alexa Prize, a competition for university students dedicated to advancing the field of conversational artificial intelligence (AI). Congratulations to Gunrock, an inspiring group of students from the University of California, Davis! Their socialbot earned an average score of 3.1 on a 5-point scale from our panel of independent judges and achieved an average conversation duration of 9 minutes and 59 seconds. As the winner, Gunrock earned a $500,000 first-place prize, which will be shared equally among the students.

We also had the opportunity to recognize our other finalist teams. The 2018 runner up was Alquist from Czech Technical University in Prague. They were presented with a $100,000 prize for their efforts. We also awarded the third-place team, Alana from Heriot-Watt University in Edinburgh, with a $50,000 prize.

At the start of the competition teams were chosen based on several criteria, including potential scientific contribution to the field, the technical merit of their approaches, the novelty of their ideas, and their ability to execute against their plan. All eight teams competed throughout the semifinals period. The 2018 socialbots improved in the second year of the challenge: during the semifinals, the average customer rating across all socialbots was 3.2 – a 10% improvement from 2017.

Our three finalists were then selected; two with the highest average ratings by Alexa customers, and one wildcard based on a combination of factors needed to drive the science forward. This year, finalists were announced live on Twitch. During the finals period, Alexa customers continued to interact with and provide feedback to the three finalists with over 1.7 million conversations in a span of 18 weeks. More than 10% of customers’ conversations with the socialbots lasted at least 10 minutes.

The finals were held at our Seattle headquarters over two days in early November and involved three Interactors who held conversations with the socialbots, as well as professionals across academia and industry who served as Judges.

Judging a conversational AI competition is difficult because conversation is inherently subjective; there isn’t a clear right or wrong response at each turn in a dialog, nor a precise definition of what makes a conversation “coherent” or “engaging”. To represent the diversity of thought and perspectives we engaged Judges with a variety of backgrounds and expertise to independently evaluate each socialbots ability.

Amazon is incredibly grateful to all the student teams who exercised our Alexa Skills Kit, Conversational Bot (CoBot) toolkit, and the AWS cloud to create socialbots for Alexa that continue to advance the science and the art required to converse coherently and engagingly with humans on popular topics and news events.

The Alexa Prize fosters an environment that encourages a diverse set of experimental approaches to building open domain conversational agents or socialbots. The participants this year have explored novel ideas in the areas of natural language understanding, dialog management, error correction, and personalization. These advances and many others are described in the 2018 Proceedings of the Alexa Prize that includes papers from all Alexa Prize teams.

Looking Ahead

The Grand Challenge, of maintaining a coherent and engaging conversation for 20 minutes, still remains! As does a $1 million unrestricted gift which will be awarded to the winning team’s university, if their socialbot meets this challenge. Could that team be yours?

I am excited to share that the Alexa Prize competition will continue in 2019! Based on feedback from universities across the globe the 2019 Alexa Prize calendar will open the application period in late winter/early spring 2019 with teams being announced in June 2019 for a fall start. This allows the competition to better align to the academic calendar. We look forward to seeing the number of entrants grow next year. I’d encourage teams from universities worldwide to visit alexaprize.com to learn more about our competition. More details will be posted in January 2019.

Teams have put forward great work this year to advance the state of the science and Rohit, myself, and everyone involved with Alexa Prize want to thank them for their partnership. We look forward to next year’s competition as we work together to create the future of conversational AI.