What Is the Role of Science in Alexa?

Science is critical to how Alexa is revolutionizing daily conveniences, from playing music and controlling your smart home, to getting information and much more just by using your voice. Our scientists and engineers develop foundational AI technologies for anyone to build intelligent conversational interfaces for any device, application, language or environment. We build machine learning algorithms, services and data-driven models for key components, such as wake word detection, automatic speech recognition, natural language understanding, contextual reasoning, dialog management, question answering, and text-to-speech, all of which contribute to the magic that is Alexa.

What Is Our Unique Approach to Science?

We believe in hiring and developing world-class talent in science and engineering, and building teams with multi-disciplinary skills with clear charters and goals. These multi-disciplinary teams employ our working backwards method to identify key long-term problems to solve on behalf of our customers and a staged approach to ensuring we make rapid progress towards our goals. The combination of world-class elastic computing resources available via AWS, large-scale heterogeneous data resources, and the team’s years of experience in building and deploying machine learning algorithms is key to innovation at scale.

What Is the Impact?

Our research is focused on delivering magical experiences for our customers through the ground-breaking Echo family of devices and third-party devices available everywhere. As a result, our conversational AI inventions are having a direct impact on the lives of millions of people. We also contribute to the advancement of conversational AI through engagements with the academic community via funded research and Grand Challenges such as the Alexa Prize. Moreover, we encourage the publication of research that will contribute to the future of AI.

Research published recently by Alexa conversational AI scientists includes:

• Multi-Dialect Acoustic Modeling Using Phone Mapping and Online i-Vectors, Interspeech 2019, Harish Arsikere, Ashtosh Sapru, Sri Garimella
• Scalable Multi Corpora Neural Language Models for ASR, Interspeech 2019, Anirudh Raju, Denis Filiminov, Gautam Tiwari, Ariya Rastrow
• Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations, Interspeech 2019, Karthik Gopalakrishnan, Behnam Hedayatnia, Qinlang Chen, Anna Gottardi, Sanjeev Kwatra, Anu Venkatesh, Raefer Gabriel, Dilek Hakkani-Tür
• Towards Achieving Robust Universal Neural Vocoding, Interspeech 2019, Jaime Lorenzo-Trueba, Thomas Drugman, Javier Latorre, Thomas Merritt, Bartosz Putrycz, Roberto Barra-Chicote, Alexis Moinet, Vatsal Aggarwal
• Acoustic Model Bootstrapping Using Semi-Supervised Learning, Interspeech 2019, Langzhou Chen, Volker Leutnant
• Two Tiered Distributed Training Algorithm for Acoustic Modeling, Interspeech 2019, Pranav Ladkat, Oleg Rybakov, Radhika Arava, Sree Hari Krishnan Parthasarathi, I-Fan Chen, Nikko Ström
• Sub-band Convolutional Neural Networks for Small-footprint Spoken Term Classification, Interspeech 2019, Chieh-Chi Kao, Ming Sun, Yixin Gao, Shiv Vitaladevuni, Chao Wang
• Model Compression on Acoustic Event Detection with Quantized Distillation, Interspeech 2019, Bowen Shi, Ming Sun, Chieh-Chi Kao, Viktor Rozgic, Spyros Matsoukas, Chao Wang
• Neural Named Entity Recognition from Subword Units, Interspeech 2019, Abdalghani Abujabal, Judith Gaspers
• Improving ASR Confidence Scores for Alexa using Acoustic and Hypothesis Embeddings, Interspeech 2019, Prakhar Swarup, Roland Maas, Sri Garimella, Sri Harish Mallidi, Björn Hoffmeister
• HyST: A Hybrid Approach for Flexible and Accurate Dialogue State Tracking, Interspeech 2019, Rahul Goel, Shachi Paul, Dilek Hakkani-Tür
• Towards Universal Dialogue Act Tagging for Task-Oriented Dialogues, Interspeech 2019, Shachi Paul, Rahul Goel, Dilek Hakkani-Tür
• Fine-Grained Robust Prosody Transfer for Single-Speaker Neural Text-to-Speech, Interspeech 2019, Viacheslav Klimkov, Srikanth Ronanki, Jonas Rohnke, Thomas Drugman
• A Study for Improving Device-Directed Speech Detection toward Frictionless Human-Machine Interaction, Interspeech 2019, Che-Wei Haung, Roland Maas, Sri Harish Mallidi, Björn Hoffmeister
• One-Versus-All Models for Asynchronous Training: An Empirical Analysis, Interspeech 2019, Rahul Gupta, Aman Alok, Shankar Ananthakrishnan
• Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversion, Interspeech 2019, Alex Sokolov, Tracy Rohlin, Ariya Rastrow
• Multimodal and Multi-view Models for Emotion Recognition, ACL 2019, Gustavo Aguilar, Viktor Rozgic, Weiran Wang, Chao Wang
• Multi-Task Networks With Universe, Group, and Task Feature Learning, ACL 2019, Shiva Pentyala, Mengwen Liu, Markus Dreyer
• Answering Complex Questions by Joining Multi-Document Evidence with Quasi Knowledge Graphs, SIGIR 2019, Xiaolu Lu, Soumajit Pramanik, Rishiraj Saha Roy, Abdalghani Abujabal, Yafang Wang, Gerhard Weikum
• Content-Based Weak Supervision for Ad-Hoc Re-Ranking, SIGIR 2019, Sean MacAvaney, Andrew Yates, Kai Hui, Ophir Frieder
• Learning Context-Dependent Label Permutations for Multi-Label Classification, ICML 2019, Jinseok Nam, Young-Bum Kim, Eneldo Loza Mencía, Sunghyun Park, Ruhi Sarikaya, Johannes Furnkranz
• Active Learning for New Domains in Natural Language Understanding, NAACL 2019, Stanislav Peshterliev, Jack Kearney, Imre Kiss
• ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters, NAACL 2019, Abdalghani Abujabal, Rishiraj Saha Roy, Mohamed Yahya, Gerhard Weikum
• Continuous Learning for Large-scale Personalized Domain Classification, NAACL 2019, Han Li, Jihwan Lee, Sidharth Mudgal, Ruhi Sarikaya, Young-Bum Kim
• Cross-Lingual Transfer Learning for Japanese Named Entity Recognition, NAACL 2019, Andrew Johnson, Penny Karanasou, Judith Gaspers, Dietrich Klakow
• Practical Semantic Parsing for Spoken Language Understanding, NAACL 2019, Marco Damonte, Rahul Goel, Tagyoung Chung
• Generating Token-Level Explanations for Natural Language Inference, NAACL 2019, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Arpit Mittal
• Learning When Not to Answer: A Ternary Reward Structure for Reinforcement Learning based Question Answering, NAACL 2019, Fréderic Godin, Anjishnu Kumar, Arpit Mittal
• Locale-agnostic Universal Domain Classification Model in Spoken Language Understanding, NAACL 2019, Jihwan Lee, Ruhi Sarikaya, Young-Bum Kim
• Neural Text Normalization with Subword Units, NAACL 2019, Courtney Mansfield, Ming Sun, Yuzong Liu, Ankur Gandhe, Björn Hoffmeister
• Scaling Multi-Domain Dialogue State Tracking via Query Reformulation, NAACL 2019, Pushpendre Rastogi, Arpit Gupta, Tongfei Chen, Lambert Mathias
• Simple Question Answering with Subgraph Ranking and Joint-Scoring, NAACL 2019, Wenbo Zhao, Tagyoung Chung, Anuj Goyal, Angeliki Metallinou
• Audio Watermarking over the Air with Modulated Self-Correlation, ICASSP 2019, Mohamed Mansour, Yuan-Yen Tai
• Cross-Lingual Transfer Learning for Spoken Language Understanding, ICASSP 2019, Judith Gaspers, Quynh Do
• Data Augmentation for Low Resource Sentiment Analysis Using Generative Adversarial Networks, ICASSP 2019, Rahul Gupta
• Deep Embeddings for Rare Audio Event Detection with Imbalanced Data, ICASSP 2019, Vipul Arora, Ming Sun, Chao Wang
• Effect of Data Reduction on Sequence-to-Sequence Neural TTS, ICASSP 2019, Javier Latorre, Jakub Lachowicz, Jaime Lorenzo-Trueba, Thomas Merritt, Thomas Drugman, Srikanth Ronanki, Klimkov Viacheslav
• End-to-End Anchored Speech Recognition, ICASSP 2019, Yiming Wang, Xing Fan, I-Fan Chen, Yuzong Liu, Tongfei Chen, Björn Hoffmeister
• Frequency Domain Multi-Channel Acoustic Modeling for Distant Speech Recognition, ICASSP 2019, Minhua Wu, Kenichi Kumatani, Shiva Sundaram, Nikko Ström, Björn Hoffmeister
• Hierarchical Residual-Pyramidal Model for Large Context Based Media Presence Detection, ICASSP 2019, Qingming Tang, Ming Sun, Chieh-Chi Kao, Viktor Rozgic, Chao Wang
• Improving Emotion Classification through Variational Inference of Latent Variables, ICASSP 2019, Srinivas Parthasarathy, Viktor Rozgic, Ming Sun, Chao Wang
• Improving Noise Robustness of Automatic Speech Recognition via Parallel Data and Teacher-Student Learning, ICASSP 2019, Ladislav Mosner, Minhua Wu, Anirudh Raju, Sree Hari Krishnan Parthasarathi, Kenichi Kumatani, Shiva Sundaram, Roland Maas, Björn Hoffmeister
• Lessons from Building Acoustic Models with a Million Hours of Speech, ICASSP 2019, Sree Hari Krishnan Parthasarathi, Nikko Ström
• Reconfigurable Multitask Audio Dynamics Processing Scheme, ICASSP 2019, Jun Yang, Amit S. Chhetri, Carlo Murgia, Philip Hilmes
• Semi-Supervised Acoustic Event Detection Based on Tri-Training, ICASSP 2019, Bowen Shi, Ming Sun, Chieh-Chi Kao, Viktor Rozgic, Spyros Matsoukas, Chao Wang
• Multi-Geometry Spatial Acoustic Modeling for Distant Speech Recognition, ICASSP 2019, Kenichi Kumatani, Minhua Wu, Shiva Sundaram, Nikko Ström, Björn Hoffmeister
• Online Embedding Compression for Text Classification Using Low Rank Matrix Factorization, AAAI 2019, Anish Acharya, Rahul Goel, Angeliki Metallinou, Inderjit Dhillon
• Unsupervised Transfer Learning for Spoken Language Understanding in Intelligent Agents, AAAI 2019, Aditya Siddhant, Anuj Goyal, Angeliki Metallinou
• Efficient Online Learning for Mapping Kernels on Linguistic Structures, AAAI 2019, Giovanni Da San Martino, Fabio Aiolli, Alessandro Sperduti, Alessandro Moschitti
• Transfer Learning Only Using the Source Model and the Target Data for Sequence Labeling Tasks, AAAI 2019, Lingzhen Chen, Alessandro Moschitti