Tushar Chugh is a graduate student at the Robotics Institute at Carnegie Mellon University (CMU). There he studies the latest in robotics, particularly how computer vision devices perceive the world around them.
One of his favorite projects was a robot named Andy. Besides having arms, Andy could discern colors and understand spatial arrangement. Andy could also respond to voice commands, like “pick up the red block and place it on top of the blue block.” Andy’s speech recognition, a CMU framework, was about to change.
When Amazon came to give some lectures at CMU, they had a raffle drawing. Chugh won the drawing and took home a new Amazon Echo as a prize. Over three days and nights without sleep, he completely integrated Andy and Alexa using the Alexa Skills Kit (ASK).
When he saw Hackster’s 2016 Internet of Voice challenge, he knew he had to enter. And in August 2016, Chugh’s Smart Cap won the prize for the Best Alexa Skills Kit with Raspberry Pi category.
According to Chugh, there are about 285 million visually-impaired people in the world. In 2012, he worked on a project to help the visually impaired navigate inside a building. His device, a belt with embedded sensing tiles, won a couple of prizes, including a Wall Street Journal Technology Innovation Award. It was ahead of its time, though, and it wasn’t yet practical to develop the technology into a commercial product.
A lot can change in four years, including Chugh’s discovery of Alexa. Besides dabbling with Alexa and Andy the robot, he has also worked with Microsoft Cognitive Services for image recognition. Chugh now saw a chance to bring a new and better “seeing device” to light.
“When I saw Alexa, I thought we can extend it and integrate [Alexa] as a separate component,” says Chugh. “I talked with a couple of organizations for the blind in India, and they agreed this kind of system would be very, very useful. That was my main motivation.”
Chugh says the hardware for the Smart Cap is basic. He used a Raspberry Pi (RPi), a battery pack, a camera and a cap on which to mount it. As for the software, it included:
The goal was straightforward. A visually-impaired user could ask Alexa what is in front of them. Alexa would vocalize the scene, allowing the person to navigate safely wherever he or she may be.
How do the pieces all fit together?
Chugh says there are two distinct parts.
First, the image capture and analysis:
Now comes the Alexa skill:
The home is rapidly evolving thanks to the proliferation of connected devices and advancements in voice recognition technology. Together, new smart home products and voice control services are giving customers greater control over their homes.
Amazon and Intel see a tremendous opportunity to bring the benefits of a personal voice experience to millions of new consumers and are collaborating to encourage developers and device manufacturers to extend natural voice interaction to more products via Amazon Alexa.
The collaboration will enable partners to build new devices with Alexa using an Intel-based smart speaker form factor reference design coming in Q1 2017, as well as make it easier to create skills that work with the Intel-based Smart Home Hub.
Intel is working with Amazon to deliver smart speaker form factor reference designs (FFRD) with Alexa that make it easier for device manufacturers to build products with high-performance, far-field voice interaction. The first FFRD will be available starting in Q1 2017 and will offer device makers:
The FFRD combines Intel’s platform technology advancements with Amazon’s ever-smarter Alexa Voice Service to accelerate innovation among device manufacturers and the developers building new skills for all Alexa-enabled products.[Read More]
We are excited to announce a new addition to the Alexa family—JAM Voice.
JAM Voice is a portable speaker system with Alexa that serves as a complete hub for music and information. It’s a Wi-Fi and Bluetooth-connected speaker that features touch-activated integration with the Alexa Voice Service (AVS), giving customers the ability to push a button and just ask Alexa to play music, check the weather, get the news, or even order a pizza.
Whether entertaining a group of friends or relaxing after work, the JAM Voice system can play music from one or many rooms in the house. You can pair multiple speakers when connected to Wi-Fi, streaming music from Amazon Music, iHeartRadio, and TuneIn just by asking Alexa. The Alexa integration also makes it easy for customers to access thousands of third-party skills, built using the Alexa Skills Kit, including smart home controls through Philips Hue, Belkin Wemo, SmartThings, Insteon, and Wink.
Buy JAM Voice on Amazon.com now.
Alexa is always getting smarter with new capabilities and services through machine learning. Your product also gains access to new capabilities with Alexa through API updates, feature launches, and custom skills. Learn how AVS can add rich voice-powered experiences to your connected products, and read how some of our partners below have integrated with Alexa already:
AVS is coming soon to the UK and Germany. Read the full announcement here.
Update December 7, 2016: Today we announced the US preview of our new Alexa Skills Kit (ASK) built-in library is available to developers. Learn more >
A year and a half ago, we released the Alexa Skills Kit, and we’ve seen developers are eager to build skills and learn to build voice experiences. Developers like yourself have published over 5,000 skills, up from just over 100 at the beginning of the year. These skills are available on millions of Alexa-enabled devices in the US, UK and Germany.
Today we announced that we will roll out a library of hundreds of new intents and slots as part of the Alexa Skills Kit in developer preview in the coming weeks (US only). These new slots and intents are the product of learnings over the last year for Alexa’s natural language understanding (NLU) that help Alexa better understand and reply to requests. With the new built-in library, we have combined those learnings with the most common requests we have seen from the developer community to offer hundreds of built-ins for use in your own skills. This is just a start and we will continue to increase the set of built-in functionality and improve their accuracy as we get feedback from all of you.
With built-in intents and slot types, you do not need to provide sample utterances to train Alexa to understand customers’ intents for your skill. We introduced the concept of built-ins earlier in the year beginning with 15 intents (such as Stop, Cancel, Help, Yes, No) and 10 slot types (such as Date, Number, City, etc.). As part of the Alexa Skills Kit we now are introducing a new built-in library that provides hundreds of built-in intents and slots – for developers as part of the Alexa Skills Kit. The syntax for these built-ins are designed to make integration of these capabilities super simple in your custom skills.
For example, let’s imagine a custom skill that allows someone to ask for the temperature in a location for the next three days. If we wanted to build this skill previously, we would have to create an interaction model that included a combination of built-in and custom intents for handling how someone would ask the question. Most likely this would include built-in slot types for city and state, a built-in slot type for the number of days, and then a lot of sample utterances to ensure Alexa was accurately understanding the question each time. We also would need to do server side type validation to ensure we were being passed the specific type of data we were looking for.
With the new built-in intents library, weather becomes an object that Alexa knows a lot about, both weather itself and its attributes, but also how a person may ask for the weather. Our interaction model now can be done with no sample utterances and a single intent! We call this new type of interaction an Intent Signature and it includes actions, entities and properties. There are numerous Intent Signatures available for use in your Alexa skills across all sorts of categories.
Stay tuned to learn more about built-in library. For more information about getting started with the Alexa Skills Kit, check out the following:
For over a year, the Alexa Fund has been investing in promising startups that are fueling speech technology advances and voice application innovation. Today, we are excited to announce the Alexa Accelerator, powered by Techstars. The accelerator is the first of its kind at Amazon and another initiative from the Alexa Fund to champion builders, developers, and entrepreneurs innovating with voice technology.
The Alexa Accelerator, powered by Techstars, will focus on Alexa domains, including, but not limited to, such areas as connected home, wearables and hearables, enterprise, communications, connected car, health and wellness, and enabling technologies. It will also give us the opportunity to work with exciting startups from around the world, and along the way, we expect to see many new applications and innovations we haven’t even imagined.
The Alexa Accelerator, powered by Techstars, will begin accepting applications to the program in January 2017. Following a rigorous review process, 10-12 selected finalists will come together in Seattle to take part in an intensive 13-week program, during which time they’ll be matched with mentors from Amazon and Techstars to develop their technologies and business models. The program begins in July 2017 and comes to an eventfulclose in October 2017 with the Alexa Accelerator, powered by Techstars Demo Day, giving the companies the opportunity to showcase their products and meet with investors.
The Alexa Accelerator, powered by Techstars, is part of the fast-growing Alexa Fund, which now includes 22 investments in companies that range from early- to late-stage. To share more about the accelerator, program leaders from the Alexa Fund and Techstars will host information sessions in places like Seattle, San Francisco, New York City, Boston, London, Berlin, and Tel Aviv over the coming months. Visit the Alexa Accelerator, powered by Techstars page to learn more and to stay tuned to updates.
Consumers want greater control over their homes—the ability to manage not only smart products like their lights and thermostats, but also the services that provide them with connectivity and original content. For service providers, this means tapping into a network that empowers customers with a growing number of capabilities and products built for managing the entire home through voice.
Today, Technicolor and Amazon announced a new collaboration that brings Amazon Web Services and Amazon Alexa together with next generation home gateways from Technicolor that will allow service providers to develop new services for connected homes faster than ever.
Technicolor is a leader in digital innovation for the media and entertainment industry that works with cable, telco and satellite operators to bring bandwidth-intensive experiences into the home. The company sees opportunities for network service providers to build a bridge between cloud and edge technology to introduce new revenue-generating services while making home networks easier to access, manage and configure through voice activated commands.
Technicolor will use AWS for home gateway applications, leverage AWS IoT and Greengrass, integrate Alexa into its new gateway products to allow users to configure network settings and more just by asking, and incorporate Amazon’s Alpine System-on-chip into its new family of gateway products.
The collaboration has clear benefits for both service providers and consumers: NSPs will gain access to a broad developer community through AWS that’s constantly building new applications for the home, improving overall quality of service, customer experience, cost control, while addressing privacy issues by empowering consumers to have greater control over their own information. For consumers, Technicolor’s new services and gateway products will offer the ability to live and interact with connected home services in an easy and natural manner—through voice. This in turn will drive demand for new, more meaningful applications, incentivizing developers to continue to push the bar when working with NSPs.[Read More]
Barely a year later, Mandy attended the Technica Hack ‘15—and won the JP Morgan Chase prize for best mobile app. For Mandy, that event started a new career. After participating alongside so many eager and helpful programmers, she knew what she wanted to do.
Just 11 months later, Mandy has won three more hackathons, including NY TechCrunch ‘16 and Manhattan AngelHack ’16. But unlike her first contest, all these prizes were for Alexa skills.
Mandy first discovered Amazon Echo while attending a 2016 developer conference in San Francisco. She watched a team working with an Echo, and it instantly appealed to her interests in both back-end software development and artificial intelligence. “It was like having my code right in front of me. I talked to my code, and the code kind of talked back to me,” says Mandy.
The day before TechCrunch New York in May, Mandy dove into all the online Alexa Skills Kit documentation she could find. The next day, nervous but determined, she created the prototype that became the Dr. Speech skill—and won the Best Use of Alexa prize.
Mandy, originally from Hong Kong, wanted to help others improve their pronunciation of challenging words, so she created a skill called Dr. Speech, which helps non-native English speakers pronounce words accurately, thereby giving them more confidence without expensive speech therapy sessions.
Mandy gets tweets from people around the world thanking her for how Dr. Speech has been instrumental in improving their pronunciation and has also inspired other developers to build self-improvement skills. Similarly, a user of Mood Journal—another of Mandy’s Alexa skills—wrote to say it helped him in battling anxiety and depression.
Humbled, Mandy repeats she loves to write software that helps people. “Every skill I write is an extension of me. Dr. Speech is about improving speech, because I strive to be a great speaker. I never imagined how my skills would have touched so many people.”[Read More]
We’re excited to introduce “Powered by Linkplay,” a turn-key audio solution that comes with an Alexa Voice Service (AVS) integration. With “Powered by Linkplay,” OEMs now have a quick and cost-effective way to launch Wi-Fi speaker products with Amazon Alexa.
The built-in Alexa experience allows customers to ask Alexa for weather reports, traffic updates, Flash Briefings, and to play their favorite music. Linkplay also gives manufacturers the ability to offer end users access to a range of streaming music providers, including Amazon Music, iHeartRadio, Spotify, TIDAL, Napster, and more. Best of all, “Powered by Linkplay” makes it easy to pair multiple speakers together to play music throughout the home.
OEMs that use the Linkplay audio solution to power their speaker products also gain access to thousands of Alexa skills, new features and capabilities through regular AVS API updates, and smart home controls with partners such as Philips Hue, Belkin Wemo, SmartThings, Insteon, and Wink.
Here are a few products that use the new “Powered by Linkplay” solution:
Contact Linkplay by email if you're interested in building a product with the "Powered by Linkplay" solution.
Not a speaker manufacturer? Alexa is always getting smarter with new capabilities and services. Learn how AVS can add rich voice-powered experiences to your connected products, and read how some of our partners below have integrated with Alexa already:
New to voice user interface design? See how the technology works by building your own Amazon Alexa prototype for Raspberry Pi.
AVS is coming soon to the UK and Germany. Read the full announcement here.
Developers have created thousands of skills for Alexa, expanding Alexa’s capabilities and offering consumers novel new voice experiences. We recently unveiled a new way for customers to browse the breadth of the Alexa skills catalog by surfacing Alexa skills on Amazon.com.
Today we are introducing a new program that allows you to nominate your favorite Alexa skills to be featured in our Community Favorites campaign. Skills that are nominated and meet the selection criteria will be featured in the Alexa app and on Amazon.com in December. This is a great way to help customers everywhere discover new, intriguing and innovative skills on their Alexa-enabled devices.
Share other innovative ways you’re using Alexa in your life. Tweet us @alexadevs with hashtag #AlexaDevStory.
We are happy to announce the Amazon Alexa Hack the Dorm Contest for university students. This is our first challenge in collaboration with MindSumo, a company that focuses on giving students first-hand experience with industry projects.
As voice technology becomes ubiquitous around us, new and current developers are quickly learning about voice user interfaces (VUI). As a student, you are well positioned to create intuitive ways for people to interact with technology and can make a big impact on the interfaces of tomorrow. This contest is your chance to show customers on millions of Alexa-enabled devices what you can build using the Alexa Skills Kit.
To win, you will build a new voice-activated Alexa skill for your university dorm. The winners will create a new skill that makes life easier and better in a university dorm. From improving accessibility features, to creating integrations with your entertainment system, or even making homework more efficient, all domains and interests will be considered. Entries will be judged on their usefulness and creativity among a few other criteria. The best part; no hardware is necessary to do this. If you do not have an Alexa-enabled device, you can test your skill with Alexa right in your browser and begin building your skill right away. See the full contest rules here.
The contest begins today and will close on December 31st at 11:59:59 PM PST. Winners will be announced on or around January 31st, 2017.
Restrictions may apply, see the contest rules on MindSumo’s page.
The Alexa Skills Kit (ASK) enables developers to easily build capabilities, called skills, for Alexa. ASK includes self-service APIs, documentation, templates, and code samples to get developers on a rapid road to publishing their Alexa skills.
Three months ago, we launched Alexa Champions, a recognition program designed to honor the most engaged developers and contributors in the community. Through their passion and knowledge for Alexa, these individuals educate and inspire other developers in the community – both online and offline.
Today we’re excited to recognize ten new Alexa Champions and to showcase their contributions to the Alexa community on our dedicated gallery. We thank them for all the knowledge they have shared with others and for the tools they have created to make it easier for developers to use the Alexa Skills Kit (ASK) and Alexa Voice Service (AVS).
Join me in extending a warm welcome to the newest Alexa Champions:
There are many ways you can share educational and inspiring content about AVS and ASK with the Alexa community through your own blog or newsletter, open-source development tools, tutorials, videos or podcasts and social media. You can also organize local meetup groups for like-minded Alexa enthusiasts and developers.[Read More]
Magic mirror, on the wall—who is the fairest one of all?
Probably the most memorable line from Disney’s 1937 classic, Snow White and the Seven Dwarfs, it may soon become a household phrase again. Modern-day magic mirrors are taking a number of forms, from toys to high tech devices offering useful information to their masters. Now, Darian Johnson has taken that concept an enormous step farther.
Darian, a technology architect with Accenture, has worked in software solution design for 17 years. Today he helps clients move their on-premise IT infrastructure into the cloud. With a recent focus solely on Amazon Web Services (AWS), it’s only natural other Amazon technologies like Alexa would pique his interest.
One night, Darian was pondering what he might build for Hackster’s 2016 Internet of Voice Challenge. He was surfing the web, when he happened on an early concept of a Magic Mirror and realized he could do even better than that. He did. In August 2016, Darian’s new Mystic Mirror won a prize in the Best Alexa Voice Service with Raspberry Pi category.
Darian says his morning routine consists of running between bedroom and bathroom, trying to get ready for work. He doesn’t have an Amazon Echo in either, but he does, however, have mirrors there. That’s another reason why an Alexa Voice Service (AVS)-enabled mirror made sense.
He set his budget at a mere $100. That covered a Raspberry Pi (RPi), a two-way mirror, a refurbished monitor and speaker, some wood planks and a few other assorted items. He determined that his device would:
You can build your own Mystic Mirror using the details on the Hackster site. But it was his software and Alexa that brought it to life.
Darian decided to voice-enable his Raspberry Pi, microphone and speaker with the Alexa Voice Service (AVS). That meant the Mystic Mirror’s master would have access to the built-in power of Alexa and over 4,000 third-party skills, developed using the Alexa Skills Kit (ASK). With just a word, they could control smart home devices, ask for a Lyft ride, play music from Amazon Prime accounts and much more. Best of all, since Alexa is getting smarter all the time, the mirror’s capabilities would constantly evolve, too.[Read More]
On September 29, 2016, Amazon announced the Alexa Prize, a $2.5 million university competition to advance conversational AI through voice. We received applications from leading universities across 22 countries. Each application was carefully reviewed by senior Amazon personnel against a rigorous set of criteria covering scientific contribution, technical merit, novelty, and ability to execute. Teams of scientists, engineers, user experience designers, and product managers read, evaluated, discussed, argued, and finally selected the ten teams who would be invited to participate in the competition. Wait, make that twelve; we received so many good applications from graduate and undergraduate students that we decided to sponsor two additional teams.
Today, we’re excited to announce the 12 teams selected to compete with an Amazon sponsorship. In alphabetical order, they are:
These teams will each receive a $100,000 research grant as a stipend, Alexa-enabled devices, free Amazon Web Services (AWS) services to support their development efforts, access to new Alexa Skills Kit (ASK) APIs, and support from the Alexa team. Teams invited to participate without sponsorship will be announced on December 12, 2016.[Read More]
Eric Olson and David Phillips, co-founders of 3PO-Labs, are both “champs” when it comes to building and testing Alexa skills. The two met while working together at a Seattle company in 2015. Finding they had common interests, they soon combined forces to “start building awesome things”—including Alexa skills and tools.
Eric, an official Alexa Champion, is primarily responsible for the Bot family of skills. These include CompliBot and InsultiBot (both co-written with David), as well as DiceBot and AstroBot. David created and maintains the Alexa Skills Kit (ASK) Responder. The two do most everything as a team, though, and together built the underlying framework for all their Alexa skills.
This fall, they’re unveiling prototyping and testing tools that will enable developers to build high-quality Alexa skills faster than ever.
Eric and David first got involved with Alexa when Eric proposed an Amazon Echo project for a company hackathon. The two dove into online documentation and started experimenting—and having fun. “After the hackathon, we just kind of kept going,” Eric said. “We weren’t planning to get serious about it.”
But over the past year, they grew more involved with the Alexa community. They ended up creating tools that could benefit the whole community. “We wrote these tools to solve problems we ran into ourselves. We ended up sharing them with other people and they became popular,” David said.
The first of these, the Alexa Skills Kit Responder, grew from David’s attempt to speed the process of testing different card response formats. Testing a response until it was just right meant you had to repeatedly modify and re-deploy code each time you changed the response. Instead, this new tool lets developers test mock skill responses without writing or deploying a single line of code. Follow the documentation to set up an Alexa skill to interface with ASK Responder, then upload any response you’d like. The ASK Responder will return it when invoked.
And that’s just the beginning. The ASK Responder’s usefulness is about to explode.
David created Responder for testing mock responses. But the two soon discovered a home automation group using the tool in an unexpected way.
Instead of a skill called “Responder,” they’ll create a skill named My Home Temp, for example. They’ll map an intent like “What is the temperature?” and have their smart home device upload a response to the ASK Responder with the temperature of the house. When the user says “Alexa, ask My Home Temp what is the temperature?” Alexa plays the uploaded response through the Echo. This creates the seamless illusion of a fully operating skill.[Read More]
This new technical tutorial by Sr Solutions Architect for Amazon Alexa, Sebastien Stormacq will show you how to use Amazon API Gateway and configure it to act as a HTTP Proxy, sitting between Alexa and your OAuth server.
Have you ever developed an Alexa skill that uses account linking? Do you remember the first time you tried to click on the “Link Account” button and feared for the result? I bet you first saw the dreadful error message: “Unable to Link your skill”. Sometimes trying to figure out what an error is, is like searching for a needle in a haystack. You have no clue.
Most of the errors that I have seen when working with developers, fall in two categories:
When you have access to the OAuth server logs, debugging the error message you see in the Alexa App is relatively easy. You just enable full HTTP trace on the server side and search for the error or the misconfiguration on the server. Full HTTP trace includes the full HTTP headers, query string and body passed by the Alexa service to your server.
With a bit of experience, catching an OAuth error in HTTP stack trace takes only a few minutes.
The problem is that most developers we are working with, have no access to the OAuth servers or the server logs. Either they are using a third party OAuth server (Login With Amazon, Login With Facebook, Login with Google and the likes), or they are working in a large enterprises where another team is operating the OAuth server. Meeting that team and asking them to change logging level or to request access to the logs can take weeks, or may not be possible at all.
This article explains how to setup an HTTP proxy between Alexa Skill Service and your OAuth server to capture all HTTP traffic and log it. By analyzing the logs, you can inspect the HTTP URLs, query strings, headers and full bodies exchanged. Setting such a proxy requires infrastructure to host the proxy: a networked server, with a runtime to deploy your code etc … this is unnecessary heavy lifting where Amazon Web Services can help.
We will use Amazon API Gateway instead and will configure it to act as an HTTP Proxy, sitting between Amazon’s Alexa Skill Service and your OAuth server.
Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. With a few clicks in the AWS Management Console, you can create an API that acts as a “front door” for applications to access data, business logic, or functionality from your back-end services.
API Gateway HTTP Proxy Integration mode is a new feature of API Gateway that was launched on Sept. 20th 2016. You can read the post by AWS Director of Evangelism, Jeff Bar’s, if you want to learn more about this.
The diagram below shows where API Gateway, with HTTP Proxy Integration, fits in the OAuth Architecture.