Multimodal design: Introduction

One of the hallmarks of a great skill is when the experience adjusts to the customer’s context, especially when it's tailored to the kind of device they’re using. An Echo Show device lets you show graphics and interactivity on a GUI display (graphical user interface). We can use the screens on these devices to enhance an experience, provide more context, and help customers accomplish their tasks even more quickly. In this section you'll learn…


Learn the fundamentals of multimodal design

Be voice-first, complement the VUI:

While a multimodal voice experience can engage customers in ways voice alone cannot, it should always complement a voice-first experience.

▢  A customer should be able to use the core functions of the skill without need of a screen.

▢  Each touch (or remote, for FireTV) target on screen has a voice command analog (not every voice command needs a target)

▢  Leverage touch interactions where voice may not be ideal (such as displaying long lists to scroll)

▢  Differentiate voice responses to surface to devices with screens; Customers may not need to hear some information if they can see it on the screen

▢  A customer can switch modalities – between voice and touch, to better serve their needs – during their skill session without friction

Create impact & delight:

Determine which moments in the customer experience that will be most memorable, significant, or climactic, and use GUI to enhance the moment. Consider which screens your customer will see most often and determine how GUI could either expedite those steps or create surprise. Leverage touch in your experiences where it provides the most value to customers.

▢  The core voice functions of the skill should have corresponding touch interactions

▢  Offer GUI shortcuts or hints to expedite the experience for steps the customer will complete most often

▢  Offer a variety of images based on customer history, seasonality, or other dynamic context, especially on screens the customer will see most often

▢  Punctuate the most significant “moments” in the customer’s experience with on-screen interactivity

▢  Provide additional context beneficial to the customer in completing their task or enjoying content

▢  Clearly display important, contextually relevant data (such as the score of a game, answer to a question, etc.) when it’s most needed

▢  Use graphics to create additional meaning or understanding for the customer, such as putting faces to key characters

Be contextual & glanceable

Customers don’t always watch the screen while they’re interacting with your skill. They may be multitasking or dividing their attention, sitting on a couch, or standing at a counter. Create glanceable designs that quickly tell the customer what they need to know most in that device’s context.

▢  Don’t include long texts that repeat exactly what Alexa is saying

▢  The dominant element on each screen should be the most important information the customer would need

▢  All touch (and remote) targets should have a matching voice command. Their labels are valid utterances to use that command

▢  On-screen elements shouldn’t compete for attention

▢  Scale graphical elements to maintain readability on each device

Design for readability, accessibility & discoverability:

▢  Screens should be readable from a 3-foot and 10-foot viewing distance for most devices

▢  GUI elements should assist the customer in discovering features and commands they may not know about

▢  Screen layouts should have a clear hierarchy of information

▢  Screens that display similar information or function similarly should have consistent layouts

▢  Check that the most important on-screen information can be read from 10 feet

▢  Check that the rest of the on-screen information can be read from 3 feet

▢  The foreground and background colors should meet contrast ratio minimums of 4.5:1; Use an online calculator to check your contrast ratios

▢  Support a customer request for “tell me what’s on screen”

▢  Handle personal information on-screen carefully (such as avoiding displaying it larger than necessary, if at all)

▢  Don't display sensitive or secure information (such as passcodes)

Next Article: