Build a Custom Capability with the Smart Home AI Toolkit

Use the Smart Home AI Toolkit to chat with the Alexa+ smart home agent to describe your device and create a new capability. Or, you can start with an existing specification and chat with the agent to fill in the gaps. The agent generates the custom capability, including the interface definition and test utterances. You save the capability to a device that represents your physical product.

Describe your device capabilities

On the New Integration page, describe the capabilities of your smart home device by chatting with the agent or uploading one or more specifications. You can ask the agent at any time what the next steps are.

To describe your device capabilities in the Smart Home AI Toolkit

1. Sign in to the Smart Home AI Toolkit.
2. On the Alexa+ Smart Home page, choose New Integration.
3. To create your custom capability, use the chat and build option or the upload option.
4. Instruct the agent to save the capability and create a device.
   Example prompt: Save this capability and create a device.
5. After the agent suggests a device name, device type, and device version, confirm the defaults or instruct the agent with valid information.

Chat and build

The chat and build path enables you to specify your device's capabilities through natural conversation, without explicitly defining directives, properties, or responses. The agent maps your plain-language descriptions to the formal structure internally.

To start, describe your smart home device in the chat box, including the type of device and what your device can do. Throughout the conversation, the agent summarizes what it understands and asks clarifying questions. You can continue to enter text in natural language or select an agent-suggested response to provide answers. Continue to chat with the agent as it requests missing or ambiguous information. You can tell the agent that you will define an item later and continue without providing that detail immediately.

In your conversation, describe the following elements of your device capabilities:

• Device type and function.
• Actions that the customer can request the device to perform, such as turn on, turn off, or adjust settings.
• Any limits, ranges, or fixed sets of options for each action, such as temperature range 60 to 80 degrees Fahrenheit.

Upload specification

If you have technical documentation, you can skip the full guided conversation and upload reference, requirements, or JSON manifest files. To upload a file, choose the attachment icon (paper clip) and browse to the file. The upload path supports the following file types: TXT, CSV, JSON, DOCX, and PDF.

The agent parses the file, generates the interface definition, and asks follow-up questions to fill any gaps. The agent never silently fills in missing values.

To complete the custom capability, answer all questions from the agent.

Review and refine the capability

After the agent has enough information, it generates the interface definition and presents it in the capability workspace. The capability view includes the following four tabs:

• Controller View – Shows the generated controller interface definition for your custom capability, including the capability namespace, version, and description. The definition includes configuration attributes, properties, directives, events, and data types.
• Json View – Displays the full controller definition in JSON. You can review and update the definition directly in the viewer.
• Test – Provides tools to generate test utterances for your controller, run a batch evaluation using the utterance, and test custom utterances.
• Documentation – Generates a technical specification for the capability. You can use the specification to implement the capability in your smart home add-on.

To update the definition, open the Json View, make changes directly in the viewer, and then choose Save. Or, you can instruct the agent to make the changes for you. The agent validates the JSON automatically as you make changes through conversation.

After you save the changes, the agent automatically marks any previous test utterances and batch evaluations as out-of-date.

Test your custom capability

After you create the custom capability, you can ask the agent to generate utterance-to-directive mappings for each directive. Use the generated utterance dataset to run a batch evaluation that tests your capability. You can run these tests in the capability workspace or the device workspace.

Batch evaluation validates that Alexa correctly understands your capability. Your custom capability must pass batch evaluation before you can publish the interface for review. If the evaluation fails, the agent gives you actionable feedback pointing to the exact directive or parameter description to fix.

Generate an utterance dataset

Use the Smart Home AI Toolkit to generate an utterance dataset for the custom capability.

To generate test utterances in the Smart Home AI Toolkit

1. Under Capabilities > your capability name, choose the Test tab.
2. To generate the utterance dataset, expand Generate Test Utterances, and then choose Generate.
   Tip: You can view the status of the utterance generation under Generate Test Utterances or you can ask the agent for a status update. To refresh the utterance generation status, choose Refresh.
3. If the generation fails, fix any errors in the capability definition.
   Tip: To fix errors, chat with the agent to add, update, or remove directives, properties, or responses.
4. To regenerate the utterance dataset, expand Generate Test Utterances, and then choose Rerun.
5. After the dataset generation completes successfully, expand Generate Test Utterances to view the generated utterances. To download the utterances, choose Download dataset.
6. Review the generated utterances. If any mappings don't reflect how customers would naturally speak, instruct the agent to update the utterances or capability, and then choose Rerun.

The downloaded dataset is a JSON file of turns in the following format:

[{
  "dialogId": "d0c0dce2837d494db0a8f69454980972",
  "turns": [{
    "utterance": "feed my dog",
    "actionPlans": [{
      "apiName": "Alexa.Custom.<capabilityName>",
      "parameters": { }
    }]
  }]
}]

Run a batch evaluation

After the dataset generation completes, use the toolkit to test the utterances.

To run a batch evaluation in the Smart Home AI Toolkit

1. Under Capabilities > your capability name, choose the Test tab.
2. To test the utterance dataset, expand Batch Evaluation, and then choose Run Evaluation.
3. If the evaluation fails, fix any errors in the capability definition.
   Tip: To fix errors, chat with the agent to add, update, or remove directives, properties, or responses.
4. If you changed the capability definition, regenerate the utterance dataset. Under Generate Test Utterances, and choose Rerun.
   Tip: The evaluation takes several minutes to complete. You can view the status of the evaluation under Batch Evaluation. To update the evaluation status, choose Refresh.

Test custom utterances

To test errors and edge cases, you can test a set of custom utterances.

To test custom utterances in the Smart Home AI Toolkit

1. Under Capabilities > your capability name, choose the Test tab.
2. Expand Custom Utterance Testing, and then enter one or more customer utterances, one per line.
3. To run the test, choose Run Custom Test.
4. If the test fails, fix any errors in the utterance or capability definition.
5. To rerun the utterances, choose Rerun.

Test in the sandbox

To validate an utterance, you can run ad-hoc testing in the sandbox at any time. Use the results to modify the generated or custom utterances.

To run end-to-end testing in the sandbox

1. To access the sandbox, choose Sandbox from the menu, or expand the Sandbox panel at the bottom of the page.
2. In the UTTERANCE INPUT field, enter one natural language utterance, and then choose Run.
3. In the RESPONSE panel, validate that Alexa sent the expected directive with the expected properties and property values.

Edit a published custom capability

You can make updates to your published custom capability by creating a new version of the definition. To make edits to an unpublished version, see Review and refine the capability.

To edit a published custom capability

1. On the menu, choose Capabilities, and then select a capability to edit.
2. In the chat window, instruct the agent to create a new version of the capability.
3. To make changes to the capability definition, chat with the agent or update the Json View and save the changes. For more details, see Review and refine the capability.
4. To test the updates, on the Test tab, generate a new utterance dataset and rerun the batch evaluation. For more details, see Test your custom capability.
5. Instruct the agent to add the new version to your device.
6. To test and publish your device for review, see Steps to publish a device with custom capabilities.

Next steps

After you create the custom capabilities for your device, you can test and publish your device for review.

Related topics

• Add Custom Capabilities with the Smart Home AI Toolkit
• List of Alexa Interfaces
• Steps to Build a Smart Home Add-on

Last updated: Jul 22, 2026