Interactive Voice Response
Automated phone routing system to connect callers to the right team
Amazon
Aug 2024 - Mar 2025
Usability Testing, Interviews, Multilingual
PROBLEM
Caller Support
Amazon Last Mile delivery drivers and delivery service providers (DSPs) struggle to connect to support teams through an inefficient call workflow, leading to wasted time and long call wait times.
Out-of-Scope Calls
The Last Mile Emergency Team (LMET) is trained to handle incoming calls concerning life-safety issues, like assaults or injuries. Calls concerning delivery issues, app issues, or minor property damage require LMET specialists to spend an average of 2.23 minutes of time to transfer the call to a different team. From October 2023 to August 2024, 40-65% of calls received by LMET were out-of-scope issues, resulting in 383,424 misdirected calls annually that require a transfer, consuming 14,271 hours of specialist time. This represents a potential annual entitlement of $321,097 (based on $22.5/hr wage).
Call Wait Times
LMET has struggled to scale with the rapid growth of Amazon's transportation network, exasperated by the influx of out-of-scope calls. Call wait times have increased 363% quarter-over-quarter from 3 minutes in Q4'23 to 15 minutes in Q3'24, with a record-high 59-minute queue time in May 2024. In Q3'24, the call abandonment rate reached a high between 14-38%, leaving important caller issues unaddressed and reducing user trust.
Caller Experience
Amazon delivery drivers and DSPs have frequently voiced discontent with the current call experience, having to wait through a 60-second introduction message before being presented with a long list of menu options. Lack of guidance and clear language have led drivers to choose the incorrect support team, leading to a frustrating experience for both caller and specialist. Because LMET specialists must spend time handling out-of-scope calls, callers with critical life-safety issues are missed or face long wait times.
GENERATIVE
Interviews
Since I hadn't designed for users from the Amazon Last Mile space in my role before, I independently conducted a series of field interviews with delivery drivers and dispatchers at local Amazon delivery stations in Phoenix. I explored their prior experiences in seeking support for emergencies or delivery issues, and established an understanding of their daily work life and struggles they encounter.
Entry Points
Due to the infrequency of serious incidents, most drivers were unaware of the Emergency Help button in the Flex delivery app's side panel, suggesting a need for improved visibility and enhanced driver training. When faced with a problem, drivers experience mental burden trying to distinguish between LMET and SDS (Shipping and Delivery Support) jurisdictions, needing a clearer entry point that guides drivers to the appropriate team based on their situation. Participants also praised the current SDS Help menu's tree-like structure for selecting delivery issues in the app and suggested implementing a similar pattern for LMET contact.
Contact Points
Drivers consistently contact their DSP dispatchers first during incidents due to their comfortable working relationship and dispatchers' need to know about schedule impacts. LMET calls are viewed more as a reporting mechanism rather than for immediate assistance, with DSPs even waiting to call until drivers return to the station. Participants unanimously expressed negative perceptions of SDS support, citing difficulties understanding agents' strong accents and the agents' limited ability to resolve problems. More experienced drivers have developed ways to avoid SDS contact by solving issues independently or by using SDS's chat-based option to prevent miscommunication.
Data Sharing
Drivers wish for a single call or action to notify all relevant parties during an incident, including their dispatcher, LMET, and SDS, to reduce their burden of deciding who to contact and when. Participants appreciated LMET's increasing ability to pull driver information automatically, and want this functionality to expand to SDS and include more robust data. This would address the major pain point of drivers spending time answering basic questions at the start of the call, which is particularly challenging for those injured or having difficulty communicating clearly.
Verbal Communication
Participants voiced concern about how well the proposed voice recognition system would handle heavy accents and speech impediments, citing accent-related issues as an already challenging hurdle with SDS responders. Cell phone reception also poses a common challenge even in urban areas like Phoenix, where drivers need to relocate their vehicle until reception improves. Some participants suggested implementing a text-based option to aid in communication, which would particularly benefit deaf/hard-of-hearing drivers or those unable to speak.
USER
Personas
The support workflow for drivers and DSPs involves multiple parties across a period of high-stakes time, each with unique needs and expectations. Although the primary customer for the project was the Last Mile Emergency Team, I ensured every user type across the workflow was involved in the design process.
Delivery Associate (DA)
Works under Delivery Service Providers (DSPs) to deliver packages to customers using Amazon delivery vans and manage deliveries through the Rabbit/Flex app.
Delivery Partner (DP)
Independent contractor who uses their own vehicle to deliver packages to customers for Amazon, using the Flex app to set their own schedule by picking up delivery "blocks".
Escalation Specialist (ES)
Agents within LMET who respond to calls from Last Mile operations reporting emergency incidents, providing immediate assistance while documenting incident details.
EVALUATIVE
Usability Testing
After guiding support engineers to create the IVR phone system using simple language and an efficient flow informed by my research interviews, I conducted of onsite usability testing with delivery drivers and dispatchers. After improving the system based on findings from the first round, I conducted a second round to test the new version. Using diverse scenarios as simulations, I had participants call a demo phone number and talk through the verbal call interface, noting positive and negative feedback.
ROUND 1
71%
Success rate
Long Pauses
Participants experienced awkward long pauses after speaking their inputs and waiting for the system to respond.
Prompts Cut Off
The IVR system would occasionally cut off midway through statement, causing participants to miss out on important information.
Prompt Wording
There was some confusion caused by inconsistent terminology that didn't match participants' mental models.
Text Option
Participants were unfamiliar with the verbal input interaction pattern of the phone system, and suggested a text-based option.
ROUND 2
95%
Success rate
Prompt Interrupted
The IVR system had difficulty handling users who spoke before prompts finished, repeating the prompt from the beginning and wasting time.
Long Pauses
The system's responsiveness improved from the first round, but there were a few instances the prompts would pause due to ambient noise.
Prompt Wording
A prompt after from the first round was deemed unnecessary by participants, and there were opportunities identified to shorten prompts for efficiency.
QUANTIFYING SUCCESS
Results
We launched the IVR system to the English language call queue in the Canada on October 14, 2024, and the United States on October 28, 2024, impacting 300,000+ users across the Last Mile organization making up 63.1% of worldwide call volume. After 1 month, the new IVR system was 15% more effective at correctly routing calls to SDS than the old call menu, reducing LMET call volume by 13.8%. This success demonstrates the system's efficiency in reducing the number of out-of-scope calls received by LMET, and improvement of organizational health and strain on escalation specialists.
EXPANSION
Spanish
Expanding on the success of the English launch in the US and Canada, our team implemented a Spanish version of the IVR system using automated and manual translation. I conducted comprehensive usability testing of the new version with Spanish-speaking delivery drivers across two Phoenix delivery stations. Working collaboratively with the Last Mile UX Research team, I documented findings in detail, coordinated participant compensation, and helped implement prompt adjustments for a more streamlined call flow. One month after launching to the Spanish US queue on March 7, 2025, the IVR system effectively reduced out-of-scope calls to LMET by 27%.
RESULTS
83%
Success rate
Correct Translations
No participant had trouble understanding the prompts and questions, though they sounded mildly formal.
Call Menu Prompts
The addition of multiple initial questions has reduced the number of drivers reaching the voice description step, more like the old call menu options.
Prompt Wording
Drivers misinterpreted the "delivery or app issue" question, occasionally saying "yes" for LMET incidents that affects their delivery, leading to incorrect routing.
Language Variances
Participants noted some vocabulary that varies by country of origin in Latin America, like alternatives for "delivery van": “camioneta”, “van”, “autobús”, “bus”