Apple previews Live Captions, Apple Watch gestures coming later this year

Ahead of WWDC beginning June 6, Apple on Tuesday announced a set of accessibility features for the iPhone, iPad, and Mac that will be coming “later this year.” The software features are for “users with disabilities to navigate, connect, and get the most out of Apple products.”

Apple did not announce how the new accessibility features will be rolled out, though they’re likely to be part of the next crop of operating systems. Apple is expected to reveal iOS 16, iPadOS 16, and macOS 13 at WWDC, but Apple’s press release did not specifically state whether the new features are part of these new operating systems or the current versions. Here are the new features coming to Apple devices:

Door Detection

The iPhone and iPad’s Magnifier app will get a new Detection Mode feature that will support Door Detection, which will help users locate doors and provide information about the door itself (such as numbers, symbols, or signs), if it is open or closed, and how the door can be handled. Door Detection can work with the Magnifier’s People Detection and Image Descriptions to help with navigating an area. Apple Maps will also have sound and haptic feedback to help find the starting point for walking directions.

Apple Watch Mirroring, Quick Actions

This feature mirrors the Apple Watch user interface on a paired iPhone, giving users access to Apple Watch controls on the iPhone instead of using the watch itself. iPhone’s Voice Control and Switch Control can be used to interact with Apple Watch, as well as voice command, sound actions, head tracking, and external Made for iPhone switches.

The new Quick Action is a double-pinch gesture that can be used for several Apple Watch functions, including answering or ending phone calls, dismissing notifications, taking pictures, play or pause in the Now Playing app, and start, pause, or resume a workout. 

Live Captions

Live Captions will be available on iPhone, iPad, and Mac on “any audio content.” Apple states FaceTime calls, video conferencing, streaming media content, and in-person conversations as examples where Live Captions will work. Live Captions will have adjustable font sizes and appear to all participants in a group video call, and typed responses can be spoken aloud. Live Captions are generated on the device, which means a user’s privacy is secured.

VoiceOver, Siri Pause Time, and more

Other new features include:

• VoiceOver support for over 20 new languages and VoiceOver on Mac will have a new Text Checker tool to look for formatting issues in written text.
• Buddy Controller combines two game controllers into one that can be used for the input of a single player.
• Siri Pause Time, which allows manual adjustment of Siri’s response time.
• Voice Control Spelling Mode allows dictation of custom spellings using letter-by-letter input.
• Sound Recognition can be adjusted to learn the sounds specific to an environment.
• Apple Books will have features to make books easier to read, as well as new themes.

Global Accessibility Awareness Day

Global Accessibility Awareness Day is on May 19, and Apple made several announcements to commemorate the day:

• SignTime is expanding to Canada. This allows Apple Store and App Support customers with on-demand American Sign Language interpreters.
• Live sessions about iPhone accessibility will be happening in Apple Stores. 
• Apple Support social channels will feature how-to content.
• The Shortcuts app on Mac and Apple Watch will have support for the Accessibility Assistant.
• Apple Fitness+ will have trainer Bakari Williams use American Sign Language to showcase accessibility features.
• New guides in Apple Maps. There is one for the National Park Foundation, Park Access for All, and a set of guides from Gallaudet University that focuses on sites that prioritize service to the Deaf community. 
• The App Store will highlight accessibility apps.
• Apple Books and Apple Podcasts will showcase stories by and about people with disabilities.
• Apple Music will offer Saylists playlists, a collection of playlists that each focus on a different sound to practice vocal sounds or speech therapy.
• Apple TV+ will showcase movies and shows of “authentic representation of people with disabilities.”