Siri AI can know what’s happening on your device screen, Apple said.
Credit:
Apple
While many other AI models can perform similar tasks, these and other demos highlighted how Apple Intelligence and Siri AI benefit from tight integration into the “personal context” of data on your device. That means Siri can search to find information across your messages and email, even if you can’t remember specifically where that information is stored, for instance. It also means that, when using Siri to write emails, this “personal context” can also customize the writing style to match your previous emails to the same person, Apple said.
Siri AI utilizes “world knowledge” by searching the web using private cloud compute to generate answers, Apple said, and integrates with app actions to figure out which tools to use to complete a task. The on-device assistant can also display on-screen understanding to tailor its assistance based on what you’re doing at that moment.
Credit:
Apple
Siri AI extracts multiple calendar reminders from a single concert schedule image.
Credit:
Apple
Apple also highlighted Siri’s new “visual intelligence,” which can give relevant responses to questions right from the camera app or extract multiple calendar updates from a complex image of a concert schedule, for instance. On VisionOS, you can also directly ask questions about physical objects you can see in the world around you and get answers from Apple Intelligence. A new “Write with Siri” feature, meanwhile, will let you use Apple Intelligence to generate text “virtually anywhere you type,” Apple said, while a new AI-powered automatic proofreading system will check your writing style system-wide (a la spell check).
On MacOS, these new Siri AI features will be integrated into Spotlight search, which can identify when a typed query should start a Siri conversation rather than just a list of files or web results. You can also ctrl-click across the OS to ask Siri questions about photos, files, or text, or even multiple files at once. And anyone still using VisionOS will be able to put a glowing, animated Siri Orb into their workspace that will be able to answer questions whenever you look at it.
