What to Expect from Siri at WWDC 2024
For years, Siri has often felt like an underwhelming virtual assistant, struggling with contextual understanding and limited integration with third-party apps. Meanwhile, competitors like Alexa and Google Assistant rapidly advanced, leaving Siri seemingly in the dust.
However, the landscape changes dramatically with the introduction of iOS 18, iPadOS 18, and macOS Sequoia. Apple has supercharged Siri with two game-changing features: the App Intents framework and Apple Intelligence. This dynamic duo transforms Siri from a mere gimmick into a deeply integrated, context-aware assistant that seamlessly interacts with your favorite apps.
At the core of this transformation is the App Intents framework, an API that empowers developers to create “assistant schemas” — detailed models describing specific app actions and data types. By adopting these schemas, apps can communicate their capabilities in a way that Apple’s advanced AI models can fully comprehend.
App Intents provide the structural foundation, but the true innovation lies in Apple Intelligence, a powerful new system unveiled at WWDC. This system infuses advanced generative AI directly into Apple’s operating systems, enabling Siri to operate on apps’ structured data, understand natural language in context, make intelligent suggestions, and even generate content — all while upholding Apple’s strong privacy standards.
To illustrate the potential, let’s consider a hypothetical library app called Library Manager, which leverages these new technologies.
Data Modeling With App Entities
For Siri to understand the domain of library management, the app must define its data entities so that Apple Intelligence can process them. This involves creating custom data structures that conform to the @AssistantEntity schema macros. For example:
@AssistantEntity(schema: .library.book)
struct BookEntity: IndexedEntity {
let id: String
let book: Book
@Property(title: "Title")
var title: String
@Property(title: "Author")
var author: String?
@Property(title: "Genre")
var genre: GenreType?
var summary: String?
var availableCopies: Int
var displayRepresentation: DisplayRepresentation {
DisplayRepresentation(title: title,
subtitle: genre?.displayRepresentation)
}
}By adhering to schemas like .library.book, Library Manager ensures its data models align with what Apple Intelligence expects, laying the groundwork for Siri to interact with the app in meaningful ways.
Finding Books
One of the fundamental features of a library app is the ability to search for books. The app can enable Siri to perform this task using the .library.findBooks schema:
@AssistantIntent(schema: .library.findBooks)
struct FindBooksIntent: FindIntent {
@Property(title: "Search Query")
var searchQuery: String?
@Dependency
var bookStore: BookStore
@MainActor
func perform() async throws -> some ReturnsValue<[BookEntity]> {
let results = try await bookStore.findBooks(matching: searchQuery)
return .result(results)
}
}This intent allows users to search for books through Siri, which can intelligently suggest options even without explicitly naming the app. For instance:
“Hey Siri, find mystery novels in Library Manager.”
Siri displays a list of mystery novels available in Library Manager.
Viewing Book Details
Once a user finds a book, they’ll likely want to view its details. The app can handle this with the .library.openBook schema:
@AssistantIntent(schema: .library.openBook)
struct OpenBookIntent: OpenIntent {
var target: BookEntity
@Dependency
var navigation: NavigationManager
@MainActor
func perform() async throws -> some IntentResult {
navigation.openBook(target.book)
return .result()
}
}This allows users to open a specific book with a simple voice command, such as:
“Hey Siri, show me the details for ‘To Kill a Mockingbird.’”
Siri opens the Library Manager app to the book’s detail page.
Intelligent Reading List Planning
Library Manager can take advantage of Apple Intelligence to offer even more advanced features like intelligent reading list planning. By using assistant schemas, the app enables Siri to handle multi-step tasks:
“Hey Siri, I want to add ‘1984’ to my reading list for next month.”
Siri doesn’t just find the book; it also checks the availability, adds the book to your reading list, and schedules a reminder to pick it up, all while staying within the conversational interface.
Interactive Widgets With WidgetKit
Beyond voice interactions, Library Manager can integrate its App Intents with WidgetKit to create intelligent, interactive widgets. For example:
struct BookSearchEntry: TimelineEntry {
let date = Date()
var searchQuery = ""
@OpenInAppIntent(schema: .library.findBooks)
var findBooksIntent: FindBooksIntent? {
FindBooksIntent(searchQuery: searchQuery)
}
This widget allows users to search for books directly from their home screen, offering a seamless, ambient experience across Apple’s platforms.
Generative Experiences
Perhaps the most exciting potential lies in generative AI. Imagine asking Siri to create a custom reading list based on your favorite genres or authors. Using Apple Intelligence, Library Manager could generate a personalized list of book recommendations in real-time, tailored to your reading preferences.
Prioritizing Privacy and Responsibility
With these powerful features come significant responsibilities. Apple emphasizes privacy, security, and ethical AI development. Apps like Library Manager must ensure user data is protected, generative outputs are safe, and transitions from older frameworks are handled smoothly.
In conclusion, with the introduction of App Intents and Apple Intelligence, Siri is evolving into a sophisticated, context-aware assistant that can profoundly enhance app experiences. For developers, the opportunities are vast — from intelligent reading list planning to interactive widgets and generative content creation, all while upholding Apple’s commitment to privacy and responsible AI.
