WWDC 2024 Highlights: Apple Intelligence, Siri Makeover, and More

Apple’s WWDC 2024 keynote brought several major announcements, including the highly anticipated Apple Intelligence and a refreshed Siri. The focus was on the company’s AI advancements, Vision Pro updates, and numerous feature enhancements.

You can watch the archive over on Apple’s events page, or you can watch the YouTube archive right here and via the embed below. And if you’re curious about what the predictions were ahead of the event, all of the pre-event speculation is still captured below that YouTube embed and a couple others highlighting Apple’s reveals. Otherwise, you can head right here for our complete rundown of everything Apple announced during this year’s WWDC.

Spotlight Search Enhancements in iOS 18

Apple is making significant strides in improving the user experience with the introduction of advanced AI features in iOS 18. Among these advancements is an enhancement to the Spotlight Search feature, which now includes semantic search capabilities, allowing for more intuitive and effective searches within apps.

Spotlight Search with Natural Language Queries

Accessed by swiping down on the Home Screen, Spotlight Search already serves as a versatile tool for discovering content on the device and on the web. With iOS 18, Apple has enhanced Spotlight with semantic search capabilities, enabling users to search for content in apps using natural language queries. This means users can now find relevant content even if their search terms do not exactly match the indexed terms.

Improved Search Through Core Spotlight Framework

Using the Core Spotlight framework, developers can make their app content searchable via Spotlight. This content is stored in a private index on the user’s device, ensuring privacy as other apps cannot access this data. The new semantic capabilities improve Spotlight’s understanding of app content, allowing for more flexible and user-friendly searches. These searches leverage machine learning models that run on the device, providing real-time results based on the user’s queries.

Enhancing Search Result Rankings

Developers can also improve the rankings of their app’s search results by sending signals to Spotlight when users engage with their content. For instance, if a user frequently searches for a specific hiking trail and marks it as a favorite, the developer can prioritize this content to appear higher in future search results. This feature ensures that users find the most relevant and personalized content quickly.

Practical Example of Semantic Search

Consider a user searching for a local hiking trail. With semantic search, Spotlight can return results related to the trail’s location or associated keywords, even if the exact name isn’t used. This feature helps users find trails more efficiently, enhancing their overall experience. By tapping on a search result, users are directly taken to the relevant page in the hiking trails app, streamlining their journey from search to content.

Integration with Siri

This update not only enhances Spotlight Search but also improves Siri’s ability to understand and interact with app content. While it may not be as flashy as AI-powered Siri or ChatGPT integration, this behind-the-scenes enhancement significantly boosts the usability of Spotlight Search, making it a more powerful tool for iPhone users.

The integration of semantic search capabilities in Spotlight Search is a testament to Apple’s commitment to enhancing user experience through practical AI applications. These updates make it easier for users to find and engage with content, providing a more intuitive and efficient search experience on iOS 18.

Top WWDC 2024 Features You Might Have Missed

Apple revealed a slew of new features and updates during its WWDC 2024 keynote, covering iOS, iPadOS, macOS, VisionOS, and introducing Apple Intelligence. While many features were highlighted onstage, numerous others are surfacing as developers explore the new betas. Here’s a rundown of some of the best features you might have missed.

Updates to Messages App

 

The Messages app in iOS 18 has received a facelift along with several new features. Users can now schedule messages to be sent later and enjoy the addition of RCS support, making it easier to communicate with friends using Android devices. This update also enhances group messaging capabilities and introduces new ways to interact with multimedia content.

iOS 18 Design Tweaks

iOS 18 introduces subtle design changes to improve user experience. These updates include refined icons, a more cohesive color scheme, and improved animations that make navigating the OS smoother and more visually appealing.

Enhanced Settings in iOS 18

Some settings in iOS 18 have been revamped for better usability. For example, privacy settings are now more accessible, allowing users to manage app permissions more intuitively. The new layout helps users quickly find and adjust their preferences without digging through multiple menus.

 

iOS 18 brings significant enhancements to language and input methods. This includes better support for multilingual users, improved predictive text, and advanced voice recognition capabilities that make dictation and voice commands more accurate and responsive.

Utility Updates

New utilities in iOS 18 include an upgraded Calculator app for the iPad that utilizes the larger screen to offer features like Math Notes, which performs calculations for you. Additionally, a new Passwords app helps users manage and organize their login credentials more efficiently.

Photos App Makeover

The Photos app has been redesigned to provide a more intuitive and enjoyable user experience. The new navigation system organizes photos by themes such as people, places, and events, making it easier to find and relive special moments. Enhanced editing tools and AI-driven suggestions further enrich the photo management experience.

 

Apple continues to lead in accessibility with iOS 18. The update includes new features to assist users with disabilities, such as voice control enhancements, better screen reader support, and customizable display options to accommodate various needs.

New Permissions for Address Book Access in iOS 18

Apple introduced significant steps to enhance user privacy in iOS 18 by restricting how apps access users’ address books. Traditionally, social apps have relied on users’ contacts to build their networks, but this practice is changing. iOS 18 introduces a new two-step permissions process for apps requesting access to users’ contacts, providing more control and privacy to users.

New Two-Step Permissions Process

The new permissions screen in iOS 18 is designed to give users more granular control over their contacts. When an app requests access to contacts, users will first be asked to allow or deny access, as before. If access is granted, users will then choose whether to share all their contacts or select specific contacts to share.

This change was highlighted during a session at Apple’s Worldwide Developers Conference, focusing on user privacy features. The two-step process ensures that users are fully aware of which contacts they are sharing and can limit access accordingly.

Ongoing Access to Contacts

Currently, when users share their contacts with an app, the app can continue to access new contacts added over time. This practice has raised privacy concerns, as users often unknowingly provide ongoing access to their address books. The new permissions screen in iOS 18 aims to address this issue by allowing users to select specific contacts to share, rather than granting blanket access.

Preventing Growth Hacks

Social apps have historically used aggressive tactics to grow their user base, often requiring full address book access. Apps like Lapse, Poparazzi, and Clubhouse have employed such methods to rapidly expand their networks. However, this approach often leads to unsustainable growth and compromises user privacy.

With iOS 18, these tactics will become more challenging. Users who opt to share their address book will be directed to a second screen where they can choose “Allow Full Access” or “Select Contacts.” This additional step discourages apps from demanding full access and promotes more transparent and user-friendly practices.

Developer Impact

The introduction of this new feature does not require developers to make any changes to their apps. The permissions screens will automatically appear if an app requests access to contacts. This seamless integration ensures that user privacy is enhanced without placing additional burdens on developers.

Industry Reactions

The response to this update has been mixed. Developer Nikita Bier, known for creating growth-hacked social apps, humorously commented on X that it was “the end of the world” for such apps. Security experts and privacy advocates, however, have welcomed the change. Security firm Mysk noted that the update would be “sad news for data harvesting apps” and praised Apple for improving user privacy.

Contact Access Button

In addition to the new permissions process, iOS 18 introduces the Contact Access Button. This feature allows apps to display search results for contacts they do not have access to, with an “Add” button next to each contact’s name. Users can then choose which contacts to provide to the app on a one-by-one basis, further enhancing control and privacy.

Apple’s iOS 18 update represents a significant shift in how apps can access and use users’ contacts. By introducing a two-step permissions process and the Contact Access Button, Apple is prioritizing user privacy and addressing long-standing concerns about data harvesting and aggressive growth tactics. This update ensures that users have greater control over their personal information, setting a new standard for privacy in mobile applications.

Apple Watch Adds Long-Requested Feature

At every keynote, there’s always a mix of big announcements and smaller, highly anticipated updates that catch the audience’s attention. This year at WWDC 2024, the most significant buzz was around Apple Intelligence. However, sometimes it’s the smaller, seemingly minor updates that generate the most excitement.

During the event, the introduction of the Calculator app for the iPad elicited one of the most enthusiastic responses, even before its impressive features were showcased. But for me, the standout update came during the watchOS segment.

A Long-Awaited Feature for Health Enthusiasts

As someone who has faced health challenges, including contracting COVID-19 four times, I have been eagerly waiting for a particular Apple Watch feature. The ability to pause your activity rings is a game-changer. It might seem minor to some, but for those who are dedicated to tracking their fitness metrics, it’s a significant update.

Previously, if you were sick, traveling, or simply unable to meet your daily exercise goals, your activity streak would be broken. Now, with watchOS 11, you can pause your activity rings. This means that if you’re unable to meet your daily goals due to illness or other reasons, you won’t lose your streak.

How the Pause Feature Works

The new feature offers flexibility in how you pause your activity rings. You can pause for a single day, specific days of the week, or even for entire months. During the pause, the system will still collect your metrics, but if you don’t close your rings, your streak remains intact.

Customizing Your Goals

In addition to the pause feature, watchOS 11 introduces further customization options for your activity rings. You can now adjust your exercise goals based on the day. For instance, if you use Sunday as a recovery day from marathon training, you can lower your exercise goal for that day. This adjustment remains until you decide to change it again.

Personal Relief and Excitement

For me, the excitement is personal. The ability to pause my activity rings means I no longer need to stress about getting in 30 minutes of exercise while battling a cold or COVID-19. This update offers a much-needed reprieve for health-conscious users who want to maintain their streaks without compromising their well-being.

In summary, while Apple’s major announcements like Apple Intelligence drew the most headlines, it’s these thoughtful, user-focused updates in watchOS 11 that can make a real difference in everyday life.

Apple’s New Features Borrowed from Third-Party Apps

Apple has been busy unveiling a plethora of new features for its devices during the WWDC 2024. Interestingly, many of these features were already available through third-party apps. This trend, often referred to as “sherlocking,” is when Apple introduces features that mimic those offered by existing third-party apps, effectively making them redundant.

What Does “Sherlocked” Mean?

The term “sherlocking” dates back to the late ’90s when Apple launched Sherlock, a search app for macOS 8. Sherlock could search the web and local files, much like a $29 app called Watson by Karelia Software. When Apple released Sherlock 3 with features similar to Watson, it rendered Karelia’s app unnecessary. Since then, “sherlocking” has referred to Apple integrating features from third-party apps into its own software.

Sherlocked Features at WWDC 2024

New Password App

Sherlocked Apps: 1Password, LastPass, BitWarden, Proton Pass

Apple introduced a standalone password manager for iOS 18, iPadOS 18, macOS Sequoia, visionOS, and Windows. While Apple has had a password manager, it was previously buried in the Settings app. This new app could reduce the need for third-party password managers, although these still offer cross-platform support and can store additional information like addresses and credit cards.

Call Recording and Transcription

Sherlocked Apps: TapeACall, Truecaller

Apple added native call recording and transcription to the Phone app. Previously, apps like TapeACall and Truecaller required users to call another number to activate recording due to Apple’s restrictions on accessing its calling stack.

Voice Transcription

Sherlocked Apps: Otter, AudioPen, Voicenotes.com

With the upcoming updates, Apple will offer voice transcription in apps like Notes. While third-party apps like Otter excel in transcribing meetings and video calls, and AudioPen and Voicenotes.com offer AI-supported summaries, Apple’s integration could challenge their dominance.

iPhone Mirroring on Macs

Sherlocked App: Bezel

Apple’s updates will enable iPhone mirroring on Macs, allowing users to control their phones remotely and seamlessly transfer files. Bezel, which required a cable connection for mirroring, may now face obsolescence.

Window Tiling on Mac

Sherlocked Apps: Magnet, Rectangle

Apple is introducing window tiling in macOS Sequoia, a feature long available on Windows and Linux. While this will reduce the need for apps like Magnet and Rectangle, these third-party apps may retain users by offering advanced customization features.

Custom Emojis

Sherlocked App: Newji

Apple will allow users to create custom emojis using prompts and its new AI engine. This functionality mimics what Newji already offers, potentially impacting its user base.

Custom Maps

Sherlocked App: AllTrails

Apple Maps will soon let U.S.-based users explore national park trails, create custom routes, and save them for offline use, features currently provided by AllTrails. However, AllTrails’ established community and broader geographic coverage might help it stay relevant.

The Future for Sherlocked Apps

Apple has to balance adding competitive features while maintaining a thriving App Store. Some sherlocked apps might survive by offering specialized or enterprise solutions, but smaller indie apps may struggle to retain users. Interestingly, Apple’s new Writing Tools suite, powered by its AI, now competes with numerous email and note-taking apps and even Grammarly by offering proofreading features.

While Apple’s integration of these features can enhance user experience, it poses challenges for third-party developers. As Apple continues to evolve, the landscape for app developers will need to adapt to stay relevant amidst the tech giant’s expanding capabilities.

Apple’s Approach to Generative AI

Among the biggest questions surrounding models like ChatGPT, Gemini, and Midjourney since their launch is what role (if any) they’ll play in our daily lives. Apple aims to address this with its own generative AI, Apple Intelligence, officially unveiled at WWDC 2024.

A Pragmatic Approach to AI

Apple’s approach to generative AI is distinct from the “bigger is better” philosophy adopted by many tech giants. Instead of creating a vast, all-encompassing AI, Apple Intelligence is designed to be a more bespoke solution, tailored specifically to enhance the user experience within Apple’s ecosystem. This means seamless integration into iOS, macOS, and other Apple operating systems, ensuring that the AI works effortlessly behind the scenes.

Keeping Models Small and Focused

One of the key strategies Apple employs is developing smaller, more focused models. These are trained on customized datasets that align closely with the specific functionalities needed by Apple users. By limiting the scope, Apple aims to improve transparency and user trust, as the system’s decisions are more predictable and easier to understand. This targeted approach also helps in maintaining high standards of privacy and security.

Third-Party Integration and User Consent

Apple acknowledges the limitations of its models by allowing integration with third-party AI systems like OpenAI’s ChatGPT and Google Gemini. When a query falls outside the scope of Apple’s models, users can opt to share information with these external systems. This opt-in requirement ensures users are aware and in control of their data sharing, aligning with Apple’s commitment to privacy.

On-Device vs. Cloud Processing

Whether processing occurs on-device or via remote servers using Private Cloud Compute, Apple maintains consistent privacy standards. Users may not always know where the processing happens, but they can rest assured that their data is handled securely. Apple’s philosophy is to prioritize efficiency and accuracy, leveraging cloud computing when necessary but defaulting to on-device processing whenever possible.

Handling of Datasets

Apple’s models are trained using a combination of licensed datasets and publicly accessible information gathered by AppleBot. Publishers can opt out of having their content used for AI training while still allowing it to appear in search results. This nuanced approach respects both user privacy and publisher preferences.

Principles of Responsible AI

Apple has outlined key principles guiding its AI development:

1. Empower Users with Intelligent Tools: Providing tools that responsibly address specific user needs while respecting their goals.
2. Represent Users Authentically: Continuously working to avoid biases and stereotypes.
3. Design with Care: Taking precautions at every stage to prevent misuse and potential harm.
4. Protect Privacy: Ensuring that private data and user interactions are not used in model training.

A Balancing Act

Apple’s bespoke approach to AI aims to offer a frictionless user experience without compromising privacy. This balance will be crucial as the current OS betas transition to general availability. Apple aims to provide varying levels of information based on user interest, ensuring transparency where possible while maintaining ease of use.

As privacy advocates and curious users demand more transparency, Apple strives to make its processes as open as possible. The goal is to provide a seamless yet secure experience, ensuring users trust and rely on Apple Intelligence in their everyday interactions.

Apple Intelligence: Practical AI for Everyday Use

Artificial intelligence (AI) has been generating excitement and controversy in equal measure. AI chatbots often “hallucinate,” generating incorrect information confidently, and AI enhancements to services like Google Search and Microsoft’s Recall feature have faced significant issues. In this context, Apple’s launch of an AI-powered iPhone with iOS 18 is notably cautious and pragmatic.

A Practical AI Approach

At WWDC 2024, Apple introduced “Apple Intelligence” as part of iOS 18, focusing on practical and useful AI features rather than overwhelming users with too many capabilities. This restrained approach aims to avoid disrupting the carefully curated user experience Apple is known for.

Integrating AI Into Everyday Apps

Instead of grandiose AI features, Apple has integrated Apple Intelligence into everyday apps and functionalities. These include writing assistance, proofreading tools, AI summaries and transcripts, prioritized notifications, smart replies, improved search, photo editing, a smarter Siri, and a refined “Do Not Disturb” mode that filters important messages.

For example, AI-enhanced photo editing tools can remove unwanted objects or people from pictures, and users can create AI images based on conversation contexts, ensuring safe and appropriate use.

Apple Intelligence in Action

Apple Intelligence’s features are designed to be practical and user-friendly:

1. Writing and Proofreading: Apple Intelligence helps with grammar and spelling errors, rewriting text in different styles, and providing common response suggestions.
2. Photo Editing: Basic photo edits like removing unwanted objects or people are made easier with AI tools.
3. Notifications and Smart Replies: AI prioritizes important notifications and suggests quick replies.
4. Enhanced Search: Improved natural language search capabilities within apps and photos.
5. Do Not Disturb: An intelligent “Do Not Disturb” mode that filters essential messages.

Seamless User Experience

Apple’s approach ensures that these AI features feel like intelligent tools rather than intrusive AI elements. By focusing on specific, solvable problems, Apple aims to provide reliable results and limit potential AI misuse and safety concerns. This approach contrasts with more ambitious AI efforts that can lead to unexpected and often problematic outcomes.

Responsible AI Integration

Apple’s AI integration carefully balances offering user guidance and providing a source of independent creation. For instance, while Apple Intelligence can help make writing more concise or summarize emails, creating extensive content like a bedtime story may require asking ChatGPT for assistance.

When it comes to image creation, Apple Intelligence relies on contextual understanding to ensure appropriate usage. For instance, creating images in a texting conversation or adding visuals in apps like Keynote, Pages, and Freeform will be guided by user interaction and limited to specific styles.

Leveraging AI for Specific Needs

Apple’s AI features are often in the background, enhancing existing apps rather than taking center stage. This approach emphasizes solving everyday problems and providing practical assistance without overshadowing the core functionalities of the device.

Apple Intelligence’s success lies in its practicality and seamless integration into the user experience. By focusing on useful, everyday applications and maintaining strict guardrails, Apple ensures that its AI enhancements are both functional and safe. This practical and less intrusive approach to AI might be why Apple Intelligence works effectively, providing value without overwhelming users.

Apple Intelligence will be available in beta this fall, offering a blend of practical AI solutions and smart tools designed to enhance the overall user experience on Apple devices.

Updates to Apple’s Messages App

The most significant updates coming to Apple’s iMessage and Messages app in iOS 18 aren’t flashy new features like AI emoji or Genmoji, nor the ability to send texts via satellite. The real game-changers are the ability to schedule messages for later delivery and support for RCS, the next-gen messaging standard set to replace SMS and greatly improve texting with Android users.

Though these updates were somewhat buried amidst the numerous AI announcements and user interface tweaks at WWDC 2024, they are poised to make a significant impact on everyday communication.

Scheduling Messages

For years, users have demanded the ability to schedule messages, leading to the creation of various workarounds, such as apps that remind you to send a text or solutions that only worked on jailbroken iPhones. With iOS 18, scheduling messages to be sent later will become a built-in functionality.

This feature, briefly mentioned at WWDC, was tucked into Apple’s press release alongside the Tapbacks upgrade, which now includes support for any emoji or sticker in iOS 18. Despite its low-key announcement, the ability to “Send Later” will be a welcome addition for those who run businesses from their iPhones or those who remember important messages in the middle of the night. This functionality will also allow users to send messages across different time zones without disturbing recipients or ensure they don’t miss sending birthday greetings.

RCS Support

Even more transformative is the Messages app’s support for RCS (Rich Communication Services), the modern messaging standard set to replace SMS. This upgrade addresses many issues iPhone users face when texting Android users, such as the lack of typing indicators, read receipts, and high-quality media in group chats. RCS will also improve reliability and media quality compared to SMS and MMS.

For a long time, Google has pushed Apple to adopt RCS to enhance communication between Android and iOS users. Media coverage and regulatory scrutiny have also played roles in Apple’s decision. For instance, a Wall Street Journal report highlighted the green vs. blue bubble debate among U.S. teens, and while EU regulators decided iMessage wasn’t popular enough to force interoperability, U.S. lawmakers’ interest in Apple’s shutdown of Beeper, an app that brought iMessage to Android, added pressure.

Due to Apple’s previous refusal to support RCS, messaging between iPhone and Android users lacked key features like typing indicators, read receipts, and end-to-end encryption, resulting in broken group chats and low-quality media. With the upcoming support for RCS, these issues will be addressed.

However, RCS messages will still display as green bubbles on Apple devices, maintaining the visual distinction between iMessage and other messaging standards. Screenshots on Apple’s website show the text box indicating “Text message + RCS” in light gray font, while the messages themselves remain green.

While Apple will support RCS later this year, initial reports suggest that end-to-end encryption (E2EE) will not be available out of the gate. Apple intends to work with the GSMA to add E2EE support to the Universal Profile for RCS, but this feature wasn’t mentioned in the initial announcement, indicating it might come later.

These updates to Apple’s Messages app will significantly enhance the user experience, making communication smoother and more reliable. The ability to schedule messages and the support for RCS will address longstanding issues, ensuring Apple stays competitive in the messaging space. Though these features might not have received the spotlight during the keynote, they are sure to be among the most appreciated by users when iOS 18 rolls out later this year.

Enhancements to the App Store

Apple’s keynote at WWDC 2024 showcased a plethora of updates, including new platform features and the introduction of Apple Intelligence, their generative AI offering. As a developer conference, it also focused heavily on tools designed to enhance app revenue and promotion capabilities for developers. Notably, Apple is introducing win-back offers, improved search suggestions, and a new commerce API.

Win-Back Subscriptions

Apple is enabling developers to re-engage customers who have previously subscribed to their apps by offering targeted win-back deals. Developers can specify which users to target, and Apple will display these offers prominently in the App Store, making it easy for users to redownload the app through these promotional cards.

“Eligible customers can discover win-back offers across the App Store, including on your product page and in editorial selections on the Today, Games, and Apps tabs, as well as within your app,” Apple explained.

Improved Search Suggestions

Later this year, Apple will enhance the App Store’s search functionality by adding Focus State. This feature will surface recent searches and provide personalized search term recommendations, improving the search experience for users.

Commerce APIs

Apple is also set to introduce new APIs designed to handle “complex monetization” scenarios. These APIs will benefit apps offering subscriptions, those with extensive content catalogs, and digital content that spans multiple apps managed by a single developer.

Featured App Nominations

Developers can now notify Apple’s App Store editorial team when launching a new app or major update. This increases their chances of being featured on various App Store pages. Developers can highlight specific content such as in-app events, preorders, or significant updates.

Additional Updates

• TestFlight Enhancements: Developers can now define criteria for users eligible to join their TestFlight, offering a more targeted testing phase. Apple is also updating the design of TestFlight invitations and introducing analytics for public TestFlight links.
• One-Time and Custom-Use Codes: Later this year, developers will be able to generate these codes to attract users to their macOS apps.
• Custom Product Pages in Apple Search Ads: Developers will soon be able to link their custom product pages on the App Store directly in Apple Search Ads.

These updates are aimed at giving developers more control over their app promotions and monetization strategies, ultimately helping them to reach and retain more users effectively.

iOS 18 Device Compatibility

Apple’s WWDC 2024 was filled with exciting announcements, particularly about iOS 18, the upcoming version of the iPhone operating system. This update introduces numerous new features, including enhanced customization options, the ability to arrange apps and widgets on the Home Screen, customizable buttons, new text effects, the ability to lock and hide apps, improved Mail inbox management, iMessages over satellite, a redesigned Photos app, and more.

Compatible Devices

Wondering if your iPhone supports iOS 18? Here’s the list of models compatible with the new update:

• iPhone 15
• iPhone 15 Plus
• iPhone 15 Pro
• iPhone 15 Pro Max
• iPhone 14
• iPhone 14 Plus
• iPhone 14 Pro
• iPhone 14 Pro Max
• iPhone 13
• iPhone 13 mini
• iPhone 13 Pro
• iPhone 13 Pro Max
• iPhone 12
• iPhone 12 mini
• iPhone 12 Pro
• iPhone 12 Pro Max
• iPhone 11
• iPhone 11 Pro
• iPhone 11 Pro Max
• iPhone XS
• iPhone XS Max
• iPhone XR
• iPhone SE (second generation or later)

Availability

iOS 18 will be available this fall as a free software update. For those eager to try it out sooner, the public beta will be available next month at beta.apple.com.

It’s important to note that some features may not be available in certain countries, languages, or devices. For example, the Safari “Highlights” feature, which automatically detects relevant information, will only be available in English in the U.S. Similarly, the new live audio transcription capability in the Notes app will be accessible on iPhone 12 and later in English in Australia, Canada, Ireland, New Zealand, South Africa, the U.K., and the U.S.

Apple Intelligence Compatibility

Apple Intelligence, the company’s new generative AI offering, will be integrated into most of Apple’s apps. It will help compose emails and respond to text messages with Smart Replies, among other functionalities.

The best experience with Apple Intelligence features will be on the iPhone 15 Pro or iPhone 15 Pro Max, thanks to their A17 Pro chip. The feature will also be available on iPad and Mac devices with M1 and later.

Apple Intelligence will be available in beta this fall.

Top AI Features Announced by Apple at WWDC 2024

Apple’s WWDC 2024 kicked off with a bang, showcasing some of the most significant updates and features coming to its devices, apps, and software. This year’s event was especially noteworthy due to Apple’s substantial investment in generative AI technology, leading to numerous exciting announcements. From an upgraded Siri to AI-generated emoji, here’s a roundup of the most noteworthy AI features revealed at WWDC 2024.

New Siri

Siri received a significant upgrade thanks to Apple’s generative AI push, branded as Apple Intelligence. With this update, Siri now sports a revamped look, including a new icon and a glowing indicator light around the screen’s edges. More importantly, Siri can handle speech stumbles better and understand context more effectively. Users can now type to Siri and receive answers to questions about using their iPhone, iPad, or Mac.

Soon, Siri will gain onscreen awareness and the ability to take actions across apps. For example, you could ask Siri to “make this photo pop” and then “add this photo” to another app. During the WWDC keynote, Apple demonstrated Siri finding a photo of a driver’s license, extracting the ID number, and entering it into a web form.

To utilize the new Siri features, users will need an Apple device that supports Apple Intelligence, specifically the iPhone 15 Pro or devices with M1 or newer chips.

ChatGPT Integration

Apple is integrating ChatGPT into Siri and other first-party apps across its operating systems. Siri users will soon be able to direct questions to ChatGPT for “expertise” when needed. Users can include photos with their questions or ask questions related to documents or PDFs.

Additionally, ChatGPT will be integrated into OS-wide tools like Writing Tools, powered by Apple Intelligence. This integration allows users to create content with ChatGPT, including images, or ask for revisions or variations on initial ideas.

These ChatGPT integrations will be available on iOS 18, iPadOS 18, and macOS Sequoia later this year, free of charge and without requiring a ChatGPT or OpenAI account. Initially, these features will be powered by GPT-4, OpenAI’s latest generative AI model.

Genmoji and Image Playground

Genmoji, coming soon to iOS 18 on devices supporting Apple Intelligence, lets users create AI-generated emoji-like images of anyone in their photo library or custom emoji. These Genmoji can be used as stickers for reacting to messages or inline within messages.

Another new image-generation capability allows users to create AI images of people they are messaging with. Apple Intelligence understands the context of conversations, enabling personalized chat with custom AI images.

The Image Playground feature, available as a standalone app and API for developers, allows users to create images using themes, costumes, accessories, places, and more. Users select the desired themes, and Image Playground generates a preview of the image.

AI Photo Editing

Apple introduced a new Clean Up tool in the Photos app, which removes unwanted people and objects from pictures. By circling or highlighting the elements to be removed, Clean Up uses AI to replace them with contextually-aware pixels, making it seem like they were never there.

Additionally, the Photos app is now better organized, thanks to AI. In iOS 18, Photos will automatically organize collections by topics like time, people, favorite memories, trips, and more. Users will also be able to use more specific terms to search across photos.

Transcribed Calls

Coming soon to the iPhone 15 Pro and newer models, iOS will offer optional recording and transcription of phone calls. This feature, which must be enabled manually and informs the other party that the call is being recorded, transcribes the conversation and provides a summary of the key points in the Notes app.

Apple also noted that users would be able to record and transcribe audio from within the Notes app.

These updates underscore Apple’s focus on enhancing user experience through practical and reliable AI features, ensuring that the technology serves users’ needs efficiently and effectively.

Apple Confirms Future Partnership with Google’s Gemini

At the WWDC 2024 keynote, Apple introduced Apple Intelligence and announced a significant partnership that brings ChatGPT access to Siri through a deal with OpenAI. Following the keynote, Senior Vice President Craig Federighi confirmed that Apple plans to work with additional third-party models, specifically mentioning Google’s Gemini.

“We’re looking forward to doing integrations with other models, including Google Gemini, for instance, in the future,” Federighi stated during a post-keynote discussion. However, he emphasized that there is “nothing to announce right now, but that’s our general direction.”

OpenAI Integration

OpenAI’s ChatGPT will be the first third-party model to integrate with Apple’s ecosystem, expected to be available later this year. Apple confirmed that users would access ChatGPT without needing to sign up for an account or pay for premium services. The integration with the updated iOS 18 version of Siri is designed to maintain user control and privacy.

Federighi elaborated on this process, explaining, “Siri, it’s really important to understand, will ask you before you go to ChatGPT. Then you can have that conversation with ChatGPT. If there’s any helpful data referenced in your question that you might want to supply to ChatGPT, we’re going to ask, ‘Do you want to send this photo?’ From a privacy point of view, you’re always in control and have total transparency.”

Looking Ahead

While OpenAI’s ChatGPT is the initial focus, Apple’s acknowledgment of future collaborations with models like Google Gemini signifies a broader strategy to enhance AI capabilities across its devices. This direction underscores Apple’s commitment to integrating advanced AI while prioritizing user privacy and control.

Elon Musk’s Reaction to Apple’s AI Moves

Elon Musk is threatening to ban iPhones from all his companies over the newly announced OpenAI integrations Apple revealed at WWDC 2024. In a series of posts on X, the Tesla, SpaceX, and xAI executive expressed concerns that integrating OpenAI’s ChatGPT at the OS level would compromise user privacy. Musk stated that if Apple proceeds with this integration, Apple devices would be banned from his businesses, and visitors would have to check their Apple devices at the door, where they’ll be “stored in a Faraday cage.”

Musk’s Concerns and Reactions

Musk’s posts suggest a misunderstanding of the relationship Apple announced with OpenAI, or at least an attempt to cast doubt on user privacy. Both Apple and OpenAI clarified that users are asked for permission before any questions, documents, or photos are sent to ChatGPT. Despite this, Musk believes OpenAI is deeply integrated into Apple’s operating system, potentially accessing personal and private data.

In iOS 18, Apple explained that Siri could route questions to ChatGPT if it deemed the AI could provide a helpful response. Users would be asked for permission to share their questions and receive answers directly within Siri, eliminating the need to open the ChatGPT app. Photos, PDFs, or other documents shared with ChatGPT would follow the same permission process.

Musk’s Preference for App-Based AI

Musk prefers that OpenAI’s capabilities remain confined to a dedicated app, not integrated into Siri. Responding to Sam Pullara, CTO at Sutter Hill Ventures, who explained that users approve each request individually and OpenAI does not have system-wide access, Musk wrote, “Then leave it as an app. This is bullshit.”

Pullara likened the integration to the current functionality of the ChatGPT app, emphasizing that on-device AI models are either Apple’s or those using Apple’s Private Cloud. Musk, however, remained unconvinced, arguing that Apple’s integration with OpenAI compromises user privacy.

Additional Criticisms and Apple’s Response

Musk responded to various posts, including one by YouTuber Marques Brownlee explaining Apple Intelligence, saying, “Apple using the words ‘protect your privacy’ while handing your data over to a third-party AI that they don’t understand and can’t themselves create is not protecting privacy at all!” He even threatened to ban Apple devices from his companies if Apple CEO Tim Cook didn’t “stop this creepy spyware.”

Apple’s announcement detailed another integration allowing system-wide access to ChatGPT within Writing Tools. Users can ask ChatGPT to compose text or generate images in various styles. This integration provides access to ChatGPT’s capabilities without needing to create an account, which is expected to drive a significant influx of requests from Apple users.

Privacy and User Control

Apple emphasized that users’ requests and information are not logged, and ChatGPT subscribers can connect their accounts to access paid features within Apple’s AI experiences. Craig Federighi, Apple’s SVP of Software Engineering, assured that users control when ChatGPT is used and are asked before any information is shared.

OpenAI reiterated similar points, noting that requests are not stored by OpenAI and users’ IP addresses are obscured. Users can choose to connect their ChatGPT account, which means their data preferences will apply under ChatGPT’s policies. This optional connection allows for the integration of paid subscription features.

Musk’s objections to Apple’s integration of OpenAI’s ChatGPT highlight ongoing concerns about user privacy and data security in the age of AI. While Apple aims to enhance user experience with AI capabilities, the controversy underscores the need for transparency and user control in implementing such technologies.

Availability of Apple Intelligence Features

At WWDC 2024, Apple introduced a plethora of new AI features under the Apple Intelligence brand. These new features, which include a revamped Siri, writing tools across the system, image generation capabilities, and GenAI-powered emoji, will be available for free. However, only a limited number of devices will have access to these features.

Device Compatibility

The devices compatible with Apple Intelligence features include:

• iPhone 15 Pro
• iPad with M1 or newer chips
• Mac with M1 or newer chips

Unfortunately, older iPhones and the standard iPhone 15 will not support these new AI capabilities.

Availability and Language Support

Apple stated that the Apple Intelligence features would be available to try in U.S. English in beta versions of iOS 18, iPadOS 18, and macOS Sequoia this fall. Some features and additional language support are planned for release in the following year.

Key Features of Apple Intelligence

1. Revamped Siri:
   • Siri has been upgraded to handle speech stumbles better and understand context more accurately.
   • Users can now type to Siri and ask it to perform tasks across apps, such as enhancing a photo and then adding it to another app.
2. Writing Tools:
   • Integrated system-wide, these tools can help compose, proofread, rewrite, and summarize text across various content, including emails and notes.
3. Image Generation Capabilities:
   • Users can create AI-generated images for various applications, including messaging and document creation.
   • The new Image Playground feature allows users to generate images based on themes, costumes, accessories, places, and more.
4. GenAI-Powered Emoji:
   • Users can create custom emoji using GenAI, which can be used as stickers or in-line with messages.

While the introduction of Apple Intelligence brings exciting new capabilities to Apple’s ecosystem, the restricted availability to newer devices highlights the ongoing challenge of balancing innovation with device compatibility. Users with the latest iPhone 15 Pro, iPads, and Macs with M1 chips will be the first to experience these advanced AI features, while others may need to upgrade to take advantage of these innovations.

Stay tuned for more updates as Apple continues to roll out these features and expand their availability in the coming year.

ChatGPT Integration in Siri

At WWDC 2024, Apple announced a groundbreaking integration of ChatGPT, OpenAI’s AI-powered chatbot, into Siri and other first-party apps across its operating systems. This collaboration aims to enhance user experiences by leveraging advanced AI capabilities.

“We’re excited to partner with Apple to bring ChatGPT to their users in a new way,” OpenAI CEO Sam Altman said in a statement. “Apple shares our commitment to safety and innovation, and this partnership aligns with OpenAI’s mission to make advanced AI accessible to everyone.”

Siri Gets Smarter with ChatGPT

With this integration, Siri can now tap into ChatGPT for expert advice and information. For instance, if you need meal ideas using ingredients from your garden, you can ask Siri, which will then consult ChatGPT to provide an answer after obtaining your permission.

Users can include photos with their questions or ask questions related to documents or PDFs. Apple has also integrated ChatGPT into system-wide tools like Writing Tools, allowing users to create content with ChatGPT, including generating images or refining initial ideas.

Availability and Features

ChatGPT integrations will be available on iOS 18, iPadOS 18, and macOS Sequoia later this year. Notably, users won’t need to create a ChatGPT or OpenAI account to access these features, which will initially be powered by GPT-4o, OpenAI’s latest generative AI model.

Subscribers to OpenAI’s premium ChatGPT plans will be able to access additional features within Siri and other Apple apps that incorporate ChatGPT. Apple ensures that privacy protections are a priority, stating that requests are not stored by OpenAI and users’ IP addresses are obscured.

Enhanced Privacy and User Control

Apple emphasized that privacy protections are “built in” to these integrations. Before sending any information to ChatGPT, Siri will ask for user permission. Additionally, requests are not stored by OpenAI, and IP addresses are obscured to protect user privacy.

This partnership marks a significant step in making advanced AI accessible and useful for everyday tasks. With ChatGPT integrated into Siri and other apps, Apple users can look forward to a more intelligent and responsive virtual assistant, enhancing their overall user experience. Stay tuned for more updates as these features roll out later this year.

Apple Intelligence: A New Playground for Images and Emojis

Apple’s latest innovation at WWDC 2024 includes a new AI-powered feature in iOS 18 that allows users to create AI-generated images of people they’re messaging with, functioning similarly to an upgraded version of Bitmoji.

Enhancing Conversations with AI

This new feature aims to make everyday conversations more engaging. Leveraging Apple Intelligence, the system can understand who you’re chatting with and lets you personalize your messages with custom AI images created on the fly.

Image Credits: Apple

For example, Apple showcased an AI Bitmoji of a friend celebrating a birthday, complete with cake, balloons, and flowers. Another example featured a mom in a superhero cape, symbolizing her child’s admiration for her abilities.

Styles and Integration

At launch, the feature will support three illustrative styles: sketch, illustration, and animation. This capability is reminiscent of what a group of former Snap researchers was developing with their startup Easel, a next-generation Bitmoji.

In addition to messaging, the image-generation feature will work across other apps like Notes, Freeform, Keynote, and Pages.

Introducing Genmoji

Later in the keynote, Apple announced Genmoji, a feature that lets users create AI-generated emoji-like images from their Photos library or custom emojis. These Genmojis can be used as stickers for reacting to messages with a Tapback or inline with your messages.

Image Playground

Apple also introduced Image Playground, available as a standalone app and an API for developers. This tool allows users to create images using concepts such as themes, costumes, accessories, and places. Users can select themes and see a preview of their image moments later.

Image Credits: Apple

There’s no need to engineer the perfect prompt. Users can swipe through image previews and create more images by selecting different styles like sketch, animation, or illustration. Apple Intelligence will also suggest concepts related to your conversation, all taking place on the device to ensure privacy.

Image Credits: Apple

Beyond Image Generation

In addition to generating images, Apple Intelligence will enhance functionality across apps. For example, users can ask it to pull up files they interacted with last week, find photos of specific people, or play a previously recommended podcast.

Apple Intelligence’s understanding of apps and actions will make it more powerful, providing personal context to interactions by understanding Calendar appointments, traffic information from Maps, and more.

Alongside these AI innovations, Apple also announced updates to Siri and introduced macOS Sequoia at WWDC 2024. These advancements demonstrate Apple’s commitment to integrating practical and engaging AI features into their ecosystem, enhancing user experience and personalization.

New Features for Apple TV+

At the recent WWDC 2024, Apple unveiled a new feature for Apple TV+ called InSight, designed to enhance the viewing experience by providing additional information about actors and music in real-time. This feature is reminiscent of Amazon’s X-Ray but with added functionality tailored to Apple’s ecosystem.

What is InSight?

InSight allows viewers to learn about actors and songs as they appear on the screen. Users can bring up actor names and character information in real-time, along with background and filmography details for specific actors. Additionally, InSight includes a Shazam-like feature that identifies songs playing in TV shows or movies, allowing users to add these songs to their Apple Music playlist.

Image Credits: Apple

How to Use InSight

Apple TV+ subscribers can use their remote to activate InSight, which will display a sidebar with actor names and character details. By selecting an actor, viewers can access an extended bio and filmography. For songs, InSight will highlight the track playing and provide an option to add it directly to an Apple Music playlist.

tvOS 18 Updates

Alongside InSight, Apple announced several updates to tvOS, the operating system for Apple TV:

• Enhanced Dialogue: Previously available only with HomePod speakers, this feature is now extended to TV speakers, receivers, AirPods, and other Bluetooth devices. It uses machine learning and computational audio to improve vocal clarity over background noise, music, and action scenes on Apple TV 4K.
• Automatic Subtitles: tvOS 18 will automatically enable subtitles in specific situations, such as when the volume is muted, the language in the show or movie doesn’t match the device language, or when replaying a scene.

tvOS 18 is set to be released later this year.

Additional Announcements

During WWDC 2024, Apple also revealed updates to other areas:

• Photos App in iOS 18: The app will see a significant revamp with new features.
• VisionOS 2: Updates for the Vision Pro were announced.
• Calculator App for iPad: Apple introduced a new calculator app for iPad with enhanced functionalities.

These updates reflect Apple’s continued commitment to improving user experience across its platforms, integrating new AI features, and enhancing the overall functionality of its ecosystem.

Siri Gets Smarter

At Apple’s WWDC 2024 in Cupertino, Siri received a significant upgrade as part of the company’s broader generative AI initiative, dubbed Apple Intelligence. This makeover introduces a more natural, relevant, and personal Siri, complete with a new look.

A New Look and Enhanced Capabilities

Image Credits: Apple

Siri now features a revamped icon and a glowing light that wraps around the edges of your device’s screen when activated. Thanks to Apple Intelligence, Siri can handle speech stumbles and better understand context. Additionally, users can now type their queries to Siri, which can answer questions about how to use various Apple devices like the iPhone, iPad, and Mac.

Onscreen Awareness and App Integration

One of the standout features is Siri’s new onscreen awareness. For instance, if a friend sends you their address, you can ask Siri to “add this address to contact card,” and it will handle the task seamlessly. Siri’s ability to take action in and across apps has been enhanced through the new App Intents API, which allows developers to enable Siri to perform actions within their apps.

Image Credits: Apple

Personal Context and Advanced Actions

Siri’s upgraded capabilities include a deep understanding of personal context, such as messages, calendar events, files, and photos. Apple demonstrated Siri finding a photo of a user’s license, extracting the ID number, and entering it into a web form automatically.

Availability and Future Updates

While the exact release dates for many of Siri’s new features are not yet clear, Apple has announced that Apple Intelligence will be available on the iPhone 15 Pro and devices with M1 or newer chips. More information is expected as WWDC continues.

The new Siri, powered by Apple Intelligence, promises to deliver a more intuitive and integrated experience for Apple users, leveraging AI to enhance everyday tasks and interactions across Apple’s ecosystem.

Introducing Apple Intelligence

Apple introduced its new generative AI initiative called Apple Intelligence. This ecosystem-wide feature promises safety and personalized experiences, aligning with rumors and expectations.

CEO Tim Cook emphasized the importance of grounding AI in personal context, including routines, relationships, and communications. He stated, “Most importantly, it has to understand you and be grounded in your personal context, like your routine, your relationships, your communications, and more. And of course, it has to be built with privacy from the ground up together. All of this goes beyond artificial intelligence. It’s personal intelligence, and it’s the next big step for Apple.”

Deep Integration Across Apple Ecosystem

Apple Intelligence is integrated into iOS, macOS, and visionOS, enhancing user experience across the board. “It has to be powerful enough to help with the things that matter most to you,” Cook added. “It has to be intuitive and easy to use. It has to be deeply integrated into your product experiences.”

SVP Craig Federighi highlighted that Apple Intelligence is grounded in personal data and context, utilizing large language and intelligence models processed locally on Apple silicon. Despite the on-device processing capabilities, some functions still rely on cloud computing via Private Cloud Compute, ensuring privacy for personal data.

Major Siri Update

One of the most significant updates is to Siri, marking its biggest transformation since its launch. The new Siri, integrated deeply into Apple’s operating systems, features a glowing border around the screen and supports typed queries, acknowledging that voice isn’t always the best interface.

App Intents and Multitasking

The new App Intents API allows Siri to take actions within apps, starting with first-party applications and expanding to third-party apps. This integration will enable advanced multitasking, such as scheduling meetings without switching between Calendar, Mail, and Maps.

Integration with Third-Party Models

Apple also announced partnerships with third-party models like OpenAI’s ChatGPT, which will be integrated into Siri and other apps. Users can ask ChatGPT questions through Siri and get responses directly, with privacy controls in place. This integration will be available without requiring a ChatGPT or OpenAI account, though premium features will be accessible for subscribers.

Features and Capabilities

Apple Intelligence includes various features, such as:

• AI-powered writing tools in Mail and third-party apps.
• Genmoji for creating custom emojis.
• Image Playground for generating images within apps like Messages, Keynote, and Pages.
• Smart replies and message composition assistance.

Device Compatibility

Apple Intelligence will roll out with the latest versions of iOS, iPadOS 18, macOS Sequoia, and visionOS 2. However, it will only be available on the iPhone 15 Pro and M1 Mac and iPad devices, leaving out older models.

Developer Tools

Developers can integrate Apple Intelligence into their apps using updated SDKs and APIs. This includes leveraging the Image Playground and AI-powered writing tools, enhancing user interaction within their apps.

Apple’s new AI initiative, Apple Intelligence, promises to revolutionize the user experience by integrating advanced generative AI features into its ecosystem. With a focus on privacy, personalization, and deep integration, Apple aims to provide powerful yet intuitive tools that enhance everyday tasks. The rollout of these features will begin with the latest operating system updates later this year.

Apple’s New Passwords App

Apple unveiled its new Passwords app, designed to streamline the management of passwords across Apple devices. Previously, iCloud users could store and sync passwords, but accessing and managing them was cumbersome, requiring navigation through the Settings app on Mac or iPhone.

A Dedicated Password Management App

With the introduction of the Passwords app, Apple aims to simplify password management. The app will be available with the next major releases: iOS 18, macOS Sequoia, and visionOS 2.

Enhanced Navigation and Features

The Passwords app features a new, user-friendly interface. A column on the left side of the app allows users to navigate their password collection more easily. Users can view all passwords, Wi-Fi passwords (a new addition), passkeys, and codes unrelated to specific websites or services.

Existing Capabilities and Improvements

iCloud Passwords already offer support for shared passwords, verification codes, and a note field for each entry. The new app will enhance these features, making it easier for users to get started with storing their sensitive information securely.

The Passwords app is part of Apple’s continuous effort to improve user experience and security. By providing a dedicated application for password management, Apple hopes to encourage more users to adopt and utilize iCloud Passwords for their sensitive information. The app’s intuitive design and added features will be available with iOS 18, macOS Sequoia, and visionOS 2 later this year.

Smart Script Feature for iPad

Apple introduced a new feature called Smart Script in iPadOS 18, designed to enhance the handwriting experience when using an Apple Pencil in the Notes app.

Improved Handwriting with On-Device Machine Learning

Smart Script leverages on-device machine learning to improve the appearance of handwriting in real-time. As you write, the feature smoothens and straightens your notes, making them more legible while retaining your unique handwriting style. This ensures that your notes look clean and professional without losing their personal touch.

Seamless Writing Flow

The Smart Script feature is designed to help you maintain your writing flow. It quickly fixes mistakes, allowing you to continue writing without interruptions. You can also paste text into Notes, and it will appear in your handwriting style. If you need to add more space to your notes, simply tap and hold with your Apple Pencil. Erasing is just as intuitive—scribble over the text you want to remove.

Additional Announcements

In addition to Smart Script, Apple also announced that the Calculator app would be coming to the iPad, a long-awaited addition for users.

Smart Script in iPadOS 18 represents a significant enhancement for users who rely on the Apple Pencil for note-taking. By improving the legibility and flow of handwritten notes, Apple continues to refine the user experience, making iPadOS an even more powerful tool for productivity and creativity. The new feature will be available with the release of iPadOS 18 later this year.

Calculator App for iPad

One clear sign that you’re at a developer conference: the crowd’s excitement over the announcement of a new Calculator app. This addition garnered the loudest applause so far at WWDC 2024. While the company still has time to unveil more of its intelligence features, the new Calculator app has already made a significant impression.

A Long-Awaited Addition

The Calculator app, a staple of iOS, is finally making its way to the iPad. This version takes advantage of the larger screen to introduce features that couldn’t fit on the iPhone. The standout addition is Math Notes, a feature that does the math for you. This makes the iPad an even more compelling choice for academic use, although some teachers may have reservations about an app that handles all the calculations.

Enhanced Functionality with Apple Pencil

Math Notes allows users to draw formulas with the Apple Pencil, and the on-board AI solves the problem. This functionality is not only innovative but also practical for students and professionals alike. Apple briefly mentioned that this feature would also be available in macOS Sequoia, with both it and iPadOS 18 set to arrive later this year.

Additional Announcements

Alongside the new Calculator app, Apple also introduced visionOS 2 for the Vision Pro and updates to the Photos app.

The new Calculator app for iPad, with its Math Notes feature, is a significant enhancement for users who rely on their devices for academic and professional tasks. By leveraging the iPad’s larger screen and the precision of the Apple Pencil, Apple continues to push the boundaries of what its devices can do. The feature’s inclusion in macOS Sequoia further extends its utility, making it a versatile tool for a wide range of users.

macOS Sequoia

At Apple’s WWDC 2024 in Cupertino, macOS received some significant updates with the introduction of macOS Sequoia. Here are the highlights of the latest version:

iPhone Mirroring

One of the standout features is the ability to mirror your iPhone’s screen on your Mac using the Continuity feature. This allows you to control your iPhone directly from your Mac laptop or desktop. Notifications from your iPhone will appear on your Mac, and iPhone audio will play through your Mac while the iPhone itself remains locked. This feature is particularly useful for demonstrating apps during video calls or presentations, as users can seamlessly switch between Mac and iPhone apps.

New Functionalities and Upgrades

• Automatic Window Tiling: macOS Sequoia introduces automatic window tiling to enhance multitasking.
• Presenter Preview Mode: For video conferencing, a new presenter preview mode with built-in background replacement has been added.
• Upgraded Keychain and Passwords App: The updated Keychain includes a cross-platform Passwords app to help manage credentials more efficiently.

Safari Enhancements

• Highlights Feature: Safari now features AI-powered highlights that automatically detect and display interesting content with a single click.
• Reader Mode: Enhanced with AI-powered summaries and tables of content.
• Viewer Option: This new feature brings videos to the forefront with system video controls.

Gaming Improvements

• Game Porting Toolkit 2: The newest version offers improved Windows compatibility and enhanced debugging tools, making game porting more seamless.

The macOS Sequoia beta will be available this month, with the final release expected in the fall.

iPhone Mirroring with macOS Sequoia

macOS Sequoia introduces a new Continuity feature that allows users to mirror their iPhone screens directly on their Macs. This capability is particularly useful for developers demonstrating apps, or for users whose iPhone is out of reach.

Key Features

• Control and Interaction: Users can control their iPhone from their Mac, using the trackpad to swipe through pages and launch apps. This feature also supports text and input entry via the Mac keyboard.
• Integrated Notifications: iPhone notifications appear alongside Mac notifications, allowing users to interact with them directly on their Mac.
• Audio Integration: iPhone audio will play through the Mac during mirroring.
• Privacy Protection: The iPhone remains locked while being mirrored, ensuring that the physical device is secure.

Practical Use Cases

This feature simplifies tasks like app demos during video calls or presentations by allowing users to seamlessly switch between their Mac and iPhone apps. It also facilitates the use of Mac and iPhone apps together, such as editing a video clip with Final Cut Pro on a Mac and dropping it into a template on an iPhone app.

Additional Announcements

• visionOS 2 for the Vision Pro
• Calculator App for iPad: A long-awaited addition with features that leverage the iPad’s larger screen.

macOS Sequoia is set to bring a new level of integration and functionality to Apple devices, enhancing productivity and user experience across the board.

Messages via Satellite

Apple’s WWDC 2024 keynote unveiled a significant upgrade to its messaging capabilities with the introduction of “Messages via Satellite.” This new feature builds on the emergency SOS functionality that has been invaluable for many users stranded in areas without cellular coverage.

How It Works

Similar to the emergency SOS feature, “Messages via Satellite” allows users to send iMessages and SMS when they have no cellular signal. The process involves locating a satellite to relay the message, requiring the user to point their phone in the right direction, as guided by an on-screen overlay.

Key Features

• Satellite Connection: When no cellular signal is available, users can switch to satellite to send their messages.
• iMessage Encryption: Messages sent via iMessage will maintain end-to-end encryption, ensuring privacy even over the satellite connection.
• Limitations: While text messages can be sent, images are not supported due to higher data requirements.

Image Credits: Apple

Privacy and Security

Apple emphasized that privacy measures will be upheld for iMessages sent via satellite, maintaining their end-to-end encryption. However, this does not extend to SMS (green bubble texts), which do not support encryption and should be considered open and less secure.

Availability

Details on when “Messages via Satellite” will be available, any potential charges, and device compatibility were not provided during the keynote. More information is expected in the coming months.

This feature is expected to be a valuable addition for users in remote areas or situations where cellular coverage is unreliable, enhancing the usability and reliability of iPhone messaging capabilities.

Photos App Enhancements

At WWDC 2024, Apple unveiled a major overhaul of its Photos app with the upcoming release of iOS 18. The updated app brings new navigation, enhanced organizational features, and more intuitive ways to discover and enjoy your favorite photos, including those of friends, family, pets, and trips.

New Design and Navigation

The redesigned Photos app aims to minimize the time spent searching for photos by placing everything within easy reach. The app now features a unified single view, with a photo grid at the top and a library organized by themes below.

Image Credits: Apple

Key Features

• Unified View: The photo grid remains at the top, allowing users to browse specific dates, months, and years. A filter button lets you narrow down the content, including filtering out screenshots.
• Intelligent Organization: Leveraging the new intelligence capabilities in iOS 18, photos are automatically organized into collections based on time, people, favorite memories, trips, and more. This intelligent organization helps users quickly find the photos that matter most.
• Recent Days: This feature orders photos by each day, filtering out clutter like photos of receipts. An autoplaying view at the top of the screen allows you to swipe between recent days to see previous collections.
• Collage View: Users can view their collections as a collage and share them with others with a single tap.
• Favorite People: The new feature groups favorite people together, making it easier to find family photos or pictures of your best friends.

Image Credits: Apple

Enhanced Discovery

• Trips Section: All your travel photos are gathered in one place, with autoplay functionality to relive trips as you browse. You can reorder collections and pin your favorites for easy access.
• Highlight Carousel: Swiping right from the grid brings up a new carousel that highlights your best content in a poster-like view. Favorited photos and other notable pictures will appear here, with the carousel surprising you with new photos to enjoy daily.

Image Credits: Apple

The revamped Photos app in iOS 18 promises a more streamlined, enjoyable, and intuitive experience, helping users better organize, discover, and share their cherished memories.

Tap to Cash

During its WWDC 2024 keynote, Apple unveiled a slew of updates for iOS 18, with one of the standout features being “Tap to Cash.” This innovative addition allows users to make payments simply by tapping two iPhones together.

As the name suggests, Tap to Cash builds on the foundation of Apple Pay’s Tap to Pay feature. Leveraging the device’s NFC functionality, it facilitates the transfer of money without sharing any personal information, enhancing privacy and security.

Apple’s introduction of this feature is notable as it simplifies peer-to-peer payments and could potentially impact the market for standalone payment devices and platforms like Square.

In addition to Tap to Cash, Apple announced several other updates, including the ability to customize icons and the introduction of a new password manager app, further enhancing user experience and security on iOS 18.

iOS 18

Apple unveiled significant updates to Siri, powered by the new Apple Intelligence framework. Siri now supports hundreds of new actions across apps, thanks to the integration of App Intents. This framework, already used for Siri, Shortcuts, and other automation features, allows Siri to perform complex tasks without the user needing to touch their device.

For example, users can now ask Siri to find an old photo from a family trip, enhance it with a “make it pop” command, and add it to a note seamlessly. Third-party developers can also build their own App Intents, expanding Siri’s capabilities even further.

Apple’s prerecorded demos showcased the effectiveness of these new features, which will be interesting to test once available. A notable new interaction method includes the ability to double-tap the multitasking line at the bottom of the phone to initiate a conversation with Siri, which is particularly useful for multi-step requests similar to ChatGPT interactions.

Locking and Hiding Apps

In iOS 18, Apple introduces a new feature that allows users to lock or hide apps, enhancing privacy and security. This feature is particularly useful when users need to hand over their phones temporarily.

• App Lock: Users can lock specific apps, requiring authentication via Face ID, Touch ID, or a passcode if someone tries to access the app.
• Hidden Apps: Users can hide apps they don’t want others to see, with these apps appearing in a hidden, locked folder. While the app lock feature is enabled, information from inside the app won’t appear in search results or notifications.

Customizable Icons and Widgets

iOS 18 brings long-requested customization features, allowing users to set app icons and widgets anywhere on the Home Screen and supporting dark mode icons with different color schemes.

• Freedom to Arrange Icons: Users are no longer restricted by the grid layout. Icons can be moved and placed freely on the Home Screen, allowing for more personalized layouts.
• Dark Mode Icons: Switching to dark mode now provides a dark look for icons. Additionally, users can pull a new customization sheet to tint icons with different colors, with iOS automatically suggesting tint colors that complement the wallpaper.
• Simplified Customization: Previously, iPhone users had to rely on complicated workarounds like using iOS Shortcuts with custom icons. Now, iOS 18 simplifies this process, making it easier to customize the interface.

ChatGPT Integration

Apple also announced the integration of ChatGPT, powered by the GPT-4 model, into Siri and other system-wide interactions. This integration brings enhanced AI capabilities to iOS 18, allowing users to leverage ChatGPT for various tasks without needing a separate account.

• Siri Integration: Users can ask Siri to utilize ChatGPT for specific queries, with permission prompts ensuring privacy and control over shared information.
• System-Wide Writing Tools: ChatGPT is integrated into writing tools across iOS, enabling users to generate, revise, and enhance content seamlessly.
• Future Integrations: Apple plans to integrate with other AI models in the future, broadening the scope of AI-powered interactions.

With these updates, iOS 18 not only enhances Siri’s capabilities and app interactions but also offers greater customization and security features for iPhone users. The integration of ChatGPT and other AI models signifies Apple’s commitment to leveraging advanced AI technologies to improve user experience.

visionOS 2

Apple announced a significant update to visionOS, the operating system for the Vision Pro headset. The upgraded visionOS — visionOS 2 — includes productivity enhancements and new immersive experiences.

Key Features of visionOS 2

1. Spatialized Photos:
   • Leveraging AI and machine learning, visionOS 2 can turn regular photos into interactive, spatial experiences. This feature allows users to transform their existing photo libraries into immersive content.
2. Enhanced Navigation:
   • A new navigation option allows users to switch to the home screen by simply tapping or pull up the control center by flipping their hand. This provides quick access to notifications, shortcuts, and other controls.
3. Travel Mode with Train Support:
   • The travel mode feature now includes support for trains, expanding the functionality initially designed for air travel.
4. Improved Mac Virtual Display:
   • Later this year, visionOS 2 will offer higher display resolutions and sizes for the Mac virtual display capability, including support for mice pairing. This enhancement allows Vision Pro users to view and interact with Mac content in an immersive desktop environment.
5. Developer Tools and APIs:
   • visionOS 2 includes new volumetric APIs for running two apps simultaneously and a new dev kit called TabletopKit for creating games on flat surfaces like board games. There are also new enterprise APIs to support business applications.

Apple Immersive Video and Content Partnerships

Apple announced that its 180-degree 8K video format, Apple Immersive Video, is getting new content partners. These include:

• Red Bull: Developing a sports series.
• Scripted Feature: Apple is working on new scripted content.
• Hardware and Workflow Integrations: Partnerships with Canon and BlackMagic to improve hardware and workflows for capturing and creating immersive content.

Global Availability of Vision Pro

Apple revealed plans to expand the availability of the Vision Pro headset to eight new countries:

• June 28: China, Japan, and Singapore, with preorders starting on June 13.
• July 12: Australia, Canada, France, Germany, and the United Kingdom, with preorders starting on June 28.

CEO Tim Cook’s Remarks

Tim Cook, Apple’s CEO, expressed excitement about the global expansion and the new features in visionOS 2:

• “The enthusiasm for Apple Vision Pro has been extraordinary, and we are thrilled to introduce the magic of spatial computing to more customers around the world.”
• He highlighted the various uses of Vision Pro, including collaboration, reliving memories in three dimensions, watching TV shows and movies in a personal cinema, and enjoying new spatial experiences.

Conclusion

visionOS 2 aims to enhance the user experience with the Vision Pro headset by introducing new AI-driven features, improved navigation, and expanded developer tools. The global rollout will allow more users to experience Apple’s advancements in spatial computing, bringing a new level of interactivity and immersion to their digital lives.

Useful AI, Not Flashy AI

Apple is gearing up for next week’s Worldwide Developers Conference (WWDC), where it is expected to introduce Apple Intelligence, its latest foray into generative AI. The tech world is abuzz with anticipation, particularly given how Google and Microsoft have recently showcased their own AI advancements. However, Apple must tread carefully to ensure it doesn’t overpromise on features, aiming instead for reliability and utility.

Prioritizing Reliability

Apple’s devices are among the most popular globally, and their AI capabilities should enhance this reputation by making these devices more useful. A critical aspect of this utility is the ability to run AI-powered features locally on-device, reducing reliance on cloud-based tools. This could lead to greater user adoption due to the increased speed and availability of these features.

For instance, offline transcriptions in Voice Memo and Notes apps could be a game-changer, enabling users to access AI functionality without an internet connection. Additionally, while features like notification summaries, basic text generation, and photo editing are expected, these functionalities already exist in many browsers, note-taking apps, and photo-editing tools. Apple’s edge will be in how seamlessly and smoothly these features are integrated into its ecosystem.

Maintaining Privacy

Apple has always positioned itself as a privacy-centric company, and this ethos will likely extend to its AI offerings. According to reports, the new AI features will be available only on the iPhone 15 Pro and devices with M1 or later chips, and they will be opt-in. This cautious approach indicates that Apple aims to avoid potential user backlash and maintain its reputation for protecting user privacy.

Recent criticisms, such as those surrounding its controversial iPad “Crush” ad, highlight the need for Apple to be sensitive to user sentiment, especially among creators. Given AI’s mixed reception in the creative community, Apple will likely introduce non-controversial, privacy-focused AI features that enhance user experience without compromising security.

Enhancing Siri

One of the most anticipated updates is a significant revamp of Siri. Currently, Siri struggles with multitasking and delivering accurate responses to user queries. For example, it fails to set multiple timers correctly when asked. While these issues might not require advanced AI to fix, they underscore the need for a more capable and intuitive assistant.

If Siri doesn’t gain deeper access to apps, Apple can still improve the user experience by introducing an AI assistant that helps users create complex Siri shortcuts for multistep tasks. This enhancement would make Siri more functional and versatile.

Strategic AI Partnerships

Rumors suggest that Apple will announce a partnership with OpenAI to integrate AI features across its operating systems. However, given the challenges associated with AI, such as hallucinations and errors, Apple will likely proceed cautiously. The company might limit its reliance on large language models (LLMs) for content generation to avoid issues related to accuracy and bias.

Many companies have faced backlash for overhyping AI features, only to deliver flawed or biased results. Apple’s cautious approach may help it avoid these pitfalls and deliver reliable, user-friendly AI tools.

As Apple prepares to unveil Apple Intelligence, it faces the challenge of meeting high expectations without overpromising. By focusing on reliability, privacy, and incremental enhancements, Apple can ensure its AI features genuinely enhance user experience. The key will be to introduce practical, seamless AI functionalities that make everyday tasks easier and more intuitive, solidifying Apple’s position as a leader in technology innovation.