Update on Recall security and privacy architecture

Overview

As AI becomes more integral to Windows, Microsoft is doing more with AI on the edge with the power of a 40+ TOPS Neural Processing Unit on Copilot+ PCs. This enables lower latency, better battery life for AI intense tasks, use of AI experiences without an internet connection and better privacy by retaining information locally.

Moving models and AI-related data processing onto the PC also creates unique security challenges that need to be accounted for in the product design. This blog outlines the security and privacy models, security architecture and technical controls implemented in Recall (preview), an all-new exclusive experience coming to Copilot+ PCs. Recall is designed to help you instantly and securely find what you’ve seen on your PC.

Security and privacy design principles

1. Recall is designed with security and privacy in mind and built on four principles aligned to the updates announced in June:
   The user is always in control.
   • Recall is an opt-in experience. During the set-up experience for Copilot+ PCs, users are given a clear option whether to opt-in to saving snapshots using Recall. If a user doesn’t proactively choose to turn it on, it will be off, and snapshots will not be taken or saved. Users can also remove Recall entirely by using the optional features settings in Windows.
2. Sensitive data in Recall is always encrypted and keys are protected.
3. Recall services that operate on snapshots and associated data are isolated.
   • Within Recall, the services that operate on screenshots and associated data or perform decryption operations reside within a secure VBS Enclave. The only information that leaves the VBS Enclave is what is requested by the user when actively using Recall.
4. Users are present and intentional about the use of Recall.
   • Recall leverages Windows Hello Enhanced Sign-in Security to authorize Recall-related operations. This includes actions like changing Recall settings and run-time authorization of access to the Recall user interface (UI). Recall also protects against malware through rate-limiting and anti-hammering measures. Recall currently supports PIN as a fallback method only after Recall is configured, and this is to avoid data loss if a secure sensor is damaged.

Recall security model

Recall snapshots and associated data are protected by secure VBS Enclaves. VBS Enclaves use the same hypervisor as Azure to segment the computer’s memory into a special protected area where information can be processed. Using Zero Trust principles, code in these enclaves can use cryptographic attestation protocols to safeguard that the environment is secure before performing sensitive operations, such as snapshot processing. This area acts like a locked box that can only be accessed after permission is granted by the user through Windows Hello. VBS Enclaves offer an isolation boundary from both kernel and administrative users.

Recall snapshots are available only after you authenticate using Windows Hello credentials. Specifically, Windows Hello Enhanced Sign-in Security biometric credentials protect your privacy and actively authenticate you to query your semantic indices and view associated snapshots.

Biometric credentials must be enrolled to search Recall content. Using VBS Enclaves with Windows Hello Enhanced Sign-in Security allows data to be briefly decrypted while you use the Recall feature to search. Authorization will time-out and require the user to authorize access for future sessions. This restricts attempts by latent malware trying to ’ride along’ with a user authentication to steal data.

Recall privacy controls

Recall is always opt-in. Snapshots are not taken or saved unless you choose to use Recall. Snapshots and associated data are stored locally on the device. Recall does not share snapshots or associated data with Microsoft or third parties, nor is it shared between different Windows users on the same device. Windows will ask for your permission before saving snapshots. You are always in control, and you can delete snapshots, pause or turn them off at any time. Any future options for the user to share data will require fully informed explicit action by the user.

Windows offers a rich set of tools to help you control your privacy and customize what gets saved for you to find later in Recall.

• In-private browsing in supported browsers is never saved.
• Users can filter out specific apps or websites viewed in supported browsers.
• Users can control how long Recall content is retained and how much disk space is allocated to snapshots.
• Sensitive content filtering is on by default and helps reduce passwords, national ID numbers and credit card numbers from being stored in Recall. Recall leverages the libraries that power Microsoft’s Purview information protection product, which is deployed in enterprises globally.
• Find something you didn’t mean to save? You can delete a time range, all content from an app or website or anything and everything found in Recall search.
• An icon in the system tray will help you know when snapshots are being saved and makes it easy to quickly pause saving snapshots.

With the Recall controls a user can store as much or as little as they would like and remain in control. Note: Like any Windows feature, some diagnostic data may be provided based on the user’s privacy settings.

Recall architecture

The core components of the Recall architecture are the following:

Secure Settings

A protected data store used within the VBS Enclave, which stores security configuration data for Recall. To make any changes to security-sensitive settings a user must authorize the actions taken within the enclave to prevent malicious tampering. In addition, the settings are secure by default, meaning if tampering is detected they will revert to secure defaults.

Semantic Index

The semantic index converts images and text into vectors for later search. These vectors may reference private information extracted from snapshots, so these vectors are encrypted by keys protected within the VBS Enclave. All query operations are performed within the VBS Enclave.

Snapshot Store

Contains the saved snapshots and associated metadata, including any launch URIs provided by apps integrating with Recall User Activity API, as well as data like the time of the snapshot, title bar string, app dwell times, etc. Each snapshot is encrypted by individual keys and those keys are protected within the VBS Enclave.

Recall User Experience

The UI experience that users leverage to find things they have done on their PC, including timeline, search and viewing specific snapshots.

Snapshot Service

Background process that provides the run time for saving new snapshots, as well as querying and processing data returned by the VBS Enclave.

Recall’s storage services reside in a VBS Enclave to protect data, keys and tampering from malware or attackers operating on the machine. Recall components such as the Recall UI operate outside the VBS Enclaves and are untrusted in this architecture.

Because the Snapshot Service must release information requested by a user by design, a key tenet of the design is to reduce the potential for exfiltration of data outside the normal use of the Recall system.

Processes outside the VBS Enclaves never directly receive access to snapshots or encryption keys and only receive data returned from the enclave after authorization. The authorization period has a timeout and anti-hammering protection that limit the impact of malicious queries. The Snapshot Service is a protected process further limiting malicious access to memory containing the data returned from the query outside the VBS Enclave. Protected processes are the same technology used to protect anti-malware and the Windows LSA host from attacks.

Lastly, the Recall VBS Enclave leverages concurrency protection and monotonic counters to prevent malicious users from overloading the system by making too many requests.

Additional architectural properties that are key to security for Recall:

Bound and verified VBS Enclaves

• Encryption keys used by Recall are cryptographically bound to the identity of the end user, sealed by a key derived from the TPM of the hardware platform and are performed entirely within the trusted boundary of Virtual Trust Level 1 (VTL1).
• Virtualization Based Security (VBS) – the hypervisor provides the secure enclave environment, which loads integrity-verified code into a confidential and isolated TEE.

Recall only operates on Copilot+ PCs

Recall only operates on Copilot+ PCs that meet the Secured-core standard and include the following capabilities by default, which are verified by Recall:

Recall security reviews

In addition to designing and architecting Recall with security, privacy and responsible AI in mind, we have also conducted a set of thorough security assessments of the feature. This includes the following efforts to ensure a thoughtful and secure approach:

•  The Microsoft Offensive Research & Security Engineering team (MORSE) has conducted months of design reviews and penetration testing on the Recall.
• A third-party security vendor was engaged to perform an independent security design review and penetration test.
• A Responsible AI Impact Assessment (RAI) was completed, which covered risks, harms and mitigations analysis across our six RAI principles (Fairness, Reliability & Safety, Privacy & Security, Inclusiveness, Transparency, Accountability). A cohesive RAI Learn and Support document was developed for increasing awareness internally, and external facing RAI content was published to drive trust and transparency with our customers.

Conclusion

Recall’s secure design and implementation provides a robust set of controls against known threats. Microsoft is committed to making the power of AI available to everyone while retaining security and privacy against even the most sophisticated attacks.

We truly believe that security is a team effort. By partnering with OEMs, app developers and others in the ecosystem, along with helping people to be better at protecting themselves, we are delivering a Windows experience that is more secure by design and secure by default. The Windows 11 Security Book is available to help you learn more about what makes it easy for users to stay secure with Windows.

To learn more about Microsoft Security solutions, visit our website. Bookmark the Microsoft Security Blog to keep up with our expert coverage on security matters. Also, follow us on LinkedIn (Microsoft Security) and X (@MSFTSecurity) for the latest news and updates on cybersecurity.

Editor’s note – Sept. 27, 2024: Information about Microsoft’s Purview protection product was updated.