New Microsoft Edge feature uses GenAI to scan PDF documents

Editor’s take: More AI functionality is coming to Edge. A new feature incorporates Copilot into the built-in PDF reader. Microsoft already has a slew of AI-powered features in its browser, but this particular one raises some questions about privacy. The PDF content is likely processed by Microsoft’s servers to generate the keywords and explanations, which means potentially sensitive document information is being transmitted and analyzed externally. Also, it’s reasonable to assume that some level of data gathering will occur to improve the AI model and user experience. Users might want to think twice before using the feature on tax forms or legal documents.

Microsoft has been on a mission to integrate AI capabilities into its browser, opening the door to all sorts of new functionality along the way. It now appears set to introduce a new feature to its built-in PDF reader, one that uses AI to scan the document to look for important keywords and phrases and then provide additional information about them.

Powered by Copilot, the feature can be accessed by clicking a new button located next to the existing “Ask Copilot” button in the PDF reader interface. It starts a scan of the entire PDF document to generate relevant keywords and phrases. Once the smart keywords are generated, users can click on any of them to open the Copilot side panel in the browser, providing more context or information related to that keyword.

This feature was first spotted by X user @Leopeva64, highlighting the community’s role in tracking Edge’s evolution.

Edge’s PDF reader now highlights “smart keywords” in the document, then you can click on each of those keywords to get additional information using Copilot:https://t.co/nTZwkUaebv
.https://t.co/7nn0D2m4ih pic.twitter.com/8TOpzNGgF6

– Leopeva64 (@Leopeva64) August 11, 2024

This new functionality is currently available in the beta version of Microsoft Edge.

To get the beta version of Microsoft’s browser, visit the Microsoft Edge Insider page and download the Beta Channel version – recognized as the most stable preview available.

The feature is just one of several AI-powered capabilities that have been added to Edge. The browser now features a smart find function that suggests related matches and words, making it easier to locate information on a webpage. Users can also create custom browser themes using an AI Theme Generator, which transforms text prompts into visual designs. Another feature uses AI to automatically name groups of tabs, so users can organize their browsing more efficiently.

Now, it appears that Microsoft is working on improving Edge’s ability to suggest websites (using AI), according to the Canary preview build. The exact nature and functionality of these suggestions remain to be seen, as they are not yet available even in test versions, but given Microsoft’s push to incorporate AI into Edge, it is likely that they will appear sooner or later.

Also, while Microsoft has not tipped its hand with plans for expanding the smart keywords feature beyond PDFs to include Word documents, Excel spreadsheets, or PowerPoint presentations, it seems a natural path for Redmond to take. Edge also sports a Compose tab for AI-enhanced writing assistance, and depending on the success of the smart keywords feature for PDFs, Microsoft might well decide to expand it to other document types.