- Microsoft has upgraded Copilot to make the AI assistant a proactive AI companion
- Announced as part of Microsoft’s 50th Anniversary Copilot Event, these new features will start rolling out today
- Features include chatbot memory, a research tool, and on-screen awareness to name just a few
Microsoft Copilot is taking the leap, transforming from a mere generative AI chatbot to a full-blown AI companion. At least, that’s what Microsoft is promising in one of the most significant Copilot updates to date.
The changes, which add a wide array of new capabilities and enhance some existing ones, were announced on Friday, April 4, as part of Microsoft’s Copilot/50th Anniversary celebration.
Granted, many of these features will be familiar to those running Google‘s Gemini and OpenAI’s ChatGPT. In fact, there’s a solid chance that some of these enhancements come directly from Microsoft’s partnership with OpenAI, which gives it access to many of the AI company’s best generative models.
Copilot is, after all, built on the foundation of OpenAI‘s GPT models.
Image 1 of 3
Among the changes Copilot fans will find on Windows (and the iOS and Android apps) is, finally, a memory. The current version of Copilot on Windows 11 can hold onto previous conversations, but it has no recollection of what you discussed. A new prompt asking about a recipe or a trip you were planning with Copilot’s help days or weeks ago, would draw a digital blank stare from the generative AI.
That changes with this update. As you chat it up, Copilot will (with permission, of course) collect key bits of info about you that indicate not just facts but preferences. Microsoft claims the new Copilot will build “richer user profiles.” It now has the potential to “learn who you are deeply.” While that might sound invasive, Microsoft says you can control exactly what you want Copilot to learn and retain.
Image 1 of 3
Personalization and memory are significant benefits but they might be outstripped by Copilot’s new agentic capabilities that Microsoft calls “Actions.” Where the previous Copilot might help you grab concert tickets and get a ride to the big show, or find a new apartment and even movers to help you lug your stuff from one place to the next, Copilot Actions can take a prompt about renting an apartment or booking storage and do all the legwork to make it happen. You’re just entering a prompt.
Of course, it’s not that simple. Copilot’s Action capabilities are not ubiquitous. Microsoft has a small collection of launch partners that includes, OpenTable, Priceline, Vrbo, Viator, Kayak, Expedia, TripAdvisor, Skyscanner, Booking.com, and 1-800-Flowers. It seems that, for now, Actions will only work if you want it to take actions through those apps and services.
Now see this
Image 1 of 2
The update is also making Copilot on Windows and mobile apps more aware. While the current version can look at photos and screenshots uploaded to the AI, the new version can use your mobile phone camera to see what you’re seeing (it’ll look at photos or live video).
Perhaps more interesting is that the Copilot’s Windows app will be able to see onscreen activity within apps. It’ll finally live up to its name, acting as a copilot on file management, Windows settings, and working on projects. It’s the kind of system awareness Apple‘s been promising from Apple Intelligence and Siri for almost a year but has yet to deliver. It’ll be interesting to see if what Microsoft delivers here with Copilot is as good as they’re advertising.
The new Pages capabilities can take disparate project files and organize them into a single Canvas. But Pages appears to be more than an organizer, it can help you brainstorm ideas and then alter and incorporate them into your own projects.
Go deep
If Pages isn’t getting your project there, you might appreciate the new Deep Research. I know, everyone is calling the research portion of their generative AI platforms “Deep [something]”. This one appears to work similarly in that it can help you accomplish multi-step research, acting as a sort of very intelligent research assistant.
Copilot is even delving into podcasts. Similar to what we’ve seen with Spotify generative AI podcasts and Google NotebookLM, it can generate podcasts based on topics of interest or other discussions you’ve had with the AI. Microsoft claims the podcast can play even as you continue to feed Copilot new prompts. It sounds a bit confusing, but also powerful and intriguing.
There’s also a shopping update, turning Copilot into a proactive personal shopper that can watch for deals, do product research, and even handle purchasing.
Finally, Copilot is taking Bing Search in what might best be described as a Google AI Overviews direction, but in a subtle and maybe more palatable way. From the looks of things, Bing search results will offer the option to take a generative deep dive inside traditional search results. It looks pretty good in the demo video, but we’re anxious to see how it works on live Bing search results.
Not all of these updates will show up at once across Windows, iOS, and Android, but many are starting to roll out today. If you already have Windows 11 running Copilot, you may want to download the latest update and give your Copilot a radical AI Companion upgrade.
Image 1 of 2
You might also like
https://cdn.mos.cms.futurecdn.net/ZccKtfPAZUoTPzbzdDXBuS-1200-80.jpg
Source link
lance.ulanoff@futurenet.com (Lance Ulanoff)
