What have we learned about Microsoft Copilot so far?

With Microsoft Copilot being available for over a year, we asked Microsoft MVPs what we’ve learned so far about Copilot. We put all their answers together into a short video montage. You can also check out their extended answers in this blog post to gain more of their awesome insights.

If you put bad prompts in, you’re going to get bad data out – Derek Cash-Peterson

I think Microsoft Copilot is really a productivity tool, right? It helps us do things quicker, faster, more efficiently. My favourite Copilot thing is to summarise meetings because I constantly take bad notes. So having something that can actually go back through and check the things that I said I was going to do or that other people are going to do, I think is really, really valuable. So that’s really my big thing for it.

The challenge is really finding the right prompts and retraining our users and retraining the people using it to be able to use the correct prompts to get the data that you want back.

We have to teach them just like we taught them how to search. So if you put bad prompts in, you’re going to get bad data out. So I think it’s training our users and helping them get their content.

The real challenge is the prompt engineering – Gokan Ozcifci

With Microsoft Copilot now I can actually focus on other stuff that I would basically spend hours doing. Let me give you a few examples. We’re here at the European Collaboration Summit, and making a presentation takes a lot of time. With Microsoft Copilot, I can just create my presentation in a couple of seconds and use it as I wish. I can even let Copilot write a piece of text.

I can use that text in my presentation or even in a professional way with Outlook. My goodness, all those emails I write in a very professional way in my language or in English. It’s phenomenal. With all the Copilot features I can focus on something else rather than the design, or the language, or XYZ.

The real challenge is the prompt engineering. Being able to know how to write your prompt in a good way in order to receive the correct information is one thing. The more you write into your prompt as context, the better the result would be.

And the second thing again, the challenge is the fact check. Being able to know if the answer is correct from the web or from a SharePoint site or OneDrive. That’s another point too.

Every time I touch Microsoft Copilot, I save an hour – Simon Hudson

I think the first thing is people like us, basically the people at the European Collaboration Summit, you know this group we’re kind of automatic evangelists. We just see the opportunities and the possibilities and we love it and get on board with it. I think we’re now realising that actually getting those late majority on board that’s gonna be quite a big ask you know.

There’s a whole new level of training and understanding and enthusiasm that we need to drive to get the adoption. I started talking to companies I work with about the days of IT illiteracy and how we expected people to be able to use a computer and an operating system and Word. Now we need to get AI literacy and actually for advanced companies, they need AI competency. And I think that’s the change, Microsoft Copilot and other AIs look really appealing, but you’ve still gotta put the leg work in to migrate people thinking to use those kind of technologies.

However, every single time I touch Microsoft Copilot, I save an hour. I’ve been measuring it, I save an hour every time, so the gains there are huge. If I could give everyone, my staff and colleagues an hour back in their day, that’s worth a chunk of money. That’s the European productivity gap fixed in one hit.

I deal a lot in the clinical world and so some of the safeguards which are entirely appropriate around things like Copilot are actually a blocker in some spaces. So often I actually do want to go and explore, you know, sexual health issues or very particular other clinical conditions. And it’s really hard to find a mechanism to get Copilot to be able to let me do the kind of content creation or the queries that I want to do against quite sensitive content.

So I think we need to explore that. There needs to be a governed mechanism by which sensitive content which would fall foul of responsible AI is allowed to be re-rooted in certain situations.

I really expect this only to grow – Adis Jugo

In the very beginning Microsoft Copilot very much seemed to me like a solution for which we are we are still looking for a problem. So we have a solution, we are trying to find the problem.

Nowadays, fast forward one year, I’ve seen a lot of use in Copilot for Microsoft 365. The classic ones which you all see is summarizing my meetings. The ones what Vesa showed us yesterday is following the meetings. Those are all really cool stuff, which we do now in Copilot.

So there are really good applications and I really expect this only to grow. I cannot imagine this is going to stay on this level. With this technology we’ve got on our hands right now, I’m really looking forward to what Microsoft Copilot can still do.

The largest challenge I see is people not having proper information architecture, people oversharing. Because if people overshare, you have the situation that, for example, if some files are in SharePoint in some open folder, even if they’re overshared, people are really not going to stumble upon them because they are in some forgotten folder. Maybe some search could find them, but with Copilot those overshared files are becoming incredibly exposed to everybody else, to people who should not know them.

I could be asking, “Hey, what’s my colleagues salary?” I shouldn’t know that. But maybe somebody has forgotten to put proper permissions on that file. So one thing we need to do there is definitely to think about security, permissions, and oversharing, and if we really bring it down to the bottom, we speak about proper information architecture and security over information architecture.

Focus on adoption of Microsoft Copilot – Karoliina Kettukari

The hype around Microsoft Copilot and AI in general is huge. Many organisations are starting to use Copilot very eagerly, but they will see that the usage drops after a while after its initial enthusiasm. So I would advise organisations to really focus on adoption. Inspiring people with use cases, user roles, how to actually use Copilot in their line of work with their everyday processes at work, and then you can really succeed with Microsoft Copilot.

I think one of the challenges is oversharing and not having the big picture. So if you just focus on Copilot as a technical product or a licence, you wont gain the benefits from it, so you need to really address the whole scene of modern work, AI strategy in general, and then you can fit Copilot into that as one stream into your whole strategy.

Microsoft Copilot won’t do all the things you want it to do, for now – Marijn Somers

On learnings about Copilot, on the one hand, that it will probably not do all the things that you want it to do, for now. That it will not take over the world and eradicate all human life, for now. So I think that’s a big learning point that we have.

But also that there are the unknowns that we have like we don’t know what we could have done with it, like last year, there were some business cases. Why would I use ChatGPT or Copilot? But now that has grown exponentially and that learning curve is awesome to see where we can all use it for and how we are using it today and tomorrow.

Make the right prompts – Elio Struyf

I think people are still exploring on what they can do, how to create the right prompts. Get things out of it. I see a lot of people using it for emails, getting summarization. This is the easiest part of getting to know Microsoft Copilot and getting your benefits out of it. But now companies are really exploring Copilot for Microsoft 365 to see what can we do with it and what we can discover. So there’s a lot of security aspects on top of it as well, where people can start thinking. Umm, should I actually say this or shouldn’t I?

The challenges are how to actually use it, if I’m user A, how am I going to use it compared to another user? And there are a lot more technical people in the company that are probably going to use it in a completely different way. So it’s the prompting is what makes it difficult and getting the adoption on creating the right prompts, but Microsoft is providing the tools as well for you with one-click summarization and so on. So they are making it easier but still some of the times people need to make the right prompts in order to get the right data out of it and that’s the most challenging part right now.

Don’t create information pollution – Emily Mancini

Nielsen Norman group, the world leaders in user research, released a great study about using artificial intelligence to write content. And I think it really highlighted that artificial intelligence is helping us identify that there’s some repetitive tasks that we do or some ways that we can start off a little bit faster. So using artificial intelligence to shorten our day and decrease that amount of repetition that we’re going through.

I think one of the biggest mistakes people are making with Microsoft Copilot is thinking that more is more so, while generative AI can be really beneficial, continually expanding your text and content doesn’t necessarily support the outcome objectives of having your end users read the content and understand it when we’re reading on the web, we read only 28 percent of it.

So please be really careful when you generate content and make sure that you only include what is needed and that you’re not creating information pollution.

Would you like to know more?

It was great hearing what Microsoft MVPs had to say about what they’ve learned so far about Microsoft Copilot. You can also hear their thoughts about the the biggest security concerns for IT admins. Thank you to all the Microsoft MVPs for their insights!

If you’re looking for ways to secure and govern Microsoft Copilot, see how Syskit Point can help with a 21-day free trial.