Copilot Agents in SharePoint deep dive for Administrators

One of the biggest announcements during Microsoft Ignite 2024 in Chicago was the General Availability of Copilot Agents for SharePoint. If you have a Copilot license (maybe even if you don’t, more on that in a minute), you have a built-in agent that is only trained on your specific SharePoint site. Given AI is only as good as the content you train it on – this should give you the best answers possible, what is there not to like?

Now let me put my admin hat on for a minute, and I think it’s safe to say that most admins were more scared than excited when Microsoft said on stage that:

• Everyone with edit permission on a site can create SharePoint agents and deploy them to Teams. 
• Each site will already have a pre-built agent available. 
• Users without Microsoft 365 Copilot licenses can use those agents – but in a Pay-as-you-go capacity.

In this blog post, we will first introduce the foundations of Copilot agents in SharePoint and then discuss the back end and administration of it. We will also answer popular questions, such as the one you might have right now: How do you turn it off if needed?

Copilot Agents in SharePoint – Default Agent with every site

Let’s start with making sure we are all on the same page and understand the maker experience – who can do what, and where is it stored?  When you have a Microsoft 365 Copilot license, you can go to any SharePoint site and you will see the Copilot icon at the top right of the screen, just a bit to the left of your name.

Clicking on it will open the Copilot pane, and by default, it will open the agent that came built with the site. Every SharePoint site has a built-in agent.

This agent is sourced on the entire website, meaning it uses all the document libraries in your site. And to answer one of the most popular questions I get – No, Copilot agents in SharePoint cannot be sourced on SharePoint Lists and will not take any information from lists. Copilot agents in SharePoint can only source information from document libraries, folders, or files.

If we go even more specific, Copilot agents in SharePoint support: 

• Office documents: DOC, DOCX, PPT, PPTX, XLSX 
• New Microsoft 365 formats: FLUID, LOOP 
• Universal formats: PDF, TXT, RTF 
• Web files: ASPX, HTM, HTML 
• OpenDocument formats: ODT, ODP 
• Images, Meeting Recordings, Videos, and OneNote notebooks will be supported in the future.  

Now, back to our agent. Given that it’s configured to only be sourced from this site, whenever you use this agent and ask it any question, all the answers will only be based on the information on this site.

Whatever question you ask the agent that has nothing to do with the site will result in the agent gently telling you that it cannot answer this question based on the site’s content.

Copilot Agents in SharePoint – Building your own Agent

Now that we know about the default agent, how do we build our own? You can start by simply clicking on the arrow near the bot’s name in the Copilot pane, and then you will be able to create a new agent.

First, you need to provide information about the agent’s identity, such as name, icon, and description. We then go to our Sources.

By default, the agent will be sourced from the entire site. However, you can change it to be only sourced from certain document libraries, folders, or even specific files.

If you wish, you can even add other sites to this agent’s knowledge sources and then decide whether you want the complete site or only some libraries, folders, or documents from that site.

As a note, when you select a hub site as a knowledge source, the agent automatically includes the hub site’s associated sites as knowledge sources of the agent, so you will get responses based on information from both the hub site and its associated sites.

 

Of course, the agent will respect the permissions of the user using it. So if Vlad configures the agent to get information from Site A and Site B, but a user who only has permissions to access Site A asks a question, the agent will not return anything from Site B. It follows the same permission basics as Search or Copilot BizChat, so you only see the information you have permissions to access.   

In my case, I configured my agent to get information from only two document libraries, and after that, it’s time to go to the Behavior step.

In the Behavior step, we tell our agent how we want it to behave. We can add information such as a welcome message, some Starter Prompts (if someone opens your agent and has no idea what to ask, those prompts will get them started), and even instructions. 

The instructions part is where you tell your agent who the audience is, what format they expect information in, what tone to use, and even, if you wish to, you can tell your agent to finish every sentence with a joke or to talk like a pirate.   

In my instructions, I told my agent to “end each answer with a Microsoft-related joke,” and it delivered! You can test it right away on the right side of the window, and I have to say I love the joke it gave me.

We could also customize Copilot Studio more extensively, but we will discuss those options in another blog post. After we save the agent, we will see it under “Recommended for your section” with our chosen name and icon.

But if you look at it again, you notice we have two sections: the “Approved for this site” section and the “Recommended for you” section. You also notice that Conference Resources has a checkbox near it. 

If we log in as Vanessa, my user that is a site member (edit permissions) and not an owner, and we look at our agents, we see there is only one agent available, and that’s the one in the “Approved” category.  I can create my own agent, but I don’t see “The Vlad Bot.”

Copilot Agents in SharePoint – Configuring approved Agents

As a site owner, I have a special power: I can configure which agents are approved for my site members and viewers to see.  The process is simple: you first must switch over to your agent, go in the properties, and then you see an option to “Set as approved”.

Once I approve the warning, my agent will now be set as approved.

And Vanessa can now see and use our “The Vlad Bot” agent in the agent picker.

Another superpower you have as an administrator is the ability to set an agent as the site default, meaning users open this agent as soon as they click the Copilot icon in the top ribbon.

This process, at least in theory, ensures that Site owners have control over what agents are visible to the users of their site. Now, as with everything SharePoint, there are some exceptions and limitations you need to know about.   

Copilot Agents in SharePoint – How it works on the backend

To better understand some of the administration challenges we have with Copilot Agents in SharePoint, we need to better understand how everything works on the backend. Every time you create a Copilot Agent, you have a .agent file that gets created in the Site Assets > Copilot library of the site. The screenshot below shows what happened before we approved the “Vlad Bot” agent.

Each agent is, in fact, a JSON file, which, if we format it nicely, allows us to see all the configurations we made earlier in the blog post. We can see the conversation starters, the bot description, and the “Capabilities,” which are the sources from which this agent can get information.

And no, Microsoft does not support modifying the JSON file directly. I did not test whether it works, but in a YouTube video I recorded with one of the developers, they confirmed that modifying the JSON is unsupported. Now, if you saw the Approved folder in the screenshot a bit above and are wondering, does SharePoint simply move Approved agents to a folder called Approved? The answer is yes.

There is one scenario where the saved location is different, and this is if the agent is created directly from the context of a document library, in which case the .agent file will be saved directly in the document library it was created from.

This seems like a simple enough way to manage agents, but because it’s so simple, it can introduce some challenges.

Why Copilot Agents in SharePoint can become a nightmare to manage

Most of the time, I am a fan of simple architecture as it makes management easier. But in this case, it can bring you some governance challenges. The challenge in our case is that the Approved folder has the same permissions as the site which means Members have Edit permissions on the Approved folder.

Therefore, Vanessa, my member, can go and move the “Vanessa Agent 2” agent inside the “Approved” Folder, and Vanessa could even fully delete the “The Vlad Bot” agent that we had approved as a site owner.

If I go back to Vlad now, our site owner, I see “Vanessa Agent 2” as Approved, and my “The Vlad Bot” is gone from the list.

In the current implementation of Copilot Agents in SharePoint, we have this problem where if everyone follows the UI, it all works well. However, if a member knows SharePoint a bit, they could potentially create a fake agent with the same name/icon as an Approved one and replace it. Nothing stops you from a permissions point of view.

The only way I found to fix it, is to break permission inheritance on the Approved folder and only give the Members group access to view the folder. This way, Members can still use the agent in the site but not modify it anymore.

This also introduces a limitation (which some might say is an advantage): Even if a member creates an agent, once it’s Approved and visible to everyone, members cannot modify the agent anymore, even if they created it. Only owners can modify those agents.

Now, a question you might have is: Can we just block members from creating agents by breaking permission inheritance on the Copilots folder in Site Assets? This way, only owners can create them. The answer technically is yes, but it’s not as pretty. 

The member would be able to do everything in the Copilot UI, but it would give an error when trying to save the agent, as seen in the screenshot below.

I also want to put a note that, I do not yet know if changing permissions on the Copilot/Approved folders is supported by Microsoft. I reached out to my contacts and waiting for a confirmation at which point we will update this blog post. 

At the time of writing this blog post, there is no way to turn off the “Create an agent” functionality for users, so we don’t have a site-level setting (I wish we did), but we do have a few other tricks up our sleeve.

More Advanced Configurations

We covered settings at the site level, but what if we do not want a site to have any agents at all or its content to even be included in any agents on other SharePoint sites? Luckily, that’s something we can do, and we will need to use a new feature announced at Ignite called Restricted Content Discoverability, or RCD for short.  This feature is included only in SharePoint Advanced Management (SAM),  but don’t worry, at Ignite Microsoft announced that SAM will be included with Copilot for Microsoft 365, so you will have this functionality for free.

I will not deep dive into RCD as a whole in this blog post as it might turn into an eBook from a length perspective, but you can learn more about RCD as a standalone feature in this YouTube video. 

You can turn on RCD on a site by running the following PowerShell cmdlet, but please watch the video or read the full description on Microsoft Learn to fully understand the other impacts outside of Copilot Agents in SharePoint.

Set-SPOSite –identity <site-url> -RestrictContentOrgWideSearch $true

Now, it can take a few days for the setting to fully apply, but after that, the site will not display the Copilot icon in the top bar anymore, as seen in the screenshot below.

Furthermore, if I try to create an agent from another site, I will not be able to select my RCD-protected site.

However, Sanjoyan Mustafi from the SharePoint Advanced Management team confirmed that existing copilot agents created on an RCD-protected site will continue to work as long as you’re on that site. However, if an agent is created on Site A and references Site B, which is RCD-protected, the agent from Site A won’t return any information from Site B.

Licensing for Copilot Agents in SharePoint

Finally, let’s talk licensing! I know licensing is never an admins’ favorite topic, but it is still an important one to know, especially as we start the new year. Let me start by saying that every user with a Microsoft 365 Copilot license will be able to create as many agents as they want and use them as much as they can.

But what about users without a Microsoft 365 Copilot license? Microsoft has a promotional offer started on January 6th, 2025, to enable every employee in your organization with SharePoint agents.

Here are the highlights:

• Applied to companies with at least 50 Microsoft 365 Copilot licenses
• It will enable any employee in the organization will be able to create, share, and interact with SharePoint agents
• There will be a limit of 10,000 queries per calendar month until June 30^th, 2025
  • Once the 10,000 monthly queries are exhausted, users will receive notifications that the service is temporarily unavailable
  • Unused queries will not roll over to the following month.
• According to Message ID MC942833, this will be ON by default, and you need to use PowerShell to disable it.

If you wish to turn off the promotion for your company, you can use the Set-SPOCopilotPromoOptInStatus PowerShell cmdlet. Microsoft shared some images in the Message Center of what the promotion will look like from an end-user perspective. You can see the invitation to use the credits and the experience when you are out for the month.

Conclusion on Copilot Agents

Copilot agents in SharePoint are an amazing tool that enables everyone to create high-quality Copilot agents on their best data. This feature was only released a few weeks ago, and in the future, we will all learn the best practices for governing those agents.

But for now, SharePoint Administrators and owners might need to take extra precautions depending on the knowledge level of their SharePoint power users since they could delete the amazing agents you just created.