How to track and control default (ready-made) and custom SharePoint agents

SharePoint Agents are poised to transform how your teams interact with information. We are nosediving into a world where employees will be able to instantly retrieve content, navigate complex workflows, and automate tasks with a simple natural language query, all within the familiar confines of SharePoint. The allure of enhanced collaboration, reduced support loads, and contextual assistance is undeniable.

But the heavy dose of reality hits when you’re an IT admin grappling with “shadow AI”. This blog will cover the pros and cons of SharePoint agents, the differences between default and custom agents, and how to monitor them. If you’re here for concrete steps on how to manage agents, navigate to this section.

What are SharePoint Agents?

A SharePoint Agent is an AI-powered virtual assistant integrated within SharePoint environments. It interacts with users in natural language to answer questions, retrieve content based on data stored in SharePoint, and connect Microsoft 365 services. These agents enhance productivity by providing contextual assistance directly within SharePoint sites or pages, improving collaboration and knowledge sharing.

Benefits of SharePoint Agents

• Improved productivity: Quickly retrieve relevant content without manual searching. 

• Enhanced collaboration: Facilitate knowledge sharing by answering questions about policies, projects, or documents. 

• Contextual assistance: Provide tailored guidance within specific SharePoint sites or document libraries. 

• Reduced support load: Helpdesk and HR agents can answer common queries instantly. 

• Customization: Agents can be scoped to specific folders, libraries, or sites for focused responses. 

• Natural language interaction: Users can ask questions in plain English making it user-friendly. (You can check the list of all the supported languages here). 

• Centralized information access: By leveraging SharePoint as a central repository, agents help overcome information fragmentation, allowing users to access data from one location. 

Security and business concerns of SharePoint Agents

• Permissions enforcement: Agents respect SharePoint site and file permissions, ensuring users only access content they are authorized to see. 

• Data privacy: Agents do not expose restricted files or sites; if a user lacks access, the agent denies the request. 

• Governance: Agents are represented as .agent files governed by SharePoint permissions, allowing control over who can create, edit, or use them. 

• Auditability: Microsoft Purview logs agent usage, including who interacted with agents and what content was accessed, supporting compliance and security audits. 

• Risk of data exposure: Improper configuration or overly broad agent scopes could risk unintended data access, so careful permission management is essential. 

The difference between Default (ready-made) Agents and Custom Agents

• Default (ready-made) Agents: Every SharePoint site automatically comes with a “ready-made agent” scoped to the content on that site. These agents require no building from site administrators or owners and appear by default, allowing site visitors to chat with them about the site’s content.

• Custom-built Agents: Users with site editing permissions can create custom agents. These allow for greater flexibility by enabling users to change the content scope (e.g., include additional SharePoint sites, specific document libraries, folders, or file types), customize their branding and purpose, and define tailored prompts for their specific needs. Custom agents are represented as .agent files and can be managed like other files in SharePoint.

Aspect	Default (Ready-Made) Agent	Custom Agent
Scope	Automatically scoped to the entire SharePoint site content	Scoped to selected sites, folders, files, or multiple sites
Creation	Pre-created for every SharePoint site, no setup required	Created by users with edit permissions to tailor scope & behavior
Customization	Cannot be customized	Can be customized with specific instructions, tone, and data sources
Agent file	No associated .agent file	Represented by .agent files stored in document libraries
Use case	General site-wide assistance	Targeted, project-specific, or function-specific assistance

The default agent provides broad, out-of-the-box support for the whole site, while custom agents allow precise control to meet specific business needs or workflows.

Monitoring and limits of SP agents for IT Admins

M365 IT admins can monitor and manage SharePoint agents through various methods:

• Manage access: Admins can manage who can access agents through Microsoft 365 Copilot licenses. They can allow or block users from using Copilot experiences on SharePoint on a per-user basis in the Microsoft 365 admin center. (You can also use Syskit Point’s License Overview report to get an overview of licenses, revoke licenses, and perform different actions.)
  
• File permissions on Agent files: Since SharePoint agents are represented as .agent files, permissions on these files govern who can access or edit them.
  
• Restricted Content Discovery Policy: SharePoint Admins can use this policy to turn off all agent-related features on individual sites, preventing the site’s content from being surfaced in Microsoft 365 Copilot or organization-wide search, but only for users who didn’t access it in the last 28 days.
  
• Sensitivity labels and DLP: Microsoft Purview Data Loss Prevention (DLP) in conjunction with sensitivity labels can prevent selected files from being used by agents.
  
• Site owner controls: Site owners can designate specific agents as ‘approved’ to make them more visible to users on their site.
  
• Auditing: SharePoint offers auditing features to track user actions, which can help in monitoring agent activities.

Current limitations

• No tenant-wide disablement for Agents alone: While broader Copilot functionality can be controlled via licensing, a specific tenant-level toggle for only SharePoint Agents has been a requested feature.
  
• Default Agent configurability: The ready-made agent that comes with each site cannot be directly edited or customized.
  
• Limited transparency into Agent data sources: It can be challenging to track what content an agent is referencing, especially when users freely build them and extend their data sources.
  
• Integration complexity: When it becomes available soon, agents will be extended with Copilot Studio, and fall under the Power Platform governance model, adding a layer of complexity to management.

Controlling default SharePoint Agents

Despite default SP agents not having a .agent file, site owners and admins still have certain control options for these default agents, primarily through SharePoint’s UI and admin settings.

Monitoring default SharePoint agents relies on aggregated usage data and audit logs rather than file-level statistics. Site owners can use site usage reports for general trends. At the same time, SharePoint admins and compliance teams can leverage PowerShell, Purview audit logs, and SharePoint Advanced Management for detailed monitoring and governance of default agent activities.

Monitoring method	Role/Permission Required	What it provides
Site Usage (Site Settings)	Site owners, members, visitors	Popular files and overall agent usage trends
SharePoint Online Management Shell	SharePoint Admins	Tenant-wide agent inventory and usage (only custom agents)
Microsoft Purview Audit Logs	Compliance Admins	Detailed logs of agent interactions including default agents
SharePoint Advanced Management	SharePoint Admins	Basic tenant-wide agent usage and policies
Microsoft 365 Admin Center	Global administrator, AI administrator, Global reader (read-only access)	Cost and consumption data for agent usage

How to track custom SharePoint agents

Syskit Point offers a custom SharePoint agents inventory report that helps organizations:

• Gain an overview: It provides a centralized view of all custom-built SharePoint Agents within your M365 tenant.
  
• Visibility into knowledge sources: This shows which data sources (e.g., specific sites, document libraries) each custom agent has access to. This is crucial for understanding the scope of information an agent can provide.
  
• Permissions overview: It details who has access to these agents, helping IT admins identify potential access risks and ensure compliance.
  
• Risk identification: By providing insights into agent locations, knowledge sources, and permissions, the report helps identify “high-risk agents” that might inadvertently expose sensitive data or contribute to sprawl within the M365 environment.
  
• Monitoring sprawl: SharePoint agents, especially custom ones, can create security blind spots and lead to an uncontrolled proliferation of agents. This report aids in monitoring and managing this sprawl.

This report enables IT admins and site owners to maintain control over AI assistant usage, improve governance, and maximize the business value of SharePoint agents.

To sum up: Control aspects of SharePoint Agents

Control Aspect	Default Agent (No .agent file)	Custom Agent (.agent file)
Creation	Auto-created per site, no file	Created by users, stored as .agent JSON files
Approval	Always approved by default, can be reset as default	Requires site owner approval to be visible
Visibility	Controlled by site owner approval and site settings	Controlled by file permissions and approval
Configuration	No direct editing or JSON file access	Editable via JSON file (unsupported to modify manually)
Disable on Site	Via Restricted Content Discovery policy	Same as default; can be disabled site-wide
Licensing Control	Managed via Microsoft 365 Copilot license settings	Same licensing controls apply.