Inconsistent SharePoint search results between Copilot Studio web and custom Direct Connect app

Hi Thomas Denis,

This behavior is not related to tool planning but is primarily caused by differences in identity context, permissions, consent lifecycle, and runtime handling between Microsoft Copilot Studio (web) and a custom Direct Connect application.

When search_result: None appears intermittently while Studio returns consistent results, it typically indicates that the SharePoint retrieval context is not reliably established in the custom channel. SharePoint-based grounding strictly relies on delegated user identity, so any mismatch in user, tenant, token scopes, or missing permissions will silently result in empty outputs rather than explicit errors. A critical factor is the handling of the connectors/consentCard; if the consent submission is altered, or if the conversation/user context is reset after approval, the connector session may not persist, leading to intermittent failures. Additionally, discrepancies in app registration (missing delegated scopes or consent), lack of a valid Microsoft 365 Copilot license (or PAYG enablement), and SharePoint indexing latency can all contribute to inconsistent retrieval.

Beyond these core factors, minor variations can also arise due to SharePoint’s relevance ranking (top-K results), prompt sensitivity, token/session expiration handling in custom implementations, and the absence of built-in retry/resiliency mechanisms that are internally managed in Studio. To achieve parity, it is essential to ensure identical user identity and tenant alignment, proper delegated permissions, correct and unmodified consent handling with session continuity, matching app scopes and consent, valid licensing, and verification that documents are indexed and discoverable via SharePoint search. In practice, when no explicit error is returned and only search_result: None is observed, the root cause is almost always a breakdown in identity propagation, permission enforcement, consent persistence, or retrieval conditions rather than an issue with the UniversalSearchTool itself.

Thanks.

Karan Shewale.

Intermittent search_result: None with SharePoint-backed agents almost always indicates a permissions, licensing, or indexing issue on the SharePoint/Microsoft 365 side rather than a tool-planning issue, especially when Copilot Studio web behaves more reliably than a custom Direct Connect app.

Based on the documented behavior, use the following checklist to align behavior between Copilot Studio web and the custom channel:

1. Verify user identity and tenant alignment
   • Ensure the identity used by the custom app is the same user and tenant as in Copilot Studio web:
     • Same Microsoft Entra user (Object ID) is passed through to the agent.
     • No cross-tenant scenario: 401/auth failures and missing results can occur if the Foundry/Power Platform user and Microsoft 365 user are not in the same tenant.
   • In the SharePoint tool troubleshooting, cross-tenant access attempts are a known cause of 401 and failures.
2. Confirm SharePoint permissions for the actual calling user
   • Generative answers over SharePoint rely on delegated permissions. The calling user must have at least read access to the relevant SharePoint sites and files; otherwise, no search results are returned and no error is surfaced.
   • This matches the symptom where search_result is None with no explicit error.
   • Validate that the user identity used in the custom app has read access to all target sites/libraries, not just the maker or test account used in Studio.
3. Check licensing for the calling user
   • For SharePoint-based tools and Copilot scenarios, a missing Microsoft 365 Copilot license (or pay-as-you-go) can cause Forbidden: Authorization Failed - User does not have valid license and effectively block retrieval.
   • Ensure the user identity used in the custom app has a valid Microsoft 365 Copilot license or pay-as-you-go enabled, as required by the SharePoint tool.
   • Be aware that license and permission changes can take time to propagate; transient failures that later disappear are consistent with propagation delays.
4. Validate connector/app registration and scopes
   • When generative answers use SharePoint, admins must configure Microsoft Entra app registration with the correct delegated scopes and grant consent.
   • If scopes are missing or consent is not granted, no results are returned and no errors are shown, which again matches search_result: None.
   • Compare:
     • The app registration and scopes used when Copilot Studio web calls the agent.
     • The app registration and token used by the custom Direct Connect app.
   • Ensure the same scopes and consent are in place for the identity path used by the custom app.
5. Connector consent card handling
   • The connectors/consentCard indicates the agent is trying to use a connector (SharePoint) that requires user consent.
   • After the user clicks Allow and the app relays the Action.Submit back to the bot, verify:
     • The submission is passed through unchanged (including any hidden fields/IDs in the Adaptive Card).
     • The conversation ID and user ID remain the same before and after consent.
   • If the consent flow is interrupted or the conversation is restarted incorrectly, subsequent tool calls may run without a valid connection, leading to intermittent search_result: None.
6. Indexing and search behavior in SharePoint
   • Generative answers only use the top three search results from SharePoint and only from supported formats and modern pages.
   • If content is newly uploaded or not yet indexed, or if search settings hide certain sites, the tool can return no results.
   • Inconsistent results (sometimes documents, sometimes none) can occur if:
     • Indexing is still in progress.
     • Queries are slightly different and fall above/below the relevance threshold.
   • Confirm that the target documents are indexed and visible in SharePoint search for the same user.
7. Behavior parity: Studio vs custom app
   • To maximize parity between Copilot Studio web and a custom Direct Connect app:
     • Use the same user identity, tenant, and licensing.
     • Ensure delegated permissions and scopes are identical.
     • Confirm that the SharePoint connector consent is completed once and then reused for subsequent conversations for that user.
   • The SharePoint tool troubleshooting guidance explicitly calls out that “Tool returns no results” when the user lacks access to SharePoint content; this is the primary parity gap to eliminate.
8. Maker-provided credentials vs user identity
   • The SharePoint tool and generative answers over SharePoint are documented as relying on user identity / delegated permissions, not application-only authentication.
   • The SharePoint tool explicitly does not support app-only OBO tokens; it requires identity passthrough.
   • Maker-provided credentials can simplify configuration but do not change the requirement that the effective identity used for SharePoint access must have the correct permissions and licensing.
   • For strict parity and least surprise, user identity passthrough is recommended; maker-provided credentials are more about simplifying setup than fixing intermittent search_result: None caused by missing user access or consent.
9. Concrete diagnostic steps
   • For the same user and prompt:
     1. Confirm the user can open the target SharePoint documents directly in the browser.
     2. Verify the user has a valid Microsoft 365 Copilot license or pay-as-you-go.
     3. Check that the app registration used by the custom app has the same delegated scopes and consent as the one used by Studio.
     4. After consent, keep the same conversation ID and user ID; test multiple queries in the same session to see if results stabilize.
     5. Use SharePoint search UI with the same user to confirm that the documents appear in search and are indexed.

If, after these checks, search_result: None still appears intermittently only in the custom channel, focus on verifying that the consent card submission is correctly relayed and that no app-only or cross-tenant token is being used in the Direct Connect path.

References:

• Generative answers pointing to SharePoint sources don't return results
• Use SharePoint tool with the agent API (preview) (rest)
• Use SharePoint tool with the agent API (preview) (typescript)
• Use SharePoint tool with the agent API (preview) (java)
• Use SharePoint tool with the agent API (preview) (python)
• Use SharePoint tool with the agent API (preview) (csharp)
• Copilot Studio Agent doesn't recognize the docs in the Knowledge Base. - Microsoft Q&A
• Copilot Studio Agent will not read entire data set in Excel or markdown - Microsoft Q&A
• Microsoft 365 Copilot Retrieval API errors with Authorization Failed - User does not have valid license - Microsoft Q&A