Add data retention and maintenance documentation for CoE Starter Kit tables

[Copilot]

The admin_EnvironmentSecurityRolePermission table grows unbounded (28+ GB reported), with no documented retention strategy or cleanup procedures. Users lack guidance on expected behavior, safe deletion criteria, and maintenance automation.

Changes

Documentation Added

• CenterofExcellenceResources/DataRetentionAndMaintenance.md (350 lines)

  • Table growth patterns and expected behavior
  • 30-90 day retention policy recommendations with rationale
  • Three cleanup approaches: Bulk Delete (recommended), Power Automate, PowerShell
  • Archival strategies: Azure Data Lake, Azure SQL, manual export
  • Storage monitoring and impact assessment
  • Links to 13 official Microsoft resources

• CenterofExcellenceResources/QuickStart-DataCleanup.md (142 lines)

  • 5-minute Bulk Delete job creation guide
  • Step-by-step with Power Platform Admin Center screenshots (descriptions)
  • Troubleshooting and quick reference card

README Updates

• Added "Additional Resources" section linking to maintenance guides
• Improves discoverability for storage management

Key Points

Retention: 30-90 days sufficient for CoE dashboards (use latest snapshots only)

Safe Deletion: Records older than retention period via createdon field

No Built-in Cleanup: Manual Bulk Delete job setup required (documented)

Dataverse Limitations: No custom partitioning/compression; delete + external archival only

Example Bulk Delete Configuration:

Table: admin_EnvironmentSecurityRolePermission
Condition: createdon older than 90 days
Schedule: Weekly, off-peak hours

Applies same principles to other growing tables: admin_SyncFlowErrors, admin_FlowActionDetail, admin_Audit.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[CoE Starter Kit - QUESTION] Alto consumo de almacenamiento en la tabla admin_EnvironmentSecurityRolePermissionBase</issue_title>
<issue_description>### Does this question already exist in our backlog?

• I have checked and confirm this is a new question.

What is your question?

We have identified that the table [admin_EnvironmentSecurityRolePermissionBase] is consuming a significant amount of storage and shows continuous growth over time.
The goal of this issue is to confirm whether this behavior is expected and to define a supported approach to reduce its size without impacting the tool’s functionality.

Is this growth expected, or is it considered anomalous behavior?

Is there a recommended retention policy for this table?

Which data can be safely deleted or archived without impacting the tool?

Are there automatic cleanup jobs that should be enabled?

Is it recommended to partition, compress, or move part of the data?

Is there official documentation covering this type of maintenance?

What solution are you experiencing the issue with?

Core

What solution version are you using?

4.50.7

What app or flow are you having the issue with?

Dataver

What method are you using to get inventory and telemetry?

None</issue_description>

<agent_instructions>Analyze the issue and provide solutions to all the questions with clear and proper explanations.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@harini-2-y Hi @Eusemazo , Thanks for raising this question!

Is this growth expected, or is it considered anomalous behavior?
Yes — this behavior is expected. The admin_EnvironmentSecurityRolePermissionBase table stores full security‑role permission snapshots each time the Security Role sync runs. The data is insert‑only, so the table will continue to grow over time as more snapshots are collected.

Is there a recommended retention policy for this table?
There is no official retention policy for this specific table. However, most customers retain 30–90 days of data, as only recent snapshots are needed for the CoE dashboards and insights.

Which data can be safely deleted or archived without impacting the tool?
You can safely delete older snapshot records (for example, anything older than 30–90 days based on createdon). Removing historical data does not impact the CoE functionality because the dashboards use only the latest snapshots.

Are there automatic cleanup jobs that should be enabled?
No — the CoE Starter Kit does not include any automatic cleanup jobs for this table. If cleanup is required, you will need to create a Dataverse Bulk Delete job or a scheduled cleanup flow to enforce retention.

Is it recommended to partition, compress, or move part of the data?
Dataverse does not support custom partitioning or compression for this table. The supported approach is:
Delete older rows, and
Optionally archive them externally (such as Azure SQL or Data Lake) before deletion, if long‑term storage is required.

Is there official documentation covering this type of maintenance?
There is currently no dedicated documentation for retention or cleanup of this table. Existing CoE documentation only explains how Security Role tracking works, not how to manage table size or historical snapshot growth.</comment_new>

Custom agent used: CoE Custom Agent
Specialized agent for triaging and fixing issues in the CoE Starter Kit. Always consult the official CoE docs and the team’s SharePoint notes first, then mine prior GitHub issues and the CoE Common Responses playbook.

• Fixes [CoE Starter Kit - QUESTION] Alto consumo de almacenamiento en la tabla admin_EnvironmentSecurityRolePermissionBase #10686

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[muditagarwal2308]

Approved