[CoE Starter Kit - QUESTION] High amount of CoE Service Account Non-Interactive login logs

[accvik]

Does this question already exist in our backlog?

• I have checked and confirm this is a new question.

What is your question?

We have identified rather high Azure Log Analytics costs caused by a large volume of non‑interactive sign‑in logs generated by the CoE Starter Kit service accounts in our organization. We would like guidance on best‑practice approaches to reduce or avoid these logs-particularly whether the CoE kit can be run using service principals instead of user accounts, and how others typically handle this. What are our options?

Ideas

• We do not want to reduce frequency of inventory scan due to keep data up to data and we do not want to introduce managed layers. That makes our update process cumbersome. Right?
• Can you use the kit with Service Principal connections for the Dataverse queries/actions (sync templates) to reduce amount?

Open for discussion and suggestions, thanks!

What solution are you experiencing the issue with?

Core

What solution version are you using?

No response

What app or flow are you having the issue with?

No response

What method are you using to get inventory and telemetry?

Cloud flows

[AmarSaiRam]

Hi @accvik

Documentation Added

ReducingAzureLogAnalyticsCosts.md - Comprehensive analysis covering:

Root cause: Power Platform Admin connector lacks Service Principal support for most operations
Five mitigation options evaluated (log exclusion, retention optimization, scan frequency reduction, Application Users, custom connectors)
Recommended approach: Azure-side log filtering via transformation rules
Implementation guides with KQL examples
Cost estimation scenarios and FAQ
ISSUE_RESPONSE_LOG_ANALYTICS_COSTS.md - Quick reference template for responding to similar issues

Documentation/README.md - Index for the new documentation directory

Recommended Solution

Exclude CoE service account logs from Azure Log Analytics collection using transformation rules:

SigninLogs
| where UserPrincipalName !in ("coe-service@contoso.com", "powerplatform-admin@contoso.com")
This approach:

Requires no CoE Starter Kit modifications (preserves managed solution for easy updates)
Maintains inventory scan frequency (data freshness unchanged)
Reduces costs by up to 100% for this log category
Simple Azure-side configuration
Alternative approaches (custom connectors, Application Users) either create unmanaged layers that complicate updates or provide only partial cost reduction.

[accvik]

Thank you so much!

Are there any drawbacks or cons regarding this solution that we should be aware of?

Actually, one more question. We need a step-by-step guide on how to add the KQL filter, are you able to provide that?

Thanks!

[AmarSaiRam]

Option 1: Filter at Query Level (No Cost Reduction, Query-Time Only)
If you just want to exclude these logs from your views and reports:

Open Azure Portal → Navigate to Log Analytics workspace
Click on "Logs" in the left menu
Enter your KQL query with a filter. For example:
kql
SigninLogs
| where TimeGenerated > ago(30d)
| where UserPrincipalName ! contains "coe-service-account" // Adjust to your service account naming pattern
| where UserPrincipalName !has "your-service-account-name@yourdomain.com"
// Add your analysis here
Note: This only filters the view but doesn't reduce ingestion costs.

Option 2: Collection-Time Filtering (Reduces Costs) Recommended
To actually reduce Log Analytics costs by preventing ingestion:

Step 1: Identify Your CoE Service Account(s)
Run a query to find the exact UserPrincipalName(s):
kql
SigninLogs
| where TimeGenerated > ago(7d)
| where AppDisplayName has "Power" or AppDisplayName has "Dynamics"
| summarize Count=count() by UserPrincipalName
| sort by Count desc
| take 20
Note the service account names generating high volumes
Step 2: Create a Data Collection Rule (DCR) Transformation
Navigate to: Azure Portal → Monitor → Data Collection Rules
Click: "+ Create" or select existing DCR for your Log Analytics workspace
Go to: "Data Sources" tab
Add or Edit: The data source for Azure AD Sign-in logs
Enable: "Data transformation"
Step 3: Add KQL Transformation
In the transformation editor, add:

kql
source
| where UserPrincipalName !has "coe-service-account@yourdomain.com"
| where UserPrincipalName !has "coe-admin@yourdomain.com"
// Add all your CoE service accounts here
Or for multiple accounts:

kql
source
| where UserPrincipalName !in~ (
"coe-service-account@yourdomain.com",
"coe-admin@yourdomain.com",
"powerplatform-service@yourdomain.com"
)
Test the transformation
Click: "Save"
Option 3: Configure Azure AD Diagnostic Settings (Most Effective)
Step 1: Navigate to Azure AD Diagnostic Settings
Azure Portal → Azure Active Directory
Monitoring → Diagnostic settings
Select your existing diagnostic setting (or create new one)
Step 2: Modify What Gets Logged
Unfortunately, Azure AD Diagnostic Settings don't support filtering at this level, so you'll need to use Option 2 (DCR Transformation) instead.

Option 4: Use Table-Level Retention & Transformation
Step 1: Navigate to Log Analytics Workspace
Azure Portal → Log Analytics workspaces → Select your workspace
Settings → Tables
Find "SigninLogs" or "AADNonInteractiveUserSignInLogs"
Step 2: Configure Table Settings
Click on the table name
Select: "Table settings" or "Manage table"
Set up transformation:
kql
SigninLogs
| where UserPrincipalName !contains "coe-"
| where UserPrincipalName !startswith "svc-"
// Adjust patterns to match your naming convention
Optionally reduce retention period for this table specifically (e.g., 30 days instead of 90 days)
Save
Option 5: Alternative Architecture Recommendations
Based on your question about using Service Principals:

✅ Yes, you can use Service Principals with the CoE Kit, which would:

Generate different log types (ServicePrincipalSignInLogs instead of NonInteractiveUserSignInLogs)
Potentially reduce logging volume
Be more aligned with best practices for automation
Steps to migrate to Service Principals:

Review CoE Kit documentation for supported authentication methods
Create App Registrations with appropriate API permissions
Update connection references in your flows
Test thoroughly before switching production

[accvik]

@AmarSaiRam We have tried above and still have some issues, can you please assist? Let me know if you are available through email/teams for a quick call/chat. Our question below:

We need help with how we are supposed to transform / filter incoming logs from Entra ID (e.g. SignInLogs, NonInteractiveSignInLogs) based on this article:
https://learn.microsoft.com/en-us/azure/azure-monitor/data-collection/data-collection-transformatio…
We want to be able to filter out unnecessary data before it reaches the Log Analytics workspace. When we are trying to create DCRs in Azure monitor, then Entra ID is not listed as a Source.
Please advise us on how this is supposed to be done.