We have an existing SPO metrics list, populated automatically by user activity, and we want to produce a straightforward report from that data.
No new data collection, no migration, no unnecessary infrastructure. And no unnecessary charges!
Reporting options
|
Option |
What it provides |
Complexity |
|---|---|---|
|
1. SharePoint list views |
Filter, sort, group and total metrics directly in SPO |
Very low |
|
2. Excel connected to SPO |
Tables, pivot tables, charts and summaries |
Low |
|
3. Power BI |
Interactive reports, charts, filtering and aggregation |
Medium |
|
4. Power Apps |
A custom, admin-only interface for viewing metrics |
Medium |
|
5. Python-generated HTML |
A standalone report generated from SPO data |
Low, given your existing tools |
|
6. Dataverse |
A separate data platform that can expose or store SPO metrics |
Higher |
What each option actually means
1. SharePoint views
We can create views of the existing metrics list, including grouping, filtering and totals for numeric columns.
No additional platform or data duplication.
The limitation is presentation. It's still a SharePoint list view, not a particularly attractive report.
2. Excel
Excel can connect to a SharePoint Online list using Power Query.
Once connected, you can create pivot tables, charts and summaries. Refresh the connection when you need updated metrics.
This is probably the simplest route to a conventional management report.
3. Power BI
Power BI Desktop can connect directly to your SPO list.
You can create reports showing, for example, searches by day, most frequently accessed pages and activity trends.
You don't need Dataverse or Power Automate.
Power BI Desktop is free for authoring. Publishing and sharing reports through the Power BI service may require additional licensing.
4. Power Apps
A canvas app can connect directly to the existing SPO list.
You could create an admin screen displaying totals, filtered records and basic charts.
However, Power Apps is primarily an application-building platform. For reporting alone, it's probably more work than necessary.
5. Python and HTML
This is close to what you're already doing.
Python reads the metrics from SharePoint, processes them and generates HTML.
Advantages include complete control over formatting, no additional reporting platform, and the ability to run the report manually.
The disadvantage is maintaining your own reporting code.
6. Dataverse
Dataverse is worth understanding, but it isn't necessary to produce this report.
A SharePoint virtual table can expose list data through Dataverse without copying it. However, virtual tables have limitations around aggregation and native Dataverse reporting features.
Alternatively, you could replicate the metrics into regular Dataverse tables, but that introduces synchronization, storage and security considerations.
For a simple report, neither approach offers an obvious advantage.
One important distinction: reporting versus collection
USER ACTIVITY
|
v
SPO Metrics List
|
+------------+------------+
| | |
v v v
Excel / BI SharePoint Python
Views HTMLAll three reporting approaches can read the same existing list.
The metrics continue accumulating regardless of which reporting tool you choose or how often you run a report.
Yes. Based on what you've already established:
-
Data source: Existing SharePoint Online metrics list.
-
Purpose: Evaluate how users find and use Plays in the Playbook.
-
Metrics: Searches, clicks, repeat searches, unique users (anonymous), and most-used Plays.
-
Existing solution: Python-generated HTML reports, run manually.
-
Management response: Positive, particularly regarding presentation.
-
Requirement: Professional-looking, browser-accessible reporting without unnecessary complexity.
Evaluation
|
Option |
Fit |
Assessment |
|---|---|---|
|
Python HTML |
Excellent |
Already working, proven presentation, no new platform. |
|
Power BI |
Excellent |
Best alternative for professional dashboards, directly connects to SPO. |
|
Excel |
Good |
Can analyze all these metrics, but less suited to browser-first presentation. |
|
SharePoint views |
Fair |
Useful for basic counts and grouping, but limited presentation. |
|
Power Apps |
Fair |
Can create a polished interface, but requires building and maintaining an app. |
|
Dataverse |
Poor |
Adds infrastructure without solving a reporting problem. |
Conclusion
Power BI is the only alternative I'd seriously evaluate against your existing Python HTML reports.
It can use the existing SPO list, present all five metrics professionally, and support interactive reporting without introducing Dataverse.
The deciding considerations are Power BI licensing, report-sharing permissions, and whether its built-in presentation offers anything valuable beyond your existing HTML.
There is no technical reason, based on the requirements you've described, to replace the Python solution.
There is a manual export step, and the SP REST API is really XML in REST clothing!
Fair criticism. SharePoint's REST API has a rather impressive ability to make modern technology feel like 2007. :)
But there's an important distinction: SharePoint REST can return JSON instead of Atom/XML. You don't need to parse XML in Python.
For example:
GET /_api/web/lists/GetByTitle('Metrics')/items
Accept: application/json;odata=nometadataThat header requests a simplified JSON response.
Your Python code could then work with ordinary dictionaries:
items = response.json()["value"]Microsoft documents JSON responses for SharePoint list items, so this is supported functionality rather than a workaround.
The real obstacle is authentication, not data format.
Before we write any code, I'd investigate whether your existing authenticated SharePoint session can be used for a read-only proof of concept.
If that works, we can concentrate on automating the export without changing anything else in your reporting system.
Authentication
Look at Microsoft Entra ID delegated authentication using my existing Microsoft 365 work account.
There are two distinct possibilities:
|
Method |
How it works |
Suitable for |
|---|---|---|
|
Interactive OAuth 2.0 (MSAL) |
I sign in through Microsoft using my work account and MFA |
Running Python reports on demand |
|
Application authentication (certificate) |
A registered application authenticates without a person signing in |
Fully unattended scheduled reports |
Both use Microsoft's existing identity infrastructure. Neither inherently requires a premium Power Platform licence.
My preference for your situation
Start with interactive delegated authentication:
-
Python initiates a Microsoft sign-in.
-
You authenticate with your existing work account.
-
Microsoft issues an access token.
-
Python uses that token to read the metrics list.
-
Your existing reporting code generates the HTML.
There is one catch: Python needs an approved application identity (client ID), and the tenant must permit the necessary delegated permissions. We can't assume your browser's existing SharePoint session is directly reusable.
But there's an important distinction
You originally asked about automating the manual export.
If you want to run the reports yourself with one command, interactive authentication is appropriate.
If you want the reports to generate automatically every night, we need unattended authentication, which generally means an Entra application registration and certificate.
Which level of automation are we pursuing: one-command execution or completely unattended scheduling?
That determines whether we can keep authentication interactive.
Hi Kaiyan,
I'm investigating automating the retrieval of metrics from an existing SharePoint Online list using Python, without additional licensing or services.
Could you confirm three things?
- Interactive authentication: Can we use an existing or new Entra ID app registration with MSAL and delegated read permissions to access SPO list data using our Microsoft 365 credentials?
- Unattended authentication: Would certificate-based application authentication be permitted if we later want to schedule the reports automatically?
- Licensing: Can we accomplish this entirely within our existing Microsoft 365 licensing, without additional costs?
The objective is simply to replace a manual list export:
SPO metrics list → Python → HTML reports
No additional platforms or infrastructure.
Dataverse
This is an interesting use case for Dataverse because the SharePoint Online (SPO) list is already collecting operational metrics, and we're exploring what Dataverse might add to the reporting side.
The first thing to understand is that Dataverse is not simply another way to view a SharePoint list. It's a separate data platform with its own tables, security, relationships, and licensing.
And that distinction matters quite a bit for your metrics.
1. Three ways to approach this
|
Approach |
What happens |
|---|---|
|
Keep data in SPO |
Your existing metrics list remains the source of truth. Reports query SharePoint directly. |
|
Copy data to Dataverse |
Power Automate or another integration copies metrics into Dataverse tables. |
|
Use a virtual table |
Dataverse exposes the SharePoint data through a supported virtual-table connector, without storing another copy. |
2. What changes when you introduce Dataverse?
Current architecture
User clicks / searches
|
v
SPO Metrics List
|
v
Existing admin report
Possible Dataverse experiment
SPO Metrics List
|
v
Dataverse Table
|
v
Power Apps / Power BI / Dataverse views3. Important discovery: SharePoint virtual tables ARE supported!
Microsoft now documents a SharePoint virtual connector provider. That means you can expose an SPO list through Dataverse without copying its records.
This is worth exploring first.
There are limitations, particularly relevant to metrics:
-
Virtual tables don't support native Dataverse charts, rollups, or calculated columns. -- > OK
-
SharePoint Person and Managed Metadata columns aren't supported by this connector. -- > OK
-
Virtual-table queries have a 1,000-record return limit, so reporting over substantial event histories needs careful consideration.
-
Five Things to Investigate
|
Area |
Question |
Why It Matters |
|---|---|---|
|
Environment |
Do we have access to a Power Platform environment with Dataverse? |
Determines whether Dataverse is available for exploration and testing. |
|
Permissions |
Do we have permission to create Dataverse tables? |
The Environment Maker role alone does not necessarily grant table-creation privileges. |
|
Licensing |
Does our licensing include Dataverse capabilities? |
Full Dataverse functionality generally requires premium licensing beyond standard Microsoft 365. |
|
Data Volume |
How many clicks and searches are recorded daily? |
Replicating every event could increase storage, processing and licensing costs. |
|
Security |
Can reporting access be restricted to administrators? |
Dataverse security roles can control access to tables and reporting data. |
Current assessment: The existing SharePoint metrics list and manually generated reports meet the immediate reporting requirement. Dataverse remains an option for future investigation rather than an immediate migration priority.
Microsoft documents these distinctions in its licensing overview and security-role guidance.
5. Where I'd begin exploring
Go to Power Apps and examine the environment selector.
Look for Tables, Solutions, and whether you have access to Dataverse. You don't need to change your production list to investigate.
One particularly useful experiment would be to expose a small copy of your metrics list through a SharePoint virtual table, then see what Dataverse can actually do with it.
I wouldn't introduce Power Automate or a second database yet.