Calling Microsoft Dynamics 365 CRM Online APIs from External Applications

If you’re working with Microsoft Dynamics 365 CRM Online, you’ve probably encountered situations where you need to integrate it with other applications like your website, custom web APIs, or tools like KingswaySoft, XrmToolBox, or Change Data Capture (CDC) tools. API integration can unlock incredible automation and data synchronization, but with great power comes a few headaches.

Let’s dive into how you can make API calls to your MS CRM Online system, what limits and challenges you need to be aware of, and how to avoid common pitfalls while following best practices.

How MS CRM Online API Calls Work

The Microsoft Dataverse Web API (formerly known as the Common Data Service API) is at the heart of connecting with MS CRM Online. It uses OData and supports CRUD (Create, Read, Update, Delete) operations for your CRM data.

Common Use Cases for CRM API Calls:

• Website Integration: Pull customer data, update records, or submit forms directly to CRM.
• Middleware Tools: Automate imports, data migration, or bulk updates using KingswaySoft or SSIS packages.
• Data Sync with External Systems: Use APIs to sync CRM data with ERP, marketing tools, or third-party systems in near-real time.
• Automation Tools: Tools like Power Automate or XrmToolBox to streamline operations without reinventing the wheel.

What Makes MS CRM Online API Calls Different?

Dynamics 365 CRM Online operates in the cloud, which means Microsoft enforces strict rules to ensure the stability of its infrastructure. These include:

1. Daily API Limits (Service Protection Limits):
   Each user and tenant in MS CRM Online has a daily API call limit based on the licenses they’re using.

   • For example, a Dynamics 365 Customer Service Enterprise license allows 40,000 daily API calls per user.
   • These limits are shared across all applications making calls on behalf of a user or application.

2. Timeouts and Query Size Restrictions:

   • The maximum request timeout is 2 minutes.
   • API calls can fail if they try to retrieve too much data at once. Large datasets need to be retrieved in pages using $top and $skip.

3. Concurrency Limits:

   • Too many concurrent calls from the same application or user can trigger throttling (HTTP 429 errors).

4. Authentication:

   • CRM Online only supports OAuth 2.0 for authentication. No basic auth! This means you’ll need to register an app in Azure AD and obtain access tokens to make API calls.

Limitations You Need to Know About

1. Throttling (HTTP 429 Errors):
   If you exceed API limits or send too many requests too quickly, Microsoft will throttle your app to ensure the CRM stays stable for all users. Throttling typically results in a “429 Too Many Requests” error.

2. Complex Queries Can Fail:
   Using too many joins, nested filters, or retrieving large data sets can lead to timeout errors or performance issues.

3. Eventual Consistency:
   Data changes made through the API might not be immediately available for reads, especially if you’re using background processes like workflows.

4. Security Role Restrictions:
   Even if you have API access, the user or application making the call is still limited by their assigned security roles in CRM.

Best Practices for MS CRM Online API Calls

1. Optimize Your Queries:

   • Only retrieve the data you need by using $select to specify columns and $filter to narrow down records. For example:

     GET https://<your-org>.crm.dynamics.com/api/data/v9.2/accounts?$select=name,revenue

2. Use Pagination for Large Datasets:

   • MS CRM Online supports server-side paging using the @odata.nextLink property in responses. This is crucial when dealing with more than 5,000 records.

3. Batch API Requests:

   • Instead of making multiple single calls, use batch operations to group several requests into one. This reduces the number of API calls and improves efficiency.

4. Implement Retry Logic:

   • Throttling is inevitable if you’re working at scale. Implement retry logic with exponential backoff when you encounter HTTP 429 errors.

5. Monitor API Usage:

   • Use the Power Platform Admin Center to monitor API call consumption and identify heavy users or apps. This helps you stay ahead of limits.

6. Use Change Tracking for Incremental Data Sync:

   • Instead of polling all records repeatedly, use Change Tracking in CRM to fetch only the records that have changed since the last sync.

7. Secure Your API Calls:

   • Use HTTPS for all communications. Ensure Azure AD tokens are stored securely, and rotate client secrets regularly.

How to Monitor API Usage in MS CRM Online

1. Power Platform Admin Center:
   Go to Admin Center > Analytics > Common Data Service (Dataverse) to see the API usage summary for your environment. You’ll get a breakdown of calls by user, application, and type.

2. Enable Logging:
   Turn on the Plug-in Trace Log to track failed calls or performance bottlenecks.

3. Use Audit Logs:
   Dynamics 365 allows you to track API interactions via the audit log. It’s particularly useful for troubleshooting data updates or failed requests.

4. Alerts for API Limits:
   Set up alerts to notify you when usage approaches the daily API limit. This can prevent unexpected disruptions.

What to Do If You Exceed the Limit

1. Analyze Your Calls:
   Use the Power Platform Admin Center to identify which users or applications are consuming the most API calls. Common culprits include poorly optimized workflows or data sync jobs.

2. Reduce API Calls:

   • Replace frequent polling with Change Tracking or webhooks.
   • Consolidate multiple small API calls into fewer, larger calls using batch operations.

3. Purchase Additional Capacity:
   If you’re regularly exceeding limits, you can buy extra capacity from Microsoft as an add-on.

4. Temporarily Slow Down Integrations:
   Adjust schedules for non-critical jobs or integrations to avoid peak times.

How to Monitor API Usage from the Microsoft Power Platform Admin Center

One of the best ways to keep an eye on your API usage in Dynamics 365 CRM Online is by leveraging the Microsoft Power Platform Admin Center. This portal provides real-time analytics and insights into API consumption for your environment. Here's how you can monitor your usage step by step:

Steps to Monitor API Usage in the Admin Center

1. Log in to the Power Platform Admin Center

   • Go to Power Platform Admin Center.
   • Use your admin credentials to log in. You’ll need global admin or system admin permissions to access detailed metrics.

2. Navigate to the Analytics Section

   • On the left-hand navigation pane, click on Analytics and select Dataverse (formerly Common Data Service).
   • Choose the Environment you want to monitor if you’re managing multiple environments (e.g., production, sandbox).

3. View API Usage Overview

   • In the Dataverse Analytics dashboard, go to the Capacity or API Usage tab.
   • This section shows:
     • Daily API Usage: A graph or table displaying how many API calls have been consumed by the environment.
     • Users and Applications: Breakdowns of which users or applications are generating the most API traffic.
     • Remaining API Capacity: See how close you are to hitting your daily API limits.

4. Filter by Time Period or User/Application

   • Use filters to narrow down API usage by:
     • Specific Users
     • Applications (e.g., integrations like KingswaySoft, Power Automate flows)
     • A specific time period (e.g., daily, weekly).

5. Identify Problem Areas

   • Look for patterns in API consumption. For example:
     • A user making excessive API calls due to misconfigured workflows.
     • An application like a middleware tool consuming more calls than expected.
   • Drill down into specific calls to identify whether they are read-heavy or write-heavy.

6. Set Up Alerts (Optional)

   • Currently, you cannot directly configure alerts from the admin center, but you can monitor daily API thresholds using Power BI dashboards or scripts. Microsoft may notify you by email if your usage consistently exceeds capacity.

7. Access Logs for Detailed Troubleshooting

   • If you’re troubleshooting failures, you can enable and access:
     • Plug-in Trace Logs: Tracks issues with custom plugins or workflows that might be contributing to excessive API usage.
     • Audit Logs: Enables detailed tracking of API actions, including which records were updated or accessed.

Key Insights You Can Gather from the Admin Portal

• Who or What Is Consuming the Most API Calls?
  The analytics dashboard shows a breakdown of API usage by user and application. This helps you pinpoint whether the problem lies with a specific integration (like KingswaySoft) or a user's actions (e.g., running an overly complex advanced find).

• Are You Approaching Your Daily API Limits?
  The dashboard provides a visual representation of your daily API consumption and how much headroom you have left before hitting the limit.

• Which API Calls Are Failing?
  If certain API calls are failing, you can cross-reference your logs to determine the cause. Look out for throttling errors (HTTP 429) or misconfigured apps making unnecessary requests.

Tips for Effective Monitoring

1. Regularly Check the Dashboard
   Make it a habit to check API usage in the admin center weekly or after deploying a new integration to catch issues early.

2. Set Thresholds for Heavy API Consumers
   If you find that specific apps (e.g., Power Automate flows) or users consistently hit the limits, optimize their behavior by reducing frequency or batching operations.

3. Plan Capacity for Busy Periods
   If you know your environment will experience high API demand (e.g., a data migration or bulk import), consider purchasing additional capacity in advance.