There could be many instances where we want to observe the cost of your Azure resources and analyze the expenses. We can definitely achieve this from Azure Portal itself from the Cost Analysis under Cost Management in our subscription.
Though in an enterprise it could be really frustrating to navigate multiple dashboards for performing our analysis and observability.
Grafana can be really helpful in such scenarios for both Ops and Management Teams. It can be a single stop solution for all our needs for managing dashboards for different insights in an enterprise. Whether it is Dev, Ops, Infra, QA, or Management Team, Grafana has something for everyone.
Ways to integrate Azure Cost Management with Grafana:
- Using Opencost / Kubecost: Both Opencost and Kubecost are tools for monitoring and optimizing Kubernetes costs. These tools are open-source and can analyze cost utilization of resources running in Azure cloud. However, it could be tricky to setup and integrate at times, based on the complexities in the cloud infrastructure and applications. Custom role and permission could also be challenging during the initial phase.
- Using Azure Consumption Rest APIs: The Azure Consumption Rest APIs give you programmatic access to cost and usage data for your Azure resources.
How to configure the Azure Cost Analysis dashboard on Grafana?
- Create an App Registration in Microsoft Entra ID on Azure Portal under your tenant.
2. Save the Client ID, Client Secret and Tenant ID of the created App Registration (in my case I have named my App Registration as “Grafana-App”):
3. Select the Subscription which you want to analyze the usage cost for, select “Identity Access Management (IAM)” then click on “Add role assignment”:
Assign the “Cost Management Contributor” role to the newly created App Registration (in my case it is “Grafana-App
4. On the Grafana instance, install the “Infinity data source” plugin and configure the same by following the documentation: https://grafana.com/docs/plugins/yesoreyeram-infinity-datasource/latest/examples/azure/
5. We can use Postman API client or CURL command to test out the Azure Consumption APIs. To test the APIs, we need to first generate the authentication token. Below is the sample:
- For Microsoft Entra ID’s authentication token:
curl --location 'https://login.microsoftonline.com/<TENANT ID>/oauth2/token' \ --form 'client_id="<CLIENT ID>"' \ --form 'client_secret="<CLIENT SECRET>"' \ --form 'resource="https://management.azure.com/"' \ --form 'grant_type="client_credentials"'
Response:
{
"token_type": "Bearer",
"expires_in": "3599",
"ext_expires_in": "3599",
"expires_on": "1716669210",
"not_before": "1716665310",
"resource": "https://management.azure.com/",
"access_token": "****************************************************"
}
- Using Azure Consumption API, for usage cost of services (Current month):
curl --location 'https://management.azure.com/subscriptions/{{subscriptionID}}/providers/Microsoft.CostManagement/query?api-version=2023-11-01' \
--header 'Authorization: bearer {{bearerToken}}' \
--header 'Content-Type: application/json' \
--data '{
"type": "Usage",
"timeframe": "MonthToDate",
"dataset": {
"granularity": "Monthly",
"aggregation": {
"totalCost": {
"name": "Cost",
"function": "Sum"
}
},
"grouping": [
{
"type": "Dimension",
"name": "ServiceName"
}
]
}
}'
Response:
{
"id": "subscriptions/{{subscriptionID}}/providers/Microsoft.CostManagement/query/e0acf38b-1234-abcd-86ee-57a62111d6d6",
"name": "e0acf38b-1234-abcd-86ee-57a62111d6d6",
"type": "Microsoft.CostManagement/query",
"location": null,
"sku": null,
"eTag": null,
"properties": {
"nextLink": null,
"columns": [
{
"name": "Cost",
"type": "Number"
},
{
"name": "BillingMonth",
"type": "Datetime"
},
{
"name": "ResourceType",
"type": "String"
},
{
"name": "Currency",
"type": "String"
}
],
"rows": [
[
45.0241476195185,
"2024-05-01T00:00:00",
"microsoft.compute/virtualmachinescalesets",
"INR"
],
[
334.380796149969,
"2024-05-01T00:00:00",
"microsoft.containerregistry/registries",
"INR"
],
[
1207.89758199245,
"2024-05-01T00:00:00",
"microsoft.network/loadbalancers",
"INR"
],
[
482.52868,
"2024-05-01T00:00:00",
"microsoft.network/publicipaddresses",
"INR"
],
[
1.98347837744,
"2024-05-01T00:00:00",
"microsoft.storage/storageaccounts",
"INR"
],
[
1634.941782,
"2024-05-01T00:00:00",
"microsoft.web/serverfarms",
"INR"
],
[
0.0,
"2024-05-01T00:00:00",
"microsoft.web/sites",
"INR"
]
]
}
}
Similar to the above, we can POST different body to get the cost of the resources in Azure based on different granularity (Daily/Monthly/Yearly).
6. Now, we can create a custom dashboard as per our requirement for cost analysis. However, I have already create a Grafana dashboard which provides cost for services in the current month and based on custom duration.
Link to my dashboard: https://grafana.com/grafana/dashboards/21134-azure-cost-analysis/
Also, two panels which can dynamically show cost for services in the current month and custom duration based on the selection from the drop-down menu (in the below images “Virtual Network” is selected):
NOTE: I am using “Transformations” in this Grafana dashboard to make the panels dynamic with variables. However, it will not work straightforward in Grafana. To make it work we need to enable the variable “transformationsVariableSupport” which supports for using variables in transformations, behind a feature toggle. You can activate it by modifying the .ini file of your local Grafana instance.
[feature_toggles] # there are currently two ways to enable feature toggles in the `grafana.ini`. # you can either pass an array of feature you want to enable to the `enable` field or # configure each toggle by setting the name of the toggle to true/false. Toggles set to true/false # will take precedence over toggles in the `enable` list. # enable = feature1,feature2 enable = # Some features are enabled by default, see: # https://grafana.com/docs/grafana/next/setup-grafana/configure-grafana/feature-toggles/ # To enable features by default, set `Expression: "true"` in: # https://github.com/grafana/grafana/blob/main/pkg/services/featuremgmt/registry.go # feature1 = true # feature2 = false transformationsVariableSupport = true
For Grafana Cloud users, we can contact support for the same.
Finally, we can setup alerts from Grafana for usage cost of resources in the current billing cycle, or of any particular service’s usage cost. However, I will take setting up of alerting from Grafana in some other blogs in future.
In case you have any questions or doubts, then please feel free to comment.
References:
- https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth-ropc
- https://learn.microsoft.com/en-us/rest/api/consumption/
- https://grafana.com/docs/plugins/yesoreyeram-infinity-datasource/latest/examples/azure/
- https://github.com/grafana/grafana/issues/79118
This blog is part of Microsoft Azure Week! Find more similar blogs on our Microsoft Azure Landing page here.
About the author:
Reference:
Kumar, G. (2024) Azure Cost Analysis Dashboards on Grafana. Available at: Azure Cost Analysis Dashboards on Grafana | LinkedIn [Accessed on 24/06/2024]
