Decoding Azure Locks — a Lifesaver or a Nightmare?

The ship is the safest when it is in port, but that is not what ships are built for.

— Paul Coelho

Assume you have a important presentation for your senior stakeholders this afternoon, you wake up fresh, glance through your emails and the stop at one email. You read something strange. For few moments you can’t believe your eyes. Then you call your team lead or a technical SME to confirm if this is true.

Yes, some automation glitch has cause an outage on system and now this is going to cost you at least, some part of your presentation probably, the practical aspects you plan to showcase.

You can relax now!!! This is just imaginary scenario created to explain the context why preventing resources from accidental deletion is paramount in high stake situations.

I am sure it might have taken place at many occasions, though the order if events or other details might have been different, as I imagined

Locks: a security and a preventive measure

Critical resources needs to be preserved or better say prevented from accidental deletions.

There are few ways to approach this use case. One is to block all such operations in first place, probably via a Azure Policy. But then most companies today, have some dedicated team for managing such policies. This brings dependency on other teams which brings unforseen delays to projects.

Some can argue, why not using principle of least privilege and not allow such access via custom RBAC roles. While this is possible option, if your team does not manages RBAC roles much like Azure Policies, it again poses similar challenges. Moreover, in most cases, automated processes using a technical user identity or Service principal which is typically used for IAC systems, needs access to all most CRUD operations permissions for valid reasons. Not having “delete permission” on this identity poses significant operational challenges.

Enter Locks: a potential solution

Locks are meant to solve challenges such as accidental user deletions and modifications by being applied on Azure resources. Basically you can lock an entire Azure subscription, resource group, or resource to protect them by overriding any user permissions.

Scope of locks

Lock applied at higher scope is inherited by resources at a lower level, for example: a lock applied at resource group level is inherited by resources within that resource group.

For a quick recap of resource hierarchy, please see below:

Unlike role-based access control (RBAC), management locks can be used to apply a restriction across all users and roles. This do provides an additional security layer as well as a preventive measure against disasters.

Modes to apply locks

Locks can be applied in two modes: CanNotDelete or ReadOnly. With first one, authorised users can read and modify a resource, but they can’t delete it while with the latter users can read a resource, but can’t delete or update it.

Locks don’t protect against all operations

Yes, locks don’t protect against all operations. For example: a CanNotDelete lock at a storage account still allows deletion of containers/blobs etc.

How this can be true?
Well, this can be understood from perspective of Azure API operations, which falls into Control plane and Data plane operations. While Azure control plane operations hit the endpointhttps://management.azure.com, Azure data plane operations depending on the service type reaches service endpoint https://mystorageaccount.blob.core.windows.net/.

It implies locks protect a resource from changes, but they don’t restrict how a resource performs its functions. This explains that while lock prevents storage account from deletion, it does not prevent blobs from being deleted which falls on data plane.

Another example to clarify: a ReadOnly lock on an Azure Postgres Flexi server protects it from deletions or modifications, while allowing you to create, update, or delete data in the server database. In other words, data plane is not impacted by locks. That’s where you data backup and restore, basically Data loss prevention (DLP)/Disaster recovery (DR) kicks in, but this is not the scope of current blog.

How the implement locks

Basically the identity applying the lock, in most cases, the one configured for automated process like CI/CD pipelines, need the permissions to manage locks i.e. applying locks and removing locks. Removing locks or unlock, is required when you want to delete the resource, basically when it is intended.

Locks can be applied in multiple ways, say from Azure portal — a real DevOps engineer will hate this Or by using a Terraform, BICEP or ARM template, or through some Azure CLI scripts/automation.

Just a quick example from Azure CLI:

To lock a resource, provide the name of the resource, its resource type, and its resource group name.

az lock create — name LockSite — lock-type CanNotDelete — resource-group exampleresourcegroup — resource-name examplesite — resource-type Microsoft.Web/sites

To delete a lock for a resource, use:

lockid=$(az lock show — name LockSite — resource-group exampleresourcegroup — resource-type Microsoft.Web/sites — resource-name examplesite — output tsv — query id)
az lock delete — ids $lockid

Locks are not so cool as they sound

There are certain constraints which one needs to be careful about when applying locks. Otherwise, locks can easily turn into nightmare for the teams troubleshooting issues (without any real bug/issue). Really non-productive, isnt it???

Yes, Azure locks can have some side effects which impacts how certain operations run in Azure.

To clarify, let see one example, some Azure services, like Azure Databricks, use managed applications to implement service, which creates two resource groups:

An unlocked resource group for service overview.
A locked resource group for service infrastructure.

You can’t delete the infrastructure resource group or its lock directly, as it’s owned by a system application. To remove it, delete the service, which will automatically delete the infrastructure resource group.

Detailed list of such constraints can be accessed here.

How to choose scope of lock then?

> Whether I should go for a Resource level lock versus a higher order lock is a point of contention. While there are no clear guidelines and depends on use case, below can serve an anchor to decide.

> Clearly a lock at RG scope seems easy to implement, even if user get the RBACs implemented via some central teams managing polices and permissions. While if we have many resources to protect, this can bit challenging for your technical identity to have multiple RBACs.

> RG scope lock obviously has a bigger blast radius and any misconfigurations has a bigger impact compared to resource level lock.

> This can be operationally challenging in development environments where churn of resources are high.

Some recommendations from my experiments with locks

Avoid locking in dev environments, they put unnecessary handcuff and stifle development velocity.

> Test the CI/CD flows when you need to unlock for any intended operation. Treat this as an exception and not the norm, else you have chose locks a a safety net to please people only.

> If something fails and you don’t understand the reason to see, refer the Microsoft documentation to check if this is a side effects of these locks.

> Many IAC tools like Terraform provides ways to provide protection against accidental deletes but then they only protect from actor or principal perspective, say your CI/CD technical user but manual errors can still happen on resource side. For details check here.

> Explore custom Azure polices if they provide a way suitable for your use case. For details check here.

> If your architecture allows consider using decentralise resource across resource groups, one such example is “critical resource group” from which you don’t want to delete resources anytime, other being “non critical” which can expect some churn. You can obviosly lock your “critical resource group” now and while implementing need based locks on other resource group.

In summary, while Azure locks provide a great way to prevent Azure resources against unintended actions especially deletions, overlooking some of the constraints they pose and how Azure locks fit in your architecture and support your use case; can have detrimental effects in the longer run. Discretion along with detailed impact assessment is a “must” especially for this Azure feature.

Thanks for reading, you can read my other blogs here or connect with me on LinkedIn. If you have any feedback, please reach out to me and I will be happy to incorporate any useful inputs/suggestions.

Resources

Lock resources: https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/lock-resources?tabs=json
Bicep, ARM and Terraform templates: https://learn.microsoft.com/en-us/azure/templates/microsoft.authorization/locks?pivots=deployment-language-bicep
Powershell, Azure CLI and Python examples: https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/lock-resources?tabs=json

Credits
Azure documentation and a bit of experience

Disclaimer
The above blog is for educational purpose please avoid using it on prod/live systems without proper design, implementation and testing in lower environments. There are obviously many pitfalls and intricacies, which can’t be covered in a small blog like this one.s. It’s a future where the possibilities are limitless, and the journey has just begun.

About the Author