Setting up soft delete on Azure Storage blobs

Soft delete is a data protection stragegy offered on Azure Storage blobs to prevent inadvertent data loss due to deletion or overwrite of blobs.

When configuring soft delete, users have to specify a retention period. This is the amount of time for which the soft-deleted data is available for recovery (even if you go ahead and turn off soft delete).

Below documentation does a great job explaining the specifics of how snapshots and deleted versions of blobs are created and made available for recovery: https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-soft-delete#saving-deleted-data

So, instead of focusing on that, in this post, I'll deep-dive into how this can be setup using various tools available at your disposal.

Configuring soft delete using Azure Portal

You can configure soft-deletion of blobs while creating a storage account as follows:

Click on the + button on the top left of the Azure Portal and select "Storage account"
Once you've filled out all items in the Basics tab, click 'Next' to navigate to the Advanced tab
Here, you can enable soft delete for blobs and configure the retention period as shown below

Soft delete configuration in Storage account create blade

Once you've done this, you can continue with creating your account and rest assured that your data is being retained as configured.

What about existing storage accounts?

If you already have a storage account you're using, you can enable soft delete on it by navigating to the Soft delete menu item of your Storage account blade as shown below:

Soft delete configuration for existing storage account

Configuring soft delete using ARM templates

To understand how to author an ARM template that enables soft delete, lets do a brief recap of how Storage accounts are structured in the Azure Resource Manager (ARM) REST API. A storage account is a resource type under the Microsoft.Storage resource provider. You would access this as Microsoft.Storage/storageAccounts/<storage-account-name>. Within a storage account, there's a sub-resource called blobServices with just a single instance named default accessed using Microsoft.Storage/storageAccounts/<storage-account-name>/blobServices/default. Soft delete configuration for blobs is defined in this sub-resource.

To mimic what we did using the Azure Portal above, we would need to define an ARM template with both the storage account resource and the blob service sub-resource. First, let's define the Storage account resource. If you plan on using ARM templates for multiple deployments, I'd recommend parameterization of any parameters that are user-specified. For example, below I've parameterized the name and location attributes.

{
  "type": "Microsoft.Storage/storageAccounts",
  "sku": {
    "name": "Standard_LRS",
    "tier": "Standard"
  },
  "kind": "StorageV2",
  "name": "[parameters('storageAccountName')]",
  "apiVersion": "2018-07-01",
  "location": "[parameters('location')]"
}

When you create a storage account, the blob service sub-resource named "default" will be pre-created for you, though it won't have soft delete enabled by default. You can then make updates to this sub-resource to enable soft delete. Below I've configured it to have a retention of 30 days. Also, make a note of the 'dependsOn' attribute. Its been set so that the blob service update happens only after storage account has been created.

{
  "name": "[concat(parameters('storageAccountName'), '/default')]",
  "type": "Microsoft.Storage/storageAccounts/blobServices",
  "apiVersion": "2018-07-01",
  "properties": {
    "deleteRetentionPolicy": {
      "enabled": true,
      "days": 30
    }
  },
  "dependsOn": [
    "[concat('Microsoft.Storage/storageAccounts/', parameters('storageAccountName'))]"
  ]
}

Now, that we've defined above resources, we can assemble the full ARM template to run this payload.

{
  "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "location": {
      "type": "string"
    },
    "storageAccountName": {
      "type": "string"
    }
  },
  "variables": {},
  "resources": [
    {
    "type": "Microsoft.Storage/storageAccounts",
    "sku": {
      "name": "Standard_LRS",
      "tier": "Standard"
    },
    "kind": "StorageV2",
    "name": "[parameters('storageAccountName')]",
    "apiVersion": "2018-07-01",
    "location": "[parameters('location')]"
    },
    {
      "name": "[concat(parameters('storageAccountName'), '/default')]",
      "type": "Microsoft.Storage/storageAccounts/blobServices",
      "apiVersion": "2018-07-01",
      "properties": {
        "deleteRetentionPolicy": {
          "enabled": true,
          "days": 30
        }
      },
      "dependsOn": [
        "[concat('Microsoft.Storage/storageAccounts/', parameters('storageAccountName'))]"
      ]
    }
  ],
  "outputs": {}
}

Deploy template using CLI

Below example creates a resource group and deploys a template from your local machine.

az group create --name ExampleGroup --location "Central US"
az group deployment create \
  --name ExampleDeployment \
  --resource-group ExampleGroup \
  --template-uri "path/to/template.json" \
  --parameters storageAccountName=demostorageaccount9 location=westus

You can find more detailed information here: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-deploy-cli

Deploy template using PowerShell

Below example is an equivalent of the CLI version above. It creates a resource group and deploys a template from your local machine.

$resourceGroupName = "ExampleGroup"

New-AzResourceGroup -Name $resourceGroupName -Location "centralus"
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
  -TemplateFile "path\to\template.json"
  -storageAccountName "demostorageaccount9"
  -location "westus"