Generate Auto-Increment id on Azure

AzureAutoNumber: High performance, distributed unique id generator for Azure environments.

Published in

ITNEXT

3 min readFeb 17, 2020

If you have worked with TableStorage or Cosmos DB, you already know they don’t offer auto-incremental ids due to consistency levels. But in many uses-cases, we need to generate human-friendly identities to make them easy to read for clients.

Ids like GUID or UUID are too ugly and painful to remember, imagine the following example:

Your order is on process. Please keep your invoice id for further tracking.Invoice id:
e3a52fee-2ad5-4c35-b9c0-4edcf438bdff

There are some workarounds to generate incremental ids on Azure, such as using Azure SQL. But by using it, you serialize all of your writes down to a single thread and throwing away all of the possible benefits of something like a queuing architecture.

After investing some time, I managed a library for it; I called it Azure AutoNumber. The project is using the latest version of Azure NuGet packages, and it supports .NET Standard 2.0 and 2.1.

How to use AutoNumber

First, Install it via NuGet:

dotnet add package AzureAutoNumber --version 1.0.0

For generating ids, you need to make a singleton instance of UniqueIdGenerator class. This class has a dependency on BlobOptimisitcDataStorage class.

var blobStorageAccount = Microsoft.WindowsAzure.Storage.CloudStorageAccount.Parse(connectionString);var blobOptimisticDataStore = new BlobOptimisticDataStore(blobStorageAccount, "unique-ids");var idGenerator = new UniqueIdGenerator(blobOptimisticDataStore);

BlobOptimisitcDataStorage will create a new container called “unique-ids” on your blob storage. As I said earlier, the instance of UniqueIdGenerator should be a singleton in every App Domain.

Now by calling the NextId method, it is easy to generate ids. The NextId method requires a parameter called ScopeName. By ScopeName, you can create different ids for different scopes.

var orderId = idGenerator.NextId("orders");
var invoiceId = idGenerator.NextId("invoices");

If you prefer to register IUniqueIdGenerator and its dependencies in Microsoft Dependency Injection, you can use the service extension of the package like the following example.

// configure the services
services.AddAutoNumber();// Inject `IUniqueIdGenerator` in constructorpublic class Foo
{
  public Foo(IUniqueIdGenerator idGenerator)
  {
      _idGenerator = idGenerator;
  }
}

The only caveat is you need to register CloudStorageAccount in DI before registering AutoNumber.

How does it work?

First, it allocates batches of ids to each running instance. Azure Blob Storage is used to coordinate these batches. Azure Blob Storage has optimistic concurrency checks via standard HTTP headers. At a persistence level, AutoNumber creates a small text file for each id scope.

var orderId = idGenerator.NextId("orders");
var invoiceId = idGenerator.NextId("invoices");

Consider the example above; these two calls will make two new text files get created or modified on a container.

- unique-ids
--- orders -> contains "1"
--- invoices -> contains "2"

So basically, for each scope, we will have a plain text file that stores the latest generated id for that scope. On the first call, it reserves a batch of ids for future uses in the app domain. Those ids in the batch will be wasted if you don’t use them. To understand it better, take a look at this diagram:

You can set the batch size easily via appsettings.json if you prefer the service registration extension method.

{
  "AutoNumber": {
    "BatchSize": 50,
    "MaxWriteAttempts": 25,
    "StorageContainerName": "unique-urls"
  }
}

or you can change it via BatchSize property:

idGenerator.BatchSize = 25;

Source Code

You can access the source code in GitHub.

Good to know

A few years a go Tatham Oddie released a library called SnowMaker. I forked his work and made lots of changes to make it available on .NET Standard (2.0 and 2.1). SnowMaker is out-dated and is using a very old version of Azure Packages. All credits of this project go to him.