Provisioning with two Azure OpenAI model deployments fails

[aaronpowell]

I'm attempting to deploy dotnet/eShop with Azure OpenAI support enabled but it fails to deploy with the following error:

{"code":"DeploymentFailed","target":"/subscriptions/294ea8c8-3674-4e75-a047-4fb1c279c1bb/resourceGroups/rg-aapowell-eshop-azureified/providers/Microsoft.Resources/deployments/openai","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.","details":[{"code":"RequestConflict","message":"Another operation is being performed on the parent resource '/subscriptions/294ea8c8-3674-4e75-a047-4fb1c279c1bb/resourceGroups/rg-aapowell-eshop-azureified/providers/Microsoft.CognitiveServices/accounts/openaihxaqj5hu4jxby'. Please try again later."}]}

The problem is this code:

builder.AddAzureOpenAI(openAIName)
            .AddDeployment(new AzureOpenAIDeployment(chatModelName, "gpt-35-turbo", "0613"))
            .AddDeployment(new AzureOpenAIDeployment(textEmbeddingName, "text-embedding-ada-002", "2"));

And the reason this happens is that when deploying models to Azure OpenAI you are can only be doing one operation on the parent (Azure OpenAI resource) at a given time, meaning that the Bicep generated will be unable to deploy (here's a sample Bicep):

targetScope = 'resourceGroup'

@description('')
param location string = resourceGroup().location

@description('')
param principalId string

@description('')
param principalType string


resource cognitiveServicesAccount_wXAGTFUId 'Microsoft.CognitiveServices/accounts@2023-05-01' = {
  name: toLower(take('openai${uniqueString(resourceGroup().id)}', 24))
  location: location
  kind: 'OpenAI'
  sku: {
    name: 'S0'
  }
  properties: {
    customSubDomainName: toLower(take(concat('openai', uniqueString(resourceGroup().id)), 24))
    publicNetworkAccess: 'Enabled'
  }
}

resource roleAssignment_Hsk8rxWY8 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  scope: cognitiveServicesAccount_wXAGTFUId
  name: guid(cognitiveServicesAccount_wXAGTFUId.id, principalId, subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'a001fd3d-188f-4b5d-821b-7da978bf7442'))
  properties: {
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'a001fd3d-188f-4b5d-821b-7da978bf7442')
    principalId: principalId
    principalType: principalType
  }
}

resource cognitiveServicesAccountDeployment_5K9aRgiZP 'Microsoft.CognitiveServices/accounts/deployments@2023-05-01' = {
  parent: cognitiveServicesAccount_wXAGTFUId
  name: 'mymodel'
  sku: {
    name: 'Basic'
    capacity: 4
  }
  properties: {
    model: {
      format: 'OpenAI'
      name: 'gpt-35-turbo'
      version: '0613'
    }
  }
}

resource cognitiveServicesAccountDeployment_mdCAJJRlf 'Microsoft.CognitiveServices/accounts/deployments@2023-05-01' = {
  parent: cognitiveServicesAccount_wXAGTFUId
  name: 'embedding-model'
  sku: {
    name: 'Basic'
    capacity: 4
  }
  properties: {
    model: {
      format: 'OpenAI'
      name: 'text-embedding-ada-002'
      version: '2'
    }
  }
}

output connectionString string = 'Endpoint=${cognitiveServicesAccount_wXAGTFUId.properties.endpoint}'

Instead, we need to generate Bicep that looks like this:

targetScope = 'resourceGroup'

@description('')
param location string = resourceGroup().location

@description('')
param principalId string

@description('')
param principalType string

param deployments array = []

resource cognitiveServicesAccount_wXAGTFUId 'Microsoft.CognitiveServices/accounts@2023-05-01' = {
  name: toLower(take('openai${uniqueString(resourceGroup().id)}', 24))
  location: location
  kind: 'OpenAI'
  sku: {
    name: 'S0'
  }
  properties: {
    customSubDomainName: toLower(take(concat('openai', uniqueString(resourceGroup().id)), 24))
    publicNetworkAccess: 'Enabled'
  }
}

resource roleAssignment_Hsk8rxWY8 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  scope: cognitiveServicesAccount_wXAGTFUId
  name: guid(cognitiveServicesAccount_wXAGTFUId.id, principalId, subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'a001fd3d-188f-4b5d-821b-7da978bf7442'))
  properties: {
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'a001fd3d-188f-4b5d-821b-7da978bf7442')
    principalId: principalId
    principalType: principalType
  }
}

@batchSize(1)
resource cognitiveServicesAccountDeployment_5K9aRgiZP 'Microsoft.CognitiveServices/accounts/deployments@2023-05-01' = [for deployment in deployments: {
  parent: account
  name: deployment.name
  properties: {
    model: deployment.model
  }
  sku: contains(deployment, 'sku') ? deployment.sku : {
    name: 'Standard'
    capacity: 20
  }
}]

output connectionString string = 'Endpoint=${cognitiveServicesAccount_wXAGTFUId.properties.endpoint}'

There are two important things to note in this Bicep change:

1. We use batchSize(1) to instruct the resource provider to only deploy change at a time from the loop
2. We use a loop rather than individual resource definitions so that we can control the behaviour

The only workaround I can find for this is to use two separate Azure OpenAI resources (or as many resources as you need models deployed).

Note: This is most likely a limitation in Azure.Provisioning, but I'm raising it here for visibility too.

[sebastienros]

@mitchdenny I see that this template had the batchSize before the new way to generate them. Do you think a solution would be to have the template have an array initialized with the deployments data and use the same [for deployment in as before with a batchSize, or is it possible (and preferable) to have subsequent deployment resources have the previous deployment as its parent?

[mitchdenny]

We can't do this with CDK unfortunately.

We could consider reverting to using a Bicep template as an interim solution until we can figure out how to make this work.

Alternatively we could go live with this limitation and tell people to crate multiple Open AI accounts.

If you deploy multiple times it also sorts itself out.

I'm also considering whether we can use explicit dependencies here.

[sebastienros]

I'm also considering whether we can use explicit dependencies here.

is this the parent property?

Update: I see it is not, the CDK expects a CognitiveService as the parent.

[mitchdenny]

Just waiting on AddDependency being unblocked in Azure.Provisioning.