Select the appropriate service for a generative AI solution

Generative AI workloads on Azure are anchored in Microsoft Foundry, the unified resource type that aggregates model deployments, agents, evaluations, and data assets inside a Foundry account and one or more Foundry projects. Practitioners face a recurring question: when does the workload belong in Azure OpenAI in Foundry Models, when does it call for a partner model hosted in the Foundry Models catalog, and when is a task specific Azure AI service the more appropriate path. The answer turns on capability fit, latency and throughput predictability, residency, and total cost rather than raw model quality alone.

Azure OpenAI in Foundry Models is the default surface for tasks that demand strong general purpose reasoning, instruction following, multimodality, or function calling. Workloads such as customer email drafting, contract clause rewriting, marketing copy generation, code completion, structured data extraction with JSON schema, and retrieval augmented generation over enterprise content all map naturally to the GPT 4.1, GPT 4o, and o series reasoning model deployments inside a Foundry project. Workloads that require image generation use DALL E 3 deployed through the same Foundry account, and workloads that need general purpose embeddings for vector search use the text-embedding-3-large and text-embedding-3-small deployments.

The Foundry Models catalog widens the lens by hosting partner foundation models and open source models behind the same authentication and SDK shape. Workloads that benefit from a specialist model (a domain tuned medical model, a small language model optimized for on device fallback, or a long context partner model) are deployed through the catalog as Models as a Service so that token billing and capacity remain managed by Microsoft. Workloads with stricter open weights requirements use the same catalog but choose a managed compute deployment that runs the model on a dedicated Azure Machine Learning compute, paying for VM time rather than tokens.

Task specific Azure AI services remain the right answer when the workload maps cleanly to a known capability such as named entity recognition, sentiment, document layout extraction, OCR, translation between widely supported languages, or speaker recognition. A generative model can perform these tasks, but the cost per call is higher, latency variance is larger, and the quality on narrow tasks is rarely superior to the dedicated service. The selection rule is to start with the most specific service that fits the capability and to fall back to Azure OpenAI in Foundry Models only when the workload requires reasoning, free form generation, or cross capability orchestration.

A second axis to evaluate is data residency and traffic routing. Standard deployments pin tokens to the region of the Foundry account. Global Standard routes prompts to whichever Microsoft data center has capacity, which is acceptable for non sensitive workloads that prioritize throughput but is rejected by many regulated tenants. Data Zone Standard and Data Zone Provisioned restrict routing to a named data zone such as the European Union or the United States and is the right choice when residency must be honored without abandoning global capacity pooling. Provisioned (PTU) reserves dedicated throughput in Provisioned Throughput Units, which is the right call when the workload is high volume, latency sensitive, or subject to a service level agreement. Batch is the right choice for asynchronous workloads such as bulk summarization or backfill embeddings, where a twenty four hour completion window is acceptable and a discount of roughly fifty percent is welcome.

Choose the appropriate AI models for your solution

Model selection inside a Foundry project follows a workflow that begins with task framing, then narrows by context window, modality, deployment SKU, region availability, and price per million tokens. The first question is whether the task needs reasoning. Chain of thought heavy work such as multi step planning, math, code synthesis with verification, and policy reasoning belongs on an o series reasoning model. Tasks that need fast generation across long context (a sales meeting summary, a research summary, an executive brief over a long policy document) belong on a GPT 4.1 or GPT 4o deployment that offers a wide context window and lower latency than reasoning models.

Multimodal needs raise the next question. Workloads that consume images, audio, or video alongside text use GPT 4o or its multimodal siblings. Workloads that only consume text but produce structured outputs benefit from JSON schema mode and structured outputs, which require a model deployment that advertises support for that feature in the model card. Workloads that need image generation route through DALL E 3, and workloads that need transcription route through whisper or the Azure Speech service depending on whether the audio is short form or streaming.

Embedding model selection follows a similar logic. Workloads that need maximum recall on long passages use text-embedding-3-large with 3072 dimensions, optionally truncated to a smaller dimension count via the dimensions parameter to save on vector index storage. Workloads that need cheap embeddings for an internal knowledge base use text-embedding-3-small with 1536 dimensions. Workloads that need cross lingual recall favor the large model, which carries stronger multilingual semantics.

💡 Exam Trap: Global Standard routes traffic to any Microsoft data center, which means a tenant that requires European Union residency cannot pick Global Standard even if its Foundry account is provisioned in westeurope. The correct answer is Data Zone Standard or Data Zone Provisioned, which keeps requests inside the European Union data zone while still pooling capacity across European regions.

💡 Exam Trap: Provisioned Throughput Units are not interchangeable with Tokens Per Minute. A question that gives a TPM target and asks about Provisioned deployment is testing whether you know that PTU capacity is sized by the PTU calculator and the same TPM number can require different PTU counts depending on the model and prompt length distribution.

Manage and protect account keys

Each Microsoft Foundry account exposes two account keys, key1 and key2. They are stored on the Foundry account resource and grant unrestricted access to every model deployment and every Foundry capability bound to the account, which makes them a high value secret. Production identity design assumes that keys are the fallback path rather than the primary path. Microsoft Entra ID authentication via managed identity or service principal is the default, and keys are reserved for break glass workflows, legacy clients that cannot integrate with the Microsoft Identity Platform, and isolated test rigs.

Keys must never be embedded in source code, container images, configuration files committed to a repository, or notebooks shared via a chat tool. Production code resolves keys at runtime from Azure Key Vault using the Azure Key Vault references in App Service or Azure Functions, the Container Apps secrets reference, or the azure-keyvault-secrets SDK with managed identity. The Foundry account exposes a rotation endpoint that regenerates key1 or key2 independently, which supports a zero downtime rotation pattern. Clients switch to key2, key1 is regenerated, clients switch back to key1, then key2 is regenerated, and the cycle continues on a fixed cadence such as every ninety days.

# Regenerate key2 on a Microsoft Foundry account
az cognitiveservices account keys regenerate \
 --name contoso-prod-aifoundry-eus2 \
 --resource-group rg-ai-prod \
 --key-name key2

# Read the active keys (script should pipe straight to Key Vault, never to a log)
az cognitiveservices account keys list \
 --name contoso-prod-aifoundry-eus2 \
 --resource-group rg-ai-prod \
 --query primaryKey -o tsv | \
 az keyvault secret set --vault-name kv-ai-prod \
 --name aifoundry-primary-key --file /dev/stdin

The Foundry account also supports disabling local authentication entirely. Setting properties.disableLocalAuth = true on the account causes every request that presents an Ocp-Apim-Subscription-Key or api-key header to fail with HTTP 401, forcing every caller to authenticate with Microsoft Entra ID. This is the strongest posture available and is the recommended default for new deployments.

resource foundryAccount 'Microsoft.CognitiveServices/accounts@2024-10-01' = {
 name: 'contoso-prod-aifoundry-eus2'
 location: 'eastus2'
 kind: 'AIServices'
 sku: { name: 'S0' }
 identity: { type: 'SystemAssigned' }
 properties: {
 customSubDomainName: 'contoso-prod-aifoundry-eus2'
 publicNetworkAccess: 'Disabled'
 disableLocalAuth: true
 networkAcls: {
 defaultAction: 'Deny'
 virtualNetworkRules: []
 ipRules: []
 }
 }
}

📊 Limit: A Microsoft Foundry account holds exactly two account keys (key1, key2) and supports independent regeneration of each, which is the structural basis for the zero downtime rotation pattern. Azure Key Vault stores secret values up to 25 KB in size and supports an unlimited version history within standard quota.

⚠️ Anti-Pattern: Storing the Foundry account key in an environment variable inside a container image or a .env file committed to source control. Even with private repositories, key exfiltration via supply chain attacks or accidental fork remains a real risk. The correct pattern is a Key Vault reference resolved at startup by the platform or by the SDK using DefaultAzureCredential.

🎯 Scenario: A team plans to launch a GPT 4.1 powered email drafting assistant for a European bank that contracts for one thousand requests per second sustained, with a maximum latency budget of eight hundred milliseconds at the ninety ninth percentile, and a regulatory rule that prohibits any data leaving the European Union. The correct posture is a Foundry account in westeurope, a Data Zone Provisioned deployment of GPT 4.1 sized via the PTU calculator, Microsoft Entra ID authentication via managed identity, and account level disableLocalAuth = true so that keys cannot be used even by accident.

Decision Anchor. Choose Azure OpenAI in Foundry Models when the workload needs general purpose reasoning, multimodal input, or free form generation. Choose a partner model from the Foundry Models catalog when the workload needs a domain specialist or open weights. Choose a task specific Azure AI service when the capability is narrowly scoped and a dedicated service offers lower cost and lower latency variance.