November 22, 2024 5 min read

Azure AI Foundry: Microsoft's Unified Platform for Enterprise AI

Azure AI Azure AI Foundry Ignite Enterprise AI

Microsoft Ignite 2024 brought a major consolidation of Azure’s AI services under a new umbrella: Azure AI Foundry. If you’ve been navigating the maze of Azure AI Studio, Azure OpenAI Service, and various Cognitive Services, this announcement brings clarity.

What is Azure AI Foundry?

Azure AI Foundry is Microsoft’s unified platform for building, deploying, and managing AI applications. It brings together:

Azure AI Foundry Portal (formerly Azure AI Studio)
Azure OpenAI Service
Azure AI Speech and Language Services
Azure AI Agent Service (new)
25+ Pre-built Application Templates
Unified SDK (Python, C#, JavaScript coming soon)

Think of it as the “Visual Studio for AI” - one place to build everything from simple chatbots to complex multi-agent systems.

The Azure AI Foundry SDK

The new SDK unifies access to all Azure AI capabilities:

from azure.ai.foundry import AIFoundryClient
from azure.identity import DefaultAzureCredential

# One client for all AI services
client = AIFoundryClient(
    credential=DefaultAzureCredential(),
    project="my-ai-project"
)

# Chat completions
response = client.chat.complete(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain data lakehouse architecture."}
    ]
)

# Embeddings
embeddings = client.embeddings.create(
    model="text-embedding-3-large",
    input=["Microsoft Fabric is a unified analytics platform."]
)

# Speech synthesis
audio = client.speech.synthesize(
    text="Your data pipeline completed successfully.",
    voice="en-US-JennyNeural"
)

No more juggling openai, azure-cognitiveservices-speech, and azure-ai-textanalytics libraries separately.

Azure AI Agent Service

The biggest new capability is the AI Agent Service - infrastructure for building autonomous AI agents:

from azure.ai.foundry.agents import AgentService, Agent, Tool

# Define tools the agent can use
query_tool = Tool(
    name="query_database",
    description="Execute SQL queries against the data warehouse",
    function=execute_sql_query
)

report_tool = Tool(
    name="generate_report",
    description="Create a report from query results",
    function=generate_report
)

# Create an agent
data_analyst_agent = Agent(
    name="DataAnalyst",
    model="gpt-4o",
    instructions="""You are a data analyst. When asked questions about business data:
    1. First understand what data is needed
    2. Query the database using the query_database tool
    3. Analyze the results
    4. Generate a report if requested""",
    tools=[query_tool, report_tool]
)

# Run the agent
agent_service = AgentService(client)
result = await agent_service.run(
    agent=data_analyst_agent,
    task="Generate a monthly sales report for Q4 2024 with regional breakdown"
)

Agents can:

Reason through multi-step tasks
Use tools to take actions
Maintain conversation context
Hand off to other agents

This is the infrastructure for building the AI assistants of the future.

Multi-Agent Orchestration

For complex scenarios, agents can work together:

from azure.ai.foundry.agents import Orchestra

# Define specialized agents
data_agent = Agent(name="DataAgent", ...)
analysis_agent = Agent(name="AnalysisAgent", ...)
reporting_agent = Agent(name="ReportingAgent", ...)

# Create an orchestra
orchestra = Orchestra(
    agents=[data_agent, analysis_agent, reporting_agent],
    routing_strategy="semantic"  # Routes tasks to best-fit agent
)

# Complex task gets distributed automatically
result = await orchestra.run(
    task="""Analyze customer churn for the past 6 months:
    1. Pull customer transaction and support ticket data
    2. Identify patterns in churned vs retained customers
    3. Create an executive summary with recommendations"""
)

Integration with Microsoft Fabric

The Fabric + AI Foundry integration is particularly interesting for data professionals:

# Query Fabric lakehouse from AI agent
fabric_tool = Tool(
    name="fabric_query",
    description="Query Microsoft Fabric lakehouse",
    function=lambda query: fabric_client.execute_sql(query)
)

# Create Power BI reports from AI
powerbi_tool = Tool(
    name="create_report",
    description="Create a Power BI report from data",
    function=lambda spec: powerbi_client.create_report(spec)
)

fabric_analyst = Agent(
    name="FabricAnalyst",
    model="gpt-4o",
    instructions="You analyze data in Microsoft Fabric and create reports.",
    tools=[fabric_tool, powerbi_tool]
)

Soon, asking questions in natural language and getting Fabric reports will be seamless.

The Model Catalog

AI Foundry provides access to models beyond OpenAI:

OpenAI: GPT-4o, GPT-4, GPT-3.5 Turbo
Microsoft: Phi-3, Orca
Meta: Llama 3.1
Mistral: Mistral Large, Mixtral
Cohere: Command, Embed
And many more…

# Use any model from the catalog
response = client.chat.complete(
    model="meta-llama-3.1-70b",  # Llama on Azure
    messages=[...]
)

# Or a smaller, faster model
response = client.chat.complete(
    model="phi-3-medium",  # Microsoft's efficient model
    messages=[...]
)

This gives you flexibility to choose the right model for cost, latency, and capability tradeoffs.

Evaluation and Monitoring

AI Foundry includes built-in evaluation:

from azure.ai.foundry.evaluation import Evaluator

evaluator = Evaluator(client)

# Evaluate response quality
eval_results = evaluator.evaluate(
    model="gpt-4o",
    test_cases=[
        {"input": "What is Fabric?", "expected": "..."},
        {"input": "Explain DAX", "expected": "..."}
    ],
    metrics=["relevance", "coherence", "groundedness"]
)

print(eval_results.summary())
# relevance: 0.92
# coherence: 0.88
# groundedness: 0.85

This is crucial for production AI - you need to measure quality continuously.

Tracing and Debugging

Full observability for AI applications:

from azure.ai.foundry.tracing import Tracer

with Tracer() as trace:
    response = client.chat.complete(
        model="gpt-4o",
        messages=[...]
    )

# View in AI Foundry portal:
# - Token usage
# - Latency breakdown
# - Tool invocations
# - Agent reasoning steps

When things go wrong in production, you can see exactly what happened.

Getting Started

Access AI Foundry Portal: Navigate to ai.azure.com
Create a Project: Organizes your AI resources
Deploy Models: Choose from the model catalog
Build with SDK: Install azure-ai-foundry package
Monitor: Use built-in dashboards and tracing

pip install azure-ai-foundry

My Take

Azure AI Foundry represents the maturation of enterprise AI infrastructure. We’ve gone from:

2022: “Call the OpenAI API”
2023: “Wrap it in Azure for security”
2024: “Build production AI systems with proper tooling”

The Agent Service and multi-agent orchestration are particularly significant. We’re moving from “AI that answers questions” to “AI that accomplishes tasks.”

For data professionals, this means our analytics platforms will increasingly include AI agents that:

Automatically investigate data quality issues
Generate reports on demand
Alert on anomalies with context
Answer business questions from data

The tools are here. Time to build.