1 min read
Azure Database Migration Service - Strategies for Database Modernization
1. Assess compatibility first Use DMA for SQL Server 2. Test migrations in non prod Validate before production 3. Plan for downtime Even online migrations…
Blog
Thoughts on data engineering, cloud architecture, and technology. 2205 articles and counting.
Showing 24 of 2205 posts — page 81 of 92
2 min read
Azure Migrate - Planning and Executing Cloud Migrations
Azure Migrate is the tool that turns a migration conversation from "we think we have about 300 servers" to "we have 287 servers, here's the dependency map…
1 min read
Azure VMware Solution - Running VMware Workloads in Azure
Private Cloud Dedicated VMware environment Clusters Groups of ESXi hosts (minimum 3) vSAN Software defined storage NSX T Software defined networking HCX…
2 min read
Azure Dedicated Host - Running VMs on Isolated Physical Servers
Azure Dedicated Host is the offering I typically reach for when a client's security or compliance team says "I can't be on shared hardware." The use cases…
2 min read
Azure Confidential Computing - Protecting Data in Use
Confidential Computing addresses the attack surface that most cloud encryption doesn't touch: data while it's actively being processed in memory. Encryption…
2 min read
Responsible AI Toolkit - Building Trustworthy ML Systems on Azure
Responsible AI went from a philosophy discussion to a practical engineering requirement in a short time. The catalysts I've seen in real projects: a loan…
2 min read
Azure ML Designer - Visual Machine Learning Pipeline Development
Azure ML Designer is the no-code visual pipeline builder for ML, and its place in the toolbox is narrower than the marketing suggests. It's genuinely useful…
1 min read
Azure ML AutoML - Automated Machine Learning at Scale
AutoML automates: Feature engineering Automatic featurization Algorithm selection Tests multiple algorithms Hyperparameter tuning Optimizes model parameters…
2 min read
MLflow Experiments - Tracking and Managing Machine Learning Models
MLflow became the experiment tracking tool I recommend to every ML team regardless of their cloud platform choice. The core value proposition is simple…
1 min read
Databricks Notebooks - Collaborative Data Science and Engineering
Notebooks support multiple languages in a single document: Python PySpark and pandas Scala Native Spark SQL Spark SQL R SparkR and local R Markdown…
1 min read
PySpark Best Practices for Production Workloads
1. Define schemas explicitly Never rely on schema inference in production 2. Use built in functions Avoid UDFs when possible 3. Write pure transformation…
2 min read
Spark SQL Fundamentals and Best Practices
Spark SQL is where I've seen the highest productivity for data engineers who already know SQL but are new to distributed processing. You write familiar SQL…
2 min read
Synapse Pipelines vs Azure Data Factory - Making the Right Choice
The Synapse Pipelines vs. ADF question comes up in almost every Synapse project I start. The honest answer depends on where the rest of your data platform…
2 min read
ADF Self-Hosted Integration Runtime - Connecting On-Premises Data
The Self-Hosted Integration Runtime is the component that makes ADF viable for the majority of real enterprise projects—the ones where the source database…
2 min read
Azure Data Factory Triggers - Orchestrating Data Pipelines
ADF trigger design is where data pipeline reliability is won or lost. Schedule triggers are obvious—run at midnight, run every hour—but the interesting…
2 min read
ADLS Gen2 Access Control with ACLs - Fine-Grained Security
Both work together - a user needs both RBAC permission to access the storage account AND appropriate ACL permissions on the specific path.
2 min read
Azure Data Lake Storage Gen2 - Building Modern Data Lakes
ADLS Gen2 is the storage layer that most enterprise data platforms in Australia are converging on. The combination of blob-scale storage (petabytes, low…
2 min read
SQL Database Indexing with Azure Cognitive Search
Azure SQL to Cognitive Search is a combination I've set up half a dozen times, and the reason it keeps appearing is that most enterprise organisations…
1 min read
Blob Storage Indexing with Azure Cognitive Search
When indexing blobs, the indexer performs several operations: 1. Enumerate blobs Find blobs matching your criteria 2. Download content Retrieve blob data 3.…
1 min read
Azure Search Indexers - Automated Data Ingestion Strategies
Indexers handle the ETL process for search: Requires a column that increases monotonically: Enable change tracking on your database: Source JSON: Source…
1 min read
Azure Cognitive Search Custom Skills - Building Intelligent Search Pipelines
Custom skills are Azure Functions that implement a specific contract.
1 min read
Power Virtual Agents - Building Conversational AI Without Code
Topics are the building blocks of your bot. Each topic handles a specific intent.
2 min read
Power Apps Model-Driven Apps - Building Enterprise Solutions
Model-driven apps are the Power Apps surface that professional developers habitually underestimate. "No-code" suggests toylike; what you get with a…
2 min read
Power Apps Canvas Apps - Building Mobile-First Experiences
Canvas apps are where I've seen Power Apps deliver the most visible value to end users: a field inspection app for a construction site, a stock-take app…