MigrationManager

Overview

MigrationManager is a full-featured schema-migration engine. It discovers entity types from CLR assemblies, compares them against the live datasource schema, generates a typed plan artifact with per-operation risk classification, enforces governance policies, runs dry-run and preflight validations, executes the plan in checkpointed steps, and emits structured telemetry and audit events.

The manager is a partial class split into the following functional modules:

Quick Start

// 1. Instantiate and target a datasource
var migration = new MigrationManager(dmeEditor);
migration.MigrateDataSource = dmeEditor.GetDataSource("ProductsDB");

// 2. Register your entity assembly (if it is not in AppDomain already)
migration.RegisterAssembly(typeof(Product).Assembly);

// 3. Build a plan (discovery-based)
var plan = migration.BuildMigrationPlan(namespaceName: "MyApp.Entities");

// 4. Inspect plan
Console.WriteLine($"Operations planned: {plan.Operations.Count}");
Console.WriteLine($"Blocking readiness issues: {plan.ReadinessIssues.Count(i => i.Severity == MigrationIssueSeverity.Error)}");

// 5. Run preflight
plan = await Task.Run(() => { plan.PreflightReport = migration.RunPreflightChecks(plan); return plan; });
if (!plan.PreflightReport.CanApply)
{
    foreach (var check in plan.PreflightReport.Checks)
        Console.WriteLine($"[{check.Decision}] {check.Code}: {check.Message}");
    return;
}

// 6. Approve (required for high-risk / protected environments)
migration.ApproveMigrationPlan(plan, approvedBy: "deploybot");

// 7. Execute
var result = migration.ExecuteMigrationPlan(plan, new MigrationExecutionPolicy());
Console.WriteLine($"Success: {result.Success}  Token: {result.ExecutionToken}");

// 8. Snapshot telemetry
var telemetry = migration.GetMigrationTelemetrySnapshot(result.ExecutionToken);
Console.WriteLine($"Success rate: {telemetry.SuccessRate:P2}");

Core Types & Models

MigrationSummary

Lightweight result returned by simple create/update operations.

MigrationIssueSeverity

public enum MigrationIssueSeverity { Info, Warning, Error }

MigrationPlanLifecycleState

public enum MigrationPlanLifecycleState
{
    Draft, Reviewed, Approved, Applied, Verified
}

MigrationPlanOperationKind

public enum MigrationPlanOperationKind
{
    None, CreateEntity, DropEntity, AddMissingColumns, DropColumn,
    AlterColumn, RenameEntity, RenameColumn, TruncateEntity, UpToDate, Error
}

MigrationPlanRiskLevel

public enum MigrationPlanRiskLevel { Low, Medium, High, Critical }

MigrationPlanOperation

MigrationPlanArtifact

The central planning artifact returned by BuildMigrationPlan*.

Assembly Registration & Discovery

Registering Assemblies

MigrationManager scans AppDomain assemblies automatically. Use RegisterAssembly to include assemblies that are loaded lazily or reside in external plugin folders.

// Register a single assembly
migration.RegisterAssembly(typeof(MyEntity).Assembly);

// Register multiple at once
migration.RegisterAssemblies(new[] { asmA, asmB });

// Inspect what is currently registered
IReadOnlyList<Assembly> registered = migration.GetRegisteredAssemblies();

DiscoverEntityTypes

// All Entity-derived types in AppDomain
List<Type> all = migration.DiscoverEntityTypes();

// Scoped to a namespace (sub-namespaces included by default)
List<Type> scoped = migration.DiscoverEntityTypes(namespaceName: "MyApp.Domain");

// Scoped to a specific assembly
List<Type> fromAsm = migration.DiscoverEntityTypes(assembly: typeof(Order).Assembly);

Discovery searches:

1. Manually registered assemblies (RegisterAssembly)
2. Entry assembly and its referenced assemblies
3. AppDomain loaded assemblies
4. DMEEditor's assembly handler (plugin directories)

AssemblyDiscoveryEvidence

Call GetDiscoveryEvidence() after any discovery-based planning call to retrieve a detailed AssemblyDiscoveryEvidence record that maps each found entity type to its source assembly. Use this for diagnostics when expected types are missing from the plan.

var evidence = migration.GetDiscoveryEvidence();
Console.WriteLine($"Scanned: {evidence.TotalAssembliesConsidered}, Skipped: {evidence.Skipped}");
foreach (var kv in evidence.EntityTypeOriginMap)
    Console.WriteLine($"{kv.Key} => {kv.Value}");

Migration Planning

BuildMigrationPlan — Discovery-Based

// Discovers all Entity-derived types, then diffs against live schema
MigrationPlanArtifact plan = migration.BuildMigrationPlan(
    namespaceName: "MyApp.Entities",  // optional namespace filter
    assembly: typeof(Order).Assembly  // optional: limit to one assembly
);

BuildMigrationPlanForTypes — Explicit

// Preferred for production — no discovery noise
var entityTypes = new[] { typeof(Order), typeof(OrderLine), typeof(Customer) };
MigrationPlanArtifact plan = migration.BuildMigrationPlanForTypes(entityTypes);

What the Planner Does

1. Calls BuildReadinessReport — validates entity names, detects duplicates, checks provider capability.
2. Builds a MigrationProviderCapabilityProfile — probes ALTER, RENAME, transactional DDL support live.
3. Creates one MigrationPlanOperation per entity: CreateEntity, AddMissingColumns, UpToDate, etc.
4. Runs EvaluateMigrationPlanPolicy with default policy options.
5. Computes PlanHash (SHA-256) for drift detection and CI diffing.
6. Generates DryRunReport, PreflightReport (stub), CompensationPlan, RollbackReadinessReport, PerformancePlan, CiValidationReport, RolloutGovernanceReport, and an initial ExecutionCheckpoint.
7. Records the plan event to the audit store and migration history.

Readiness Analysis

BuildReadinessReport is called automatically during planning but can also be invoked standalone for pre-planning health checks.

MigrationReadinessReport Properties

MigrationReadinessIssue

// Inspect readiness issues from a plan artifact
foreach (var issue in plan.ReadinessIssues.Where(i => i.Severity == MigrationIssueSeverity.Error))
{
    Console.WriteLine($"[{issue.Code}] {issue.Message}");
    Console.WriteLine($"  Recommendation: {issue.Recommendation}");
}

Provider Capabilities

BuildProviderCapabilityProfile live-probes the datasource helper to determine which DDL operations are actually supported at runtime, not just assumed by provider type.

MigrationProviderCapabilityProfile

Provider Notes

Dry Run & Preflight

GenerateDryRunReport

Produces SQL previews for each planned operation without touching the database.

MigrationDryRunReport dry = migration.GenerateDryRunReport(plan);
foreach (var op in dry.Operations)
{
    Console.WriteLine($"{op.Kind} {op.EntityName} [{op.RiskLevel}]");
    foreach (var ddl in op.DdlStatements)
        Console.WriteLine($"  SQL: {ddl}");
    foreach (var diag in op.Diagnostics)
        Console.WriteLine($"  ! {diag}");
}
Console.WriteLine($"Has blocking issues: {dry.HasBlockingIssues}");

RunPreflightChecks

Performs live checks before executing. Returns PreflightReport.CanApply == false on any block.

MigrationPreflightReport preflight = migration.RunPreflightChecks(plan);
if (!preflight.CanApply)
{
    foreach (var check in preflight.Checks.Where(c => c.Decision == MigrationPolicyDecision.Block))
        Console.WriteLine($"BLOCKED [{check.Code}]: {check.Message}");
}

Preflight checks run in order:

1. preflight-connectivity — live probe of the datasource connection
2. preflight-policy — re-runs policy evaluation against current options
3. preflight-schema-drift — detects if schema changed since the plan was built
4. preflight-operation-window — warns when heavy DDL ops exceed session-safe count

Policy Evaluation

EvaluateMigrationPlanPolicy enforces organisation-defined rules over a plan. It returns a MigrationPolicyEvaluation with a rolled-up Decision (Pass / Warn / Block).

var options = new MigrationPolicyOptions
{
    EnvironmentTier = "production",             // marks as protected
    RequireApprovalForHighRisk = true,
    RequireApprovalForCriticalRisk = true,
    BlockDestructiveInProtectedEnvironments = true,
    AllowDestructiveOverrideInProtectedEnvironments = true,
    Approver = "jane.doe@company.com",
    OverrideReason = "Approved in change ticket CHG-4721"
};

var eval = migration.EvaluateMigrationPlanPolicy(plan, options);
Console.WriteLine(eval.Decision);          // Pass / Warn / Block
Console.WriteLine(eval.RequiresManualApproval);

foreach (var finding in eval.Findings)
    Console.WriteLine($"[{finding.Decision}] {finding.RuleId}: {finding.Message}");

Built-In Policy Rules

Execution

ExecuteMigrationPlan applies the plan in checkpointed steps. It respects a MigrationExecutionPolicy and can be resumed from any intermediate state using an executionToken.

Execution Flow

1. Creates or restores an ExecutionCheckpoint (keyed by executionToken).
2. Verifies high-risk ops have compensation actions.
3. Checks rollback readiness (backup evidence required for protected execution).
4. Runs RunPreflightChecks — blocks on any gate failure.
5. Iterates steps ordered by Sequence, honouring DependsOn ordering.
6. Returns MigrationExecutionResult with success flag, rollback outcome, and checkpoint snapshot.

MigrationExecutionPolicy

var policy = new MigrationExecutionPolicy
{
    ContinueOnNonCriticalFailure = false,  // stop on first step failure
    MaxRetries = 2,
    DryRunOnly = false                     // set true to simulate without DDL
};

Resuming After Failure

// Save the token from the first run
string token = result.ExecutionToken;

// Resume — already-Completed/Skipped steps are automatically bypassed
var resumed = migration.ExecuteMigrationPlan(plan, policy, executionToken: token);

MigrationExecutionResult

Rollback & Compensation

BuildCompensationPlan

Iterates all high-risk operations and generates a MigrationCompensationAction for each with rollback SQL preview, rollback mode, and a step-by-step playbook.

MigrationCompensationPlan comp = migration.BuildCompensationPlan(plan);
foreach (var action in comp.Actions)
{
    Console.WriteLine($"[{action.ActionId}] {action.OperationKind} on {action.EntityName}");
    Console.WriteLine($"  Rollback mode: {action.RollbackMode}");
    Console.WriteLine($"  Preview SQL: {action.RollbackSqlPreview}");
    Console.WriteLine($"  Playbook: {string.Join(" | ", action.CompensationPlaybook)}");
}

CheckRollbackReadiness

Reports whether the plan has sufficient backup evidence to safely proceed. High-risk plans without backup confirmation are blocked from execution.

var report = migration.CheckRollbackReadiness(
    plan,
    backupConfirmed: true,
    restoreTestEvidenceProvided: true,
    restoreTestEvidence: "Restore test completed at 2025-01-15 02:00 UTC by ops-bot");

Console.WriteLine($"IsReady: {report.IsReady}");
foreach (var check in report.Checks)
    Console.WriteLine($"[{check.Decision}] {check.Code}: {check.Message}");

Entity Operations

For simpler migration use-cases, EnsureEntity provides a direct create-or-update interface without building a full plan artifact.

EnsureEntity(EntityStructure)

var entity = new EntityStructure
{
    EntityName = "Orders",
    Fields = new List<EntityField>
    {
        new EntityField { FieldName = "OrderId", Fieldtype = "System.Int32" },
        new EntityField { FieldName = "CustomerId", Fieldtype = "System.String" },
        new EntityField { FieldName = "Total", Fieldtype = "System.Decimal" }
    }
};

// Creates entity if missing; adds any missing columns if it already exists
IErrorsInfo result = migration.EnsureEntity(entity, createIfMissing: true, addMissingColumns: true);
if (result.Flag == Errors.Ok)
    Console.WriteLine(result.Message);
else
    Console.WriteLine($"Failed: {result.Message}");

EnsureEntity(Type)

// Converts POCO via classCreator and delegates to EntityStructure overload
IErrorsInfo result = migration.EnsureEntity(
    typeof(Order),
    createIfMissing: true,
    addMissingColumns: true,
    detectRelationships: true);

Performance & Scale Planning

BuildPerformancePlan annotates each operation with lock-impact score, estimated runtime, and execution window mode, then rolls up maintenance-window guidance.

MigrationPerformancePlan perf = migration.BuildPerformancePlan(plan);

Console.WriteLine($"Planned window: {perf.Kpis.PlannedMigrationWindowMinutes} min");
Console.WriteLine($"Lock timeout policy: {perf.Policy.LockTimeoutMilliseconds} ms");
Console.WriteLine($"Throttled mode: {perf.Policy.EnableThrottledMode}");

foreach (var annotation in perf.OperationAnnotations)
    Console.WriteLine($"{annotation.EntityName}: lock={annotation.EstimatedLockImpactScore}/10, {annotation.WindowMode}");

MigrationPerformancePolicy Defaults

Observability & Telemetry

Telemetry Snapshot

MigrationTelemetrySnapshot snap = migration.GetMigrationTelemetrySnapshot(executionToken);

Console.WriteLine($"Plans run: {snap.Metrics.PlanCount}");
Console.WriteLine($"Executions: {snap.Metrics.ExecutionCount}");
Console.WriteLine($"Success rate: {snap.SuccessRate:P2}");
Console.WriteLine($"Failure rate: {snap.FailureRate:P2}");
Console.WriteLine($"Rollbacks: {snap.Metrics.RollbackCount}");
Console.WriteLine($"Policy blocks: {snap.Metrics.PolicyBlockCount}");
Console.WriteLine($"Avg step duration: {snap.Metrics.AverageStepDurationMilliseconds:F1} ms");

Diagnostics

// All diagnostics for a run
var diags = migration.GetMigrationDiagnostics(executionToken);
// Filter to errors only
var errors = migration.GetMigrationDiagnostics(
    executionToken,
    minimumSeverity: MigrationDiagnosticSeverity.Error);

Audit Events

var audits = migration.GetMigrationAuditEvents(executionToken);
foreach (var evt in audits)
    Console.WriteLine($"[{evt.TimestampUtc:O}] {evt.EventType} by {evt.ExecutedBy}: {evt.Result}");

Approving a Plan

// Transitions LifecycleState from Draft/Reviewed → Approved
// and records an audit event
plan = migration.ApproveMigrationPlan(plan, approvedBy: "alice@company.com", notes: "CHG-4721");

Rollout Governance

EvaluateRolloutGovernance evaluates whether a migration plan is safe to promote through a deployment wave, using live telemetry KPIs and hard-stop rules.

MigrationRolloutWave

public enum MigrationRolloutWave
{
    Wave1NonCritical,  // safe for non-critical datasources
    Wave2Standard,     // general production
    Wave3Critical      // critical/regulated systems
}

Governance Gates

var request = new MigrationRolloutGovernanceRequest
{
    Wave = MigrationRolloutWave.Wave2Standard,
    ReviewedBy = "platform-team",
    Thresholds = new MigrationRolloutKpiThresholds { MinSuccessRate = 0.98 }
};
var gov = migration.EvaluateRolloutGovernance(plan, request);
Console.WriteLine($"CanPromote: {gov.CanPromote}");
foreach (var gate in gov.Gates)
    Console.WriteLine($"[{gate.Decision}] {gate.Gate}: {gate.Observed} vs threshold {gate.Threshold}");

CI/CD Automation

ValidatePlanForCi

Runs plan lint, policy, dry-run, and portability gates. Returns CanMerge boolean for PR workflows.

MigrationCiValidationReport ci = migration.ValidatePlanForCi(plan);
if (!ci.CanMerge)
{
    foreach (var gate in ci.Gates.Where(g => g.Decision == MigrationPolicyDecision.Block))
        Console.WriteLine($"BLOCKED: {gate.Gate} — {gate.Message}");
    Environment.Exit(1);
}

BuildMigrationPlanDiff

Computes a textual diff of operation signatures between two plan artifacts for PR annotations.

string diff = migration.BuildMigrationPlanDiff(previousPlan, currentPlan);
Console.WriteLine(diff);

CI Pipeline Example

// In your CI build step
var migration = new MigrationManager(dmeEditor);
migration.MigrateDataSource = dmeEditor.GetDataSource(connectionName);
var plan = migration.BuildMigrationPlanForTypes(entityTypes);

var ci = migration.ValidatePlanForCi(plan, new MigrationPolicyOptions
{
    EnvironmentTier = "staging",
    RequireApprovalForHighRisk = false  // relax for staging
});

if (!ci.CanMerge)
    throw new Exception("Migration plan failed CI validation gates.");

// Serialize plan hash for deployment artifact tracking
Console.WriteLine($"::set-output name=plan_hash::{plan.PlanHash}");

DDL Evidence & Migration History

Every executed DDL operation is recorded as a DdlOperationEvidence entry with a SHA-256 hash of the SQL statement for tamper-detection and audit purposes.

// Read the DDL evidence log
IReadOnlyList<DdlOperationEvidence> evidence = migration.GetDdlEvidence();
foreach (var entry in evidence)
{
    Console.WriteLine($"[{entry.TimestampUtc:O}] {entry.OperationName} on {entry.EntityName}");
    Console.WriteLine($"  Outcome: {entry.Outcome}  SQL hash: {entry.SqlHash}");
    Console.WriteLine($"  Reason: {entry.ReasonCode}  Helper: {entry.HelperSource}");
}

DdlOperationEvidence Properties

End-to-End Examples

Example 1 — Production-Safe Migration

var migration = new MigrationManager(dmeEditor);
migration.MigrateDataSource = dmeEditor.GetDataSource("proddb");
migration.RegisterAssembly(typeof(OrderLine).Assembly);

// Build the plan
var plan = migration.BuildMigrationPlanForTypes(
    new[] { typeof(Order), typeof(OrderLine), typeof(Customer) });

// Abort if readiness is blocking
if (plan.ReadinessIssues.Any(i => i.Severity == MigrationIssueSeverity.Error))
    throw new Exception("Readiness errors must be resolved before migration.");

// Set production policy options
var policyOptions = new MigrationPolicyOptions
{
    EnvironmentTier = "production",
    RequireApprovalForHighRisk = true,
    BlockDestructiveInProtectedEnvironments = true,
    Approver = "john.smith",
    OverrideReason = "CHG-9001 approved by change board"
};
plan.PolicyEvaluation = migration.EvaluateMigrationPlanPolicy(plan, policyOptions);
if (plan.PolicyEvaluation.Decision == MigrationPolicyDecision.Block)
    throw new Exception("Policy evaluation blocked the migration plan.");

// Confirm rollback readiness
plan.RollbackReadinessReport = migration.CheckRollbackReadiness(
    plan,
    backupConfirmed: true,
    restoreTestEvidenceProvided: true,
    restoreTestEvidence: "Restore test run by DBA team 2025-06-01");

// Approve and execute
migration.ApproveMigrationPlan(plan, approvedBy: "john.smith");
var result = migration.ExecuteMigrationPlan(plan, new MigrationExecutionPolicy());
if (!result.Success)
    Console.Error.WriteLine($"Migration failed: {result.Message}");
else
    Console.WriteLine($"Migration completed. Token: {result.ExecutionToken}");

Example 2 — Rapid Development (No Policy)

var migration = new MigrationManager(dmeEditor) { MigrateDataSource = devDs };

// Single-call ensure — no plan artifact needed
migration.EnsureEntity(typeof(BlogPost),
    createIfMissing: true,
    addMissingColumns: true,
    detectRelationships: false);

Example 3 — CI Validation in a Build Pipeline

var plan = migration.BuildMigrationPlanForTypes(entityTypes);
var ci = migration.ValidatePlanForCi(plan);
if (!ci.CanMerge)
{
    foreach (var g in ci.Gates.Where(x => x.Decision == MigrationPolicyDecision.Block))
        Console.Error.WriteLine($"CI BLOCK — {g.Gate}: {g.Message}");
    Environment.Exit(1);
}
Console.WriteLine($"CI passed. Plan hash: {plan.PlanHash}");

Example 4 — Governance Before Promotion

var snap = migration.GetMigrationTelemetrySnapshot(execToken);
var gov = migration.EvaluateRolloutGovernance(plan, new MigrationRolloutGovernanceRequest
{
    Wave = MigrationRolloutWave.Wave3Critical,
    IsCriticalDataSource = true,
    ReviewedBy = "platform-lead"
});
if (!gov.CanPromote)
    throw new Exception($"Governance hard-stop: {gov.HardStopReason}");