Mapping Manager

Overview

MappingManager (namespace TheTechIdea.Beep.Editor.Mapping) is a static partial class that manages all field-level mappings between source and destination entities. It spans 8 partial files covering every phase of the mapping lifecycle:

MappingManager (static partial class)
├── MappingManager.cs                  ← CRUD: CreateEntityMap / Load / Save / Map / Apply
├── MappingManager.Conventions.cs      ← AutoMapByConvention, NameMatchMode, MappingDiff
├── MappingManager.AutoMatching.cs     ← AutoMapByConventionWithScoring, confidence engine
├── MappingManager.ConversionPipeline.cs ← Typed conversion, field transform chains, policy
├── MappingManager.ObjectGraph.cs      ← Nested POCO / collection graph mapping
├── MappingManager.Rules.cs            ← Rule-set validation, execution traces, priorities
├── MappingManager.ValidationScoring.cs ← Quality score (0-100), drift detection
└── MappingManager.Governance.cs       ← Versioning, approval state, audit trail

AutoObjMapper (separate class — lightweight POCO mapper)
└── Core / Configuration / Interfaces / Extensions / Utilities

Core data types (from Beep.Workflow)

Quick Start

Create and apply a mapping

// 1. Create mapping from source → destination entity
var (errInfo, mapping) = MappingManager.CreateEntityMap(
    dmeEditor,
    srcEntityName:  "Contacts",
    srcDataSourceName: "ExternalCRM",
    destEntityName: "Customers",
    destDataSourceName: "MainDB");

// 2. Apply the mapping to a record
var result = MappingManager.MapObject(dmeEditor, mapping, sourceRecord, "Customers", "MainDB");

// 3. Save the mapping
MappingManager.SaveMapping(dmeEditor, "Customers", "MainDB", mapping);

Auto-map with confidence scoring

var options = new AutoMatchOptions
{
    AutoAcceptThreshold = 0.85,
    EnableFuzzyDistanceMatching = true,
    EnableSynonymMatching = true
};

var (mapping, report) = MappingManager.AutoMapByConventionWithScoring(
    dmeEditor,
    "Contacts", "ExternalCRM",
    "Customers", "MainDB",
    NameMatchMode.CaseInsensitive,
    options);

Console.WriteLine($"Auto-accepted: {report.AutoAcceptedCount}");
Console.WriteLine($"Review needed: {report.ReviewRequiredCount}");
Console.WriteLine($"Rejected:      {report.RejectedCount}");

Entity Map API

Core CRUD and application methods on MappingManager (static).

Create methods

Load / Save

Map / Apply

Intelligent Auto-Matching

AutoMapByConventionWithScoring uses a multi-strategy scoring engine to propose field matches with a confidence score (0–1). Each destination field receives a ranked list of candidates.

AutoMatchOptions

MatchConfidenceDecision enum

MappingAutoMatchReport

FieldMatchSuggestion / FieldMatchCandidate

foreach (var suggestion in report.Suggestions)
{
    Console.WriteLine($"Dest: {suggestion.DestinationFieldName}  Ambiguous: {suggestion.IsAmbiguous}");
    foreach (var c in suggestion.Candidates)
        Console.WriteLine($"  Source: {c.SourceFieldName}  Score: {c.ConfidenceScore:P0}  Decision: {c.Decision}");
}

Convention-Based Mapping

AutoMapByConvention builds a mapping without confidence scoring — useful for simple same-schema migrations or when you just need a functional starting point.

NameMatchMode enum

// Simple convention map
var mapping = MappingManager.AutoMapByConvention(
    dmeEditor,
    "Contacts", "CRM",
    "Customers", "MainDB",
    NameMatchMode.FuzzyPrefix);

// Diff two mapping versions
var diff = MappingManager.GetMappingDiff(baselineMapping, updatedMapping);
Console.WriteLine($"Added: {diff.Added.Count}  Removed: {diff.Removed.Count}  Changed: {diff.Changed.Count}");

Conversion Pipeline

Each field assignment passes through a configurable conversion pipeline: policy → transform chain → type coercion. All state is stored in thread-safe ConcurrentDictionary keyed by "{destDS}::{destEntity}" or "{destDS}::{destEntity}::{field}".

MappingConversionPolicy

Policy methods

// Set per-entity policy
MappingManager.SetConversionPolicy("MainDB", "Customers", new MappingConversionPolicy
{
    NullHandlingMode = MappingNullHandlingMode.SubstituteDefault,
    NullSubstituteValue = string.Empty,
    TrimStrings = true,
    StringCaseMode = MappingStringCaseMode.Lower,
    FailureMode = MappingConversionFailureMode.WarnAndContinue
});

var policy = MappingManager.GetConversionPolicy("MainDB", "Customers");

Field transform chains

A MappingFieldTransformChain is a named sequence of FieldTransformSteps applied to a single destination field before type coercion. Each step references a registered transform function.

// Register a custom transform
MappingManager.RegisterTransform("padleft10", (value, arg) =>
    value?.ToString()?.PadLeft(10, '0') ?? string.Empty);

// Assign a chain to a specific field
MappingManager.SetFieldTransformChain("MainDB", "Customers", "CustomerCode",
    new MappingFieldTransformChain
    {
        Steps = new List<FieldTransformStep>
        {
            new FieldTransformStep { Name = "trim",      Argument = "" },
            new FieldTransformStep { Name = "uppercase", Argument = "" },
            new FieldTransformStep { Name = "padleft10", Argument = "" }
        }
    });

var chain = MappingManager.GetFieldTransformChain("MainDB", "Customers", "CustomerCode");

Object Graph Mapping

MapObjectGraph traverses nested POCOs and collections, respecting a configurable max depth and cycle detection. Use for Domain Object → Domain Object mapping where fields may be nested paths (e.g. "Address.City" → "ShippingCity").

ObjectGraphMappingOptions

ObjectGraphMappingResult

var opts = new ObjectGraphMappingOptions
{
    MaxDepth = 3,
    CollectionMode = CollectionMappingMode.MergeByKey,
    CollectionMergeKeyPropertyName = "ProductId"
};

var (destObj, result) = MappingManager.MapObjectGraph(
    dmeEditor,
    destinationEntityName: "OrderDto",
    selectedMapping: mapDetail,
    sourceObject: sourceOrder,
    options: opts);

Console.WriteLine($"Assignments: {result.Assignments}  Skipped: {result.Skipped}");
foreach (var w in result.Warnings) Console.WriteLine($"  WARN: {w}");

Rule Engine Integration

Each Mapping_rep_fields can carry a Rules string that is evaluated by the Beep rules tokenizer at mapping time. MappingManager.Rules.cs handles rule-set validation and execution tracing.

MappingRulePriority enum

MappingRuleDecision enum

Rule validation

// Validate all rules in a mapping
var ruleResult = MappingManager.ValidateRuleSet(mapping);
if (!ruleResult.IsValid)
    foreach (var e in ruleResult.Errors)
        Console.WriteLine($"ERROR: {e}");

foreach (var w in ruleResult.Warnings)
    Console.WriteLine($"WARN: {w}");

MappingRuleExecutionTrace

Returned internally per field assignment — useful for troubleshooting. Properties:

• Priority — which rule tier fired
• Decision — Map / Skip / UseDefault
• SourcePath / TargetPath — resolved paths
• ConditionMatched — whether the rule's when clause evaluated to true
• Message — human-readable description of the decision

Validation & Quality Scoring

ValidateMappingWithScore produces a 0–100 quality score and a band label, plus a field-level MappingDriftReport comparing the mapping against the current entity structures.

MappingQualityBand enum

MappingQualityReport

MappingIssueSeverity enum

var quality = MappingManager.ValidateMappingWithScore(dmeEditor, mapping, productionThreshold: 75);

Console.WriteLine($"Score: {quality.Score}/100  Band: {quality.Band}  Ready: {quality.MeetsProductionThreshold}");

foreach (var issue in quality.Issues.Where(i => i.Severity == MappingIssueSeverity.Error))
    Console.WriteLine($"[ERROR] {issue.Code}: {issue.Message}  → {issue.Recommendation}");

if (quality.DriftReport.DriftDetected)
    foreach (var entry in quality.DriftReport.Entries)
        Console.WriteLine($"Drift: {entry.ChangeType} on '{entry.FieldName}': {entry.Recommendation}");

Governance & Versioning

The governance subsystem attaches a MappingGovernanceRecord to each entity mapping, tracking version history, approval state, and a full audit trail. All state is persisted to the config store as JSON. Changes made inside a BeginGovernanceScope are automatically stamped with the author and reason.

MappingApprovalState enum

MappingGovernanceRecord

Governance API

// Stamp changes with author context
using (MappingManager.BeginGovernanceScope("alice", "Q2 schema alignment", MappingApprovalState.Review))
{
    var (_, mapping) = MappingManager.CreateEntityMap(dmeEditor, "Contacts", "CRM", "Customers", "MainDB");
    MappingManager.SaveMapping(dmeEditor, "Customers", "MainDB", mapping);
}

// Approve for production
MappingManager.UpdateMappingApprovalState(
    dmeEditor, "Customers", "MainDB",
    MappingApprovalState.Approved,
    actor: "bob", reason: "Code review passed");

// Inspect history
var history = MappingManager.GetMappingVersionHistory(dmeEditor, "Customers", "MainDB");
foreach (var v in history)
    Console.WriteLine($"v{v.Version}  {v.Author}  {v.ApprovalState}  {v.ChangeReason}");

Performance & Caching

The performance partial compiles mapping plans and property accessors into delegates and caches them to avoid repeated reflection. Caches are bounded to prevent unbounded growth.

// Invalidate compiled plans for a specific entity (e.g. after schema change)
MappingManager.InvalidateMappingCaches("MainDB", "Customers");

AutoObjMapper — POCO Mapper

AutoObjMapper is a lightweight, expression-compiled POCO-to-POCO mapper. It compiles assignment actions per (TSource, TDest) type pair and caches them. Inspired by AutoMapper / TinyMapper but without external dependencies.

IAutoObjMapper API

ITypeMapConfiguration<TSource, TDest> fluent API

var mapper = new AutoObjMapper();

// Configure type-specific behavior
mapper.Configure(cfg =>
{
    cfg.For<ContactDto, CustomerEntity>()
       .Ignore("CreatedAt")
       .ForMember("FullName", src => $"{src.FirstName} {src.LastName}")
       .BeforeMap((src, dest) => dest.ImportedAt = DateTime.UtcNow)
       .AfterMap((src, dest) => dest.IsActive = true);
});

// Map single object
var customer = mapper.Map<ContactDto, CustomerEntity>(contactDto);

// Map into existing destination
var existing = GetExistingCustomer(id);
mapper.Map<ContactDto, CustomerEntity>(contactDto, existing);

// Inspect stats
var stats = mapper.GetStatistics();
Console.WriteLine($"Cached mappers: {stats.CachedMappersCount}");

AutoObjMapperOptions

// Create mapper with custom options
var mapper = new AutoObjMapper(new AutoObjMapperOptions
{
    IgnoreNullSourceValues = true,  // skip null source → don't overwrite dest
    MapPrivateMembers = false       // default; only public properties
});

Examples

End-to-end: auto-match → validate → govern → apply

// 1. Auto-match with confidence scoring
var (mapping, report) = MappingManager.AutoMapByConventionWithScoring(
    dmeEditor, "Contacts", "CRM", "Customers", "MainDB",
    NameMatchMode.CaseInsensitive,
    new AutoMatchOptions { AutoAcceptThreshold = 0.88, EnableSynonymMatching = true });

// 2. Configure conversion policy
MappingManager.SetConversionPolicy("MainDB", "Customers", new MappingConversionPolicy
{
    TrimStrings = true,
    StringCaseMode = MappingStringCaseMode.Lower,
    FailureMode = MappingConversionFailureMode.WarnAndContinue
});

// 3. Validate quality
var quality = MappingManager.ValidateMappingWithScore(dmeEditor, mapping, productionThreshold: 75);
if (!quality.MeetsProductionThreshold)
{
    Console.WriteLine($"Score {quality.Score} below threshold — review issues:");
    foreach (var i in quality.Issues) Console.WriteLine($"  [{i.Severity}] {i.Message}");
    return;
}

// 4. Stamp author context and save
using (MappingManager.BeginGovernanceScope("alice", "initial CRM import mapping"))
    MappingManager.SaveMapping(dmeEditor, "Customers", "MainDB", mapping);

// 5. Approve
MappingManager.UpdateMappingApprovalState(
    dmeEditor, "Customers", "MainDB",
    MappingApprovalState.Approved, "bob", "QA sign-off");

// 6. Apply to records
var result = MappingManager.MapObject(dmeEditor, mapping, sourceContact, "Customers", "MainDB");

Custom field transforms on a specific column

// Register transforms once (e.g. at startup)
MappingManager.RegisterTransform("strip_dashes", (v, _) => v?.ToString()?.Replace("-", "") ?? "");
MappingManager.RegisterTransform("uppercase",    (v, _) => v?.ToString()?.ToUpper() ?? "");

// Bind to the SKU field
MappingManager.SetFieldTransformChain("MainDB", "Products", "SKU",
    new MappingFieldTransformChain
    {
        Steps = new List<FieldTransformStep>
        {
            new FieldTransformStep { Name = "trim" },
            new FieldTransformStep { Name = "strip_dashes" },
            new FieldTransformStep { Name = "uppercase" }
        }
    });

AutoObjMapper with before/after hooks

var mapper = new AutoObjMapper();
mapper.Configure(cfg =>
{
    cfg.For<OrderApiDto, OrderEntity>()
       .ForMember("TotalAmount", src => src.Lines.Sum(l => l.Price * l.Qty))
       .BeforeMap((src, dest) => dest.CreatedAt = DateTime.UtcNow)
       .AfterMap((src, dest) => dest.Status = OrderStatus.New);
});

var orders = apiDtos.Select(dto => mapper.Map<OrderApiDto, OrderEntity>(dto)).ToList();

Check mapping drift after schema update

var (_, updatedMapping) = MappingManager.CreateEntityMap(dmeEditor, "Contacts", "CRM", "Customers", "MainDB");
var diff = MappingManager.GetMappingDiff(savedMapping, updatedMapping);

if (diff.HasChanges)
{
    Console.WriteLine($"Added:   {diff.Added.Count}");
    Console.WriteLine($"Removed: {diff.Removed.Count}");
    Console.WriteLine($"Changed: {diff.Changed.Count}");
    // Re-run quality check before re-approving
}

Full API Reference

MappingManager — Complete Static API

Create Methods

Load / Save

Map / Apply / Transform

Auto-Matching

Validation & Quality

Governance

Rules

Conversion Pipeline

Performance / Caching

AutoObjMapper Complete Interfaces

IAutoObjMapper

IMapperConfiguration

ITypeMapConfiguration<TSource, TDest>

ITypeMapBase

AutoObjMapperOptions

Complete Enums Reference

Helper Classes

Configuration Types

Source File Map (30 files)