DMEEditor

Source of truth (2026)

Main editor implementation: DataManagementEngineStandard/Editor/DM/DMEEditor.cs.
Universal helper integration: DataManagementEngineStandard/Editor/DM/DMEEditor.UniversalDataSourceHelpers.cs.
Data source retrieval and orchestration APIs are validated against current editor methods.
Workflow surface is accessed via dmeEditor.WorkFlowEditor.

Overview
Core Properties
Data Source Management
Logging and Error Handling
ETL Operations
Configuration Management
Usage Examples
Best Practices

Overview

The DMEEditor class is the central orchestrator of the Beep Data Management Engine. It serves as the main entry point for all data operations, providing a unified interface to manage data sources, execute ETL processes, handle configurations, and coordinate complex data operations across multiple systems.

Core Purpose

DMEEditor encapsulates all functionality of data management into a single, powerful interface that can handle enterprise-level scenarios with ease.

Signature-Verified Snapshot (Editor/DM)

public partial class DMEEditor : IDMEEditor, IDisposable
{
    public bool ContainerMode { get; set; }
    public string ContainerName { get; set; }
    public List<IDataSource> DataSources { get; set; }
    public IETL ETL { get; set; }
    public IConfigEditor ConfigEditor { get; set; }
    public IDataTypesHelper typesHelper { get; set; }
    public IUtil Utilfunction { get; set; }
    public IAssemblyHandler assemblyHandler { get; set; }
    public IDMLogger Logger { get; set; }
    public IWorkFlowEditor WorkFlowEditor { get; set; }

    public virtual IDataSource GetDataSourceById(string identifier, bool useGuid = false);
    public virtual EntityStructure GetEntityStructure(string entityname, string datasourcename);
    public void AddLogMessage(string logType, string logMessage, DateTime logDate, int recordId, string miscData, Errors flag);
}

// Universal helper integration surface
public partial class DMEEditor
{
    public IDataSourceHelper GetDataSourceHelper(DataSourceType datasourceType);
    public void RegisterDataSourceHelper(DataSourceType datasourceType, IDataSourceHelper helper);
}

Data Source Management

Create, manage, and coordinate multiple data sources simultaneously

ETL Operations

Extract, Transform, and Load data between different systems

Logging & Monitoring

Comprehensive logging and error tracking across all operations

Configuration

Centralized configuration management for all components

Core Properties

▶ Essential Components

Property	Type	Description
DataSources	List<IDataSource>	Collection of all managed data sources
ETL	IETL	Extract, Transform, Load processing engine
ConfigEditor	IConfigEditor	Configuration management interface
Logger	IDMLogger	Centralized logging system
ErrorObject	IErrorsInfo	Global error tracking and reporting
WorkFlowEditor	IWorkFlowEditor	Workflow and process management
typesHelper	IDataTypesHelper	Data type mapping and conversion utilities
assemblyHandler	IAssemblyHandler	Dynamic assembly loading and management

▶ Container & Multi-Tenancy

Property	Type	Description
ContainerMode	bool	Enable multi-tenant application support
ContainerName	string	Name of the current container/tenant
progress	IProgress<PassedArgs>	Progress reporting for long-running operations

Data Source Management

▶ Core Data Source Methods

IDataSource GetDataSource(string dataSourceName)

Retrieves or creates a data source by name. Automatically handles connection creation and entity loading.

IDataSource GetDataSourceUsingGuidID(string guidId)

Retrieves or creates a data source using its GUID identifier for more robust identification.

ConnectionState OpenDataSource(string dataSourceName)

Opens a connection to the specified data source and returns the connection state.

bool CloseDataSource(string dataSourceName)

Safely closes a data source connection and releases resources.

▶ Data Source Creation

IDataSource CreateNewDataSourceConnection(string dataSourceName)

Creates a new data source connection using existing configuration.

IDataSource CreateNewDataSourceConnection(ConnectionProperties connectionProps, string dataSourceName)

Creates a new data source with custom connection properties.

IDataSource CreateLocalDataSourceConnection(ConnectionProperties dataConnection, string dataSourceName, string classHandler)

Creates a local database data source with specified class handler.

Data Source Management Example

// Get or create a data source
IDataSource customerDb = dmeEditor.GetDataSource("CustomerDatabase");

// Check if data source exists
if (dmeEditor.CheckDataSourceExist("CustomerDatabase"))
{
    // Open the connection
    var connectionState = dmeEditor.OpenDataSource("CustomerDatabase");
    
    if (connectionState == ConnectionState.Open)
    {
        // Get entity structure
        var customerStructure = dmeEditor.GetEntityStructure("Customers", "CustomerDatabase");
        
        // Perform operations...
        
        // Close when done
        dmeEditor.CloseDataSource("CustomerDatabase");
    }
}

Logging and Error Handling

▶ Logging Methods

void AddLogMessage(string logType, string message, DateTime logDate, int recordId, string miscData, Errors flag)

Comprehensive logging method with detailed context information and error classification.

void AddLogMessage(string message)

Simple logging method for basic message recording.

void RaiseEvent(object sender, PassedArgs args)

Raises global events for component communication and notification.

Error Handling Best Practice

Always check the ErrorObject.Flag property after operations to ensure proper error handling and logging.

Logging Example

try
{
    // Perform data operation
    var result = dmeEditor.GetDataSource("CustomerDB");
    
    if (dmeEditor.ErrorObject.Flag == Errors.Ok)
    {
        dmeEditor.AddLogMessage("Success", "Data source loaded successfully", 
                              DateTime.Now, 0, "CustomerDB", Errors.Ok);
    }
}
catch (Exception ex)
{
    dmeEditor.AddLogMessage("Error", $"Failed to load data source: {ex.Message}", 
                          DateTime.Now, 0, "CustomerDB", Errors.Failed);
}

ETL Operations

DMEEditor provides direct access to ETL operations through its ETL property:

▶ ETL Access Methods

Method	Description
ETL.CopyEntityData()	Copy data between entities
ETL.CreateScriptHeader()	Generate ETL script headers
ETL.RunCreateScript()	Execute entity creation scripts
ETL.RunImportScript()	Execute data import operations

ETL Operation Example

// Create ETL script for data migration
var sourceDs = dmeEditor.GetDataSource("SourceDB");
var destDs = dmeEditor.GetDataSource("DestinationDB");

// Generate create entity scripts
var entities = new List<string> { "Customers", "Orders", "Products" };
var scripts = dmeEditor.ETL.GetCreateEntityScript(sourceDs, entities, 
                                                 progress, cancellationToken, true);

// Set up the ETL script
dmeEditor.ETL.Script.ScriptDetails.AddRange(scripts);

// Execute the migration
var result = await dmeEditor.ETL.RunCreateScript(progress, cancellationToken);

Configuration Management

DMEEditor provides centralized configuration management through the ConfigEditor property:

▶ Configuration Properties

Property	Description
ConfigEditor.DataConnections	Manage data source connections
ConfigEditor.DataSourcesClasses	Data source class definitions
ConfigEditor.DataDriversClasses	Driver configurations
ConfigEditor.JsonLoader	JSON serialization utilities

Usage Examples

Complete DMEEditor Workflow

// Initialize DMEEditor (typically done through dependency injection)
var dmeEditor = new DMEEditor(logger, utilFunction, errorObject, 
                             configEditor, assemblyHandler);

// Create and configure a new data connection
var connectionProps = new ConnectionProperties
{
    ConnectionName = "ProductionDB",
    DatabaseType = DataSourceType.SqlServer,
    ConnectionString = "Server=prod-server;Database=Production;Integrated Security=true;",
    GuidID = Guid.NewGuid().ToString()
};

// Add connection to configuration
dmeEditor.ConfigEditor.AddDataConnection(connectionProps);

// Get and open the data source
var dataSource = dmeEditor.GetDataSource("ProductionDB");
var connectionState = dmeEditor.OpenDataSource("ProductionDB");

if (connectionState == ConnectionState.Open)
{
    // Get entity structure
    var customerStructure = dmeEditor.GetEntityStructure("Customers", "ProductionDB");
    
    // Retrieve data with error handling
    var customerData = dmeEditor.GetData(dataSource, customerStructure);
    
    if (dmeEditor.ErrorObject.Flag == Errors.Ok)
    {
        dmeEditor.AddLogMessage("Success", "Customer data retrieved successfully", 
                              DateTime.Now, 0, null, Errors.Ok);
        
        // Process data...
    }
    else
    {
        dmeEditor.AddLogMessage("Error", "Failed to retrieve customer data", 
                              DateTime.Now, 0, null, Errors.Failed);
    }
    
    // Clean up
    dmeEditor.CloseDataSource("ProductionDB");
}

// Dispose when done
dmeEditor.Dispose();

Multi-DataSource Operations

// Working with multiple data sources
var sourceDb = dmeEditor.GetDataSource("LegacySystem");
var targetDb = dmeEditor.GetDataSource("ModernSystem");

// Open both connections
dmeEditor.OpenDataSource("LegacySystem");
dmeEditor.OpenDataSource("ModernSystem");

// Copy data between systems
var result = dmeEditor.ETL.CopyEntityData(
    sourceDb, targetDb, 
    "LegacyCustomers", "NewCustomers", 
    progress, cancellationToken);

if (dmeEditor.ErrorObject.Flag == Errors.Ok)
{
    dmeEditor.AddLogMessage("ETL", "Data migration completed successfully", 
                          DateTime.Now, 0, null, Errors.Ok);
}

Best Practices

Resource Management

Always dispose of DMEEditor instances when done
Close data source connections explicitly
Use using statements for automatic disposal
Monitor memory usage in long-running operations

Error Handling

Check ErrorObject.Flag after every operation
Use comprehensive logging for debugging
Implement proper exception handling
Use progress reporting for long operations

Performance Optimization

Reuse DMEEditor instances when possible
Keep data source connections open for batch operations
Use async methods for I/O operations
Implement proper caching strategies

Home ETL Editor

Table of Contents

Overview

Signature-Verified Snapshot (Editor/DM)

Data Source Management

ETL Operations

Logging & Monitoring

Configuration

Core Properties

Data Source Management

Data Source Management Example

Logging and Error Handling

Logging Example

ETL Operations

ETL Operation Example

Configuration Management

Usage Examples

Complete DMEEditor Workflow

Multi-DataSource Operations

Best Practices