Registration & Setup - Beep Data Management Engine Documentation

Overview

The RegisterBeep class sets up the Beep Data Management Engine for direct data operations. Once configured, you work with DMEEditor to manage data sources, create connections, and perform data operations across multiple database types.

🎯 Purpose

RegisterBeep provides the foundation for data management - after setup, you use dmeEditor directly for all data operations.

1ï¸âƒ£ Bootstrap

Register services with AddBeepServices() and get IDMEEditor

2ï¸âƒ£ Initialize

Add default configurations for drivers, mappings, and queries

3ï¸âƒ£ Connect

Create connection configurations for your data sources

4ï¸âƒ£ Operate

Perform CRUD operations and data management tasks

⚙️ Initial Setup & Registration

âš ï¸ CRITICAL SEQUENCE

The initialization must follow this exact order:

Initialize DMEEditor via AddBeepServices() / BeepService.Configure()
Load ConnectionHelper drivers using GetAllConnectionConfigs() - THIS IS MANDATORY
Configure ConfigEditor with specialized managers
Create specific connections for your application

Step 1: Basic Service Setup

Corrected (2026). Earlier revisions of this page showed an Autofac ContainerBuilder registering DMEEditor, ConfigEditor and friends by type. That was never correct: BeepDM does not reference Autofac, and DMEEditor cannot be constructed by convention — it needs a logger, util, error object, config editor and assembly handler wired in a specific order. BeepService exists to do exactly that. Use AddBeepServices() (or new BeepService()) as shown below.

using Microsoft.Extensions.DependencyInjection;
using TheTechIdea.Beep;
using TheTechIdea.Beep.ConfigUtil;
using TheTechIdea.Beep.Services;
using TheTechIdea.Beep.Utilities;

static void Main()
{
    var services = new ServiceCollection();

    // Register and bootstrap BeepDM in one call.
    // BeepService constructs ConfigEditor, Util, the assembly handler and DMEEditor,
    // then registers them all in the service collection.
    IBeepService beep = services.AddBeepServices()
        .WithDirectory(AppContext.BaseDirectory)
        .WithAppRepo("MyApp")
        .WithConfigType(BeepConfigType.Application)
        .WithMapping()
        .WithAssemblyLoading()
        .AsSingleton()
        .Build();

    var dmeEditor = beep.DMEEditor;

    Console.WriteLine("✅ BeepDM core services initialized successfully");

    // STEP 2: Load drivers and plugins (skip if WithAssemblyLoading() was used)
    beep.LoadAssemblies();

    // STEP 3: Create your application connections
    CreateApplicationConnections(dmeEditor);
}

Three equivalent entry points exist, all in RegisterBeepinServiceCollection.cs:

// 1. Fluent builder (shown above)
services.AddBeepServices().WithAppRepo("MyApp").AsSingleton().Build();

// 2. Options delegate
services.AddBeepServices(o =>
{
    o.DirectoryPath = AppContext.BaseDirectory;
    o.AppRepoName   = "MyApp";
    o.ConfigType    = BeepConfigType.Application;
});

// 3. Positional shorthand (addAsSingleton defaults to true)
services.Register(AppContext.BaseDirectory, "MyApp", BeepConfigType.Application);

Step 2: ConnectionHelper Driver Initialization (CRITICAL)

/// <summary>
/// STEP 2: Initialize all connection drivers from ConnectionHelper
/// This MUST be done BEFORE creating any connections
/// </summary>
public static void InitializeConnectionDrivers(IDMEEditor dmeEditor)
{
    try
    {
        Console.WriteLine("🔧 Initializing connection drivers from ConnectionHelper...");
        
        // Get all 39 predefined connection configurations from ConnectionHelper
        var allConnectionConfigs = ConnectionHelper.GetAllConnectionConfigs();
        
        Console.WriteLine($"📦 Loading {allConnectionConfigs.Count} connection driver configurations:");
        
        // Add each configuration to the DMEEditor
        foreach (var config in allConnectionConfigs)
        {
            // Avoid duplicates by checking GuidID
            if (!dmeEditor.ConfigEditor.DataDriversClasses.Any(d => d.GuidID == config.GuidID))
            {
                dmeEditor.ConfigEditor.DataDriversClasses.Add(config);
            }
        }
        
        // Save configurations to persistent storage
        dmeEditor.ConfigEditor.SaveConnectionDriversConfigValues();
        
        Console.WriteLine($"✅ Successfully loaded {allConnectionConfigs.Count} driver configurations");
        
        // Show what was loaded by category
        var driversByCategory = allConnectionConfigs.GroupBy(c => c.DatasourceCategory);
        foreach (var categoryGroup in driversByCategory)
        {
            Console.WriteLine($"   📁 {categoryGroup.Key}: {categoryGroup.Count()} drivers");
        }
        
        Console.WriteLine("📊 Available driver categories:");
        Console.WriteLine("   🏢 RDBMS: Enterprise databases (SQL Server, PostgreSQL, MySQL, DB2, etc.)");
        Console.WriteLine("   📄 FILE: File formats (CSV, Excel, JSON, Text)");
        Console.WriteLine("   🗄️ NoSQL: Document & NoSQL databases (MongoDB, CouchDB, etc.)");
        Console.WriteLine("   🧠 VectorDB: AI/ML vector databases (ChromaDB, Pinecone, Qdrant, etc.)");
        Console.WriteLine("   🌐 WebAPI: REST and web service connections");
        Console.WriteLine("   ☁️ Cloud: Cloud and streaming services");
        
        // Verify critical drivers are available
        VerifyEssentialDrivers(dmeEditor);
    }
    catch (Exception ex)
    {
        Console.WriteLine($"❌ CRITICAL ERROR: Failed to initialize connection drivers: {ex.Message}");
        Console.WriteLine("💡 Cannot proceed without connection drivers. Application will not function properly.");
        throw;
    }
}

/// <summary>
/// Verify that essential drivers are available for basic operations
/// </summary>
private static void VerifyEssentialDrivers(IDMEEditor dmeEditor)
{
    var essentialDrivers = new[]
    {
        DataSourceType.SqlLite,    // Most commonly used embedded database
        DataSourceType.CSV,        // File operations
        DataSourceType.SqlServer,  // Enterprise database
        DataSourceType.Xls         // Excel files
    };
    
    Console.WriteLine("🔍 Verifying essential drivers...");
    
    foreach (var driverType in essentialDrivers)
    {
        var driver = dmeEditor.ConfigEditor.DataDriversClasses
            .FirstOrDefault(d => d.DatasourceType == driverType);
        
        if (driver != null)
        {
            Console.WriteLine($"   ✅ {driverType}: {driver.PackageName} v{driver.version}");
        }
        else
        {
            Console.WriteLine($"   ❌ {driverType}: NOT FOUND");
        }
    }
}

ConnectionHelper Methods Available

// ConnectionHelper provides these key static methods:

// 1. Get ALL driver configurations (MOST IMPORTANT)
var allConfigs = ConnectionHelper.GetAllConnectionConfigs();
// Returns List<ConnectionDriversConfig> with 39 preconfigured drivers

// 2. Create specific driver configurations
var sqlServerConfig = ConnectionHelper.CreateSqlServerConfig();
var sqliteConfig = ConnectionHelper.CreateSQLiteConfig();
var csvConfig = ConnectionHelper.CreateCSVDataSourceConfig();
var excelConfig = ConnectionHelper.CreateTxtXlsCSVFileSourceConfig();
var mongoConfig = ConnectionHelper.CreateMongoDBConfig();
var chromaConfig = ConnectionHelper.CreateChromaDBConfig();
var pineconeConfig = ConnectionHelper.CreatePineConeConfig();
// ... and 32 more specific Create methods

// 3. Connection string management
string connectionString = ConnectionHelper.ReplaceValueFromConnectionString(
    driverConfig, connectionProperties, dmeEditor);

// 4. Connection validation
bool isValid = ConnectionHelper.IsConnectionStringValid(connectionString, dataSourceType);

// 5. Security features
string secureString = ConnectionHelper.SecureConnectionString(connectionString);

// 6. Link connections to drivers
var linkedConfig = ConnectionHelper.LinkConnection2Drivers(connectionProperties, configEditor);

// EXAMPLE: Using individual Create methods (alternative to GetAllConnectionConfigs)
public static void LoadSpecificDrivers(IDMEEditor dmeEditor)
{
    // Instead of loading all 39, you can load specific ones
    var driversToLoad = new List<ConnectionDriversConfig>
    {
        ConnectionHelper.CreateSqlServerConfig(),
        ConnectionHelper.CreateSQLiteConfig(),
        ConnectionHelper.CreateCSVDataSourceConfig(),
        ConnectionHelper.CreateMongoDBConfig(),
        ConnectionHelper.CreateChromaDBConfig()
    };
    
    foreach (var config in driversToLoad)
    {
        if (!dmeEditor.ConfigEditor.DataDriversClasses.Any(d => d.GuidID == config.GuidID))
        {
            dmeEditor.ConfigEditor.DataDriversClasses.Add(config);
        }
    }
    
    dmeEditor.ConfigEditor.SaveConnectionDriversConfigValues();
    Console.WriteLine($"✅ Loaded {driversToLoad.Count} specific drivers");
}

✅ ConnectionHelper Benefits

39 Pre-configured Drivers: Complete coverage of all supported data sources
Production Ready: All configurations tested and optimized
Connection String Templates: Parameterized templates with proper escaping
Security Features: Built-in password masking and validation
Type Safety: Proper enum mappings and category classifications
Version Management: Handles driver version compatibility

Windows Forms Application Pattern

// Program.cs for Windows Forms
using Microsoft.Extensions.DependencyInjection;
using System;
using System.Windows.Forms;
using TheTechIdea.Beep;
using TheTechIdea.Beep.ConfigUtil;
using TheTechIdea.Beep.Services;

[STAThread]
static void Main()
{
    Application.EnableVisualStyles();
    Application.SetCompatibleTextRenderingDefault(false);

    try
    {
        var services = new ServiceCollection();

        // Singleton is the right lifetime for desktop
        IBeepService beep = services.AddBeepServices()
            .WithDirectory(AppContext.BaseDirectory)
            .WithAppRepo("MyWinFormsApp")
            .WithConfigType(BeepConfigType.Application)
            .WithMapping()
            .AsSingleton()
            .Build();

        // Load drivers and plugins
        beep.LoadAssemblies();

        var provider = services.BuildServiceProvider();

        // Start main form
        Application.Run(new MainForm(beep.DMEEditor));
    }
    catch (Exception ex)
    {
        MessageBox.Show($"Application initialization failed: {ex.Message}",
                       "Startup Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
    }
}

Singleton is the correct lifetime for a desktop process. See Service Registration & Lifetimes for the full builder surface.

WPF Application Pattern

// App.xaml.cs for WPF
using Microsoft.Extensions.DependencyInjection;
using System.Windows;
using TheTechIdea.Beep;
using TheTechIdea.Beep.ConfigUtil;
using TheTechIdea.Beep.Services;

public partial class App : Application
{
    private IBeepService beep;
    private IDMEEditor dmeEditor;

    protected override void OnStartup(StartupEventArgs e)
    {
        base.OnStartup(e);

        try
        {
            var services = new ServiceCollection();

            beep = services.AddBeepServices()
                .WithDirectory(AppContext.BaseDirectory)
                .WithAppRepo("MyWpfApp")
                .WithConfigType(BeepConfigType.Application)
                .WithMapping()
                .AsSingleton()
                .Build();

            // Load drivers and plugins
            beep.LoadAssemblies();

            dmeEditor = beep.DMEEditor;

            // Create main window
            var mainWindow = new MainWindow(dmeEditor);
            mainWindow.Show();
        }
        catch (Exception ex)
        {
            MessageBox.Show($"Application initialization failed: {ex.Message}", 
                           "Startup Error", MessageBoxButton.OK, MessageBoxImage.Error);
            this.Shutdown(1);
        }
    }
    
    protected override void OnExit(ExitEventArgs e)
    {
        // Dispose the service, not the editor directly - BeepService.Dispose()
        // disposes DMEEditor, ConfigEditor and the assembly handler for you.
        beep?.Dispose();
        base.OnExit(e);
    }
}

❌ Common Mistake - Wrong Order

This is the WRONG way - trying to create connections before loading drivers:

// BAD - This will fail with "Driver not found" errors
var beep = services.AddBeepServices()
    .WithAppRepo("MyApp")
    .WithAssemblyLoading(false)   // ❌ drivers never loaded
    .AsSingleton()
    .Build();
var dmeEditor = beep.DMEEditor;

// Trying to create connection BEFORE loading drivers
var connection = new ConnectionProperties { /* ... */ };
dmeEditor.ConfigEditor.AddDataConnection(connection); // ❌ FAILS!

// Loading drivers AFTER trying to use them
var configs = ConnectionHelper.GetAllConnectionConfigs(); // ❌ TOO LATE!

✅ Correct Order

// GOOD - Proper sequence
var beep = services.AddBeepServices()
    .WithDirectory(AppContext.BaseDirectory)
    .WithAppRepo("MyApp")
    .WithConfigType(BeepConfigType.Application)
    .AsSingleton()
    .Build();

var dmeEditor = beep.DMEEditor;

// Load drivers FIRST
beep.LoadAssemblies();   // scans plugins and restores driver config

// EnvironmentService.AddAllConnectionConfigurations() already seeds these during
// Configure(), but you can top up explicitly if you skipped it:
var configs = ConnectionHelper.GetAllConnectionConfigs();
foreach (var config in configs)
{
    dmeEditor.ConfigEditor.DataDriversClasses.Add(config);
}

// NOW you can create connections
var connection = new ConnectionProperties { /* ... */ };
dmeEditor.ConfigEditor.AddDataConnection(connection); // ✅ WORKS!

⚙️ ConfigEditor & Its Specialized Managers

Step 3: After loading the connection drivers in Step 2, the ConfigEditor can be properly configured to manage all configuration aspects of the Beep Data Management Engine. It uses specialized manager classes to handle different configuration responsibilities, providing a clean separation of concerns.

📝 Prerequisites

Before configuring ConfigEditor, ensure you have:

✅ Initialized DMEEditor via AddBeepServices() (Step 1)
✅ Loaded connection drivers from ConnectionHelper (Step 2)
🎯 Ready to configure ConfigEditor specialized managers (Step 3)

📁 ConfigPathManager

Manages configuration file paths and directory structure

🔗 DataConnectionManager

Handles data source connection lifecycle and configuration

📝 QueryManager

Manages saved queries and SQL repositories

🗺️ EntityMappingManager

Handles entity structure and field mapping configurations

🧩 ComponentConfigManager

Manages component, driver, and plugin configurations

🕘 MigrationHistoryManager

Tracks per-datasource schema migration history files

These six managers live in DataManagementEngineStandard/ConfigUtil/Managers/. ConfigEditor is a façade that delegates to them — always write configuration through the façade rather than hand-rolling JSON.

Step 3: ConfigEditor Configuration (After Driver Loading)

/// <summary>
/// STEP 3: Configure ConfigEditor after connection drivers are loaded
/// This step sets up specialized managers and persistent configuration
/// </summary>
public static void ConfigureConfigEditor(IDMEEditor dmeEditor)
{
    try
    {
        Console.WriteLine("⚙️ Configuring ConfigEditor with specialized managers...");
        
        var configEditor = dmeEditor.ConfigEditor;
        
        // Verify that drivers were loaded in Step 2
        if (configEditor.DataDriversClasses.Count == 0)
        {
            throw new InvalidOperationException("No connection drivers found. Did you run InitializeConnectionDrivers() first?");
        }
        
        Console.WriteLine($"✅ Found {configEditor.DataDriversClasses.Count} connection drivers loaded");
        
        // Set configuration type and ensure proper paths
        configEditor.ConfigType = BeepConfigType.DataConnections;
        
        // Set up configuration path if not already set
        if (string.IsNullOrEmpty(configEditor.ConfigPath))
        {
            configEditor.ConfigPath = Path.Combine(
                Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), 
                "BeepDataEngine", 
                "Config"
            );
        }
        
        // Ensure configuration directory exists
        Directory.CreateDirectory(configEditor.ConfigPath);
        Console.WriteLine($"📁 Config directory: {configEditor.ConfigPath}");
        
        // Initialize configuration collections if they're null
        InitializeConfigCollections(configEditor);
        
        // Load existing configuration values from persistent storage
        configEditor.LoadConfigValues();
        
        // Add default queries and mappings
        AddDefaultConfigurations(configEditor);
        
        Console.WriteLine("✅ ConfigEditor configuration completed");
        ShowConfigSummary(configEditor);
    }
    catch (Exception ex)
    {
        Console.WriteLine($"❌ Failed to configure ConfigEditor: {ex.Message}");
        throw;
    }
}

/// <summary>
/// Initialize configuration collections to ensure they're not null
/// </summary>
private static void InitializeConfigCollections(IConfigEditor configEditor)
{
    configEditor.DataConnections ??= new List<ConnectionProperties>();
    configEditor.DataSourcesClasses ??= new List<AssemblyClassDefinition>();
    // DataDriversClasses should already be populated from Step 2
    configEditor.EntityMaps ??= new List<EntityDataMap>();
    configEditor.DataTypesMap ??= new List<DataTypesMap>();
    configEditor.QueryList ??= new List<QuerySqlRepo>();
    configEditor.WorkFlowStepsConfig ??= new List<WorkFlowStepsConfig>();
    
    Console.WriteLine("📋 Configuration collections initialized");
}

/// <summary>
/// Add default configurations like common queries and mappings
/// </summary>
private static void AddDefaultConfigurations(IConfigEditor configEditor)
{
    // Add common SQL queries if none exist
    if (configEditor.QueryList.Count == 0)
    {
        AddCommonQueries(configEditor);
    }
    
    // Add basic entity mappings if none exist
    if (configEditor.EntityMaps.Count == 0)
    {
        AddBasicEntityMappings(configEditor);
    }
    
    Console.WriteLine("🔧 Default configurations added");
}

/// <summary>
/// Show summary of current configuration state
/// </summary>
private static void ShowConfigSummary(IConfigEditor configEditor)
{
    Console.WriteLine("📊 Configuration Summary:");
    Console.WriteLine($"   🔗 Data Connections: {configEditor.DataConnections.Count}");
    Console.WriteLine($"   🧩 Driver Classes: {configEditor.DataDriversClasses.Count}");
    Console.WriteLine($"   🗺️ Entity Maps: {configEditor.EntityMaps.Count}");
    Console.WriteLine($"   📝 Queries: {configEditor.QueryList.Count}");
    Console.WriteLine($"   📊 Data Type Maps: {configEditor.DataTypesMap.Count}");
    Console.WriteLine($"   ⚙️ Workflow Steps: {configEditor.WorkFlowStepsConfig.Count}");
}

✅ ConfigEditor Benefits

Specialized Managers: Each configuration aspect has a dedicated manager
Persistent Storage: All configurations are saved to disk automatically
Consistent Interface: Unified API for all configuration operations
Type Safety: Strongly-typed configuration objects
Extensible: Easy to add new configuration types and managers
Thread Safe: Safe for concurrent access from multiple threads

📝 Configuration Persistence

The ConfigEditor automatically saves configurations to JSON files in the application data folder:

DataConnections.json: All data source connections
ConnectionConfig.json: Driver configurations
Mapping/{datasource}/{entity}_Mapping.json: Entity mapping definitions
QueryList.json: Saved SQL queries
WorkFlow/DataWorkFlow.json: Workflow configurations

🔗 Connection Management

Based on real implementation patterns from the sample application:

SQLite Database Connection

public static bool CreateConnectionForSqlite(IDMEEditor dmeEditor)
{
    try
    {
        // Get the connection driver configuration
        var config = dmeEditor.ConfigEditor.DataDriversClasses
            .FirstOrDefault(p => p.DatasourceType == DataSourceType.SqlLite);
        
        if (config == null)
        {
            Console.WriteLine("❌ SQLite driver not found");
            return false;
        }
        
        // Create connection properties
        var connectionProperties = new ConnectionProperties
        {
            ConnectionString = "Data Source=./Beep/dbfiles/northwind.db",
            ConnectionName = "northwind.db",
            DriverName = config.PackageName,
            DriverVersion = config.version,
            DatabaseType = DataSourceType.SqlLite,
            Category = DatasourceCategory.RDBMS
        };
        
        // Add to connection list
        dmeEditor.ConfigEditor.AddDataConnection(connectionProperties);
        
        // Get and test the data source
        var sqliteDataSource = dmeEditor.GetDataSource("northwind.db");
        if (sqliteDataSource != null)
        {
            sqliteDataSource.Openconnection();
            
            if (sqliteDataSource.ConnectionStatus == ConnectionState.Open)
            {
                Console.WriteLine("✅ SQLite connection established");
                return true;
            }
        }
        
        Console.WriteLine("❌ Failed to connect to SQLite database");
        return false;
    }
    catch (Exception ex)
    {
        Console.WriteLine($"❌ SQLite connection error: {ex.Message}");
        return false;
    }
}

Excel File Connection

public static bool CreateConnectionForXls(IDMEEditor dmeEditor)
{
    try
    {
        var config = dmeEditor.ConfigEditor.DataDriversClasses
            .FirstOrDefault(p => p.DatasourceType == DataSourceType.Xls);
        
        if (config == null)
        {
            Console.WriteLine("❌ Excel driver not found");
            return false;
        }
        
        var connectionProperties = new ConnectionProperties
        {
            FileName = "country.xls",
            FilePath = "./dbfiles",
            ConnectionName = "country.xls",
            DriverName = config.PackageName,
            DriverVersion = config.version,
            DatabaseType = DataSourceType.Xls,
            Ext = "xls",
            Category = DatasourceCategory.FILE
        };
        
        dmeEditor.ConfigEditor.AddDataConnection(connectionProperties);
        
        var xlsDataSource = dmeEditor.GetDataSource("country.xls");
        if (xlsDataSource != null)
        {
            xlsDataSource.Openconnection();
            
            if (xlsDataSource.ConnectionStatus == ConnectionState.Open)
            {
                Console.WriteLine("✅ Excel file connection established");
                return true;
            }
        }
        
        Console.WriteLine("❌ Failed to connect to Excel file");
        return false;
    }
    catch (Exception ex)
    {
        Console.WriteLine($"❌ Excel connection error: {ex.Message}");
        return false;
    }
}

CSV File Connection

public static bool CreateConnectionForCSV(IDMEEditor dmeEditor)
{
    try
    {
        var config = dmeEditor.ConfigEditor.DataDriversClasses
            .FirstOrDefault(p => p.DatasourceType == DataSourceType.CSV);
        
        if (config == null)
        {
            Console.WriteLine("❌ CSV driver not found");
            return false;
        }
        
        var connectionProperties = new ConnectionProperties
        {
            FileName = "iris.csv",
            FilePath = "./dbfiles",
            ConnectionName = "iris.csv",
            DriverName = config.PackageName,
            DriverVersion = config.version,
            DatabaseType = DataSourceType.CSV,
            Ext = "csv",
            Category = DatasourceCategory.FILE
        };
        
        dmeEditor.ConfigEditor.AddDataConnection(connectionProperties);
        
        var csvDataSource = dmeEditor.GetDataSource("iris.csv");
        if (csvDataSource != null)
        {
            csvDataSource.Openconnection();
            
            if (csvDataSource.ConnectionStatus == ConnectionState.Open)
            {
                Console.WriteLine("✅ CSV file connection established");
                return true;
            }
        }
        
        Console.WriteLine("❌ Failed to connect to CSV file");
        return false;
    }
    catch (Exception ex)
    {
        Console.WriteLine($"❌ CSV connection error: {ex.Message}");
        return false;
    }
}

Generic Connection Creator

public static bool CreateConnection(IDMEEditor dmeEditor, string connectionName, DataSourceType dataSourceType, 
    string connectionString = null, string filePath = null, string fileName = null)
{
    try
    {
        // Get driver configuration
        var config = dmeEditor.ConfigEditor.DataDriversClasses
            .FirstOrDefault(p => p.DatasourceType == dataSourceType);
        
        if (config == null)
        {
            Console.WriteLine($"❌ Driver for {dataSourceType} not found");
            return false;
        }
        
        // Build connection properties based on data source type
        var connectionProperties = new ConnectionProperties
        {
            ConnectionName = connectionName,
            DriverName = config.PackageName,
            DriverVersion = config.version,
            DatabaseType = dataSourceType,
            Category = GetDataSourceCategory(dataSourceType)
        };
        
        // Set type-specific properties
        if (dataSourceType == DataSourceType.SqlLite || 
            dataSourceType == DataSourceType.SqlServer ||
            dataSourceType == DataSourceType.MySQL ||
            dataSourceType == DataSourceType.Postgre)
        {
            connectionProperties.ConnectionString = connectionString;
        }
        else if (IsFileBasedDataSource(dataSourceType))
        {
            connectionProperties.FilePath = filePath;
            connectionProperties.FileName = fileName;
            connectionProperties.Ext = Path.GetExtension(fileName)?.TrimStart('.');
        }
        
        // Add connection and test
        dmeEditor.ConfigEditor.AddDataConnection(connectionProperties);
        
        var dataSource = dmeEditor.GetDataSource(connectionName);
        if (dataSource != null)
        {
            dataSource.Openconnection();
            
            if (dataSource.ConnectionStatus == ConnectionState.Open)
            {
                Console.WriteLine($"✅ {dataSourceType} connection '{connectionName}' established");
                return true;
            }
        }
        
        Console.WriteLine($"❌ Failed to connect to {dataSourceType} '{connectionName}'");
        return false;
    }
    catch (Exception ex)
    {
        Console.WriteLine($"❌ Connection error for {connectionName}: {ex.Message}");
        return false;
    }
}

private static DatasourceCategory GetDataSourceCategory(DataSourceType dataSourceType)
{
    return dataSourceType switch
    {
        DataSourceType.SqlServer or DataSourceType.MySQL or 
        DataSourceType.Postgre or DataSourceType.SqlLite => DatasourceCategory.RDBMS,
        
        DataSourceType.CSV or DataSourceType.Xls or 
        DataSourceType.Text => DatasourceCategory.FILE,
        
        DataSourceType.MongoDB => DatasourceCategory.NOSQL,
        
        DataSourceType.WebAPI or DataSourceType.REST => DatasourceCategory.WEBAPI,
        
        _ => DatasourceCategory.RDBMS
    };
}

private static bool IsFileBasedDataSource(DataSourceType dataSourceType)
{
    return dataSourceType is DataSourceType.CSV or DataSourceType.Xls or 
           DataSourceType.Text or DataSourceType.Json;
}

ðŸ’¾ Data Operations

Basic Data Access Pattern

// Core pattern for all data operations
public static void WorkWithDataSource(IDMEEditor dmeEditor, string connectionName)
{
    try
    {
        // Get data source from DMEEditor
        var dataSource = dmeEditor.GetDataSource(connectionName);
        
        if (dataSource == null)
        {
            Console.WriteLine($"❌ Data source '{connectionName}' not found");
            return;
        }
        
        // Ensure connection is open
        dataSource.Openconnection();
        
        if (dataSource.ConnectionStatus != ConnectionState.Open)
        {
            Console.WriteLine($"❌ Failed to open connection to '{connectionName}'");
            return;
        }
        
        Console.WriteLine($"✅ Connected to '{connectionName}'");
        
        // Get list of entities (tables/sheets/collections)
        var entities = dataSource.GetEntitesList();
        Console.WriteLine($"📋 Found {entities.Count} entities:");
        
        foreach (var entity in entities)
        {
            Console.WriteLine($"   â€¢ {entity}");
        }
        
        // Work with first entity if available
        if (entities.Count > 0)
        {
            string firstEntity = entities[0];
            
            // Get entity structure (schema information)
            var structure = dataSource.GetEntityStructure(firstEntity, true);
            Console.WriteLine($"📊 Entity '{firstEntity}' has {structure.Fields?.Count ?? 0} fields");
            
            // Get data from entity
            var data = dataSource.GetEntity(firstEntity, null);
            
            if (data != null)
            {
                var dataList = data as IList ?? new List<object> { data };
                Console.WriteLine($"📈 Retrieved {dataList.Count} records from '{firstEntity}'");
                
                // Process the data
                ProcessEntityData(firstEntity, data, structure);
            }
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine($"❌ Error working with data source: {ex.Message}");
        dmeEditor.AddLogMessage("Beep", 
            $"Data operation error: {ex.Message}", 
            DateTime.Now, -1, null, Errors.Failed);
    }
    finally
    {
        // Always clean up
        dataSource?.Closeconnection();
    }
}

Working with Multiple Data Sources

public static void LoadDataFromMultipleSources(IDMEEditor dmeEditor)
{
    try
    {
        // Work with different data source types
        var dataSources = new Dictionary<string, string>
        {
            { "northwind.db", "SQLite Database" },
            { "country.xls", "Excel File" },
            { "iris.csv", "CSV File" }
        };
        
        foreach (var kvp in dataSources)
        {
            Console.WriteLine($"\n🔍 Processing {kvp.Value} ({kvp.Key})...");
            
            var dataSource = dmeEditor.GetDataSource(kvp.Key);
            if (dataSource != null)
            {
                dataSource.Openconnection();
                
                if (dataSource.ConnectionStatus == ConnectionState.Open)
                {
                    var entities = dataSource.GetEntitesList();
                    Console.WriteLine($"   Found {entities.Count} entities");
                    
                    // Get sample data from first entity
                    if (entities.Count > 0)
                    {
                        var sampleData = dataSource.GetEntity(entities[0], null);
                        var dataList = sampleData as IList ?? new List<object> { sampleData };
                        Console.WriteLine($"   Sample entity '{entities[0]}' has {dataList.Count} records");
                    }
                }
                else
                {
                    Console.WriteLine($"   ❌ Could not connect to {kvp.Value}");
                }
                
                dataSource.Closeconnection();
            }
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine($"❌ Error processing multiple data sources: {ex.Message}");
    }
}

Data Movement Between Sources

// Real implementation from BeepSharedFunctions
public static IErrorsInfo MoveEntity(IDMEEditor dmeEditor, string sourceName, string destName, 
    string entityName, Progress<PassedArgs> progress = null)
{
    IDataSource sourceDataSource = null;
    IDataSource destDataSource = null;
    
    try
    {
        // Get source and destination data sources
        sourceDataSource = dmeEditor.GetDataSource(sourceName);
        destDataSource = dmeEditor.GetDataSource(destName);
        
        if (sourceDataSource == null || destDataSource == null)
        {
            var errorMsg = $"Source '{sourceName}' or destination '{destName}' not found";
            dmeEditor.AddLogMessage("Beep", errorMsg, 
                DateTime.Now, 0, "", Errors.Failed);
            return dmeEditor.ErrorObject;
        }
        
        // Open connections
        sourceDataSource.Openconnection();
        destDataSource.Openconnection();
        
        if (sourceDataSource.ConnectionStatus != ConnectionState.Open)
        {
            throw new InvalidOperationException($"Cannot open source connection to '{sourceName}'");
        }
        
        if (destDataSource.ConnectionStatus != ConnectionState.Open)
        {
            throw new InvalidOperationException($"Cannot open destination connection to '{destName}'");
        }
        
        Console.WriteLine($"🔄 Moving entity '{entityName}' from {sourceName} to {destName}");
        
        // Get entity structure from source
        var srcEntityStructure = sourceDataSource.GetEntityStructure(entityName, true);
        if (srcEntityStructure == null)
        {
            throw new InvalidOperationException($"Entity '{entityName}' not found in source '{sourceName}'");
        }
        
        // Get data from source
        var sourceData = sourceDataSource.GetEntity(entityName, null);
        if (sourceData == null)
        {
            Console.WriteLine($"âš ï¸ No data found in source entity '{entityName}'");
            return dmeEditor.ErrorObject;
        }
        
        // Create entity in destination if it doesn't exist
        if (!destDataSource.CheckEntityExist(entityName))
        {
            Console.WriteLine($"📋 Creating entity '{entityName}' in destination");
            var createResult = destDataSource.CreateEntityAs(srcEntityStructure);
            
            if (createResult.Flag != Errors.Ok)
            {
                throw new InvalidOperationException($"Failed to create entity in destination: {createResult.Message}");
            }
        }
        
        // Move the data with progress reporting
        var updateResult = destDataSource.UpdateEntities(entityName, sourceData, progress);
        
        if (updateResult.Flag == Errors.Ok)
        {
            var dataList = sourceData as IList ?? new List<object> { sourceData };
            Console.WriteLine($"✅ Successfully moved {dataList.Count} records");
        }
        else
        {
            Console.WriteLine($"❌ Failed to move data: {updateResult.Message}");
        }
        
        return updateResult;
    }
    catch (Exception ex)
    {
        var errorMsg = $"Error moving entity '{entityName}': {ex.Message}";
        Console.WriteLine($"❌ {errorMsg}");
        
        dmeEditor.AddLogMessage("Beep", errorMsg, 
            DateTime.Now, -1, ex.StackTrace, Errors.Failed);
        
        return dmeEditor.ErrorObject;
    }
    finally
    {
        // Clean up connections
        sourceDataSource?.Closeconnection();
        destDataSource?.Closeconnection();
    }
}

ðŸ–¥ï¸ UI Integration Examples

Windows Forms with BeepGrid

public partial class DataExplorerForm : Form
{
    private readonly IDMEEditor dmeEditor;
    private IDataSource currentDataSource;
    
    public DataExplorerForm(IDMEEditor editor)
    {
        InitializeComponent();
        dmeEditor = editor;
        
        // Configure BeepGrid with services
        ConfigureBeepGrid();
        
        // Load available data sources
        LoadDataSources();
    }
    
    private void ConfigureBeepGrid()
    {
        beepGrid1.SetConfig(
            dmeEditor,                      // Data management editor
            dmeEditor.Logger,               // Logger
            dmeEditor.util,                 // Utilities
            new string[] { },               // Additional parameters
            dmeEditor.Passedarguments,      // Passed arguments
            dmeEditor.ErrorObject           // Error handling
        );
        
        Console.WriteLine("🎛️ BeepGrid configured with DME services");
    }
    
    private void LoadDataSources()
    {
        try
        {
            dataSourcesComboBox.Items.Clear();
            
            // Get all configured data sources
            var dataSources = dmeEditor.ConfigEditor.DataConnections;
            
            foreach (var connection in dataSources)
            {
                dataSourcesComboBox.Items.Add(connection.ConnectionName);
            }
            
            Console.WriteLine($"📋 Loaded {dataSources.Count} data sources");
        }
        catch (Exception ex)
        {
            MessageBox.Show($"Error loading data sources: {ex.Message}", 
                "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
        }
    }
    
    private void OnDataSourceSelected(object sender, EventArgs e)
    {
        if (dataSourcesComboBox.SelectedItem == null) return;
        
        try
        {
            string selectedDataSource = dataSourcesComboBox.SelectedItem.ToString();
            
            // Get and open data source
            currentDataSource = dmeEditor.GetDataSource(selectedDataSource);
            
            if (currentDataSource != null)
            {
                currentDataSource.Openconnection();
                
                if (currentDataSource.ConnectionStatus == ConnectionState.Open)
                {
                    LoadEntities();
                    Console.WriteLine($"✅ Connected to {selectedDataSource}");
                }
                else
                {
                    MessageBox.Show($"Failed to connect to {selectedDataSource}", 
                        "Connection Error", MessageBoxButtons.OK, MessageBoxIcon.Warning);
                }
            }
        }
        catch (Exception ex)
        {
            MessageBox.Show($"Error connecting to data source: {ex.Message}", 
                "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
        }
    }
    
    private void LoadEntities()
    {
        try
        {
            entitiesComboBox.Items.Clear();
            
            if (currentDataSource != null && 
                currentDataSource.ConnectionStatus == ConnectionState.Open)
            {
                var entities = currentDataSource.GetEntitesList();
                
                foreach (string entity in entities)
                {
                    entitiesComboBox.Items.Add(entity);
                }
                
                Console.WriteLine($"📊 Loaded {entities.Count} entities");
            }
        }
        catch (Exception ex)
        {
            MessageBox.Show($"Error loading entities: {ex.Message}", 
                "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
        }
    }
    
    private void OnEntitySelected(object sender, EventArgs e)
    {
        if (entitiesComboBox.SelectedItem == null || currentDataSource == null) return;
        
        try
        {
            string entityName = entitiesComboBox.SelectedItem.ToString();
            
            // Get data and structure
            var data = currentDataSource.GetEntity(entityName, null);
            var structure = currentDataSource.GetEntityStructure(entityName, true);
            
            // Bind to BeepGrid
            beepGrid1.ResetData(data, structure);
            
            // Update status
            var dataList = data as IList ?? new List<object> { data };
            statusLabel.Text = $"Loaded {dataList.Count} records from {entityName}";
            
            Console.WriteLine($"📈 Loaded {dataList.Count} records into grid");
        }
        catch (Exception ex)
        {
            MessageBox.Show($"Error loading entity data: {ex.Message}", 
                "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
        }
    }
    
    protected override void OnFormClosed(FormClosedEventArgs e)
    {
        try
        {
            // Clean up data source connection
            currentDataSource?.Closeconnection();
            Console.WriteLine("ðŸ”Œ Data source connection closed");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"âš ï¸ Error closing connection: {ex.Message}");
        }
        
        base.OnFormClosed(e);
    }
}

💡 UI Integration Tips

Always configure Beep controls with the service instances
Handle connection errors gracefully with user-friendly messages
Clean up data source connections when forms close
Use progress indicators for long-running operations

Usage Examples

Shared Functions Class (Production Pattern)

/// <summary>
/// Shared functions for data management operations
/// Based on actual implementation in the sample application
/// </summary>
public static class BeepSharedFunctions
{
    public static IDMEEditor dmeEditor;
    public static IDataSource SourceDataSource;
    public static IDataSource DestinationDataSource;
    
    /// <summary>
    /// Get connection driver configuration for a specific data source type
    /// </summary>
    public static ConnectionDriversConfig GetConnectionDriversConfig(DataSourceType type)
    {
        return dmeEditor.ConfigEditor.DataDriversClasses
            .FirstOrDefault(p => p.DatasourceType == type);
    }
    
    /// <summary>
    /// Create connections for commonly used data sources
    /// </summary>
    public static void CreateStandardConnections()
    {
        try
        {
            Console.WriteLine("🔗 Creating standard data connections...");
            
            var results = new Dictionary<string, bool>
            {
                ["SQLite"] = CreateConnectionForSqlite(),
                ["Excel"] = CreateConnectionForXls(),
                ["CSV"] = CreateConnectionForCSV()
            };
            
            foreach (var result in results)
            {
                var status = result.Value ? "✅" : "❌";
                Console.WriteLine($"   {status} {result.Key}: {(result.Value ? "Connected" : "Failed")}");
            }
            
            var successCount = results.Values.Count(r => r);
            Console.WriteLine($"📊 {successCount}/{results.Count} connections established");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"❌ Error creating connections: {ex.Message}");
        }
    }
    
    /// <summary>
    /// Get all available data sources with their status
    /// </summary>
    public static List<DataSourceInfo> GetDataSourceStatus()
    {
        var dataSourceInfoList = new List<DataSourceInfo>();
        
        try
        {
            var connections = dmeEditor.ConfigEditor.DataConnections;
            
            foreach (var connection in connections)
            {
                var dataSource = dmeEditor.GetDataSource(connection.ConnectionName);
                var info = new DataSourceInfo
                {
                    Name = connection.ConnectionName,
                    Type = connection.DatabaseType.ToString(),
                    IsConnected = dataSource?.ConnectionStatus == ConnectionState.Open,
                    EntityCount = 0
                };
                
                // Get entity count if connected
                if (info.IsConnected)
                {
                    try
                    {
                        var entities = dataSource.GetEntitesList();
                        info.EntityCount = entities?.Count ?? 0;
                    }
                    catch
                    {
                        info.EntityCount = -1; // Error getting entities
                    }
                }
                
                dataSourceInfoList.Add(info);
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine($"❌ Error getting data source status: {ex.Message}");
        }
        
        return dataSourceInfoList;
    }
    
    /// <summary>
    /// Close all open data source connections
    /// </summary>
    public static void CloseAllConnections()
    {
        try
        {
            var connections = dmeEditor.ConfigEditor.DataConnections;
            int closedCount = 0;
            
            foreach (var connection in connections)
            {
                var dataSource = dmeEditor.GetDataSource(connection.ConnectionName);
                if (dataSource?.ConnectionStatus == ConnectionState.Open)
                {
                    dataSource.Closeconnection();
                    closedCount++;
                }
            }
            
            Console.WriteLine($"ðŸ”Œ Closed {closedCount} data source connections");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"❌ Error closing connections: {ex.Message}");
        }
    }
    
    /// <summary>
    /// Test connectivity to all configured data sources
    /// </summary>
    public static Dictionary<string, bool> TestAllConnections()
    {
        var results = new Dictionary<string, bool>();
        
        try
        {
            var connections = dmeEditor.ConfigEditor.DataConnections;
            
            Console.WriteLine("🧪 Testing all data source connections...");
            
            foreach (var connection in connections)
            {
                try
                {
                    var dataSource = dmeEditor.GetDataSource(connection.ConnectionName);
                    if (dataSource != null)
                    {
                        dataSource.Openconnection();
                        bool isConnected = dataSource.ConnectionStatus == ConnectionState.Open;
                        results[connection.ConnectionName] = isConnected;
                        
                        if (isConnected)
                        {
                            dataSource.Closeconnection();
                        }
                    }
                    else
                    {
                        results[connection.ConnectionName] = false;
                    }
                }
                catch
                {
                    results[connection.ConnectionName] = false;
                }
            }
            
            var successCount = results.Values.Count(r => r);
            Console.WriteLine($"📊 Connection test results: {successCount}/{results.Count} successful");
        }
        catch ( Exception ex)
        {
            Console.WriteLine($"❌ Error testing connections: {ex.Message}");
        }
        
        return results;
    }
}

/// <summary>
/// Data source information for status reporting
/// </summary>
public class DataSourceInfo
{
    public string Name { get; set; }
    public string Type { get; set; }
    public bool IsConnected { get; set; }
    public int EntityCount { get; set; }
    
    public override string ToString()
    {
        var status = IsConnected ? "Connected" : "Disconnected";
        var entities = EntityCount >= 0 ? $", {EntityCount} entities" : "";
        return $"{Name} ({Type}) - {status}{entities}";
    }
}

Best Practices

🔍 Connection Verification

Always check connection status before operations

var dataSource = dmeEditor.GetDataSource("MyDatabase");
if (dataSource?.ConnectionStatus == ConnectionState.Open)
{
    // Safe to perform operations
}

âš ï¸ Error Handling

Implement comprehensive error handling and logging

try
{
    var data = dataSource.GetEntity("Customers", null);
}
catch (Exception ex)
{
    dmeEditor.AddLogMessage("Beep", 
        $"Error: {ex.Message}", DateTime.Now, -1, null, Errors.Failed);
}

ðŸ§¹ Resource Cleanup

Always close connections to free resources

protected override void OnFormClosed(FormClosedEventArgs e)
{
    BeepSharedFunctions.CloseAllConnections();
    base.OnFormClosed(e);
}

🎯 DMEEditor Usage

Use DMEEditor as the main interface for all operations

// Access through dmeEditor directly
var dataSource = dmeEditor.GetDataSource(connectionName);

âš ï¸ Common Pitfalls

Missing Driver Check: Always verify drivers exist before creating connections
Connection Leaks: Not closing connections can exhaust resources
Error Swallowing: Catching exceptions without proper logging
Configuration Order: Trying to get data sources before adding connections

✅ Success Checklist

Beep services are properly registered in DI container
ConfigEditor is initialized with appropriate configuration type
Connection configurations are created before accessing data sources
Connection status is verified before performing operations
Comprehensive error handling and logging is implemented
Resources are properly disposed when applications close
UI controls are configured with Beep service instances

Complete Setup Example

// Complete setup following all best practices
public class DataApplicationManager
{
    private readonly IDMEEditor _dmeEditor;
    private readonly List<IDataSource> openDataSources;
    private IBeepService _beep;

    public DataApplicationManager()
    {
        openDataSources = new List<IDataSource>();
        _dmeEditor = InitializeBeepEngine();
        
        // CRITICAL: Initialize ConnectionHelper configurations FIRST
        InitializeConnectionDrivers();
        
        CreateConnections();
    }
    
    private IDMEEditor InitializeBeepEngine()
    {
        try
        {
            // BeepService builds and wires every core component for you
            var services = new ServiceCollection();
            _beep = services.AddBeepServices()
                .WithDirectory(AppContext.BaseDirectory)
                .WithAppRepo("DataApplicationManager")
                .WithConfigType(BeepConfigType.Application)
                .WithMapping()
                .AsSingleton()
                .Build();

            _beep.LoadAssemblies();
            var dmeEditor = _beep.DMEEditor;

            // Configure() swallows exceptions, so verify the result
            if (dmeEditor == null)
            {
                throw new InvalidOperationException("Failed to initialize DMEEditor");
            }
            
            Console.WriteLine("✅ DMEEditor initialized successfully");
            return dmeEditor;
        }
        catch (Exception ex)
        {
            Console.WriteLine($"❌ Failed to initialize DMEEditor: {ex.Message}");
            throw;
        }
    }
    
    private void InitializeConnectionDrivers()
    {
        try
        {
            Console.WriteLine("🔧 Initializing connection drivers from ConnectionHelper...");
            
            // Get all 39 predefined connection configurations
            var allConfigs = ConnectionHelper.GetAllConnectionConfigs();
            
            // Add each configuration to the DMEEditor
            foreach (var config in allConfigs)
            {
                if (!_dmeEditor.ConfigEditor.DataDriversClasses.Any(d => d.GuidID == config.GuidID))
                {
                    _dmeEditor.ConfigEditor.DataDriversClasses.Add(config);
                }
            }
            
            // Save configurations to persistent storage
            _dmeEditor.ConfigEditor.SaveConnectionDriversConfigValues();
            
            Console.WriteLine($"✅ Loaded {allConfigs.Count} connection driver configurations");
            
            // Verify critical drivers are available
            var driverTypes = allConfigs.GroupBy(c => c.DatasourceCategory)
                .Select(g => $"{g.Key}: {g.Count()} drivers")
                .ToArray();
            
            Console.WriteLine("📊 Available driver categories:");
            Console.WriteLine("   🏢 RDBMS: Enterprise databases (SQL Server, PostgreSQL, MySQL, DB2, etc.)");
            Console.WriteLine("   📄 FILE: File formats (CSV, Excel, JSON, Text)");
            Console.WriteLine("   🗄️ NoSQL: Document & NoSQL databases (MongoDB, CouchDB, etc.)");
            Console.WriteLine("   🧠 VectorDB: AI/ML vector databases (ChromaDB, Pinecone, Qdrant, etc.)");
            Console.WriteLine("   🌐 WebAPI: REST and web service connections");
            Console.WriteLine("   ☁️ Cloud: Cloud and streaming services");
            
            // Verify critical drivers are available
            VerifyEssentialDrivers(dmeEditor);
        }
        catch (Exception ex)
        {
            Console.WriteLine($"❌ Failed to create standard connections: {ex.Message}");
        }
    }
    
    private void CreateConnections()
    {
        Console.WriteLine("🔗 Creating application connections...");
        
        // Now that drivers are loaded, we can create connections
        CreateSqlServerConnection();
        CreateSQLiteConnection();
        CreateCsvConnection();
        CreateExcelConnection();
        
        // Test all connections
        var testResults = TestAllConnections();
        var successfulConnections = testResults.Where(r => r.Value).Select(r => r.Key);
        
        Console.WriteLine($"🎯 Successfully connected to: {string.Join(", ", successfulConnections)}");
    }
    
    private void CreateSqlServerConnection()
    {
        try
        {
            // Get SQL Server driver configuration (now available from ConnectionHelper)
            var sqlServerConfig = _dmeEditor.ConfigEditor.DataDriversClasses
                .FirstOrDefault(d => d.DatasourceType == DataSourceType.SqlServer);
            
            if (sqlServerConfig == null)
            {
                Console.WriteLine("❌ SQL Server driver not found. Ensure ConnectionHelper.GetAllConnectionConfigs() was called.");
                return;
            }
            
            var connectionProps = new ConnectionProperties
            {
                ConnectionName = "MainDatabase",
                DatabaseType = DataSourceType.SqlServer,
                Category = DatasourceCategory.RDBMS,
                Host = Environment.GetEnvironmentVariable("DB_SERVER") ?? "localhost",
                Database = Environment.GetEnvironmentVariable("DB_NAME") ?? "MainDB",
                UserID = Environment.GetEnvironmentVariable("DB_USER") ?? "sa",
                Password = Environment.GetEnvironmentVariable("DB_PASS") ?? "password",
                DriverName = sqlServerConfig.PackageName,
                DriverVersion = sqlServerConfig.version
            };
            
            // Use ConnectionHelper to build proper connection string
            connectionProps.ConnectionString = ConnectionHelper.ReplaceValueFromConnectionString(
                sqlServerConfig, connectionProps, _dmeEditor);
            
            _dmeEditor.ConfigEditor.AddDataConnection(connectionProps);
            Console.WriteLine("✅ SQL Server connection configured");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"❌ Failed to create SQL Server connection: {ex.Message}");
        }
    }
    
    private void CreateSQLiteConnection()
    {
        try
        {
            // Get SQLite driver configuration
            var sqliteConfig = _dmeEditor.ConfigEditor.DataDriversClasses
                .FirstOrDefault(d => d.DatasourceType == DataSourceType.SqlLite);
            
            if (sqliteConfig == null)
            {
                Console.WriteLine("❌ SQLite driver not found");
                return;
            }
            
            var connectionProps = new ConnectionProperties
            {
                ConnectionName = "LocalDatabase",
                DatabaseType = DataSourceType.SqlLite,
                Category = DatasourceCategory.RDBMS,
                ConnectionString = "Data Source=./Data/app.db;Version=3;",
                DriverName = sqliteConfig.PackageName,
                DriverVersion = sqliteConfig.version
            };
            
            _dmeEditor.ConfigEditor.AddDataConnection(connectionProps);
            Console.WriteLine("✅ SQLite connection configured");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"❌ Failed to create SQLite connection: {ex.Message}");
        }
    }
    
    private void CreateCsvConnection()
    {
        try
        {
            // Get CSV driver configuration
            var csvConfig = _dmeEditor.ConfigEditor.DataDriversClasses
                .FirstOrDefault(d => d.DatasourceType == DataSourceType.CSV);
            
            if (csvConfig == null)
            {
                Console.WriteLine("❌ CSV driver not found");
                return;
            }
            
            var connectionProps = new ConnectionProperties
            {
                ConnectionName = "CsvData",
                DatabaseType = DataSourceType.CSV,
                Category = DatasourceCategory.FILE,
                FilePath = "./Data/Csv",
                FileName = "customers.csv",
                DriverName = csvConfig.PackageName,
                DriverVersion = csvConfig.version
            };
            
            _dmeEditor.ConfigEditor.AddDataConnection(connectionProps);
            Console.WriteLine("✅ CSV connection configured");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"❌ Failed to create CSV connection: {ex.Message}");
        }
    }
    
    private void CreateExcelConnection()
    {
        try
        {
            // Get Excel driver configuration
            var excelConfig = _dmeEditor.ConfigEditor.DataDriversClasses
                .FirstOrDefault(d => d.DatasourceType == DataSourceType.Xls);
            
            if (excelConfig == null)
            {
                Console.WriteLine("❌ Excel driver not found");
                return;
            }
            
            var connectionProps = new ConnectionProperties
            {
                ConnectionName = "ExcelReports",
                DatabaseType = DataSourceType.Xls,
                Category = DatasourceCategory.FILE,
                FilePath = "./Data/Excel",
                FileName = "reports.xlsx",
                DriverName = excelConfig.PackageName,
                DriverVersion = excelConfig.version
            };
            
            _dmeEditor.ConfigEditor.AddDataConnection(connectionProps);
            Console.WriteLine("✅ Excel connection configured");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"❌ Failed to create Excel connection: {ex.Message}");
        }
    }
    
    private Dictionary<string, bool> TestAllConnections()
    {
        var results = new Dictionary<string, bool>();
        
        try
        {
            var connections = _dmeEditor.ConfigEditor.DataConnections;
            
            Console.WriteLine("🧪 Testing all data source connections...");
            
            foreach (var connection in connections)
            {
                try
                {
                    var dataSource = _dmeEditor.GetDataSource(connection.ConnectionName);
                    if (dataSource != null)
                    {
                        dataSource.Openconnection();
                        bool isConnected = dataSource.ConnectionStatus == System.Data.ConnectionState.Open;
                        results[connection.ConnectionName] = isConnected;
                        
                        if (isConnected)
                        {
                            dataSource.Closeconnection();
                        }
                    }
                    else
                    {
                        results[connection.ConnectionName] = false;
                    }
                }
                catch
                {
                    results[connection.ConnectionName] = false;
                }
            }
            
            var successCount = results.Values.Count(r => r);
            Console.WriteLine($"📊 Connection test results: {successCount}/{results.Count} successful");
        }
        catch ( Exception ex)
        {
            Console.WriteLine($"❌ Error testing connections: {ex.Message}");
        }
        
        return results;
    }
    
    public IDataSource GetDataSource(string connectionName)
    {
        try
        {
            var dataSource = _dmeEditor.GetDataSource(connectionName);
            if (dataSource != null)
            {
                dataSource.Openconnection();
                if (dataSource.ConnectionStatus == System.Data.ConnectionState.Open)
                {
                    openDataSources.Add(dataSource);
                    return dataSource;
                }
            }
            return null;
        }
        catch (Exception ex)
        {
            Console.WriteLine($"❌ Error getting data source '{connectionName}': {ex.Message}");
            return null;
        }
    }
    
    public void Dispose()
    {
        try
        {
            foreach (var dataSource in openDataSources)
            {
                dataSource?.Closeconnection();
            }
            openDataSources.Clear();
            
            _dmeEditor?.Dispose();
            
            Console.WriteLine("ðŸ§¹ All resources cleaned up successfully");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"âš ï¸ Error during cleanup: {ex.Message}");
        }
    }
}

🚀 Registration & Setup

📚 Table of Contents

Overview

1ï¸âƒ£ Bootstrap

2ï¸âƒ£ Initialize

3ï¸âƒ£ Connect

4ï¸âƒ£ Operate

⚙️ Initial Setup & Registration

Step 1: Basic Service Setup

Step 2: ConnectionHelper Driver Initialization (CRITICAL)

ConnectionHelper Methods Available

Windows Forms Application Pattern

WPF Application Pattern

⚙️ ConfigEditor & Its Specialized Managers

📁 ConfigPathManager

🔗 DataConnectionManager

📝 QueryManager

🗺️ EntityMappingManager

🧩 ComponentConfigManager

🕘 MigrationHistoryManager

Step 3: ConfigEditor Configuration (After Driver Loading)

🔗 Connection Management

SQLite Database Connection

Excel File Connection

CSV File Connection

Generic Connection Creator

ðŸ’¾ Data Operations

Basic Data Access Pattern

Working with Multiple Data Sources

Data Movement Between Sources

ðŸ–¥ï¸ UI Integration Examples

Windows Forms with BeepGrid

Usage Examples

Shared Functions Class (Production Pattern)

Best Practices

🔍 Connection Verification

âš ï¸ Error Handling

ðŸ§¹ Resource Cleanup

🎯 DMEEditor Usage

Complete Setup Example

🚀 Registration & Setup

📚 Table of Contents

Overview

1ï¸âƒ£ Bootstrap

2ï¸âƒ£ Initialize

3ï¸âƒ£ Connect

4ï¸âƒ£ Operate

⚙️ Initial Setup & Registration

Step 1: Basic Service Setup

Step 2: ConnectionHelper Driver Initialization (CRITICAL)

ConnectionHelper Methods Available

Windows Forms Application Pattern

WPF Application Pattern

⚙️ ConfigEditor & Its Specialized Managers

📁 ConfigPathManager

🔗 DataConnectionManager

📝 QueryManager

🗺️ EntityMappingManager

🧩 ComponentConfigManager

🕘 MigrationHistoryManager

Step 3: ConfigEditor Configuration (After Driver Loading)

🔗 Connection Management

SQLite Database Connection

Excel File Connection

CSV File Connection

Generic Connection Creator

ðŸ’¾ Data Operations

Basic Data Access Pattern

Working with Multiple Data Sources

Data Movement Between Sources

ðŸ–¥ï¸ UI Integration Examples

Windows Forms with BeepGrid

Usage Examples

Shared Functions Class (Production Pattern)

Best Practices

🔍 Connection Verification

âš ï¸ Error Handling

ðŸ§¹ Resource Cleanup

🎯 DMEEditor Usage

Complete Setup Example

1ï¸âƒ£ Bootstrap

2ï¸âƒ£ Initialize

3ï¸âƒ£ Connect

4ï¸âƒ£ Operate

ðŸ–¥ï¸ UI Integration Examples

âš ï¸ Error Handling