ADO.NET bulk copy

In modern database access programming, the use of an object-oriented approach similar to the one available throughout EF is obvious. Although in most scenarios EF performs very well, there are specific cases when we want to use ADO.NET, for instance, when we want use some specific feature of the data provider or the database itself. Obviously, this choice is available at the cost of sacrificing the simplicity and reliability of dealing with an object-oriented data access layer. Although for custom database features we have to deal with the SQL language, often such features are available as data provider-specific features within ADO.NET classes.

Bulk copy is a great feature offered by the System.Data.SqlClient data provider. It allows the insertion of a lot of data within a database table without having to fulfill all INSERT validations or execution times. This boosts the throughput of the whole operation heavily. Here's an example of bulk copying:

//create a temporary data-table for new data
using (var dt = new DataTable())
{
    //the name of columns is not important
    //the type is useful
    //the right order is mandatory
    dt.Columns.Add("", typeof(int));
    dt.Columns.Add("", typeof(int));
    dt.Columns.Add("", typeof(string));
    dt.Columns.Add("", typeof(DateTime));

    for (int i = 0; i < 1000000; i++)
        dt.Rows.Add(new object[]
            {
                null, //primary key is identity on DB
                1, //the customer id
                string.Format("2015/001/{0}",i), //the number
                new DateTime(2015,1,1).AddDays(i), //the date
            });

    //the target connection
    using (var cn = new SqlConnection("data source=(local);initial catalog=TestDB;integrated security=true;"))
    {
        //open the connection
        cn.Open();

        //the bulk copy engine
        var machine = new SqlBulkCopy(cn);
        //the batch size - if desired
        machine.BatchSize = 1000;
        //the destination table name
        machine.DestinationTableName = "Invoice";
        //start the bulk copy
        machine.WriteToServer(dt);
    }
}

Using the Bulk Insert feature is the best choice when we need to add a lot of data. This feature is often used in data import/export even by the database. For instance, SQL Server Integration Services makes extensive use of the Bulk Insert feature for its ETL workflows.

Note

More information about Bulk Insert can be found at https://msdn.microsoft.com/en-us/library/ms188365.aspx.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
52.15.135.175