Sharding databases for performance

There are a few scenarios where it may be appropriate to partition data horizontally across several servers, with performance being the most obvious. In this recipe, I'll show you how we can use NHibernate Shards to split our data set across three databases.

Getting ready

  1. Download the latest NHibernate Shards binary from SourceForge at http://sourceforge.net/projects/nhcontrib/files/.
  2. Extract NHibernate.Shards.dll from the downloaded ZIP file to your solution's Lib folder.
  3. Complete the Eg.Core model and mappings from Chapter 1.
  4. In SQL Server, create three new, blank databases named Shard1, Shard2, and Shard3.

How to do it...

  1. In the Entity base class, change the type of the Id property from Guid to String.
  2. In Product.hbm.xml, change the Id generator from guid.comb to NHibernate.Shards.Id.ShardedUUIDGenerator, NHibernate.Shards.
  3. Follow the same procedure for ActorRole.hbm.xml.
  4. Use the NHibernate Schema Tool explained in Chapter 2 to build the database schema for each of the three databases.
  5. Create a new class library project named Eg.Shards.Runner.
  6. Add a reference to the Eg.Core model, log4net.dll, NHibernate.dll, NHibernate.ByteCode.dll, and NHibernate.Shards.dll.
  7. Add an App.config file with the following connection strings:
    <?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <connectionStrings>
    <add name="Shard1" connectionString="Server=.SQLExpress; Database=Shard1; Trusted_Connection=SSPI"/>
    <add name="Shard2" connectionString="Server=.SQLExpress; Database=Shard2; Trusted_Connection=SSPI"/>
    <add name="Shard3" connectionString="Server=.SQLExpress; Database=Shard3; Trusted_Connection=SSPI"/>
  </connectionStrings>
</configuration>
  8. Add a new class named ShardConfiguration with the following code:
    public class ShardConfiguration
{

  private Configuration GetConfiguration(
    string connStrName, 
    int shardId)
  {
    var cfg = new Configuration()
      .SessionFactoryName("SessionFactory" 
          + shardId.ToString())
      .Proxy(p => 
        p.ProxyFactoryFactory<ProxyFactoryFactory>())
      .DataBaseIntegration(db =>
           {
             db.Dialect<MsSql2008Dialect>();
             db.ConnectionStringName = connStrName;
           })
      .AddAssembly("Eg.Core")
      .SetProperty(
        ShardedEnvironment.ShardIdProperty,
        shardId.ToString());
    return cfg;
  }

  private IShardConfiguration GetShardCfg(
    string connStrName,
    int shardId)
  {
    var cfg = GetConfiguration(connStrName, shardId);
    return new ConfigurationToShardConfigurationAdapter(

      cfg);
  }

  private IList<IShardConfiguration> GetShardCfg(
    IEnumerable<string> connStrNames)
  {
    var cfg = new List<IShardConfiguration>();
    int shardId = 1;
    foreach (var connStrName in connStrNames)
      cfg.Add(GetShardCfg(connStrName, shardId++));
    return cfg;
  }

  public IShardedSessionFactory GetSessionFactory(
    IEnumerable<string> connStrNames,
    IShardStrategyFactory shardStrategyFactory)
  {
    var prototypeCfg = GetConfiguration(
      connStrNames.First(), 1);

    var cfg = new ShardedConfiguration(
      prototypeCfg,
      GetShardCfg(connStrNames),
      shardStrategyFactory);

    return cfg.BuildShardedSessionFactory();
  }


}
  9. Add a new class named ShardStrategyFactory with the following code:
    public class ShardStrategyFactory : IShardStrategyFactory
{

  public IShardStrategy NewShardStrategy(
    ICollection<ShardId> shardIds)
  {
    return new ShardStrategyImpl(
      GetSelectionStrategy(shardIds),
      GetResolutionStrategy(shardIds),
      GetAccessStrategy(shardIds));
  }

  private static IShardSelectionStrategy 
    GetSelectionStrategy(
      ICollection<ShardId> shardIds)
  {
    var loadBalancer =
      new RoundRobinShardLoadBalancer(shardIds);
    return new RoundRobinShardSelectionStrategy(
      loadBalancer);
  }

  private static IShardResolutionStrategy 
    GetResolutionStrategy(
      ICollection<ShardId> shardIds)
  {
    return new AllShardsShardResolutionStrategy(
      shardIds);
  }

  private static IShardAccessStrategy 
    GetAccessStrategy(
      ICollection<ShardId> shardIds)
  {
    return new SequentialShardAccessStrategy();
  }

}
  10. In Program.cs, use the following code:
    static void Main(string[] args)
{

  NHibernateProfiler.Initialize();

  var connStrNames = new List<string>();
  connStrNames.Add("Shard1");
  connStrNames.Add("Shard2");
  connStrNames.Add("Shard3");

  var shardStrategy = new ShardStrategy();

  var sessionFactory = new ShardConfiguration()
    .GetSessionFactory(connStrNames, shardStrategy);

  ClearDB(sessionFactory);

  var p1 = new Product()
             {
               Name = "Water Hose",
               Description = "50 ft.",

               UnitPrice = 17.46M
             };

  var p2 = new Product()
             {
               Name = "Water Sprinkler",
               Description = "Rust resistant plastic",
               UnitPrice = 4.95M
             };

  var p3 = new Product()
             {
               Name = "Beach Ball",
               Description = "Hours of fun",
               UnitPrice = 3.45M
             };


  using (var session = sessionFactory.OpenSession())
  {
    using (var tx = session.BeginTransaction())
    {
      session.Save(p1);
      session.Save(p2);
      session.Save(p3);
      tx.Commit();
    }
    session.Close();
  }

  using (var session = sessionFactory.OpenSession())
  {
    using (var tx = session.BeginTransaction())
    {
      var query = 
        "from Product p where upper(p.Name) " +
        "like '%WATER%'";
      var products = session.CreateQuery(query)
        .List();

      foreach (Product p in products)

        Console.WriteLine(p.Name);

      tx.Commit();
    }
    session.Close();
  }

  Console.ReadKey();
}

private static void ClearDB(ISessionFactory sessionFactory)
{
  using (var s = sessionFactory.OpenSession())
  {
    using (var tx = s.BeginTransaction())
    {
      var products = s.CreateQuery("from Product")
        .List();
      foreach (Product product in products)
        s.Delete(product);
      tx.Commit();
    }
    s.Close();
  }
}
  11. Build and run the application.
  12. Inspect the product table in each of the three databases. You should find one product in each.

How it works...

NHibernate Shards allows you to split your data across several databases, named shards, while hiding this additional complexity behind the familiar NHibernate APIs. In this recipe, we use the sharded UUID POID generator, which generates UUIDs with a four-digit shard ID, followed by a 28 hexadecimal digit unique ID. A typical ID looks like this: 0001000069334c47a07afd3f6f46d587. You can provide your own POID generator, provided the shard ID is somehow encoded in the persistent object's IDs.

The ShardConfiguration class configures a session factory for each shard. These session factories are grouped together with an implementation of IShardStrategyFactory to build an IShardedSessionFactory. A sharded session factory implements the familiar ISessionFactory interface, so the impact on your larger application is minimal.

An implementation of IShardStrategyFactory must return three strategies to control the operation of NHibernate Shards. First, the IShardSelectionStrategy assigns each new entity to a shard. In this recipe, we use a simple round-robin technique that spreads the data across each shard equally. The first entity is assigned to shard 1, the second to shard 2, the third to shard 3, the fourth to shard 1, and so on. Next, the IShardResolutionStrategy is used to determine the correct shard given an entity name and entity ID. In this example, we use the AllShardsShardResolutionStrategy, which doesn't attempt to determine the correct shard. Instead, all shards are queried for an entity. We could provide our own implementation to get the shard ID from the first 4 characters of the entity ID. This would allow us to determine which shard contains the entity we want and query only that shard, reducing the load on each database. Finally, the IShardAccessStrategy determines how the shards will be accessed. In this example, we use the SequentialShardAccessStrategy, so the first shard will be queried, then the next, and so on. NHibernate Shards also includes a parallel strategy.

Once we've built a sharded session factory, the application code looks like any other NHibernate application. However, there are a few caveats. NHibernate Shards doesn't support many of the lesser-used features of NHibernate. For example, session.Delete("from Products"); throws a NotImplementedException. Additionally, sharded sessions expect to be explicitly Closed before being Disposed. Finally, NHibernate Shards doesn't support object graphs spread across shard boundaries. The idea of well-defined boundaries between object graphs fits well with the Domain-Driven Design pattern of aggregate roots and is generally considered a good NHibernate practice even without sharding.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.16.137.117