Partitioning types and expressions

The partitioning is based on a value that is calculated for each row using a partitioning expression. The partitioning type is a method that is used to assign each row to a particular partition based on the partitioning expression. For example, the RANGE type assigns a range of values to each partition. When a row is inserted, its partitioning expression is calculated. The row will be written into the partition that matches the proper range.

Partitioning expressions

A partitioning expression is a SQL expression that returns a positive integer value or NULL. It is possible to use a temporal column in the partitioned expression as long as an integer value is returned. However, the return value cannot depend on the current timezone, so the TIMESTAMP and YEAR columns are not allowed. Some partitioning types can use expressions that return other data of different data types, such as DATE or CHAR; this will be discussed later.

The partitioning expressions must also return a deterministic nonconstant value. Stored functions and user-defined functions are not allowed, even if they are deterministic.

The / operator is not allowed because it can return a FLOAT value, even if both operands are INTEGER. The DIV and MOD operators (integer division and division's rest) are supported. Bit operators are not supported.

A partitioning expression should be able to operate as fast as possible. Ideally, it should include only one column and, if necessary, the function that is needed to obtain an integer value. In practice, more columns and calculations are sometimes necessary. However, note that the performance of the expression affects inserts, updates, and deletes.

The following temporal functions are optimized to be used in a partitioning expression:

  • YEAR()
  • TO_DAYS()
  • TO_SECONDS()

Other functions, such as MONTH(), are not optimized for this purpose but can still be good candidates.

To achieve good performance with the HASH partitioning type, only one column should be used in the expression. A strict relationship should also exist between the column values and the expression return values: a change in a column value should cause a change in the return value that is directly proportional.

Some examples of fast partitioning expressions are as follows:

id
id MOD 8
YEAR(date)
ORD(name)

In the following sections, we will see how to use partitioning expressions.

Indexes and primary keys

With partitioned tables, the primary keys and the unique keys are subject to an important limitation. Each unique key, including the primary key, must include all columns that are necessary to calculate the partitioning expression.

In practice, this often means the following:

  • The primary key must include columns of the partitioning expression
  • Only a limited number of unique keys are allowed

For example, suppose that we want to create a table that contains data about all the employees of a company. The nonpartitioned table would look like the following code:

CREATE TABLE employee (
  id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
  first_name VARCHAR(50) NOT NULL,
  last_name VARCHAR(50) NOT NULL,
  email VARCHAR(200) NOT NULL,
  vat_id VARCHAR(15),
  hire_date DATE NOT NULL,
  PRIMARY KEY (id),
  UNIQUE unq_email (email),
  UNIQUE unq_vat (vat_id),
  INDEX idx_hire (hire_date)
)
  ENGINE = InnoDB;

Then, we want to partition the table. For some reason, although this is not important now, we want the proper partition of each row to be determined by YEAR(hire_date). However, we will need to modify the table in two ways:

  • First, we need a primary key that includes hire_date because that column is used by the partitioning expression. However, hire_date is not sufficient as a primary key because its values are not unique: multiple employees can be hired the same day. So, our primary key definition will become: PRIMARY KEY (id, hire_date). Note that this key is now longer than the previously defined primary key. Since all indexes in an InnoDB table include the primary key, all indexes will be longer.
  • In this table, a unique index may or may not be useful for queries but it is useful as a constraint because it forces the email and vat_id values to be unique. However, in partitioned tables, all unique keys must contain all columns used by the partitioned expression. You can do this in three ways. We can add hire_date to the unique indexes, but they would not guarantee the uniqueness of those fields anymore. You can add the unique columns to the expression, but then this will become much slower and will slow down inserts and updates. The most common solution is simply avoiding unique indexes, which is what we will do in our case.

With the KEY partitioning type, the primary key is mandatory. It is, however, possible to partition a table without a primary key using any other partitioning type. However, a table without keys will probably be slow. Foreign keys are not allowed for partitioned tables. There are no restrictions for normal non-unique indexes.

Partition names

Some partitioning types require that some properties be specified for each partition. With other partitioning types, we can merely declare how many partitions we want for a table. In the first case, we must specify a name for each partition; in the latter case, MariaDB automatically assigns names.

Automatic partition names consist of a p followed by a partition progressive number, that starts from 0. Even if there are more than 10 partitions, there is no leading 0. Partition names are case insensitive and have a maximum length of 61 characters, which is slightly less than other objects' identifiers.

When we must specify partition names, it is a common practice to use the same criteria used for automatic names. However, this also allows us to assign meaningful names in some cases. For example, if a table has one partition that contains recent data and many partitions that contain historical data, the partition that contains the most recent data can be named current. This technique also allows us to explicitly name a partition in our queries.

Partitioning types

In this section, we will describe the partitioning types, discuss how they work, and show how to create tables using the desired type.

The RANGE and LIST basic types are quite similar and support the same administrative operations. There are also slightly different types called RANGE COLUMNS and LIST COLUMNS.

The HASH and KEY types are quite similar too, and there are slightly different types than the HASH and KEY types called LINEAR HASH and LINEAR KEY.

The RANGE type

The RANGE partitioning type assigns a different range of values to each partition. For each range, we only specify a higher bound. The first range starts with NULL, which is the lowest possible value. The other ranges start with the value that immediately follows the higher bound of the previous partition.

Consider the following example:

CREATE TABLE article (
  id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
  date DATE NOT NULL,
  author VARCHAR(100),
  language TINYINT UNSIGNED,
  text TEXT,
  PRIMARY KEY (id, date)
)
  ENGINE = InnoDB
PARTITION BY RANGE (YEAR(date)) (
  PARTITION p0 VALUES LESS THAN (1990),
  PARTITION p1 VALUES LESS THAN (2000),
  PARTITION p3 VALUES LESS THAN (2010),
  PARTITION current VALUES LESS THAN (2020)
);

The last partition holds values that are less than 2020. This will not be a problem for some years (the current year is 2014), but some day it may be problem. If we try to insert a row for which the partitioning expression returns an out-of-range value, we will get an error like the following:

ERROR 1292 (22007): Incorrect date value: '2020' for column 'date' at row 1

The error can be suppressed with the IGNORE clause, but the row will not be inserted.

However, the following syntax can be used to allow partition values that are too high to be stored in the previous partitions:

PARTITION p_name VALUES LESS THAN (MAXVALUE)

The parenthesis here are optional.

In the preceding example, we have a table that contains articles. The table is very large and we want to partition it to process common queries faster. Since the table contains very old articles, RANGE seems like a good partitioning type to use for the following reasons:

  • The YEAR(date) type is an efficient partitioning expression.
  • Common queries will only involve one partition.
  • Some historical queries may involve only one partition or only a set of partitions.
  • In future, we may want to delete very old data. In that case, we can probably do it in an easy and efficient way by dropping the oldest partition.

The LIST type

The LIST partitioning type is quite similar to RANGE though LIST assigns to each partition a list of values rather than a range. Each partition must be assigned at least one value. The order of partitions is not relevant for the LIST type. All possible values must be explicitly names; there is no way to put all unmentioned values into a partition, which is similar to what is done with MAXVALUE for the RANGE type. When trying to insert a value that is not assigned to a partition, we get an error shown as follows:

Error (Code 1526): Table has no partition for value 100

The following example shows how to create a LIST partitioned table:

CREATE TABLE article (
  id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
  date DATE NOT NULL,
  author VARCHAR(100),
  language TINYINT UNSIGNED,
  text TEXT,
  PRIMARY KEY (id, language)
)
  ENGINE = InnoDB
PARTITION BY LIST (language) (
  PARTITION p0 VALUES IN (1),
  PARTITION p1 VALUES IN (2, 3),
  PARTITION p2 VALUES IN (4, 5, 6, 7, 8, 9, 10, 11),
  PARTITION px VALUES IN (NULL)
);

In this example, language acts a foreign key to a table that stores information about languages (probably just an incremental ID and their name). The reason why the foreign key has not been declared is that foreign keys are not supported for partitioned tables.

Here, the partitioning expression is a simple column name. This works because language is an integer column that only contains positive values. Of course, a CHAR column could be used too, for example, to store the language's ISO code; however, in that case, the expression should translate its string value into a number.

Despite the language column being an integer, the values cannot be sorted as per logic, thus splitting them into ranges would make no sense. There are several reasons why creating a LIST partitioned table based on the language can be a good choice:

  • Common queries probably show articles in the selected languages, thus partitioning by language allows such queries to use only one partition.
  • There can be a query that finds the languages in which a given article is available. This query is likely to be very fast, so we do not care if it accesses all partitions.
  • If a high proportion of content is available in a language, that language can be stored in a dedicated partition. Other partitions can group several languages where a limited portion of content is available.
  • Depending on our audience, it is possible that at least two partitions are frequently read. This is very unlikely to happen if we partition the articles by date, like we did in the previous example. Thus, partitioning by language allows us to split the most accessed partitions into different disks. This technique is described later in the Partitions' physical files section.
  • If the content in a language grows sensibly, it is easy to move that language into a new partition.

The COLUMNS keyword

The RANGE and LIST partitioning types have two variations called RANGE COLUMNS and LIST COLUMNS. When they are used, there is no single partitioning expression: the values are assigned to partitions based on a list of columns. This list is sometimes called a partitioned columns list in the documentation and in technical articles. No functions, operators, or any kind of elaboration is allowed to transform the columns values. However, for COLUMNS partitioning types, more data types are allowed:

  • All integer types (but only positive values are allowed)
  • The DATE and DATETIME data types
  • The VARCHAR , CHAR, VARBINARY, and BINARY data types

This is very important because as the use of functions is not allowed, there is no way to convert a noninteger value into an integer.

Note that the example used for the LIST type could be a LIST COLUMNS type too because it only included a column name.

The RANGE COLUMNS type is primarily useful when the range of values need to be based on multiple columns to obtain a good distribution of values. The following example shows the syntax to be used:

CREATE TABLE article (
  id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
  date DATE NOT NULL,
  year CHAR(4) NOT NULL,
  author VARCHAR(100),
  language TINYINT UNSIGNED,
  text TEXT,
  PRIMARY KEY (id, language, year)
)
  ENGINE = InnoDB
PARTITION BY RANGE COLUMNS (language, year) (
  -- old english articles
  PARTITION p0 VALUES LESS THAN (1, '2010'),
  -- recent english articles
  PARTITION p1 VALUES LESS THAN (1, '2020'),
  -- old non-english articles
  PARTITION p2 VALUES LESS THAN (100, '2010'),
  -- recent non-english articles
  PARTITION p3 VALUES LESS THAN (MAXVALUE, MAXVALUE)
);

This partitioning strategy reflects a very common situation:

  • We have abundant content in English and much less in other languages. Thus, the most reasonable division seems to be English or non-English.
  • Within these groups, we want to separate recent content from the non-recent ones.

Note that we cannot use YEAR(date), so we choose to duplicate the year information into a string column. With all characters being digits, the order will not change, provided that all values consist of four characters.

Note

A limitation is also worthy to be noted—we can only use the MAVALUE keyword for the last partition. It would seem reasonable, for example, to assign the values (1, MAXVALUE) to the second partition, but this would be a syntax error.

The LIST COLUMNS type is useful to match a list of values against a noninteger column. Let's go back to the example used for the LIST type. We store languages in the form of integer values, but maybe it's preferable to store a two-character ISO code. While this is not possible with the LIST type, LIST COLUMNS allows us to do this:

CREATE TABLE article (
  id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
  date DATE NOT NULL,
  author VARCHAR(100),
  language TINYINT UNSIGNED,
  text TEXT,
  PRIMARY KEY (id, language)
)
  ENGINE = InnoDB
PARTITION BY LIST (language) (
  PARTITION p0 VALUES IN ('en'),
  PARTITION p1 VALUES IN ('fr', 'de'),
  
PARTITION p2 VALUES IN ('es', 'it', 'ir', 'is', 'nl', 'ru', 'ro', 'sr'),
  PARTITION px VALUES IN (NULL)
);

The HASH and KEY types

The HASH and KEY partitioning types are similar. Their purpose is to provide a more uniform row distribution among partitions. Each different value returned by the partitioning expression has equal probabilities to be assigned to each partition. A group of contiguous values will be assigned to different partitions. However, if the distribution of unique values has very high peaks (that is, a limited set of values occur very often), it is possible that some partitions will sensibly contain more data than others. The KEY and HASH types work better if the distribution of unique values is uniform.

With HASH and KEY, the target partition for new rows is determined automatically by the server. It uses the following formula:

target_partition = expression_result MOD number__of_partitions

In the previous syntax, MOD is the modulus operator that returns the rest of a division.

The choice between HASH and KEY depends on the partitioning function that we want to use. The difference between these types is similar to the difference between LIST and LIST COLUMNS, which makes it easier to remember.

  • The HASH type accepts any partitioning function that returns a positive integer or NULL.
  • The KEY type accepts a single column of any of the types allowed for LIST COLUMNS. No calculations are allowed on that column.

With the KEY type, a hash of the return value is calculated. The hash uses the PASSWORD() function, which uses a variation of an SHA algorithm.

Usually, the following syntax is used to create a table partitioned by HASH:

CREATE TABLE article (
  id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
  date DATE NOT NULL,
  author VARCHAR(100),
  language TINYINT UNSIGNED,
  text TEXT,
  PRIMARY KEY (id, date)
)
  ENGINE = InnoDB
PARTITION BY HASH(YEAR(date))
PARTITIONS 8;

In fact, most of the time we only want to determine the partitioning expression and the number of partitions. In this case, we have chosen YEAR(date) as the partitioning expression so that all articles published in the same year will be in the same partition. This will allow most queries to access only one partition.

The following syntax is also allowed:


PARTITION BY HASH(MONTH(date)) (
    PARTITION p0,
    PARTITION p1
);

This allows us to specify a name for each partition. A path for each partition's files can also be specified, as explained later in this chapter in the Partitions' physical files section.

To create a table partitioned by KEY, use the following code:

CREATE TABLE article (
  id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
  date DATE NOT NULL,
  author VARCHAR(100),
  language TINYINT UNSIGNED,
  text TEXT,
  PRIMARY KEY (id, date)
)
  ENGINE = InnoDB
PARTITION BY KEY(id)
PARTITIONS 8;

In this example, we partitioned the table with an AUTO_INCREMENT key, which will provide a distribution of the most uniform values.

Again, we can specify a name for each partition, like we did in the HASH example.

Otherwise, we can use an even more concise syntax. Since id is the primary key, it need not be explicitly named for KEY partitioning:


PARTITION BY KEY()
PARTITIONS 8;

The LINEAR keyword

The LINEAR HASH and LINEAR KEY partitioning types are identical to HASH and KEY, except that a much more complicated algorithm is used to select new rows to target partitions. The linear formula is less efficient for normal database operations, such as row reads and writes. However, it makes some administering operations much faster. If the performance of partition creation, drop, split, and merge are a problem, using the LINEAR keyword can be a solution.

Splitting into subpartitions

Main partitions can be split into multiple subpartitions. Only one subpartitioning level is supported, which means that subpartitions cannot be further split. Each partition must have the same number of subpartitions.

A table with subpartitions can only be partitioned by RANGE or LIST and subpartitioned by HASH or KEY. The COLUMNS and LINEAR keywords are allowed. Using any other combination of types produces the following error:

ERROR 1500 (HY000): It is only possible to mix RANGE/LIST partitioning with HASH/KEY partitioning for subpartitioning

The syntax to create a subpartitioned table is as follows:

CREATE TABLE article (
  id INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
  date DATE NOT NULL,
  author VARCHAR(100),
  language TINYINT UNSIGNED,
  text TEXT,
  PRIMARY KEY (id, date)
)
  ENGINE = InnoDB
PARTITION BY RANGE COLUMNS (date)
SUBPARTITION BY LINEAR HASH (id) (
  PARTITION p0 VALUES LESS THAN ('2000-01-01') (
    SUBPARTITION s0,
    SUBPARTITION s1
  ),
  PARTITION p1 VALUES LESS THAN ('2020-01-01') (
    SUBPARTITION s2,
    SUBPARTITION s3
  )
);

Note that each subpartition must have a unique name within the table. Each partition can be stored in a different path, as explained in the Partitions' physical files section.

A concise syntax can also be used, if we do not need to specify names or paths for subpartitions:


PARTITION BY RANGE COLUMNS (date)
SUBPARTITION BY LINEAR KEY (id)
SUBPARTITIONS 2 (
  PARTITION p0 VALUES LESS THAN ('2000-01-01'),
  PARTITION p1 VALUES LESS THAN ('2020-01-01')
);

Note

Note that, in this case, the subpartitioning expression must be explicitly specified. The primary key is used by default for KEY partitions but not for subpartitions.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.143.22.23