Locks

Locks ensure data isolation as described in the ACID principle. Hive has supported concurrency access and locking mechanisms since v0.7.0 and updated to a new lock manager in v0.13.0. There are two types of lock provided as follows:

  • Shared lock: Also called S lock, it allows being shared concurrently. This is acquired when a table/partition is read. 
  • Exclusive lock: Also called X lock. This is acquired for all other operations that modify the table/partition.
For partition tables, only a shared lock is acquired if the change is only applicable to the newly-created partitions. An exclusive lock is acquired on the table if the change is applicable to all partitions. In addition, an exclusive lock on the table globally affects all partitions. For more information regarding locks, see https://cwiki.apache.org/confluence/display/Hive/Locking.

To enable locking, make sure the two properties are set in a Hive session or hive-site.xml (refer Transaction, section above):

  • hive.support.concurrency = true
  • hive.txn.manager = org.apache.hadoop.hive.ql.lockmgr.DbTxnManager

Any query must acquire proper locks before being allowed to perform corresponding lock-permitted operations. When the query is SELECT, it will get an S lock. Concurrent SELECT statements on the same table will get multiple S locks and run in parallel. When the query is INSERT, it will get an X lock. Concurrent INSERT statements will only get one X lock, so an INSERT has to wait until the lock is released by the other INSERT. In addition, a table can only have one X lock. When trying to get an X lock, there should no other locks on the table, or else the operation that requires an X lock, such as INSERT, ALTER, has to wait and retry (the hive.lock.sleep.between.retries property controls the retry time). 

By using the new lock manager, DbTxnManager, locks can only be acquired/released from a query implicitly. To see the locks on the table, use the SHOW LOCKS/SHOW LOCKS table_name statement:

-- Show all locks when running merge into above
> SHOW LOCKS;
+--------+----------+-----------------+------------+------------+
| lockid | database | table | lock_state | lock_type |
+--------+----------+-----------------+------------+------------+
| 19.1 | default | employee_update | ACQUIRED | SHARED_READ|
| 19.2 | default | employee_trans | ACQUIRED |SHARED_WRITE|
+--------+----------+-----------------+-------------------------+
3 rows selected (0.059 seconds)
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.133.116.137