Locks ensure data isolation as described in the ACID principle. Hive has supported concurrency access and locking mechanisms since v0.7.0 and updated to a new lock manager in v0.13.0. There are two types of lock provided as follows:
- Shared lock: Also called S lock, it allows being shared concurrently. This is acquired when a table/partition is read.
- Exclusive lock: Also called X lock. This is acquired for all other operations that modify the table/partition.
To enable locking, make sure the two properties are set in a Hive session or hive-site.xml (refer Transaction, section above):
- hive.support.concurrency = true
- hive.txn.manager = org.apache.hadoop.hive.ql.lockmgr.DbTxnManager
Any query must acquire proper locks before being allowed to perform corresponding lock-permitted operations. When the query is SELECT, it will get an S lock. Concurrent SELECT statements on the same table will get multiple S locks and run in parallel. When the query is INSERT, it will get an X lock. Concurrent INSERT statements will only get one X lock, so an INSERT has to wait until the lock is released by the other INSERT. In addition, a table can only have one X lock. When trying to get an X lock, there should no other locks on the table, or else the operation that requires an X lock, such as INSERT, ALTER, has to wait and retry (the hive.lock.sleep.between.retries property controls the retry time).
By using the new lock manager, DbTxnManager, locks can only be acquired/released from a query implicitly. To see the locks on the table, use the SHOW LOCKS/SHOW LOCKS table_name statement:
-- Show all locks when running merge into above > SHOW LOCKS;
+--------+----------+-----------------+------------+------------+
| lockid | database | table | lock_state | lock_type |
+--------+----------+-----------------+------------+------------+
| 19.1 | default | employee_update | ACQUIRED | SHARED_READ|
| 19.2 | default | employee_trans | ACQUIRED |SHARED_WRITE|
+--------+----------+-----------------+-------------------------+
3 rows selected (0.059 seconds)