HCatalog (see https://cwiki.apache.org/confluence/display/Hive/HCatalog) is a metadata management system for Hadoop data. It stores consistent schema information for Hadoop ecosystem tools, such as Pig, Hive, and MapReduce. By default, HCatalog supports data in the format of RCFile, CSV, JSON, SequenceFile, ORC file, and a customized format if InputFormat, OutputFormat, and SerDe are implemented. By using HCatalog, users are able to directly create, edit, and expose (via its REST API) metadata, which becomes effective immediately in all tools sharing the same piece of metadata. At first, HCatalog was a separate Apache project from Hive. Eventually, HCatalog became part of the Hive project in 2013 starting with Hive v0.11.0. HCatalog is built on top of the Hive metastore and incorporates support for HQL DDL. It provides read and write interfaces and HCatLoader and HCatStorer. For Pig, it implements Pig's load and store interfaces. HCatalog also provides an interface for MapReduce programs by using HCatInputFormat and HCatOutputFormat, which are very similar to other customized formats, by implementing Hadoop's InputFormat and OutputFormat.

In addition, HCatalog provides a REST API from a component called WebHCat so that HTTP requests can be made from other applications to access the metadata of Hadoop MapReduce/Yarn, Pig, and Hive through HCatalog. There is no Hive-specific REST interface since HCatalog uses Hive's metastore. Therefore, HCatalog can define metadata for Hive directly through its CLI. The HCatalog CLI supports HQL SHOW/DESCRIBE statement and the majority of Hive DDL, except the following statements, which require triggering MapReduce jobs:

• CREATE TABLE ... AS SELECT
• ALTER INDEX ... REBUILD
• ALTER TABLE ... CONCATENATE
• ALTER TABLE ARCHIVE/UNARCHIVE PARTITION
• ANALYZE TABLE ... COMPUTE STATISTICS
• IMPORT/EXPORT