Apache Hadoop now include an in built Key Management Server (KMS) that secures the transport protocol over HTTP. It provides both client and server REST APIs for securing the communication channel.

The Hadoop Key Management Server is basically a Jetty application that includes support for java key store that can hold multiple keys and also includes API to access and manage key metadata. From functional security perspective, it includes Access Control List (ACL) based access as well as support for multiple authentication and authorization protocols like Kerberos, Active Directory  and LDAP coupled with SSL  based channel security. Hadoop KMS include end to end data encryptions that covers both data at rest and data in motion. As soon as data is written into HDFS, it is encrypted using specific algorithm and assigned to a security zone.

Figure 16: Hadoop Key Management Server