In this recipe, we will cover the authentication and authorization mechanism used by Ceph. Users are either individuals or system actors such as applications, which use Ceph clients to interact with the Ceph Storage Cluster daemons. The following diagram illustrates this flow:
Ceph provides two authentication modes. They are as follows:
- None: With this mode, any user can access the Ceph cluster without authentication. This mode is disabled by default. Cryptographic authentication, which includes encrypting and decrypting user keys, has some computational costs. You can disable the Ceph authentication if you are very sure that your network infrastructure is secure, the clients/Ceph cluster nodes have established trust, and you want to save some computation by disabling authentication. However, this is not recommended, and you might be at risk of a man-in-the-middle attack. Still, if you are interested in disabling the Ceph authentication, you can do it by adding the following parameters in the global section of your Ceph configuration file on all the nodes, followed by the Ceph service restart:
auth cluster required = none auth service required = none auth client required = none
- Cephx: To identify users and protect against man-in-the-middle attacks, Ceph provides its Cephx authentication system to authenticate users and daemons. The Cephx protocol works similar to Kerberos to some extent and allows clients to access the Ceph cluster. It's worth knowing that the Cephx protocol does not do data encryption. In a Ceph cluster, the Cephx protocol is enabled by default. If you have disabled Cephx by adding the preceding auth options to your cluster configuration file, then you can enable Cephx in two ways. One is to simply remove all auth entries from the cluster configuration file, which are
none, or you can explicitly enable Cephx by adding the following options in the cluster configuration file and restarting the Ceph services:auth cluster required = cephx auth service required = cephx auth client required = cephx
Now that we have covered the different authentication modes of Ceph, let's understand how authentication and authorization works within Ceph.
To access the Ceph cluster, an actor/user/application invokes the Ceph client to contact the cluster's monitor node. Usually, a Ceph cluster has more than one monitor, and a Ceph client can connect to any monitor node to initiate the authentication process. This multi monitor architecture of Ceph removes a single point of failure situation during the authentication process.
To use Cephx, an administrator, that is, client.admin, must create a user account on the Ceph cluster. To create a user account, the client.admin user invokes the ceph auth get-or-create key command. The Ceph auth subsystem generates a username and a secret key, stores this information on the Ceph monitor, and returns the user's secret key to the client.admin user that has invoked the user creation command. The Ceph sys-admin should share this username and secret key with the Ceph client that wants to use the Ceph storage service in a secure manner. The following diagram visualizes this entire process:
In the last recipe, we learned the process of user creation and how a user's secret keys are stored across all the cluster nodes. We will now examine how users are authenticated by Ceph and allowed access to cluster nodes.
In order to access the Ceph cluster, the client first contacts the Ceph monitor node and passes only its user name. The Cephx protocol works in such a way that both parties are able to prove to each other that they have a copy of the key without actually revealing it. This is the reason that a client only sends its user name but not its secret key.
The monitor then generates a session key for the user and encrypts it with the secret key associated with that user. Then, the monitor transmits the encrypted session key back to the client. The client then decrypts the payload with its key to retrieve the session key. This session key remains valid for that user for the current session.
Using the session key, the client requests for a ticket from the Ceph monitor. The Ceph monitor verifies the session key and then generates a ticket, encrypted with the user's secret key, and transmits this to the user. The client decrypts the ticket and uses it to sign requests to OSDs and metadata servers throughout the cluster.
The Cephx protocol authenticates ongoing communications between the client and the Ceph nodes. Each message sent between a client and Ceph nodes, subsequent to the initial authentication, is signed using a ticket that the monitors, OSDs, and metadata nodes can verify with their shared secret key. Also, Cephx tickets expire, so an attacker cannot use an expired ticket or session key to gain access to the Ceph cluster. The following diagram illustrates the entire authentication process that has been explained here:
In the last recipe, we covered the authentication process used by Ceph. In this recipe, we will examine its authorization process. Once a user is authenticated, he is authorized for different types of access, activities, or roles. Ceph uses the term capabilities, which is abbreviated to caps. Capabilities are the rights a user gets that define the level of access he has to operate the cluster. The capability syntax looks like the following:
{daemon-type} 'allow {capability}' [{daemon-type} 'allow {capability}']
- Monitor caps: Includes the
r,w,x, parameters, andallow profiles {cap}. For example:mon 'allow rwx' or mon 'allow profile osd' - OSD caps: Includes
r,w,x,class-read,class-write, andprofile osd. For example:osd 'allow rwx' or osd 'allow class-read, allow rwx pool=rbd' - MDS caps: Only requires
allow. For example:mds 'allow'
Let's understand each capability:
allow: This impliesrwonly for MDS.r: This gives the user read access, which is required with the monitor to read CRUSH maps.w: This gives the user write access to objects.x: This gives the user the ability to call class methods, including read and write, and also, the rights to perform auth operations on monitors.Note
Ceph can be extended by creating shared object classes called
Ceph Classes. Ceph can load.soclasses stored in the OSD classdir. For a class, you can create new object methods that have the ability to call native methods in the Ceph object store, for example, the objects that you have defined in your class can call native Ceph methods such asreadandwrite.class-read: This is a subset ofxthat allows users to call class read methods.class-write: This is a subset ofxthat allows users to call class write methods.*: This gives users full permission (r,w, andx) on a specific pool as well as to execute admin commands.profile osd: This allows users to connect as an OSD to other OSDs or monitors. Used for the OSD heartbeat traffic and status reporting.profile mds: This allows users to connect as an MDS to other MDSs.profile bootstrap-osd: This allows users to bootstrap an OSD. For example,ceph-deployandceph-disktools use theclient.bootstrap-osduser, which has permission to add keys and bootstrap an OSD.profile bootstrap-mds: This allows the user to bootstrap the metadata server. For example, theceph-deploytool uses theclient.bootstrap-mdsuser to add keys and bootstrap the metadata server.
A user can be the individual user of an application, such as cinder/nova in the case of OpenStack. Creating users allows you to control what can access your Ceph Storage Cluster, its pools, and the data within the pools. In Ceph, a user should have a type, which is always client, and an ID, which can be any name. So, a valid user name syntax in Ceph is TYPE.ID, that is, client.<name>, for example, client.admin or client.cinder.
In the following recipe, we will discuss more about Ceph user management by running some commands:
- To list the users in your cluster, execute the following command:
# ceph auth listThe output of this command shows that for each daemon type, Ceph creates a user with different capabilities. It also lists the
client.adminuser, which is the cluster admin user. - To retrieve a specific user, for example,
client.admin, execute the following:# ceph auth get client.admin
- Create a user,
client.hari:# ceph auth get-or-create client.hari
This will create the user,
client.hari, with no capabilities, and a user with no caps is of no use. - Add capabilities to the
client.hariuser:# ceph auth caps client.hari mon 'allow r' osd 'allow rwx pool=rbd'
- List the user's capabilities:
# ceph auth get client.hari