Search in book...
Toggle Font Controls
Create new playlist

Name your new playlist

Playlist description (optional)
Sign In

Email address

Password

Forgot Password?

or

Continue with Facebook

Continue with Google
Sign Up

Full Name

Email address

Confirm Email Address

Password

or

Continue with Facebook

Continue with Google

Object storage using the Ceph RADOS gateway

Object storage, as the name suggests, manages data in the form of objects. Each object stores data, metadata, and a unique identifier. Object storage cannot be directly accessed by operating systems as a local or remote filesystem. It can only be accessed via API at application level. Ceph provides an object storage interface known as the RADOS gateway, which has been built on top of the Ceph RADOS layer. The RADOS gateway provides applications with RESTful S3- or Swift-compatible API interfaces to store data in the form of objects into a Ceph cluster.

In a production environment, if you have a huge workload for Ceph object storage, you should configure the RADOS gateway on a physical dedicated machine, else you can consider using any of the monitor nodes as the RADOS gateway. We will now perform a basic RADOS gateway configuration to use Ceph storage cluster as object storage.

Setting up a virtual machine

In a usual Ceph-based setup, the RADOS gateway is configured on a machine other than MON and OSD. However, if you have limited hardware, you can use MON machines to configure RGW. In this demonstration, we will create a separate virtual machine for Ceph RGW:

Create a new VirtualBox virtual machine for the RADOS gateway:

# VboxManage createvm --name ceph-rgw --ostype RedHat_64 --register
# VBoxManage modifyvm ceph-rgw --memory 1024 --nic1 nat  --nic2 hostonly --hostonlyadapter2 vboxnet1

# VBoxManage storagectl ceph-rgw --name "IDE Controller" --add ide --controller PIIX4 --hostiocache on --bootable on
# VBoxManage storageattach ceph-rgw --storagectl "IDE Controller" --type dvddrive --port 0 --device 0 --medium /downloads/CentOS-6.4-x86_64-bin-DVD1.iso 
# VBoxManage storagectl ceph-rgw --name "SATA Controller" --add sata --controller IntelAHCI --hostiocache on --bootable on
# VBoxManage createhd --filename OS-ceph-rgw.vdi --size 10240
# VBoxManage storageattach ceph-rgw --storagectl "SATA Controller" --port 0 --device 0 --type hdd --medium OS-ceph-rgw.vdi
# VBoxManage startvm ceph-rgw --type gui

Once the virtual machine is created and started, install the CentOS operating system by following the OS installation documentation available at https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Installation_Guide/index.html. During the installation process, provide the hostname as ceph-client1.
Once you have successfully installed the operating system, edit the network configuration of the machine and restart network services:
- Edit the /etc/sysconfig/network-scripts/ifcfg-eth0 file and add:
```
ONBOOT=yes
BOOTPROTO=dhcp
```
- Edit the /etc/sysconfig/network-scripts/ifcfg-eth1 file and add:
```
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.57.110
NETMASK=255.255.255.0
```
- Edit the /etc/hosts file and add:
```
192.168.57.101 ceph-node1
192.168.57.102 ceph-node2
192.168.57.103 ceph-node3
192.168.57.200 ceph-client1
192.168.57.110 ceph-rgw
```

Installing the RADOS gateway

The last section was about setting up a virtual machine for RGW. In this section, we will learn to install and configure RGW:

Ceph object storage requires Apache and FastCGI; it is recommended to install 100-continue optimized versions for Apache and FastCGI provided by the Ceph community.

Execute the following commands on the RADOS gateway node ceph-rgw, unless otherwise specified. Create a ceph-apache repository file, ceph-apache.repo, for YUM under the /etc/yum.repos.d directory:

# vim /etc/yum.repos.d/ceph-apache.repo
## replace {distro} with OS distribution type , ex centos6 , rhel6 etc. You can grab this code at publishers website.
              [apache2-ceph-noarch]
name=Apache noarch packages for Ceph
baseurl=http://gitbuilder.ceph.com/apache2-rpm-{distro}-x86_64-basic/ref/master
enabled=1
priority=2
gpgcheck=1
type=rpm-md
gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/autobuild.asc

[apache2-ceph-source]
name=Apache source packages for Ceph
baseurl=http://gitbuilder.ceph.com/apache2-rpm-{distro}-x86_64-basic/ref/master
enabled=0
priority=2
gpgcheck=1
type=rpm-md
gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/autobuild.asc

Create the ceph-fastcgi.repo file under the /etc/yum.repos.d directory:

# vim /etc/yum.repos.d/ceph-fastcgi.repo
## replace {distro}with OS distribution type , ex centos6 , rhel6 etc. You can grab this code at publishers website.

[fastcgi-ceph-basearch]
name=FastCGI basearch packages for Ceph
baseurl=http://gitbuilder.ceph.com/mod_fastcgi-rpm-{distro}-x86_64-basic/ref/master
enabled=1
priority=2
gpgcheck=1
type=rpm-md
gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/autobuild.asc

[fastcgi-ceph-noarch]
name=FastCGI noarch packages for Ceph
baseurl=http://gitbuilder.ceph.com/mod_fastcgi-rpm-{distro}-x86_64-basic/ref/master
enabled=1
priority=2
gpgcheck=1
type=rpm-md
gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/autobuild.asc

[fastcgi-ceph-source]
name=FastCGI source packages for Ceph
baseurl=http://gitbuilder.ceph.com/mod_fastcgi-rpm-{distro}-x86_64-basic/ref/master
enabled=0
priority=2
gpgcheck=1
type=rpm-md
gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/autobuild.asc

Create the ceph.repo file under the /etc/yum.repos.d directory:

# vim /etc/yum.repos.d/ceph.repo
## You can grab this code at publishers website.
[Ceph]
name=Ceph packages for $basearch
baseurl=http://ceph.com/rpm-firefly/el6/$basearch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc

[Ceph-noarch]
name=Ceph noarch packages
baseurl=http://ceph.com/rpm-firefly/el6/noarch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc

[ceph-source]
name=Ceph source packages
baseurl=http://ceph.com/rpm-firefly/el6/SRPMS
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc

Install yum-plugin-priorities:
```
# yum install yum-plugin-priorities
```
Install the apache (httpd), fastcgi (mod_fastcgi), ceph-radosgw, and ceph packages:
```
# yum install httpd mod_fastcgi ceph-radosgw ceph
```
Set FQDN for ceph-rgw host:
- Edit /etc/hosts and add IP, FQDN, and hostname in the # <rgw_ip_addr> <FQDN> <Hostname> format:
```
192.168.57.110  ceph-rgw.objectstore.com  ceph-rgw
```
- Edit /etc/sysconfig/network and set HOSTNAME as FQDN:
```
HOSTNAME=ceph-rgw.objectstore.com
```
- Check hostname and FQDN:
```
# hostname
# hostname -f
```

Configuring the RADOS gateway

The configuration of the RADOS gateway includes Apache and FastCGI configuration and Ceph key generation. Perform the following steps:

Configure Apache by editing /etc/httpd/conf/httpd.conf:
- Set ServerName <FQDN>
- Ensure the following line is present and uncommented:
```
LoadModule rewrite_module modules/mod_rewrite.so
```
Configure FastCGI by editing /etc/httpd/conf.d/fastcgi.conf:
- Ensure that the FastCGI modules are enabled:
```
LoadModule fastcgi_module modules/mod_fastcgi.so
```
- Turn off FastCgiWrapper
Create a Ceph object gateway script with the following content, change ownership, and allow executable permission. You can match the changes from the author's version of the s3gw.fcgi file provided with this book:
```
# vim /var/www/html/s3gw.fcgi
#!/bin/sh
exec /usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway

# chmod +x /var/www/html/s3gw.fcgi
# chown apache:apache /var/www/html/s3gw.fcgi
```

Create the gateway configuration file rgw.conf under /etc/httpd/conf.d with the following contents. Replace {fqdn} with servers fqdn (hostname -f) and {email.address} with the e-mail address of the server administrator. You can match the changes from the author's version of the rgw.conf file provided with this book:

FastCgiExternalServer /var/www/html/s3gw.fcgi -socket /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
<VirtualHost *:80>
        ServerName {fqdn}
        <!--Remove the comment. Add a server alias with *.{fqdn} for S3 subdomains-->
        <!--ServerAlias *.{fqdn}-->
        ServerAdmin {email.address}
        DocumentRoot /var/www/html
        RewriteEngine On
        RewriteRule  ^/(.*) /s3gw.fcgi?%{QUERY_STRING} [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
        <IfModule mod_fastcgi.c>
        <Directory /var/www/html>
                        Options +ExecCGI
                        AllowOverride All
                        SetHandler fastcgi-script
                        Order allow,deny
                        Allow from all
                        AuthBasicAuthoritative Off
                </Directory>
        </IfModule>
        AllowEncodedSlashes On
        ErrorLog /var/log/httpd/error.log
        CustomLog /var/log/httpd/access.log combined
        ServerSignature Off
      </VirtualHost>
# vim /etc/httpd/conf.d/rgw.conf

Create the RADOS gateway user and keyring for Ceph, log in to any of the Ceph monitor nodes, and execute the following:

Create the keyring:

# ceph-authtool --create-keyring /etc/ceph/ceph.client.radosgw.keyring
# chmod +r /etc/ceph/ceph.client.radosgw.keyring

Generate the gateway user and key for the RADOS gateway instance; our RADOS gateway instance name is gateway:
```
# ceph-authtool /etc/ceph/ceph.client.radosgw.keyring -n client.radosgw.gateway --gen-key
```

Add capabilities to the key:

# ceph-authtool -n client.radosgw.gateway --cap osd 'allow rwx' --cap mon 'allow rw' /etc/ceph/ceph.client.radosgw.keyring

Add the key to the Ceph cluster:

# ceph -k /etc/ceph/ceph.client.admin.keyring auth add client.radosgw.gateway -i /etc/ceph/ceph.client.radosgw.keyring

Distribute the key to the Ceph RADOS gateway node:

# scp /etc/ceph/ceph.client.radosgw.keyring ceph-rgw:/etc/ceph/ceph.client.radosgw.keyring

Create a pool for the RADOS gateway
```
# ceph osd pool create .rgw 128 128
```

Create the Ceph RADOS gateway data directory:

# mkdir -p /var/lib/ceph/radosgw/ceph-radosgw.gateway

Add a gateway configuration to Ceph, add the following configuration to Ceph monitor's ceph.conf file, and move this ceph.conf file to the RADOS gateway node. Make sure that the hostname is the RADOS gateway hostname, and not FQDN:

[client.radosgw.gateway]
host = ceph-rgw
keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
log file = /var/log/ceph/client.radosgw.gateway.log
rgw dns name = ceph-rgw.objectstore.com  ## This would be used for S3 API
rgw print continue = false
# scp /etc/ceph/ceph.conf ceph-rgw:/etc/ceph/ceph.conf

Adjust the ownership and permission on the RADOS gateway node for /var/log/httpd, /var/run/ceph, and /var/log/ceph. Set SELinux to Permissive:
```
# chown apache:apache /var/log/httpd
# chown apache:apache /var/run/ceph
# chown apache:apache /var/log/ceph
# setenforce 0
```
Start the Apache and Ceph RADOS gateway services. If you encounter any warning, you can ignore them at this point:
- Start the Apache service:
```
# service httpd start
```
- Start the ceph-radosgw service:
```
# service ceph-radosgw start
```

Verify the configuration:

Perform an HTTP GET request on radosgw FQDN using curl:
```
# curl http://ceph-rgw.objectstore.com
```

You should get a response similar to the one shown in the following snippet. It shows that your configuration is correct:

<?xml version="1.0" encoding="UTF-8"?>
 <ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Owner>
<ID>anonymous</ID>
<DisplayName></DisplayName>
</Owner><Buckets></Buckets>
</ListAllMyBucketsResult>

Submitting an HTTP request to the browser of an radosgw node can also do the verification.

Creating a radosgw user

To use the Ceph object storage, we need to create users for the RADOS gateway. These user accounts will be identified by access and secret keys, which can be used by clients to perform operations on the Ceph object storage.

Now, let's create a RADOS gateway user and access object storage:

Make sure the RADOS gateway machine (ceph-rgw) is able to access a Ceph cluster. Copy the Ceph keyring to the RADOS gateway machine from the monitor node:
```
# scp ceph.client.admin.keyring ceph-rgw:/etc/ceph
```
Execute Ceph cluster commands from ceph-rgw to ensure cluster accessibility:
```
# ceph -s 
```
Create a RADOS gateway user. This will also create an access_key and secret_key for the user, which will be required to access the Ceph object storage:
```
#  radosgw-admin user create --uid=mona --display-name="Monika Singh" [email protected]
```

Accessing the Ceph object storage

The Ceph object storage supports S3- and Swift-compatible APIs; to make use of object storage capabilities of Ceph, we need to configure S3 or Swift interfaces. We will now perform a basic configuration for these interfaces one by one. For advanced configurations, check their respective documentations.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.

Table of Contents for Object storage using the Ceph RADOS gateway

Create new playlist

Sign In