Make sure that your Ceph cluster is in full health by checking with the ceph -s command. We will be upgrading OSD by first removing it from the cluster and then letting Ceph recover the data onto the new BlueStore OSD. By taking advantage of this hot maintenance capability in Ceph, you can repeat this procedure across all the OSDs in your cluster.
In this example, we will remove osd.2, which is residing on the /dev/sdb disk on the OSD node, by performing the following steps:
- Use the following command:
sudo ceph osd out 2
The preceding command gives the following output:
- Log into the OSD node that contains the OSD you wish to recreate. We will stop the service and unmount the XFS partition:
systemctl stop ceph-osd@2
umount /dev/sdb1
- Go back to one of your monitors and now remove the OSD using the following commands:
sudo ceph osd crush remove osd.2
The preceding command gives the following output:
sudo ceph auth del osd.2
The preceding command gives the following output:
sudo ceph osd rm osd.2
The preceding command gives the following output:
- Check the status of your Ceph cluster with the ceph -s command. You should now see that the OSD has been removed. Once recovery has completed, you can now recreate the disk as a BlueStore OSD.
- Go back to your OSD node and run the following ceph-disk command to wipe the partition details from the disk:
sudo ceph-disk zap /dev/sdb
The preceding command gives the following output:
- Now issue the ceph-disk command to create the bluestore OSD. In this example, we will not be storing the WAL and DB on separate disks, so we do not need to specify those options:
ceph-disk prepare --bluestore /dev/sdb
The preceding command gives the following output:
- And finally activate OSD:
ceph-disk activate /dev/sdb1
Returning to our monitor node, we can now run another ceph -s and see that a new OSD has been created and data is starting to be backfilled to it.
As you can see, the overall procedure is very simple and is identical to the steps required to replace a failed disk.