Profiling Ceph memory

Memory profiling is the process of dynamic program analysis to determine a program's memory consumption and identify ways to optimize it. In this recipe, we discuss how you can use memory profilers on the Ceph daemons for memory investigation.

How to do it…

  1. Start the memory profiler on a specific daemon:
    # ceph tell osd.0 heap start_profiler
    

    Note

    To auto-start profiler as soon as the ceph OSD daemon starts, set the environment variable as CEPH_HEAP_PROFILER_INIT=true.

  2. It's a good idea to keep the profiler running for a few hours so that it can collect as much information related to memory footprint as possible. At the same time, you can also generate some load on the cluster.
  3. Next, print heap statistics about the memory footprint that the profiler has collected:
    # ceph tell osd.0 heap stats
    
    How to do it…
  4. You can also dump heap stats on a file that can be used later; by default, it will create the dump file as /var/log/ceph/osd.0.profile.0001.heap:
    # ceph tell osd.0 heap dump
    
    How to do it…
  5. To read this dump file, you will require google-perftools:
    # yum install -y google-perftools
    
  6. To view the profiler logs:
    # pprof --text {path-to-daemon}  {log-path/filename}
    # pprof --text /usr/bin/ceph-osd /var/log/ceph/osd.0.profile.0001.heap
    
  7. For granule comparison, generate several profile dump files for the same daemon, and use the Google profiler tool to compare it:
    # pprof --text --base /var/log/ceph/osd.0.profile.0001.heap /usr/bin/ceph-osd /var/log/ceph/osd.0.profile.0002.heap
    
  8. Release memory that TCMALLOC has allocated but is not being used by Ceph:
    # ceph tell osd.0 heap release
    
  9. Once you are finished, stop the profiler:
    # ceph tell osd.0 heap stop_profiler
    

The Ceph daemons process has matured much, and you might not really need memory profilers for analysis, unless you encounter a bug that's causing memory leaks. You can use the previously discussed procedure to figure out memory issues with the Ceph daemons.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.220.124.177