Troubleshooting Calamari

Troubleshooting Calamari is sometimes very tricky. In this recipe, I have collected some recipes that will help you troubleshoot your Calamari environment.

How to do it…

  1. Check if the Calamari server, that is, the salt-master, could reach salt-minions:
    # salt '*' test.ping
    
  2. Check if salt-master can query the Ceph cluster and get cluster information:
    # salt '*' ceph.get_heartbeats output
    
  3. Check if the salt-minion node (Ceph cluster node) could reach the salt-master:
    # salt-minion -l debug
    
  4. Salt-minion throws an error, The Salt Master server's public key did not authenticate!, under the salt-minion log file, that is, /var/log/salt.
    How to do it…
  5. To solve this problem, we need to delete the minion salt-key as well as the master public key and recreate both of them using the following commands:
    # rm -rf /etc/salt/pki/minion/minion_master.pub
    # systemctl stop salt-minion
    

    Now, perform the following set of steps:

    1. On salt-master, delete the minion key:
      # salt-key -L
      # slat-key -D <minion name>
      
    2. Start the salt-minion service:
      # systemctl start salt-minion
      
    3. On salt-master, accept the new key for minion:
      # salt-key -L
      # salt-key -A
      
    4. Finally, on salt-minion, use # salt-minion -l and make sure you do not get this error again.
  6. Execute the calamari-ctl initialize command that will throw the could not connect to server: Connection refused error as follows:
    How to do it…

    Then, perform the set of steps mentioned as follows:

    1. First, check if the postgres services are running:
      # systemctl status postgres
      
    2. Log in to postgres and check whether the Calamari database exists or not:
      # sudo -u postgres psql
      List postgres databases # l
      
      How to do it…
    3. You might see that there is no database called calamari; now you should invoke salt to create a calamari user and database in postgres:
      # salt-call --local state.template /opt/calamari/salt-local/postgres.sls
      
    4. Finally, re-run # calamari-ctl initialize; it should work this time.
  7. The calamari-ctl initialize command line fails to throw the Updating already connected nodes. failed with rc=2 error, as seen in the following screenshot:
    How to do it…
  8. To fix this error, edit the /opt/calamari/venv/lib/python2.7/site-packages/calamari_cthulhu-0.1-py2.7.egg/cthulhu/calamari_ctl.py file, and comment out line 255, which says update_connected_minions().
    How to do it…
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.220.139.168