Create a Ceph Cluster with DeepSea

Kai Wagner — Tue, 21 Feb 2017 08:45:47 GMT

How to setup a ceph cluster with DeepSea on top of SUSE Leap VMs.

Install three VMs and configure "/etc/hosts" -> easiest way to do is to add all nodes in "/etc/hosts" on all VMs. Make sure FQDN are set, too
Install "salt-master" on one VM "zypper in salt-master". Enable the service: "systemctl enable salt-master"
Install "salt-minion" on ALL VMs "zypper in salt-minion". Enable the service: "systemctl enable salt-minion"
Configure "/etc/salt/minion" on all systems to point to the right master and restart minions "systemctl restart salt-minion"
Accept keys on the salt-master "salt-key -A -y"
Install the build tools on the salt-master VM "zypper in git make"
Clone the DeepSea repo "git clone https://github.com/SUSE/DeepSea.git" on the salt-master VM
Inside the DeepSea folder run "make install"
On the salt master run "systemctl restart salt-master"
Run "chown -R salt /srv/pillar/*"
Run "salt-run state.orch ceph.stage.0" or "salt-run state.orch ceph.stage.prep"
Run "salt-run state.orch ceph.stage.1" or "salt-run state.orch ceph.stage.discovery"

Create "/srv/pillar/ceph/proposals/policy.cfg" and run "chown salt:salt /srv/pillar/ceph/proposals/policy.cfg". Example:

# Cluster assignment
cluster-ceph/cluster/*.sls
# Hardware Profile
profile-*-1/cluster/*.sls
profile-*-1/stack/default/ceph/minions/*yml
# Common configuration
config/stack/default/global.yml
config/stack/default/ceph/cluster.yml
# Role assignment
role-master/cluster/*.sls
role-mon/cluster/*.sls
role-mon/stack/default/ceph/minions/*.yml

Run "salt-run state.orch ceph.stage.2" or "salt-run state.orch ceph.stage.configure"
Run "DEV_ENV=true salt-run state.orch ceph.stage.3" or "DEV_ENV=true salt-run state.orch ceph.stage.deploy" -> "DEV_ENV is only needed if you have <=3 VMs"
You should change the pg(p)_num to get rid of the "to few pgs per osd" error "ceph osd pool set rbd pg_num 120" and "ceph osd pool set rbd pgp_num 120"

You now should have an up and running cluster.

Now you could add the openATTIC OBS (openSUSE Build Service) repo and install openATTIC and the openattic-deployment package

Add the needed Repo from OBS "zypper ar http://download.opensuse.org/repositories/filesystems:/openATTIC/openSUSE_Leap_42.2/filesystems:openATTIC.repo"
Install openATTIC "zypper in openattic openattic-module-ceph-deployment" -> you have to break dependencies because deepsea isn't shipped with leap by default
Run "chmod 655 /etc/ceph/ceph.*"
Run "openattic install"

You're done :)

Update 2017-04-07

Fixed some typos and added some more steps to do.

openATTIC (Posts about ceph howto cluster deepsea)

Create a Ceph Cluster with DeepSea