🧰 Ceph Operations Cheat Sheet¶
This page provides quick reference commands and practices for monitoring, troubleshooting, and maintaining your Ceph cluster in the homelab.
🔍 Monitoring Cluster Health¶
-
Cluster status:
ceph -s -
Detailed health report:
ceph health detail -
Monitor quorum status:
ceph quorum_status --format json-pretty -
OSD layout:
ceph osd tree -
Pool usage and capacity:
ceph df
📦 Pool & Placement Groups¶
-
List pools:
ceph osd pool ls -
Pool details:
ceph osd pool get <pool-name> all -
Placement group statistics:
ceph pg stat
⚙️ OSD Management¶
-
List OSDs:
ceph osd ls -
Check OSD status:
ceph osd stat -
Restart an OSD service:
systemctl restart ceph-osd@<id> -
Mark OSD out/in:
ceph osd out <id> ceph osd in <id>
🛠 Troubleshooting Common Issues¶
OSD Down / Out¶
- Restart the OSD service:
systemctl restart ceph-osd@<id> - Replace failed disk and recreate OSD if necessary.
PGs Stuck / Inactive¶
- Verify network connectivity between nodes.
- Ensure all MONs are healthy:
ceph quorum_status - Restart Ceph services if needed.
Slow Requests¶
- Check Ceph network latency:
ping <node-ip> iperf3 -c <node-ip> - Verify disk health:
smartctl -a /dev/sdb
Full or Near-Full Cluster¶
- Add more OSDs (additional disks).
- Adjust pool size or PG count.
- Clean up unused images or snapshots.
MON Quorum Loss¶
- Ensure at least 2 of 3 MONs are running.
- Restart MONs:
systemctl restart ceph-mon@<hostname>
🧹 Maintenance & Housekeeping¶
-
Check logs:
journalctl -u ceph-mon@<hostname> journalctl -u ceph-osd@<id> -
Rebalance cluster:
ceph osd reweight-by-utilization -
Scrub pools (data consistency check):
ceph osd scrub <id>
🚀 Best Practices¶
- Dedicate a separate network for Ceph traffic.
- Monitor cluster health daily with
ceph -s. - Use Proxmox Backup Server alongside Ceph for snapshots and deduplication.
- Keep OSDs balanced and monitor PG counts as the cluster grows.