Make Your Own S3 Object Storage

Leverage on-promise object storage with Ceph

Photo by chuttersnap on Unsplash

Object Storage

Ceph

Ceph architecture

Creating our storage cluster

Installing ceph-deploy

user@ceph ~ # wget -q -O- ‘https://download.ceph.com/keys/release.asc' | sudo apt-key add –
user@ceph ~ # echo deb $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph.list
user@ceph ~ # sudo apt update
user@ceph ~ # sudo apt install ceph-deploy
user@ceph ~ # useradd -d /home/cephmgt -m cephmgt
user@ceph ~ # passwd cephmgt
user@ceph ~ # echo "cephmgt ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/cephmgt
user@ceph ~ # chmod 0440 /etc/sudoers.d/cephmgt
user@ceph ~ # ssh-keygen
user@ceph ~ # for i in {1..6}; do ssh-copy-id cephmgt@ceph0${i}; done
user@ceph ~ # cat .ssh/config
Host ceph01
Hostname ceph01
User ceph-mgt

Data cluster

user@ceph ~ # mkdir my-cluster
user@ceph ~ # cd my-cluster
user@ceph ~ # ceph-deploy new ceph01 ceph02 ceph03 ceph04 ceph05 ceph06
user@ceph ~ # ceph-deploy install ceph01 ceph02 ceph03 ceph04 ceph05 ceph06
user@ceph ~ # ceph-deploy mon create-initial
user@ceph ~ # ceph-deploy admin ceph01 ceph02 ceph03 ceph04 ceph05 ceph06
user@ceph ~ # ceph-deploy mon add ceph01 ceph02 ceph03 ceph04 ceph05 ceph06
user@ceph ~ # ceph-deploy osd create ceph01:sdb ceph02:sdb ceph03:sdb ceph04:sdb ceph05:sdb ceph06:sdb
user@ceph ~ # ceph-deploy rgw create ceph01 ceph02 ceph03 ceph04 ceph05 ceph06
Example of node cluster

Data Pool

user@ceph ~ # ceph osd pool create my-userfiles 64
user@ceph ~ # ceph osd pool set my-userfiles size 2
user@ceph ~ # ceph osd pool set my-userfiles min_size 1

S3 API configuration (radosGW)

user@ceph ~ # sudo radosgw-admin user create --uid="my-api" --display-name="My API"
"user": "my-api","access_key": "XXXXXXXXXXXXX","secret_key": "XXXXXXXXXXXX"
user@ceph ~ # sudo radosgw-admin caps add --uid="supfile-api" --caps="users=*;buckets=*;metadata=*;usage=*;zone=*" 
user@ceph ~ # sudo radosgw-admin quota set --uid="supfile-api" --quota-scope=bucket --max-size=30G
user@ceph ~ # sudo radosgw-admin quota enable --quota-scope=bucket --uid="supfile-api"

Conclusion

🦄 Freelance Software Engineer 🚀 Indie maker/Tech entrepreneur https://alexandrelion.com