this is a followup on: https://gist.github.com/yuvalif/d50bcc3d27121732ffbbafe7b7146112 for an RGW of version "squid" or above, where bucket notifications and topics are synced between zones
- deploy a zonegroup with 2 zones:
MON=1 OSD=1 MDS=0 MGR=0 ../src/test/rgw/test-rgw-multisite.sh 2
- export credentials:
export AWS_ACCESS_KEY_ID=1234567890
export AWS_SECRET_ACCESS_KEY=pencil
- download a python server:
wget https://gist.githubusercontent.com/mdonkers/63e115cc0c79b4f6b8b3a6b797e485c7/raw/a6a1d090ac8549dac8f2bd607bd64925de997d40/server.py
- run an HTTP receiver "per zone". in one terminal run:
python server.py 10900
- and in another terminal run:
python server.py 11900
- on zone 1 create a topic for the zone 1 HTTP receiver:
AWS_DEFAULT_REGION=zg1 aws --endpoint-url http://localhost:8101 sns create-topic --name=fishtopic1 --attributes='{"push-endpoint": "http://localhost:10900"}'
- on zone 2 create a topic for the zone 2 HTTP receiver:
AWS_DEFAULT_REGION=zg2 aws --endpoint-url http://localhost:8201 sns create-topic --name=fishtopic2 --attributes='{"push-endpoint": "http://localhost:11900"}'
it does not really matter on which zone the topic is created, since the topic information is synced however, the above is what we expect from the application that sets the topics
- create a bucket:
aws --endpoint-url http://localhost:8101 s3 mb s3://mybucket
- create a sync notification for both topics on the above bucket:
aws --endpoint-url http://localhost:8101 s3api put-bucket-notification-configuration --bucket mybucket --notification-configuration='{"TopicConfigurations": [{"Id": "notif1", "TopicArn": "arn:aws:sns:zg1::fishtopic1", "Events": ["s3:ObjectSynced:*"]}]}'
aws --endpoint-url http://localhost:8201 s3api put-bucket-notification-configuration --bucket mybucket --notification-configuration='{"TopicConfigurations": [{"Id": "notif1", "TopicArn": "arn:aws:sns:zg1::fishtopic2", "Events": ["s3:ObjectSynced:*"]}]}'
- upload a file to zone 1:
head -c 1M < /dev/urandom > myfile-zg1-1
aws --endpoint-url http://localhost:8101 s3 cp myfile-zg1-1 s3://mybucket
- once the object is synced, the RGW in zone 2 will send updates to both HTTP receivers
use the
"x-amz-id-2"field in the notification to verify the originating RGW
- upload a file to zone 2:
head -c 1M < /dev/urandom > myfile-zg1-2
aws --endpoint-url http://localhost:8201 s3 cp myfile-zg1-2 s3://mybucket
- same behavior is observed
- set negative zone filter on the notification. each zone exclude itself:
aws --endpoint-url http://localhost:8101 s3api put-bucket-notification-configuration --bucket mybucket --notification-configuration='{"TopicConfigurations": [{"Id": "notif1", "TopicArn": "arn:aws:sns:zg1::fishtopic1", "Filter": {"Zones": {"FilterRules": [{"Name": "zg1-1", "Type": "OUT"}]}}, "Events": ["s3:ObjectSynced:*"]}]}'
aws --endpoint-url http://localhost:8201 s3api put-bucket-notification-configuration --bucket mybucket --notification-configuration='{"TopicConfigurations": [{"Id": "notif1", "TopicArn": "arn:aws:sns:zg1::fishtopic2", "Filter": {"Zones": {"FilterRules": [{"Name": "zg1-2", "Type": "OUT"}]}}, "Events": ["s3:ObjectSynced:*"]}]}'
- upload files to zone 1 and zone 2, and verify that:
- when uploading a file to zone 1, only the zone 2 HTTP receiver get a notification from zone 2 RGW
- when uploading a file to zone 2, only the zone 1 HTTP receiver get a notification from zone 1 RGW
Yes @yuvalif, setting the notification to only honor object sync on respective zones with an IN makes more logical sense.
We only expect topics to receive events registered on that particular zone.