Site icon Ron Ekins' – Oracle Technology, DevOps and Kubernetes Blog

How to Replicate FlashBlade S3 Objects to Oracle Cloud Infrastructure (OCI) Object Storage

FlashBlade to OCI Object Replication

Pure Storage FlashBlade to Oracle Cloud Infrastructure (OCI) Object Replication

Introduction

The popularity and adoption of Object Storage continues to grow, with more and more applications and databases using or having an option to utilise Object Storage.

I have previously shared a number of posts discussing S3 including:

In this post I will show how we can replicate object data from an on-premises Pure Storage FlashBlade Object Store bucket to an Oracle Cloud Infrastructure (OCI) Object Storage private bucket.

Potential Use Cases

There are a number of reasons why you may want to replicate on-premises data to a Cloud Object store, including:

Benefits of Object Storage

Using Oracle Object Store as a staging area for data loading or data access has a number of benefits, these include:

Oracle Cloud Infrastructure (OCI)

Before we start, if you don’t already have an OCI account, you may want to consider creating an account. The OCI Always Free Service provides a number of free services including 10 GB Standard Object Storage and 10 GB Infrequent Access Object Storage.

Ok, let’s go!

Create OCI Bucket

Logon to Oracle Cloud Infrastructure (OCI) and from the Dashboard select Object Storage or via the hamburger menu navigate to Storage -> Object Storage -> Buckets.

Select your required Compartment, and click Create Bucket

Object Storage & Archive Storage

Provide a Bucket Name, and confirm storage options, for this post I will use the Standard Default Storage Tier. OCI also provides an archive storage tier for long-term storage which may be of interest as it is one-tenth of the cost of Standard object storage

Please Note: you need to enable Object Versioning as this is required for FlashBlade replication.

Object Storage & Archive Storage – Create Bucket

Note: Bucket names must be unique, if you have used the name even in another Compartment expect to see an error message.

By default the bucket is Private, however if required you can change the visibility by clicking on the 3 vertical dots on the right hand side and updating it to Public.

Object Storage & Archive Storage

Customer Secret Key

Before we can replicate to our OCI private bucket from the on-premises Pure Storage FlashBlade we need to provide authentication.

To do this from the OCI dashboard, Click on your Profile icon (top right), navigate to: 

User Settings -> Customer Secret Keys and click Generate Secret Key providing a descriptive name.

Customer Secret Keys

Enter secret key Name

Generate Secret Key

Once generated you will need to select Copy, this will be the aws_secret_access_key, which we will be using later.

Generated Key

Now, click on the newly created Secret Name to obtain the aws_access_key, again save this for later.

Customer Secret Keys

Determine Object Storage Namespace Name

From the OCI dashboard, Click on your Profile icon (top right), navigate to Tenancy

The Object Storage Namespace and default S3 compartments should be visible.

Tenancy details

Alternatively, you can use the OCI command line interface, for example.

% oci os ns get
{
  "data": "<Object Storage Namespace>"
}

Test OCI Object Storage URL

To test access to the OCI Object Storage Bucket we can use the AWS s3api command line interface.

Update your local ~/.aws/credentials file with a profile and query the OCI Object Storage URL, for example.

% aws s3api list-buckets --query "Buckets[].Name" --endpoint-url https://<Object Storage Namespace>.compat.objectstorage.uk-london-1.oraclecloud.com  --profile oci
[
    "FlashBlade"
]

The OCI Bucket URL format is https://<Object Storage Namespace>.compat.objectstorage.<region>.oraclecloud.com

A full list of OCI Object Storage endpoints is available in the OCI Documentation.

Pure Storage FlashBlade

Now the FlashBlade side.

Create S3 Target Connection

Logon to the Pure Storage FlashBlade and navigate to Storage -> S3 Target Connections

Click the “+” to the right of Connect S3 Target

Enter Name and Address

Connect S3 Target

Add Remote S3 Credentials

Navigate to Protection -> Object Replication -> Remote Credentials

Click the “+” to the right of Create Remote Credentials

Select the OCI Target from the pick list for Remote Array or Target

Enter a Name for the connection, for example: OCI-UK-London1

Paste the OCI Access Key ID and Secret Key we created earlier.

Click “Create”

Create Remote Credentials

Setup Bucket Replication to OCI Object Storage

Navigate to Protection -> Object Replication -> Bucket Replica Link

Select Local Bucket Name, Remote Array or Target, Remote Credential and provide the OCI Remote Bucket name.

From the FlashBlade we can now see we have established a Bucket Replica Link.

FlashBlade Object Store

Test Replication

To test the Replication Link upload a test file to the FlashBlade bucket, for example.

$ aws s3api put-object --bucket oracle-bucket --key VagrantAPEX.mp4 --body /home/oracle/osbws/VagrantAPEX.mp4 --endpoint-url ${ENDPOINT} --profile default

OCI Object Storage

And returning to OCI, navigate to OCI Object Storage -> Bucket Details and confirm replication.

Object Storage Bucket Objects

Summary

In this post I have shared some of the use cases and reasons why we may want to replicate on-premises data to the Cloud. I have also demonstrated how we can replicate an on-premises Pure Storage FlashBlade Object Store bucket to an Oracle Cloud Infrastructure (OCI) Object Storage private bucket.

Hope you found this post useful.

[twitter-follow screen_name=’RonEkins’ show_count=’yes’]

Exit mobile version