Introduction
The popularity and adoption of Object Storage continues to grow, with more and more applications and databases using or having an option to utilise Object Storage.
I have previously shared a number of posts discussing S3 including:
- Performing Oracle RMAN backups to S3 to Amazon S3 Object Storage
- Performing Oracle RMAN backups to SS to on-premises FlashBlade S3 Object Storage
- How to access OCI Object Storage from your on-premises Oracle 19c Database
In this post I will show how we can replicate object data from an on-premises Pure Storage FlashBlade Object Store bucket to an Oracle Cloud Infrastructure (OCI) Object Storage private bucket.
Potential Use Cases
There are a number of reasons why you may want to replicate on-premises data to a Cloud Object store, including:
- Data Protection
- Replicating Oracle RMAN backups to Cloud Object Storage
- Data Archiving
- Replicating on-premises Oracle RMAN backups to Cloud Archive Object Storage
- Database Cloud Migration
- Transferring on-premises data to Cloud Object Storage for migration using for example:
- SQL Developer Data Import Wizard
- SQL Developer Migration Workbench
- Oracle Data Pump
- Database Links
- DBMS_CLOUD package
- External Tables
- Transferring on-premises data to Cloud Object Storage for migration using for example:
- Data Warehouse and Data Lake workloads
- Transferring on-premises data for use with data warehouse and data lake solutions
- Data Mobility
- Replicating data for the creation of Cloud development and test environments.
Benefits of Object Storage
Using Oracle Object Store as a staging area for data loading or data access has a number of benefits, these include:
- Simplifying architectures; simplifying ELT/E-TL on Object Storage
- Options to Query-in-Place from ADM
- Object Storage is independent from the data processing engines
- Less expensive storage costs
Oracle Cloud Infrastructure (OCI)
Before we start, if you don’t already have an OCI account, you may want to consider creating an account. The OCI Always Free Service provides a number of free services including 10 GB Standard Object Storage and 10 GB Infrequent Access Object Storage.
Ok, let’s go!
Create OCI Bucket
Logon to Oracle Cloud Infrastructure (OCI) and from the Dashboard select Object Storage or via the hamburger menu navigate to Storage -> Object Storage -> Buckets.
Select your required Compartment, and click Create Bucket

Provide a Bucket Name, and confirm storage options, for this post I will use the Standard Default Storage Tier. OCI also provides an archive storage tier for long-term storage which may be of interest as it is one-tenth of the cost of Standard object storage
Please Note: you need to enable Object Versioning as this is required for FlashBlade replication.

Note: Bucket names must be unique, if you have used the name even in another Compartment expect to see an error message.
By default the bucket is Private, however if required you can change the visibility by clicking on the 3 vertical dots on the right hand side and updating it to Public.

Customer Secret Key
Before we can replicate to our OCI private bucket from the on-premises Pure Storage FlashBlade we need to provide authentication.
To do this from the OCI dashboard, Click on your Profile icon (top right), navigate to:
User Settings -> Customer Secret Keys and click Generate Secret Key providing a descriptive name.

Enter secret key Name

Once generated you will need to select Copy, this will be the aws_secret_access_key, which we will be using later.

Now, click on the newly created Secret Name to obtain the aws_access_key, again save this for later.

Determine Object Storage Namespace Name
From the OCI dashboard, Click on your Profile icon (top right), navigate to Tenancy
The Object Storage Namespace and default S3 compartments should be visible.

Alternatively, you can use the OCI command line interface, for example.
% oci os ns get { "data": "<Object Storage Namespace>" }
Test OCI Object Storage URL
To test access to the OCI Object Storage Bucket we can use the AWS s3api command line interface.
Update your local ~/.aws/credentials file with a profile and query the OCI Object Storage URL, for example.
% aws s3api list-buckets --query "Buckets[].Name" --endpoint-url https://<Object Storage Namespace>.compat.objectstorage.uk-london-1.oraclecloud.com --profile oci [ "FlashBlade" ]
The OCI Bucket URL format is https://<Object Storage Namespace>.compat.objectstorage.<region>.oraclecloud.com
A full list of OCI Object Storage endpoints is available in the OCI Documentation.
Pure Storage FlashBlade
Now the FlashBlade side.
Create S3 Target Connection
Logon to the Pure Storage FlashBlade and navigate to Storage -> S3 Target Connections
Click the “+” to the right of Connect S3 Target
Enter Name and Address

Add Remote S3 Credentials
Navigate to Protection -> Object Replication -> Remote Credentials
Click the “+” to the right of Create Remote Credentials
Select the OCI Target from the pick list for Remote Array or Target
Enter a Name for the connection, for example: OCI-UK-London1
Paste the OCI Access Key ID and Secret Key we created earlier.
Click “Create”

Setup Bucket Replication to OCI Object Storage
Navigate to Protection -> Object Replication -> Bucket Replica Link
Select Local Bucket Name, Remote Array or Target, Remote Credential and provide the OCI Remote Bucket name.

From the FlashBlade we can now see we have established a Bucket Replica Link.

Test Replication
To test the Replication Link upload a test file to the FlashBlade bucket, for example.
$ aws s3api put-object --bucket oracle-bucket --key VagrantAPEX.mp4 --body /home/oracle/osbws/VagrantAPEX.mp4 --endpoint-url ${ENDPOINT} --profile default

OCI Object Storage
And returning to OCI, navigate to OCI Object Storage -> Bucket Details and confirm replication.

Summary
In this post I have shared some of the use cases and reasons why we may want to replicate on-premises data to the Cloud. I have also demonstrated how we can replicate an on-premises Pure Storage FlashBlade Object Store bucket to an Oracle Cloud Infrastructure (OCI) Object Storage private bucket.
Hope you found this post useful.
[twitter-follow screen_name=’RonEkins’ show_count=’yes’]
Leave a Reply