This is the step 3 of adding "Auto Scaling" to the Oracle Database Cloud Service.
If you want to check the other steps, please click the below links respectively.
- Step 1: Gather OCI DB System CPU Usage and write it into a custom log
- Step 2: Configure OCI Logging service to ingest the custom log
- Step 3: Configure OCI Service Connector Hub service to transfer the custom log to OCI Functions
- Step 4: Configure OCI Functions to act accordingly based on the CPU usage in the custom log
Configure OCI Service Connector Hub service to transfer the custom log to OCI Functions
By completing step 1: Gather OCI DB System CPU Usage and write it into a custom log and step 2: Configure OCI Logging service to ingest the custom log, we have successfully gathered, ingested and uploaded the custom log into OCI Logging service. As mentioned at the very beginning, our final target is using OCI Functions to scale up the DB System according to the CPU Usage information in the log.
So obviously, next action that we need to do is to transfer the ingested custom log from OCI Logging service to the OCI Functions. Service Connector Hub is the right service to do so.
Below the a brief introduction of Service Connector Hub excerpted from the official document.
Service Connector Hub is a cloud message bus platform that offers a single pane of glass for describing, executing, and monitoring movement of data between services in Oracle Cloud Infrastructure.
Service Connector Hub orchestrates data movement between services in Oracle Cloud Infrastructure.
Data is moved using service connectors. A service connector specifies the source service that contains the data to be moved, tasks to run on the data, and the target service for delivery of data when tasks are complete.
Below is the flow of data in Service Connector Hub. As we can see, Logging is supported as source, and Functions is supported as target. That's the right one we need.
[Figure: flow of data in SCH]
Prerequisite
To create a Service Connector Hub, we need existing Logging and Functions as source and target. The Custom log in OCI Logging service has been created in step 2, so we don't need to worry about the source. But for the target side, we have not created the real Function to handle the custom log yet.
But, We should have completed some basic tasks of OCI Functions. Otherwise, it will be very hard for you to move to next step: Configure OCI Functions to act accordingly based on the CPU usage in the custom log.
So, as a prerequisite, please follow the document Configuring Your Client Environment for Function Development and complete the Quick Start Guide.
There are three options to setup the Functions development environment. I choose the local machine option and you can choose any one as you wish.
If you have completed the Quick Start Guide, there should be an Application and a Function exists. Once we invoke the Function, it should return a message.
In below screenshot, the Function sayhi is a simple function and it will response "Hello, " when we invoke it. In next section we will use this Function to test the Service Connector Hub.
[Figure: function - sayhi]
Create a Service Connector Hub
Frankly, creating a Service Connector Hub is quite easy, as long as we have prepared the source and target.
Follow the steps of the below screenshots to create a SCH.
[Figure: visit SCH]
[Figure: Create SCH]
When inputting the details of the SCH, we select the created Custom Log as source and the simple Function as target.
[Figure: create SCH, details]
Verify the Service Connector Hub
From the web console, we know that the SCH is ACTIVE, that means it is running. But is it running as expected? Is it transferring logs from Logging to the Function? Most important, is the Function being invoked?
To verify whether the SCH is working, we can check the metrics of the SCH like below.
[Figure: SCH Metrics]
To verify whether the Function is invoked, we can enable Log of the Application and check the log in Logging service.
[Figure: Enable Function Application Log]
From the Function invoke log, we can see that the Function has been invoked successfully.
[Figure: Function invoke log]
Transfer Custom Log from Logging to Object Storage
Remember the Log Retention in Logging service? The longest retention we can select is 6 months, what if we want to retain the logs longer? The easiest way is to use Service Connector Hub. We can simply create a SCH to connect Logging and Object Storage, and transfer the logs to a specific bucket.
[Figure: Create SCH, connect Logging to Object Storage]
Next
[Figure: Step 3 completed]
Until now, we have completed the step 3: Configure OCI Service Connector Hub service to transfer the custom log to OCI Functions. We created a Service Connector Hub to transfer the custom log from Logging to the Functions and Object Storage, and we confirmed that the SCH is working as expected.
In next step, we will continue to complete the final step, step 4: Configure OCI Functions to act accordingly based on the CPU usage in the custom log