This post is part of the OCI Logging – Complete Hands-on Series. Make sure to check out the other posts as well.
I have an OCI Container Engine for Kubernetes (OKE) Cluster running with 3 worker nodes. I want to ingest logs into OCI Logging from the nodes.
We will achieve this by creating a Custom Log.
Check Monitoring Agent
The OCI Unified Monitoring Agent is automatically installed on all OCI instances. You should check that the Agent is installed on your worker nodes either from the OCI Console:
From the Instance page of your OKE Worker Node, check that Custom Logs Monitoring plugin is enabled in the Oracle Cloud Agent tab
Or you can connect to the worker node and check the status of the Agent:
systemctl status unified-monitoring-agent
If the Agent is not installed, please check the Installation steps.
Before creating the Custom Log, we need to define a Host Group which will point to the VMs from which we want to get logs from. The Host Groups can be defined either by using Dynamic Groups or User Groups.
We’ll be defining our Host Group with Dynamic Groups and we’ll point it to our compartment where we have the OKE Cluster:
Make sure it points to the compartment where you have the OKE Cluster running, so we can take logs from all the worker node that exist right now, but also from any other node that will be created in the future without making any modifications to the Logging configuration.
Create the Custom Log
Now that we have our Host Group defined as a Dynamic Group, we can navigate to the Main Menu -> Observability & Management -> Logs (under the Logging Category) and Click on the Create Custom Log at the top of your screen:
In the first screen, give the log a name, place it in a compartment and in a Log Group.
In this screen, we’ll define the Agent Configuration for our log: Give the Agent Configuration a name, a description and select the Host Group created earlier as Dynamic Group
Also, click on Create to automatically create the necesary policy to allow the instances in the Dynamic Group to use the Logging Service if prompted
Next we define the Log Input. We’ll select Log Path input type and we’ll give this log input a name, specify the file path(s) (you can also use * wildcard in your file path definition) and click on Create Custom Log.
I will be using /var/log/* and /var/log/container/*.log filepaths as an example
You can easily add multiple File Paths in one Log Input, or you can create multiple Log Inputs by clicking the + Another log input button.
No need to select a parser for this example.
You can now check the logs directly from the OCI Console:
Expand a log entry using the little arrow on the right-hand side and you can see the whole content of your log plus some additional paramters added by OCI. For example the source – what node it came from and other important info.
You can check out this post on How to search logs efficiently in OCI and this one for a more complete view of How to configure custom logs in OCI for any type of workload.