This post is part of the OCI Logging – Complete Hands-on Series. Make sure to check out the other posts as well.
Intro
Once you have configured any type of logs in OCI, it can get difficult finding anything in the millions of lines of logs.
We’ll look first at all the places you can find your logs in OCI and then we’ll try out some tips & tricks on searching the logs using OCI Logging Search feature.
As always, you can check some of the other blog posts from the series if you want to see how to configure OCI Logging for different scenarios or how to use Machine Learning Algorithms to visualize your logs, and more.
Where can you find your logs?
There are multiple places in OCI where we can view the logs (OCI Service Logs, Custom Logs, and Audit Logs):
- You can find all of your logs in one place under Observability & Management –> Logging –> Logs
- You can use the Search feature from Observability & Management –> Logging –> Search menu for all logs
- Compute Custom Logs for a specific instance ca be found directly from the Compute Instance page under Custom Logs
- All OCI Service Logs can be found on each of the service’s pages (that support Service Logs, list of services can be found here) – example for Load Balancer Service Logs:
- The Audit menu for the OCI Audit logs
OCI Logging tips & tricks
First, some obvious but sometimes forgotten tips
- Use Log Groups to group your logs in an efficient matter (by project, by services, or whatever way makes sense for you). It will make your life easier when trying to find something as you can choose to view all logs in a group, for example.
- Use consistent Log Names. Try to establish a clear log naming convention for all your logs. Example: serviceName_logName
- When creating a Log Agent Configuration for a Custom Log, we define Host Groups using Dynamic Groups or User Groups. This is how to define where we want to take logs from. Make sure that your definition of the Dynamic Group covers all of your instances – for example, define your Dynamic Group to match a compartment, so whenever new Instances are created inside that compartment, you won’t have to modify your Custom Log Definition and logs will automatically be ingested from the new instances (if they have the same path of course). Check out How to configure custom logs in OCI for any type of workload for more info
Searching tips
- First of all, select only the logs you want to search. You can do the selection on 3 layers: compartment level, log group level, or log level. You can also combine your selection, let’s say you want to search through the logs from a whole compartment, but also from a specific log that is in another compartment. It’s up to you.
As you can see, you can select either a whole compartment, a whole Log Group or an individual Log to search through. You can also combine multiple logs from different compartments or Log Groups
- Filter logs – the easy way: When viewing the logs, you can expand a logline and then just click on a specific key-value pair and use the “Filter” tool. For example: if you have logs from multiple instances in a log, click on the instance_ocid key and filter match to see logs only from that specific instance.
You can select any parameter to filter on – or to exclude from your search using Filter Not Matching
- Another way to filter the logs is to use the search bar. Here you can access all parameters available in your log entries to search or you can just type a text and will search in your log content automatically.
As you can see, we can access all paramters easily and filter on each one of them
- You can configure the Summary View of your log messages to add other fields as well, like the source of the log (i.e. the name of my compute instances in the case of custom logs).
You can add and remove fields from the summary view easily to cusomise your view
- If you have specific patterns or messages that you need to filter on more than one time – save your searches for later use and then you can just go to Saved Searches and you’re good to go.
You can now go to Saved Searches and see directly the filtered logs
- Visualize the logs in different ways to better understand your records. For example, let’s look at how to visualize Accept and Reject requests from your network flow logs.
Check out different ways to group and vizialize your log records
For extended search & visualization capabilities, send the logs to Logging Analytics. Check out the following article: Analyze logs in OCI with Logging Analytics to make your life easier (soon)
