You are currently viewing How to create custom dashboards using logging data in OCI

How to create custom dashboards using logging data in OCI

This post is part of the OCI Logging – Complete Hands-on Series. Make sure to check out the other posts as well.

Intro

It’s great to have the logs ingested and available to you when needed, but it’s even better to have some insightful dashboards for a quick overview.

In this article, we’ll set up some simple dashboards based on the logs ingested with OCI Logging.

Configuration Overview

In order to create these nice dashboards based on the ingested logs, we have to, of course, ingest some logs first. I will be using some custom logs from an OCI instance and from an OKE Cluster, but also some service logs like Networking Flow logs and Object Storage Write Access Event logs.

If you need any guidance on this, check out these previous articles: How to gain insight into your OCI Native Resources with Service Logs, How to configure custom logs in OCI for any type of workload, and How to get logs from OKE nodes with OCI Logging.

In this article we’ll configure the following visualizations:

  • Chart with error logs from OKE/Compute Instances grouped by source – Easily visualize and identify if you have any errors and from what machine/node
  • Table view of error logs from OKE/Compute Instances – For easy access to the log messages
  • Networking Flow Logs grouped by Action (accept or reject) – Have an overview of Accept and Reject traffic in your subnet
  • Uploaded and deleted objects count in your Object Storage Buckets – Have an idea of how many objects are uploaded and deleted in your buckets over a period of time
  • Overview over deleted resources in tenancy – Have a clear image of what resources and how many were deleted over a period of time

These are just some examples to get you started. Using this base, you can create your own dashboards based on your use case.

Configure Logs

Custom Logs

For the first two dashboards, I am using logs from an OKE cluster but also logs from a Compute Instance I have running in OCI. I will not go into details about how to configure these logs here, so if you need any help, please check this blog for OKE logs and this blog for Compute Instance Logs.

Service Logs

For the other dashboards, I am using Networking Flow Logs, Object Storage Write Logs, and Audit logs. All of these are Service logs that can be activated with a click of a button.  For general information about OCI Service Logs, check out this blog post.

Networking Flow Logs Activation

You can repeat this process for all the subnets you wish to activate the networking flow logs for.

Object Storage Write Access Logs Activation

You can repeat this process for all buckets you wish to activate the logs for.

Audit Logs Activation

Audit logs in OCI are activated by default in all tenancies and cannot be deactivated or altered.

These are some examples, you can activate or create any logs you want/need.

Configure Dashboards

OCI Dashboards is a relatively new feature that can bring a lot of value. If you’re interested in learning more about OCI Dashboards, check out this blog post or this other blog. 

You can easily find the Dashboards on your OCI Homepage here:

When you click on Dashboard you’ll probably see a default dashboard provided to you.

Let’s click on New Dashboard, give your Dashboard a name, a description, place it in a compartment and in a Dashboard Group.

Tip: If you’re using multiple compartments for different environments or different projects, don’t hesitate to create multiple dashboards for each one

Once you’ve created your dashboard, you have an empty canvas where you can add widgets using the Add Widget button.

In this blog post, we’ll play with the Logging chart and Logging data table, but feel free to try the other widgets as well.

Configure Logging Chart for Error Messages

The first chart will be a logging chart that will filter all the custom logs for any of them that contain the word *ERROR*. I will set the visualization parameters so they are grouped by source to easily identify from where the error message came from.

  1. Give your chart a name, a description, and make sure to select the correct region of your logs
  2. Set the visualization parameters to datetime for the X Axis, and source for Group By. You can play around with the Filter by time toggle if you don’t have any results visible. Don’t worry about this as you’ll be able to toggle for different periods after you create the chart as well
  3. Make sure to select the right logs – in my case I’ve selected two Log Groups where I have the OKE Logs but also my Compute Instance Logs
  4. Filter the logs so you get only the ones with Errors using data.message = *ERROR* filter

Configure table chart with error logs

If you want to have quick access to the error logs, you can create a Logging Data Table visualization

  1. Give your chart a name, a description, and make sure to select the correct region of your logs
  2. Make sure to only select the logs that you wish to search
  3. Filter the logs so you get only the ones with Errors using data.message = *ERROR* filter

Configure Networking Flow logs grouped by action

In this chart, I wanted to get a quick view of the ACCEPTED and REJECTED requests that come to my public subnet.

  1. Give your chart a name, a description, and make sure to select the correct region of your logs
  2. Set the visualization parameters as follows: Visualization type: Pie, Group by: data.action
  3. Make sure to choose only the networking flow log created earlier

Configure Object Storage Buckets Upload & Delete counter chart

In this chart, I want to visualize the count of how many objects are being uploaded and deleted from my buckets.

  1. Give your chart a name, a description, and make sure to select the correct region of your logs
  2. Set the visualization parameters as follows: Visualization Type: Stacked Bar, X Axis: data.message, Group By: data.bucketName
  3. Make sure to select only the Bucket Write Logs

Configure an overview chart of deleted resources in tenancy

To get a quick overview of what resources are being deleted in your tenancy, we can quickly setup a chart to visualize all the deleted resources from the tenancy, or from a compartment.

  1. Give your chart a name, a description, and make sure to select the correct region of your logs
  2. Set the visualization parameters as follows: Visualization Type: Donut, Group By: type
  3. Make sure to select the Audit logs either from a compartment if you want the actions from that specific compartment, or from the root compartment if you want to see the actions for the whole tenancy
  4. Filter to see only DELETE actions using the filter: data.request.action = DELETE

Configure Layout

You can now play around with the Filter by time, Auto Refresh, Edit Layout, or Duplicate Dashboard features.

Filter by time: If you select a different period of time in the Filter by time menu, all of your dashboards will display the data for the selected period of time.

Auto-refresh: You can select to auto-refresh every 5 mins, or you can just use the refresh button to refresh now.

Edit Layout: This will let you move around your widgets so they display in the order you want them to.

Duplicate Dashboards: Under the Dashboards Actions menu, you’ll also find Duplicate, which lets you duplicate this dashboard if for example, you plan to do this for multiple compartments so you don’t do everything from scratch again.

Conclusions

OCI Dashboards is a great feature that you can take advantage of to create simple but efficient visualizations tailored-made for your use case.

Ionut Adrian Vladu

I enjoy building python scripts for…everything! I am a Cloud enthusiast and I like to keep up with technology. When I'm not behind a computer, I like taking photos -- Visit My 500px profile -- or sit back and enjoy Formula 1 race weekends. Currently, working as a Tech Cloud Specialist @ Oracle
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments