Configuring OCI Unified Monitoring Agent for Custom Logs

Managing custom application logs in enterprise cloud environments requires a robust, scalable, and secure collection pipeline. Oracle Cloud Infrastructure (OCI) provides this capability through its Unified Monitoring Agent. Based on the open-source Fluentd data collector, this agent allows you to ingest logs from virtual machine instances directly into the OCI Logging service. In this guide, we will walk you through the architectural components, identity pre-requisites, configuration steps, and troubleshooting techniques required to set up custom log ingestion successfully.

1. Architecture Overview

The logging pipeline consists of three main components:

• The Source: Apache HTTP Server running on Oracle Linux 9, writing to /var/log/httpd/access_log and /var/log/httpd/error_log.
• The Transport: The Unified Monitoring Agent (based on Fluentd), which tails local files and batches data for upload.
• The Destination: OCI Logging service in the oci region.

2. Pre-Requisites (IAM & Identity)

For the agent to successfully authenticate and upload logs using Instance Principal, the following OCI Identity components must be in place:

• Dynamic Group: Create a group (e.g., dg-unified-monitoring-agent) with a matching rule for the instance:

  ANY {instance.compartment.id = 'ocid1.compartment.oc1..aaaaaaaaqnd'}

• IAM Policy: Grant the group permission to interact with the Logging API:

  allow dynamic-group dg-unified-monitoring-agent to use log-content in tenancy

3. Agent Configuration (OCI Console)

The agent configuration is centrally managed via the OCI Console under Observability & Management > Logging > Agent Configurations:

1. Host Groups: Select the Dynamic Group created in Step 2.
2. Log Inputs:
   • Access Logs: Path /var/log/httpd/access_log.
   • Error Logs: Path /var/log/httpd/error_log.
3. Log Destination: Select the Compartment, Log Group, and Log Object where the data should reside.

4. Troubleshooting Commands

If logs stop appearing in the Log Explorer, use these commands on the target instance to verify the local state and communication:

• Check Agent Sync: Verify that the fluentd.conf file has been successfully populated by the OCI control plane:

  cat /etc/unified-monitoring-agent/conf.d/fluentd_config/fluentd.conf

• Verify File Tailing: Check if the agent is actively “following” the logs and registering changes:

  sudo tail -f /var/log/unified-monitoring-agent/unified-monitoring-agent.log

• Force Manual Flush: The agent batches logs every 180 seconds by default. Restart the service to force an immediate configuration reload and data upload:

  sudo systemctl restart unified-monitoring-agent

• Test Connectivity: Ensure the instance can reach the local regional ingestion endpoint (for example, Dubai):

  curl -I https://ingestion.logging.me-dubai-1.oci.oraclecloud.com

Conclusion

By leveraging the OCI Unified Monitoring Agent, you can centralize your application and web server logs within OCI Logging, making it easier to analyze systems, detect anomalies, and meet compliance standards. If you encounter issues during your rollout, remember to verify your instance’s dynamic group membership, confirm your IAM policies, and inspect the local agent logs for detailed error tracking. For more advanced configurations, including multi-host setups and custom parsing, refer to the official Oracle Documentation.