Oracle ADR partition full due to large trace files

If you have an Oracle database 12c (12.2.0.1+) system, you may notice the size of the partition where ADR resides increases rapidly. And when you dig into it, you will find most of space is likely used by trace files. Basically there are two issues since Oracle 12.2.0.1:

Large size of trace files created during the RMAN restore process. Depending on your database size, individual trace file could be in a range of megabytes to gigbytes. Within those files, you will see the following messages:

*** MODULE NAME:(restore incr datafile) 2020-02-02T02:17:30.032315-05:00 kcbtse_print_kod: printing kcbtekod structure 0x7ffc8a3cfccc

2. Large number of trace files created by job processes. Those files are only 4k each, but there could be dozens to hundreds generated each day. This might not be an issue since they are really small. You can identify those files (file name containing j00x) with the following messages:

SCHED 02-09 10:05:19.461 1 00 11275 J000 0(kkjspub):Slave was not marked as idle

Basically Oracle has already identify them as defects and they are fixed in 19.1 and with the Patch 30352715 respectively.

RMAN BACKUP Command Always Generates Trace File Having “kcbtse_print_kod” Message (Doc ID 2311975.1)
Job Processes Fill The Filesystem With Trace Files After Installing Patch 29541742 (Doc ID 2606144.1)

And if the partition is big enough, you can rely on ADR SHORTP_POLICY to delete trace files older than 30 (default value) days. You can reduce the value of SHORTP_POLICY to shorten the retention period if free space is under pressure. Or if you don’t want to change your policy, another option is to use cron jobs on a Linux system to do tricks like the following — finding trace files containing the key message and delete them if they are older than 1 day:

0 * * * * find <ADR trace dir> -name “*.trc” -mtime +1 -exec grep -l -m 1 “Slave was not marked as idle” {} \; -delete >/dev/null 2>&1
0 * * * * find <ADR trace dir> -name “*.trc” -mtime +1 -exec grep -l -m 1 “kcbtse_print_kod” {} \; -delete >/dev/null 2>&1