While I was doing housekeeping working to prepare ODA upgrade, I noticed some large trace files for ASM and MGMTDB databases as showed below:
[grid@joeoda001 trace]$ du -sh *gen* *mmon*
1.6G +ASM1_gen0_20481.trc
250M +ASM1_gen0_20481.trm
1.5G +ASM1_mmon_20571.trc
243M +ASM1_mmon_20571.trm
[grid@joeoda001 grid]$ du -sh diag/rdbms/_mgmtdb/-MGMTDB/trace/*|grep -v K
22M diag/rdbms/_mgmtdb/-MGMTDB/trace/alert_-MGMTDB.log
161M diag/rdbms/_mgmtdb/-MGMTDB/trace/-MGMTDB_dbrm_23332.trc
16M diag/rdbms/_mgmtdb/-MGMTDB/trace/-MGMTDB_dbrm_23332.trm
1.7G diag/rdbms/_mgmtdb/-MGMTDB/trace/-MGMTDB_gen0_23224.trc
240M diag/rdbms/_mgmtdb/-MGMTDB/trace/-MGMTDB_gen0_23224.trm
91M diag/rdbms/_mgmtdb/-MGMTDB/trace/-MGMTDB_mmnl_23517.trc
2.2M diag/rdbms/_mgmtdb/-MGMTDB/trace/-MGMTDB_mmnl_23517.trm
1.7G diag/rdbms/_mgmtdb/-MGMTDB/trace/-MGMTDB_mmon_23507.trc
240M diag/rdbms/_mgmtdb/-MGMTDB/trace/-MGMTDB_mmon_23507.trm
Inspecting those files — file name containing mmon or gen0 shows that the common thing was they had been filled with the following messages:
*** 2018-11-13T15:55:56.292514+00:00 (CDB$ROOT(1))
AUTO SGA: kmgs_parameter_update_timeout gen0 1 mmon alive 1
*** 2018-11-13T15:55:59.297524+00:00 (CDB$ROOT(1))
AUTO SGA: kmgs_parameter_update_timeout gen0 1 mmon alive 1
And the timestamp was from when the process was started. Since the process has been running for that long without a restart, the trace file has been kept updating and growing.
Searching in MOS, it turned out to be a bug of Oracle. And the Patch 25415713 is there for it.
Reference:
- AUTO SGA: kmgs_parameter_update_timeout gen0 0 mmon alive 1 Excessive Trace Files were generated on one node after Migration to 12.2 and MMON trace file grows (Doc ID 2298766.1)
- MMON trace file grows and is flooded with: kmgs_parameter_update_timeout Gen0 messages (Doc ID 2319931.1)
- Unpublished Bug 25415713 – MMON TRACE FILE GROWS WHEN NO TRACES ARE ENABLED