17457924. LOG ANOMALY DETECTION simplified abstract (INTERNATIONAL BUSINESS MACHINES CORPORATION)
Contents
LOG ANOMALY DETECTION
Organization Name
INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor(s)
Sahil Bansal of Kurukshetra (IN)
Xiaotong Liu of San Jose CA (US)
LOG ANOMALY DETECTION - A simplified explanation of the abstract
This abstract first appeared for US patent application 17457924 titled 'LOG ANOMALY DETECTION
Simplified Explanation
The patent application describes a computer system that can classify log lines as erroneous or non-erroneous, templatize them, and cluster them into template clusters. It then identifies anomalous log lines based on factors like log maturity, encountered template clusters, and the ratio of erroneous to non-erroneous log lines. The system validates the identified anomalous log lines with a site reliability engineer and trains a log anomaly model using the validated log lines. Finally, it can identify subsequent log lines as anomalous using the trained model.
- Computer system classifies log lines as erroneous or non-erroneous
- Log lines are templatized and clustered into template clusters
- Anomalous log lines are identified based on various factors
- Identified anomalous log lines are validated by a site reliability engineer
- Log anomaly model is trained using the validated log lines
- Trained model can identify subsequent log lines as anomalous
Potential Applications
- Log analysis and monitoring in software development and IT operations
- Identifying and troubleshooting errors in system logs
- Improving system reliability and performance through proactive anomaly detection
Problems Solved
- Manual analysis and classification of log lines can be time-consuming and error-prone
- Identifying anomalous log lines can be challenging without a systematic approach
- Lack of automated tools for log analysis and anomaly detection
Benefits
- Efficient and accurate classification of log lines as erroneous or non-erroneous
- Automated templatization and clustering of log lines for easier analysis
- Proactive identification of anomalous log lines for timely troubleshooting
- Improved system reliability and performance through proactive anomaly detection
Original Abstract Submitted
One or more computer processors classify each log line in a plurality of unlabeled log lines as an erroneous log line or a non-erroneous log line; templatize each classified erroneous log line and non-erroneous log line in the plurality of unlabeled log lines; cluster erroneous log templates into erroneous log template clusters and non-erroneous log templates into non-erroneous log template clusters; identify one or more log lines as anomalous utilizing a plurality of factors including a log maturity, a number of encountered log template clusters, and a ratio of classified erroneous log lines to classified non-erroneous log lines; responsive to one or more identified anomalous log lines, validate the identified anomalous log lines utilizing a site reliability engineer and human-in-the-loop validation; train a log anomaly model utilizing one or more validated log lines; and identify a subsequent log line as anomalous utilizing the trained log anomaly model.
