17849105. AUTOMATIC THRESHOLDING FOR CLASSIFICATION MODELS simplified abstract (Microsoft Technology Licensing, LLC)
Contents
AUTOMATIC THRESHOLDING FOR CLASSIFICATION MODELS
Organization Name
Microsoft Technology Licensing, LLC
Inventor(s)
Avi Caciularu of Tel Aviv (IL)
Noam Koenigstein of Tel Aviv (IL)
AUTOMATIC THRESHOLDING FOR CLASSIFICATION MODELS - A simplified explanation of the abstract
This abstract first appeared for US patent application 17849105 titled 'AUTOMATIC THRESHOLDING FOR CLASSIFICATION MODELS
Simplified Explanation
The patent application describes a method for automatically generating threshold values for machine learning models based on a desired precision or recall performance. Here are the key points:
- The method involves executing a trained machine learning model against a dataset using different threshold values.
- The accuracy of the model is determined based on the execution, and evaluation metrics are modeled using the accuracy results.
- The method calculates the probability that a modeled evaluation metric value has a relationship with a target metric value.
- If the probability meets a certain confidence level, the threshold value is added to a set of candidate threshold values.
- The final threshold value is selected from the set of candidate values based on the largest second modeled evaluation metric value.
Potential applications of this technology:
- Improving the performance of machine learning models by automatically generating optimal threshold values.
- Enhancing precision or recall performance in various domains such as fraud detection, medical diagnosis, or recommendation systems.
Problems solved by this technology:
- Manual selection of threshold values for machine learning models can be time-consuming and subjective.
- This method automates the process and ensures that the threshold values are optimized for the desired precision or recall performance.
Benefits of this technology:
- Saves time and effort by automating the threshold value generation process.
- Improves the performance of machine learning models by selecting optimal threshold values.
- Provides a more objective approach to threshold selection, reducing the risk of human bias.
Original Abstract Submitted
Embodiments are described for automatically generating threshold values based on a target metric value that specifies a desired precision or recall performance of an ML model. For instance, a trained ML model is executed against a data set using possible threshold values. Accuracy metric(s) of the ML model is determined based on the execution. Using the accuracy metric(s), evaluation metrics are modeled. A probability that a first modeled evaluation metric value has a relationship with a target metric value is determined. A determination is made that the probability has a relationship with a confidence level. Responsive to determining that the probability has the relationship with the confidence level, the threshold value is added to a set of candidate threshold values. The threshold value from among the set of candidate threshold values is selected by selecting the candidate threshold value associated with the largest second modeled evaluation metric value.
