DETECTING MODEL DEVIATION WITH INTER-GROUP METRICS

Abstract: a computer model is monitored during operation to evaluate performance of the model with respect to different groups evaluated by the model. performance for each group is evaluated to determine an inter-group performance metric describing how model predictions across groups differs. a threshold for excess inter-group performance differences can be calibrated using withheld training data or out-of-time data to provide a statistical guarantee for detecting meaningful variation in inter-group performance metric differences. when the inter-group performance exceeds the threshold, the computer model may be considered to deviate from expected behavior and the monitoring can act to correct its operation, for example, by modifying actions that may otherwise occur due to model predictions or by initiating model retraining.

Inventor(s): Jesse Cole Cresswell, George Frazer Stein, Farnush Farhadi Hassan Kiadeh, Zhaoyan Liu, Ji Xin

CPC Classification: G06F11/3447 ({Performance evaluation by modeling})

Search for rejections for patent application number 20250165375