Metrics in Machine Learning

In the context of machine learning, a metric is any number that we care about. An objective is a specific type of metric that a machine learning system attempts to optimize.

Technical Metrics

Accuracy is the most common (and easy to understand) metric but tracking only accuracy will paint an incomplete picture of how your model is performing. There are several other well-established metrics that provide deeper insight into model performance.

Metrics are often specific to the type of machine learning problem or model. Important and widely adopted metrics include: Accuracy, Loss, Confusion Matrix, AUC (Area Under ROC curve), Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and R Square.

Correlating to Business Metrics

Machine learning metrics are often directly correlated to business metric. One example would be assigning a dollar value to false positives in a classification model. Here’s a great example of how AirBnB measures the performance of their fraud prediction algorithm in dollars.