Machine learning models, such as the Ember benchmark model used for malware classification, can experience a decline in predictive performance over time, a phenomenon known as model degradation. This issue is particularly significant in information security due to the constantly evolving nature of malware. To assess the extent of model degradation, the Ember model was trained on datasets from different months in 2017, and its performance was evaluated using the area under the ROC curve (AUC) scores on subsequent test sets. Results showed that model performance decreased when tested on data that was further in time from its training data, underscoring the need for regular retraining to maintain accuracy. Although the size of the training sets was normalized to ensure consistency, the degradation pattern persisted, suggesting it is not solely dependent on dataset size. Understanding model degradation helps in deciding optimal retraining intervals for ML models, not only in malware detection but across various applications, emphasizing its importance for researchers and practitioners in maintaining model reliability.