Is AI making your teams better, or just busier?

AI adoption programs often fail to fundamentally change team operations, as initial increases in tool accessibility and usage do not necessarily translate to improved capabilities. The focus on usage metrics can be misleading, as they do not measure the effectiveness or impact of AI on team performance. Instead, two key performance indicators (KPIs) are recommended: one that assesses whether AI is enabling new outcomes previously unattainable, and another that evaluates how integrated AI is in daily operations. These KPIs are scored monthly by team leads based on evidence, not assertions, with an emphasis on continuous improvement rather than initial high scores. The goal is to determine whether AI is genuinely enhancing team capabilities or simply making existing processes faster. At companies like Ably, honest self-assessment reveals most teams start at a low capability score, with gradual progress reflecting genuine integration of AI into workflows rather than superficial adoption.