Bayesian Neural Networks (BNNs) differ from traditional Artificial Neural Networks by offering the ability to express uncertainty in predictions, which is critical for handling out-of-distribution data and enhancing AI security. Implemented using the JAX framework, BNNs transform the inference problem into an optimization problem by approximating the posterior distribution of parameters through Variational Inference, minimizing the KL divergence between the variational and true posterior distributions. The article provides a step-by-step guide on implementing a BNN for digit recognition, highlighting the importance of adjusting hyperparameters like beta and initial variance to optimize training. Through practical examples, it demonstrates how BNNs can identify instances where the model is uncertain, thereby offering a robust solution to avoid misleading predictions. The exploration underscores the significance of uncertainty estimation in AI safety and provides insights into effectively training BNNs to manage out-of-distribution examples.