Semantic search helps chatbots answer more questions by attaching a semantic search backend to a customer assistant chatbot. This allows the bot to handle a broader range of inquiries, providing direct answers and intelligently finding content even with misspellings or paraphrases. The fallback handler is used to extend the bot's capabilities, running user queries against customer review corpora and displaying up to two matches if the results score strongly enough. The performance of complex systems must be analyzed probabilistically, and NLP-powered chatbots are no exception. Evaluation of the chatbot was done using a set of 63 questions, with improvements seen in true positive rate, F1 score, and precision, with significant increases achieved easily by accessing existing reviews with semantic search. However, conclusions about precision should be drawn with caution due to the presence of confidence intervals. The addition of semantic search brings its own challenges, including dealing with misaligned facts from customer reviews and annotator disagreement.