February 2025 Summaries

Credal offers an enterprise-wide turnkey solution that connects disparate data sources in real-time and inherits existing permission schemas to provide best-in-class data integration and governance. The platform also features security-first enterprise chat and search applications that are LLM-agnostic, supporting the latest Anthropic and OpenAI models. Credal provides well-documented APIs for last-mile customizations, enabling developers to deploy tailor-made AI solutions directly into existing applications while consolidating tracking for security, cost, and compliance in a single place. The company has released new SDKs for Typescript and Python to simplify the integration of its APIs into applications, streamlining the development process.

The legal industry is ripe for transformation by generative AI, with many lawyers believing it will significantly impact the practice of law. Law firms are experimenting with various solutions to automate routine tasks such as legal research, contract analysis, and discovery. Generative AI tools like Claude and GPT are being tested to see if they can assist lawyers in tasks such as drafting, data extraction, and summarization. These tools have shown impressive ability to handle routine legal tasks, potentially shaving hours off time-consuming tasks and improving work product. However, it is essential for lawyers to use these tools judiciously, verifying accuracy and ensuring compliance with applicable rules. As the field of generative AI continues to evolve, law firms can benefit from exploring different models or adopting a system that integrates multiple AI models to remain nimble and seize upon the latest advances.

The author of the text discusses the performance of large language models (LLMs) in generating responses. They analyze their dataset of over two million LLM calls and find that the average query takes around 12.7 seconds, with a significant portion of queries taking more than 43 seconds. The authors aim to provide actionable advice on how to speed up LLM processes, focusing on optimizing prompt length and token output. They build a simple linear model to predict response time based on input parameters such as prompt tokens and completion tokens. The analysis reveals that reducing output tokens can significantly improve response speed, with each additional output token costing around 54 milliseconds. The authors also explore the impact of model choice, finding that GPT-3.5-turbo is a faster model than its predecessors, especially for single-token responses. They observe a spike in query volume during working hours and evenings, which can be mitigated by optimizing prompt length and using more efficient models. The study concludes with recommendations for businesses looking to leverage LLMs, including the importance of securing access to these powerful tools.

Generative artificial intelligence technology is providing a competitive advantage for early adopters across many industries, while businesses that haven't made the transition yet are at risk of falling behind. To integrate AI into their tech stacks without compromising data security, business leaders must carefully consider how to deploy this technology in a way that protects sensitive information. Consumer-facing versions of large language model tools like ChatGPT offer a convenient way to explore the capabilities of LLMs, but using these tools at the enterprise level carries significant privacy concerns due to potential breaches and exposure of sensitive data. To mitigate these risks, businesses should consider accessing LLM tools through an API or hosting them on premises, which offers greater protection and control over data encryption and access controls. Enterprises that want to safely use AI tools in the cloud can explore options like Azure OpenAI and Amazon Bedrock, which offer secure infrastructure designed with enterprises in mind, and comply with applicable laws and regulations by devising thorough acceptable use policies and providing training on the risks associated with using AI.

Credal's AI Copilots are designed to handle various tasks such as customer support and contract review, and the company has introduced two new features: User Inputs and Suggested Questions, making it easier for users to set up and use their Copilots. These features allow users to save prompts that would be frequently used in a given Copilot, eliminating the need to repeatedly paste the same prompts, and recommend which document should be added to the Copilot. With these new features, setting up and utilizing Credal's AI Copilots has become even easier, enhancing productivity for businesses dealing with complex data such as financial documents or incident reports.

Credal is a platform that enables enterprises to use Large Language Models (LLMs) like ChatGPT for Retrieval Augmented Generation (RAG) and AI agent applications. To run RAG, enterprises need data from internal SaaS tools such as GDrive, Slack, Confluence, and MongoDB Atlas provides an enterprise-ready solution with its unified query API, allowing control over powerful Foundation Model prompts. Credal connects to MongoDB Atlas out of the box, supports major auth schemes, and allows loading data from various sources like GDrive, Slack, or Confluence via UI or APIs. Once connected, data can be used in a chat with an LLM, and custom prompts and tools can be set up to create curated workflow assistants. Credal provides out-of-the-box functionality, customization options, and control over the data pipeline, including crawling sources, interpreting source formats, and permissions systems.

At Credal, they've observed that enterprises adopting AI often go through four distinct stages, from initial excitement and experimentation with open-source libraries to common challenges such as privacy, security, and compliance. Enterprises tend to adopt one of two strategies: either building their own internal wrapper around a third-party AI provider or buying an existing platform. The former approach is typically used for core business processes, while the latter is preferred for simpler use cases. Credal emphasizes the importance of having a multi-LLM strategy and working with a model-agnostic platform that allows for easy configuration and customization. However, enterprises face several challenges, including data security, use case discovery and value quantification, duplicate use cases and data fragmentation, legal and regulatory barriers, human resources challenges and training, and unpredictability in RAG and difficulty of debugging.

The use of generative artificial intelligence (AI) tools in the workplace presents both opportunities and challenges. These tools offer infinite possibilities for employees, creating uncertainty about how to control legal and business risks. To address this, businesses can implement AI audit logs, which provide visibility and traceability of AI use, including prompts, data shared, and security policies triggered. Analyzing these logs helps businesses analyze risks, review and enhance their security measures, and provide valuable insights on effective AI usage. By doing so, organizations can make informed decisions, optimize AI usage, and cultivate a culture of shared learning and progress.

The author and co-founder of Credal.ai, Jack Fischer, reflect on their experience in YCombinator's W23 batch. They discuss the challenges of building a successful startup, including solving real problems for customers, changing user behavior, and overcoming the bar to entry. The authors emphasize the importance of being aware of one's own limitations and biases as founders, particularly when it comes to sales and fundraising. They also stress the need to move quickly and be willing to pivot or change direction if necessary, in order to increase the chances of success. Additionally, they highlight the value of humility and engagement with investors, recognizing that not all investors are created equal and that a good investor knows how to build a successful company.

Enterprises are starting to use Large Language Models (LLMs) but we're still in the early days. LLMs can be used in complex scenarios like anti-money laundering (AML) contexts where sensitive data needs to be handled with care, dealing with technical issues such as data integration, prompt injections, permissions, and auditability. In an AML context, LLMs can screen customers to ensure they're not high-risk businesses, but this process requires careful consideration of how to represent data properly, including classifying fields as "Text" or "Metadata", and managing permissions and access controls. The use of LLMs in regulated industries also raises security concerns such as prompt injections, which can be exploited by malicious actors. However, with the right guardrails and policies in place, enterprises can harness the power of LLMs to drive decision-making and safeguard operations, gaining an advantage over their competitors.

Credal's Bulk Analysis is a powerful tool that enables users to get detailed insights from large collections of documents with just a few clicks. This feature empowers non-technical users to extract valuable information from generative AI models, unlocking new workflows and product visions for organizations. By creating a Document Collection and configuring a Copilot, users can scale up their analysis to thousands of documents at once, receiving detailed answers to every question. The results can be interpreted in various ways, including downloading as a CSV file or chatting with the results in a conversational manner. This approach differs from traditional RAG (Read-All-Gather) methods, which may not be suitable for all types of questions, particularly those requiring analysis across large datasets.

To determine the best AI tool for a company, one must navigate the fragmented market and consider key factors such as precision and thoroughness in data integration capabilities, no-code customization options, access controls, industry compliance standards, flexible deployment options, and robust support. A well-suited AI tool should be able to navigate extensive data repositories, offer detailed audit logs, and ensure transparency in all interactions while providing a smooth onboarding process and effortless customization.

Credal's Salesforce integration helps streamline customer service and sales workflows by centralizing data into a single platform and providing intelligent insights. This integration enables various business functions, including call coaching, customer check-in, helpdesk copilot, CSAT survey analysis, product feedback synthesis, and customer onboarding. To configure the integration, users must sign in with Salesforce, select fields to pull in, configure metadata, select collection, review document collections, and create a new copilot tailored to their needs. The copilot can be configured to use search documents for relevant context and enable smart filtering, allowing users to build custom workflows and improve business efficiency.

The text discusses the concept of Retrieval Augmented Generation (RAG) applications, which are a type of Generative Artificial Intelligence (Gen AI) system that uses contextual information to answer user questions. RAG systems aim to provide more accurate and relevant responses by combining human-provided context with large language models' capabilities. The article highlights the challenges of building such systems, including data retrieval, indexing, and search techniques, as well as the importance of choosing the right vector database and embedding model. It also touches on the need for user query rewriting, hybrid search, reranking, and reducing duplicates to improve system performance. The author emphasizes that RAG development is similar to software development and requires careful consideration of various factors to create a production-worthy system.

Credal has been working with a large software company to deploy Large Language Models (LLMs) in their sales call data. The goal was to get more visibility into the signal that sales teams were getting from these calls, especially around new product offerings. Credal's solution involved connecting data from various systems, including Chorus for recording sales conversations and Snowflake for storing contextual data from Salesforce. The company used Credal's Snowflake Connector to make this connection easy, allowing any approved LLM to access the data on behalf of an authorized user. To manage permissions, they used row-level access policies in Snowflake, which provided a single column with a list of OKTA group IDs defining who should have access to the data. Credal also implemented intelligent retrieval strategies, such as pinning documents and using reranking and hybrid search, to improve the effectiveness of the LLMs. By connecting structured data to LLMs, companies can unlock more complex use cases and move away from toy chatbots into high-value business applications.

To paraphrase Jane Austen, the use of AI in enterprises is becoming increasingly prevalent as companies seek to balance the need for innovation with regulatory obligations. Credal has already enabled fast-moving enterprises to safely execute tens of thousands of prompts, powering AI workflows from internal knowledge management to data analysis and software engineering. The #1 use case for AI at the enterprise is helping employees understand what's happening, such as receiving straightforward answers in Slack or having a "fill-in-the-blank" exercise with LLM-like tools. Employees are using AI to turn lengthy, unstructured text into concise, structured data, and to analyze employee sentiment. The rapid advancements in AI are getting us closer to providing real-time insights and actionable recommendations for businesses to make data-driven decisions. However, ensuring data security and privacy is a top priority, particularly when it comes to sensitive code or intellectual property.

The author of the text discusses their experience with Generative AI and the challenges they faced while building an LLM-based system. They share their learnings on how to improve the performance of LLMs, particularly in handling complex data sources such as long documents and tables. The key takeaways include the importance of clean data representation, model attention being limited, and the need for a well-designed prompting strategy. The author also highlights the limitations of current LLMs in handling dates and nuances in text representations. They propose using GPT-3.5 or Claude with their huge context window to improve performance on long documents. Additionally, they discuss the importance of human-computer symbiosis in building effective AI systems and share their experience with creating a custom "AI expert" that can be used in various applications.

This writer joined Credal as the first business hire focused on Growth Technology and Marketing (GTM), aiming to create a 'copilot' using Credal's own platform. The copilot is essentially an expert assistant that provides contextual information on questions, citing sources. To build the copilot, the writer used Credal's documentation and set up an "Onboarding Buddy" with various data sources, including internal documents and Slack channels. Initially, the copilot struggled to provide accurate answers due to overloading its context window with too much data. After reconfiguring the data sources, the copilot successfully answered questions, meeting both the writer's primary goal of familiarizing themselves with Credal and secondary goals related to understanding retrieval augmented generation (RAG) and LLMs. The writer found the experience fun and educational, highlighting that Credal provides everything needed to supercharge businesses using generative AI securely.

This is a summary of the text about implementing Retrieval Augmented Generation (RAG) for HR teams to answer employee questions about company policies using AI tools. The process involves creating a knowledge base, connecting it to an AI tool, and configuring the tool to refer to the knowledge base when answering user questions. The key steps include getting data into a VectorDB, hooking up the VectorDB to the chat interface wrapper, and adding controls over how the AI operates and responds. The HR team or subject-matter-expert needs to steer and control the responses, ensuring that good, relevant previous responses are used to guide the AI, and that bad previous responses are not fed into the AI learning algorithms. The tool also needs to protect sensitive company data by automatically redacting PII before it leaves the organization's boundaries, ensuring Zero Day Retention policies are in place with third-party models, and providing API exportable Audit Logs.

The healthcare sector faces significant challenges due to outdated record-keeping methods, with many providers relying on technologies developed over a decade ago or no technology at all. This gap presents an opportunity for the integration of artificial intelligence systems, such as "Retrieval Augmented Generation" and generative AI systems more broadly, in a safe and responsible manner. Two case studies demonstrate how AI technology can enhance operational efficiency in healthcare by streamlining processes, reducing costs, and enabling providers to devote more resources to direct patient care. The first use case showcases the potential for AI systems to revolutionize patient admissions across various healthcare settings, while the second use case highlights the importance of integrating modern tech solutions into billing operations. Key technologies such as AWS Textract, Credal, and Retool are utilized to build powerful and secure applications for healthcare in a short period of time, enabling rapid development and deployment of secure, compliant, and efficient healthcare applications that meet the dynamic needs of the industry.

Credal's experience in driving AI adoption across enterprises suggests that achieving over 90% organization-wide adoption requires intentional strategy and effective effort. This involves selecting the right people and teams to lead the initiative, providing secure platforms with access and guardrails for users, and enabling innovative employees to experiment and build tools. A governance program that clearly outlines what is allowed and what is not can also accelerate adoption by reducing uncertainty among users. Furthermore, meeting users where they are by integrating AI tooling into existing platforms and letting them discover the technology organically can drive faster adoption. Picking AI-first partners/vendors and teaching employees through regular "show and tells," hackathons, and encouraging experimentation can also achieve 3x the adoption rate compared to traditional methods. By following these strategies, organizations can realize the phenomenal productivity gains from AI adoption.

Creating an "HR department for AI agents" is crucial for organizations to successfully deploy and integrate AI across their operations. This concept mirrors the traditional HR department's responsibilities, but with a focus on embedding AI throughout the organization. Establishing clear ownership and applying core HR principles ensures AI is effectively integrated for long-term success. Similar to regular employee onboarding, employee enablement is designed to bring everyone up to a baseline level of AI literacy. Performance monitoring is also essential to ensure AI platforms are working as expected. By mapping AI processes onto established HR concepts, organizations can create a clear and accountable framework for acquiring and retaining critical resources, such as AI capabilities. This approach helps sustain funding, executive attention, and a roadmap for unlocking actual productivity gains from generative AI.

We're excited to launch Quick Searches, a new feature that integrates Credal queries with existing app launchers to create shortcuts. A customer requested this integration, and we liked the idea, thinking it would help users seamlessly integrate Credal into their workflows without visiting the web UI each time. The new feature is live today for all Credal users, who are invited to provide feedback; those interested in using Credal can start with a sales inquiry. Quick Searches allows users to quickly access Credal queries through apps like Alfred or Raycast, and it's part of our ongoing efforts to customize features based on customer requests.