February 2024 Summaries

The CLARANS algorithm is a variation of k-medoids that can be used for clustering large graph data. It offers several advantages, including the ability to work with any distance metric and the fact that medoids returned by the algorithm can serve as typical examples for the clusters they define. However, it also has some drawbacks, such as requiring the calculation of distances between all pairs of nodes in the graph, which can be slow for large graphs. The CLARANS algorithm was developed to extend k-medoids to larger datasets than were practical with earlier algorithms, and it uses a randomized search approach to find good approximations of the optimal solution. The algorithm has been implemented in Python and Neo4j Graph Data Science, and it has been tested on two graph datasets, one small and one large, showing promising results compared to other k-medoid algorithms like FasterPAM. CLARANS is a good choice for medium and large graphs where pre-computation of distances between all pairs of nodes is not feasible.

The author of the text has implemented a JSON-based agent using the Mixtral 8x7b LLM, which interacts with a graph database Neo4j through a semantic layer. The agent uses predefined tools to access information and perform actions, such as recommending movies or answering questions about actors. The author used the LangChain framework to implement the agent, which provides a JSON-based prompt structure for the LLM to use when calling tools. The output of the agent is structured in a specific way, including a "Thought" section where the LLM explains its actions and an "Observation" section where it provides tool outputs. The author also implemented a dummy smalltalk tool to handle cases where no action is required. The code for the agent is available as a Langchain template and Jupyter notebook.

PyNeoInstance is a user-friendly Python library for Neo4j that allows easy loading and reading of data in a graph by handling tasks such as driver creation, multiprocessing, and simple configuration with YAML. The library supports the use of a free Neo4j AuraDB instance and requires familiarity with Cypher Query Language, Python, and Notebooks. It provides a demo GitHub repo for users to follow along and use as a starting point for their own projects. PyNeoInstance allows for batching and parallel loading of data into the graph, which greatly decreases ingestion time by taking advantage of machine's multiprocessing capabilities. The library also supports analysis queries, enabling users to take a look at their graph and retrieve results as Pandas DataFrames.

The author presents a simple yet effective way to improve similarity search over text embeddings in GenAI solutions without using techniques like model fine-tuning or prompt engineering. The approach involves adding symbols, such as ###, to the beginning of questions to boost relevant results and increase distance between high and low scores. This technique is tested on three cloud providers: AWS Bedrock, Azure OpenAI, and Google VertexAI, with varying degrees of success. The author highlights the importance of considering trivial things like special characters/symbols in text embeddings, which can significantly impact outcomes. The article provides a starting point for evaluating embedding search and improving GenAI solutions.

The author of the article aims to transform the traditional learning process by making it dynamic and engaging through conversational AI. The goal is to revolutionize how people interact with YouTube playlists, enabling active engagement in dynamic conversations inspired by playlist content. To achieve this, the author uses two cutting-edge technologies: LangChain, an open-source framework simplifying the orchestration of Large Language Models (LLMs), and Neo4j, a robust graph database designed for optimal node and relationship traversal. The author processes YouTube playlists using LangChain and Neo4j to extract valuable information from video captions, construct a conversational chain that leads users through a dialogue rooted in playlist content, and perform queries to provide personalized educational dialogues. The article concludes by showcasing the graphical presentation of the conversation history chain using Neo4j Aura database and highlighting the potential of this technology for transforming YouTube learning into dynamic conversations.

The Neo4j community has been actively engaged with various updates and new features, including improvements to the Datashare platform for investigative journalists, LLM-powered Conversation Interfaces, Object Mapping with .NET, and the Cypher Workbench. The upcoming Training Series in March will cover a wide range of topics from Introduction to RAG App Deployment to Geospatial Analytics. Upcoming events include NODES 2023: Watch Recordings, Live Stream: Neo4j Live: Social Science Meets Knowledge Graphs on February 22, and conferences in Oakland, CA and Orlando, FL. Featured speaker Abhishek Das will be giving a session at NODES on exploring the Neo4j Workspace, while new features have been introduced to the Datashare platform, enabling journalists to connect the dots with graphs. Additionally, there is a guide available on creating LLM-powered applications that allow users to query Neo4j graph databases in natural language, and an enhancement has been made to the .NET driver for Neo4j that simplifies object mapping. The Cypher Workbench is also now available under Neo4j Labs, offering tools for conceptualising, modelling, and working with graphs and the Cypher query language.

The Neo4j Driver for .NET has introduced a new feature that allows C# objects to be quickly constructed from query results with minimal boilerplate. This feature uses convention-based mapping, where property names are used to decide which data from the record to use, or custom configuration can be supplied to specify exactly how to map from a record to an object type. The default mapper can also give "hints" to expand the situations covered. Additionally, there are two ways to create custom mappings: by implementing the IRecordMapper interface for class-based mapping and by implementing the IMappingProvider interface for property-by-property configuration. The feature is still in preview and would appreciate feedback from real-world users.

The issue with the slow Cypher statement lies in the large number of relationships being expanded, particularly with the Neo4j Fundamentals course having a high number of enrollments. The query plan identified that starting at the (:Course) node was the best place to begin the traversal, which is sensible given the schema. However, when 40k :FOR_COURSE and 40k :HAS_ENROLMENT relationships are expanded, the query time increases significantly due to the large amount of data being checked. To improve performance, using indexes can help reduce the number of database hits. By adding a USING INDEX clause to the query, the statement was able to register only 592 total db hits in 404 ms, resulting in a 1385% performance improvement.

The upcoming week features a variety of events and tutorials for developers and Neo4j users, including the recording of Nodes 2023, live streams on CypherGUI administration, and meetups in Brisbane. Tomaz Bratanic is featured as a speaker, discussing Knowledge Graphs and their potential to enhance interaction between Language Models and Graph Databases. The tutorial "VECTOR DATABASE: Semantic Search, RAG and Index Vector Databases" by Erica Brown compares Vector Databases with Graph Databases for RAG, while Anansi's Visual Metadata Management Tool aims to simplify data webs and ensure effective decision-making in data-driven enterprises. A project integrating Streamlit and Neo4j has also been launched, allowing users to track income and expenses, and a tweet from Haystack promotes the integration of Neo4j with its document store using vector search indexes.

This article discusses the creation of a song finder tool using Neo4j's Gen AI capabilities. The tool is designed to find songs based on a synopsis of what they are about, and it uses vector embeddings to search for similar texts in a database of lyrics. The author demonstrates how to create this tool in four lines of code, using Cypher queries to interact with the Neo4j graph database. The article highlights the benefits of using Neo4j's Gen AI capabilities, including its ability to handle large datasets and provide accurate results. It also notes that while ChatGPT is excellent at finding songs by lyrics, a custom tool can be more reliable and efficient, especially for searching specific genres or albums.

Neo4j is featuring a method to improve embeddings using Neo4j APOC ML procedures, which enhances similarity search in GenAI solutions without relying on conventional techniques. The company has also introduced new algorithms for Directed Acyclic Graphs (DAGs), including Longest Path and Topological Sort. Additionally, there are tutorials available on how to get started with Neo4j on Docker and LangChain templates. Various meetups and events are being organized, including NODES 2023: Watch Recordings, Live Stream: Going Meta: Episode 25, JFokus 2024 in Stockholm, Sweden, Meetup: Meet us in Berlin, DE on February 8 & in Kansas City, MO on February 9. There is also a GraphSummit Series and a GraphAcademy Live session on importing CSV Data. The company's featured Node speaker is Karina Isla-Rios, who discusses the use of graphs in representing clinically meaningful glycomics data to help fight cancer. A tweet from Tomaz Bratanic highlights the need for publicly available text2cypher datasets for finetuning LLMs.

Langchain v0.1 has been released with significant changes that introduce a new structure for the core, making it leaner and more concise. This update includes changes to how LLM integrations and community tools are organized, as well as deprecations of certain methods and classes. The API has also been standardized through the use of invoke instead of the deprecated __call__ method, and parameters should now be passed as a dictionary through the invoke method. Additionally, the process of creating and working with agents has been updated, requiring a 3-step process that includes creating an agent prompt, creating an agent, and creating an agent executor. These changes offer greater flexibility and improved dependency management, making Langchain v0.1 an important update for users looking to integrate LLMs alongside Neo4j.