January 2023 Summaries

The Neo4j community has been actively sharing knowledge and expertise through various presentations, articles, and webinars on different topics such as exploring graph databases visually with Jupyter notebooks, geospatial search functionality, closeness centrality algorithms, graph pattern matching, conversational artificial intelligence, database integration, app development, project management, and knowledge graphs. The community is also highlighting upcoming events like Neo4j Live, Webinar: Full Stack GraphQL Applications, and the Full Stack GraphQL Book Club. Additionally, a recent prototype has been developed using Neo4j technology for anomaly detection in pandemic intelligence data, showcasing its potential in graph-based approaches.

ERP systems handle multiple data types for various sectors, resulting in massive data systems with large-scale data user networks. To manage this complex data, a holistic approach is needed to visualize or modify the data. Graph technology can help businesses derive more value from their random and often discursive data by providing a single view of all entities with relationships across them. Neo4j, a leading graph database, offers various benefits such as improved visualization, analysis, forecasting, and planning capabilities for ERP systems. It provides operational excellence within the business and improves customer experience by utilizing high-performance graph technology on top of various ERP data domains. Graph technology can also help businesses optimize their supply chain operations, uncover vulnerabilities, and boost overall supply chain resilience.

The author of an article shares their experience with optimizing Cypher queries in Neo4j using parameters. They initially used a Python loop to construct queries with merge operations for every synonym they wanted to add, but this approach resulted in the Neo4j desktop app crashing after 1500 requests due to excessive query construction and optimization. The author then replaced this approach with a new method that uses Cypher Unwind and parameters, which allows the database to cache previously executed queries, reducing performance improvement benefits. An experiment was conducted to compare the two methods, showing that while both approaches perform similarly in most cases, the parameterized method provides clearer performance improvements when caching is enabled. The author concludes that using parameters is a better approach for optimizing Cypher queries and has successfully implemented it on a large dataset of over 5000 requests.

The author of the blog post is a Back End Developer at CytoSMART who built a full-text search system for a graph database containing millions of chemical compounds. The goal was to link different synonyms of the same compound to each other, allowing users to find similar chemicals. To solve this challenge, the developer used Neo4j as the graph database, Lucene for full-text searching, and Python scripts to wrangle and load data into the database. After loading 197M synonyms and 57M compounds, the developer created a full-text index on the synonym nodes and implemented several query options, including basic fuzzy matching, one synonym per compound, and optimized queries with limiting results to reduce processing time. The system also handles cases where a synonym has multiple compounds associated with it, returning all related compounds without duplicates.

The author of the text is a back-end developer at CytoSMART who aims to update their graph database with chemical synonyms from PubChem. The author has already downloaded 197M nodes of all chemical synonyms, but recognizes that this number is still too low and wants to connect compounds from NCI (National Cancer Institute) to their database using NSC numbers as synonyms. To achieve this, the developer uses two endpoints provided by PubChem: PUG (Power User Gateway) and RDF (Resource Description Framework). The author creates a backend function to utilize these endpoints efficiently, retrieving data from both APIs and combining the results to create a list of compounds with their corresponding synonyms. The developer then updates their database using this information, removing incorrect synonyms and adding correct ones. Finally, they apply this method to update NCI60 data, which includes 56,685 unique NSC numbers, and achieve an improvement in matching at least one synonym with a compound for 55578 out of 56685 NSC numbers.

The author of the blog post implements a collaborative filtering algorithm in a graph database to predict chemical-cell interaction, specifically GI50 measurements. The dataset used is the NCI60 dataset, which contains information on the growth inhibition of 60 cell lines by various chemicals. The author simplifies the graph to focus only on two nodes: compounds and cell lines, with one relationship: GI50. The algorithm works by finding similar chemicals based on their shared cell lines and then voting between these similar ones on missing links. The prediction consists of three steps: finding similar chemicals, voting between them on missing links, and removing the chemical being predicted from the results. The author tests the algorithm using the NCI60 dataset and compares the predictions with actual GI50 measurements. While not perfect, the results show promise, especially when considering the smaller range of predicted values compared to all HCT-15 GI50 measurements.

Many organizations are moving away from Relational Database Management Systems (RDBMS) due to their inflexible structure, which can make it difficult to capture complex relationships between data points. Graph databases, on the other hand, offer a more intuitive and natural way of working with connected data by representing it as nodes and relationships. To fully leverage graph analytics, organizations need a robust data orchestration strategy that ensures their data is clean, organized, and ready for analysis. This process can be challenging at large scales, but using modern data platforms like graph database technology can make it more efficient. Graph analytics provide visual representations of data and relationships, allowing data scientists to quickly understand the structure and content of their data and identify patterns and trends. The partnership between Modak Nabu and Neo4j provides a powerful solution for data preparation, visualization, and orchestration, enabling organizations to prepare their data quickly and effectively for analysis using graph technology.

The Neo4j Aura Team created an internal portal called The SRE Portal, which allows them to monitor thousands of scalable, reliable Neo4j graph databases running in hundreds of different locations. This portal provides a user-friendly interface for engineers to analyze issues and take actions without having to manually connect to Kubernetes clusters or describe pod statuses. The team has built a Go CLI App using Cobra and Viper that automates frequently needed processes behind shiny, colorful buttons, allowing engineers to easily create backups, revert databases, and trigger Java Heap Dumps for analysis. This framework enables the business to respond to issues more effectively, gather information, and analyze problems without having to repeat complex processes. The SRE Portal is a tool that helps the Neo4j Aura Team mitigate misbehaviors in their graph databases by providing a centralized platform for monitoring and incident management.

OpenAI semantic search is being used with a graph database of PDF files to save the generated embedding vectors in Neo4j nodes and calculate the cosine similarity at the database level using graph data science. A biomedical engineer is exploring a chemical graph database with Neo4j Dash, which enables fuzzy full-text search with an input field and result field. Graph algorithms and visualization are being discussed for clinical care support of pneumonia. Pythonistas can learn about Nodes 2022, while automating deployment of Neo4j Java extensions on Google Cloud is also possible. Knowledge graphs are being connected to the high-performance graph database system Neo4j as a user-friendly way to store and query them, allowing for rapid knowledge graph construction. Upcoming events include discovering AuraDB, porting Global-Chem Graph Infrastructure into Neo4j, and deploying a full stack GraphQL application. Several Neo4j-related extensions and tools are being discussed, including Quarkus Neo4j, Gephi 0.10, and built-in JDBC connector alpha.

Yokogawa, a Japanese company with over 100 years of legacy assets, is leveraging Neo4j's graph database to drive digital transformation and unify disparate information. The company's products and customers work under extreme temperature and pressure conditions, requiring safety as their primary concern. With around 75% of engineers being operational technology experts, the remaining 25% are IT experts dealing with data islands that need integration into a knowledge graph. Yokogawa's key challenges include digitally transforming automation systems and unifying disparate data sources distributed across multiple databases, tech stacks, and infrastructures. To address these challenges, they built their different information models into a knowledge graph using Neo4j, enabling them to transfer knowledge in the form of graphs, templatize information, and use it across different refineries and customers. The company is now exploring Graph Databases and Tooling to support intuitive and generic tech stacks throughout the model life cycle, with plans to use Arrows and Neo4j's Data Importer for depiction and translation of information models into the graph database.

Graph database technology offers a new way of seeing the world by reimagining connections and approaching challenges from a different dimension, potentially producing an orders-of-magnitude change in possible solutions. It's not just another passing trend, but a rising tide that development teams and businesses can't afford to pass up. Graph databases are scalable, flexible, and agile, allowing them to keep up with changing business requirements and evolving data models. They use native graph storage and processing, which is more efficient than relational or object-oriented databases. The connections between data points take priority in graph databases, making relationships and connections as important as individual data points. This approach simplifies data models while increasing expressiveness. Graph technology is unique in its properties, including native graph storage and processing, which are essential for handling connected data. By understanding graph databases, businesses can retain competitive advantage and solve real-world problems.

In a recent conversation, two employees at Fractal 5, a social media startup, shared their experiences with graph technology and Neo4j. Sara Masdottir, a developer, used Neo4j to gather data and gain insights into the company's product, leveraging its GraphQL API to analyze consumer behavior. She chose Neo4j after realizing that her previous NoSQL database solution was too complex and difficult to maintain. Bjartur Hjaltason, a machine learning engineer, found Neo4j intuitive and visually appealing, using it to explore data patterns and optimize his algorithms. Both employees advise starting simple with Neo4j and taking advantage of its documentation and support resources. They also see the future of graph technology as bright, with Neo4j's advancements in speed and adoption paving the way for widespread adoption across various industries.

The author is trying to model a graph database using ChatGPT to track supplements and their influence on processes in the body for their son with Down Syndrome. The goal is to create a database that can store information about supplements, their ingredients, effects, and interactions, as well as reference medical publications and blood test results. The author uses ChatGPT to generate a basic data model by asking specific questions, which eventually leads to a more detailed understanding of the database structure and relationships between nodes. Although the generated model is not perfect, it provides a good starting point for building an application, and the author sees this as an opportunity to make parts of building applications easier and fun. The experience highlights the potential benefits of using tools like ChatGPT in data modeling and development, especially for those who may not have extensive experience with graph databases or Neo4j.

The text discusses various topics related to graph databases, including the use of NLP for website analysis, conversational artificial intelligence with Neo4j and the Unreal Engine, graph machine learning, facial recognition, end-of-life management, and tutorials on getting started with graph databases. It also mentions upcoming events and projects of interest, such as a new Neo4j extension for VS Code, a project using OSMnx with OpenStreetMap data in Neo4j, and video tutorials by Dairon Pérez Frías on Neo4j.

The Neo4j VS Code Extension allows developers to write and execute Cypher queries directly within their IDE, eliminating the need to constantly switch between code editors and other tools. This extension targets common pain points such as copying and pasting code, debugging issues, and highlighting incomplete Cypher statements. Users can install the extension through the Visual Studio Marketplace and access a list of database connections by clicking on the Neo4j logo in the Activity Bar. The extension provides features like running Cypher queries in read or write transactions, using multi-cursor functionality, and setting parameters for queries. Developers can provide feedback and suggestions through Twitter, Mastodon, LinkedIn, or open an Issue on the GitHub repository.

Standard Chartered Bank`, a large financial institution with 85,000 employees across 1,200 branches in 49 different markets, uses knowledge graphs to combat internal and external security threats. The bank's operations, data analytics, and insights team created a knowledge graph to provide contextual intelligence and address the challenge of managing diverse customer needs globally while securely protecting their privacy. By leveraging Neo4j, they developed a self-service capability called SCypher, which enables end-users to consume insights faster than with traditional querying technologies. The tool has been successful in scaling to multiple departments and use cases, including threat intelligence, insider threat prevention, and improving business operations. The knowledge graph provides tangible benefits such as identifying more security threats before they have negative impacts, resolving those threats sooner, and preventing potential breaches, while reducing staff time spent compiling security reports.

Neo4j is being used in various applications to create graph data projections for analysis and machine learning. The company's Graph Data Science library allows developers to create an in-memory graph projection of data from Covidgraph, a biomedical graph by HealthECCO, and apply multiple algorithms for analysis and machine learning. Additionally, the company provides tutorials and resources on how to debug Spring Data Neo4j app connections and create knowledge graphs of recent Nobel Prizes in Physics, Chemistry, Medicine, and Literature. The company also hosts sessions and podcasts that cover topics such as graph analytics and knowledge graphs, including a presentation by Corey Lanum on presenting graph analytics in an easily understandable way. Furthermore, the company offers free resources and tutorials to help developers build modern applications with GraphQL.

The author of this text is a back-end developer at CytoSMART, exploring how to combine three biochemical datasets in a graph database. The goal is to find connections between different measurements in different experiments and conditions. To achieve this, the author designs a graph structure for biomedical research, which includes two node types (compounds and experiments) and one relationship type (is attribute of). The author then populates the database with NCI60 measurements, a dataset containing 50k+ chemicals tested on ~60 cell lines. The author introduces cell lines to connect to the experiment, using a script to select the best match for each compound. The author also picks an NSC synonym for each compound, connecting it to one and only one synonym. After filtering and selecting the data, the final script combines all three datasets, resulting in 4m conditions and 4.6k experiments.

Neo4j has been recognized for the first time in the 2022 Gartner Magic Quadrant for Cloud Database Management Systems, marking a significant milestone for both the company and the broader category of graph databases. This inclusion highlights Neo4j's leadership and innovation in the field, as well as the growing adoption of graph technologies anticipated to rise significantly by 2025. Neo4j celebrates other achievements, such as the launch of Neo4j 5, continued growth beyond $100 million in annual recurring revenue, and a historic funding round in 2021. The company encourages users to explore graph databases with Neo4j Aura and offers a complimentary copy of the Gartner report, while emphasizing that Gartner's research should be considered within the context of the full document and does not constitute an endorsement.

The discussion between Robin and Anand Sundarraman, Senior Vice President of DBS Bank, highlights the implementation of knowledge graphs in a bank's data platform to achieve customer 360. The team brought together siloed customer data using graph databases and knowledge graphs, enabling sophisticated questions and deep insights. They used semantics and ontologies to automate steps, automated some manual tasks, and applied data science algorithms. The knowledge graph has helped identify products to sell to customers based on transaction patterns and buying behavior, as well as connect with corporate buyers and suppliers. DBS's internal term for this project is "knowledge quotient," representing the real-world information in bankers' heads. Anand suggests starting small with a specific use case or problem to solve, then expanding from there, and emphasizes the importance of graph data science (GDS) in enhancing knowledge graphs.

The GraphAcademy courses were new in 2022, and the platform has been expanded with many exciting product releases. The community's feedback is extremely important to the developers, who are continuously improving their products based on user input. Two different surveys have been created to gather more information from users, focusing on JSON data and GraphQL development. Additionally, several deep dives into use cases were published in 2022, covering topics such as supply chain data modeling with Neo4j Graph Data Science, creating a knowledge graph of chemical compounds, and using graph algorithms to improve machine learning accuracy. These articles provide valuable insights into the applications of graph data science and its benefits for various industries.