February 2019 Summaries

The Neo4j company has updated its free online training course, "Introduction to Neo4j," to provide a more comprehensive and hands-on introduction to graph databases and the Neo4j graph database platform. The new 8-hour course is designed for beginner students and includes 16 hands-on exercises that can be completed in either Neo4j Desktop or a Neo4j Sandbox. To complete the course, students need to install Neo4j Desktop on their machine or create a Neo4j Sandbox, and they will receive a certificate of completion if they pass all quizzes correctly. The updated course is essential for becoming a Neo4j Certified Developer and covers topics such as Cypher queries, CRUD operations, constraints, indexes, and importing CSV data into Neo4j.

The Weakly Connected Components algorithm, also known as Union Find, is a graph analytics technique used to identify groups of connected nodes in an undirected graph where each node is reachable from any other node in the same group. This algorithm differs from the Strongly Connected Components algorithm in that it only requires a path to exist between pairs of nodes in one direction, whereas SCC needs a path to exist in both directions. Union Find is often used as a preprocessing step for directed graphs and can be applied to various domains such as testing whether a graph is connected, analyzing citation networks, and keeping track of clusters of database records. The algorithm was first described by Bernard A. Galler and Michael J. Fischer in 1964 and can be computed using breadth-first search or depth-first search algorithms. By applying Union Find, developers can quickly identify disconnected groups and enable running other algorithms independently on identified clusters, making it a powerful tool for graph analytics and community detection.

Neo4j has been actively engaged in various activities, including an online meetup where the BBC GoodFood graph was showcased. Additionally, there have been several community members featured, such as Calin Constantinov, a Neo4j Certified Developer and SAP Hybris Java Technical Lead. The APOC library has also released its winter version with new features and improved documentation. Furthermore, articles have been published on integrating Neo4j with Kafka, automating deployments on cloud platforms like AWS and GCP, and building a software dependency graph using graphs. These resources are available for the community to learn from and implement in their projects.

Alex Babeanu, an Identity Solutions Architect at Nulli, discussed his work with graph databases like Neo4j in managing identity and access management (IAM) complexity. With the exponential increase in identities due to smartphones, online services, and smart devices, graph databases bring sanity to this growing problem. Babeanu's team uses Neo4j for system integrations and designing solutions, finding it essential for modeling interconnected identities and deriving access policies. He attributed his choice of Neo4j to its ability to handle the dramatic increase in identities, which is a nightmare to manage with traditional methods like LDAP or SQL databases. Babeanu found Neo4j's simplicity in transforming complex requirements into manageable models fascinating. As he believes that graphs and AI will be crucial moving forward, particularly in IAM, he envisions a future where these technologies will become even more prevalent, especially for the identity community.

I'm a nostalgic child of the '80s who fondly remembers reading Choose Your Own Adventure books. Recently, I loaded one of these books into Neo4j Desktop to visualize its narrative structure as a graph. The author mapped out the story by creating nodes for pages and relationships between them to represent choices. Using this graph model, I was able to answer various questions about the book's structure, such as finding loops or unreachable pages. The graph also allowed me to query the shortest path to an ending page or the longest path from the start. This exercise has shown how graph databases like Neo4j can be used for pathfinding use cases, and I envision their potential applications in analyzing authorial intent or tracking narrative complexity.

Gousto, a UK-based recipe box service, uses Neo4j to create a personalized recommendation engine that suggests recipes based on customer preferences and ingredient similarity. The company's data journey began with external data sources such as Google Analytics and has since evolved into a rich data ecosystem supported by Amazon Redshift, Periscope, Salesforce, Airflow, Snowplough, and Neo4j. Gousto's marketing attribution model helps allocate resources effectively, while its forecasting model determines orders for ingredients based on weather, holidays, and customer trends. The company also uses a warehouse optimization system to efficiently pack boxes, and personalization is crucial in the merchandising process. A hybrid recommendation engine combining collaborative filtering and content-based models, LightFM, was developed to overcome the "cold start" problem and provide more accurate recommendations. Neo4j's graph ontology allows Gousto to capture complex relationships between recipes and ingredients, enabling the creation of similarity scores and inferences from data. The company is now exploring ways to leverage Snowplow data for personalization and AI-based recipe development.

The Strongly Connected Components algorithm is a graph analysis technique used to identify groups of nodes in a directed graph where each node is reachable from every other node in the same group, following the direction of relationships. This algorithm is often applied early in a graph analysis process to give an idea of how the graph is structured and can be used as a preprocessing step for other algorithms that work only on strongly connected graphs. It has been widely used in various applications such as analyzing powerful transnational corporations, computing routing performance in multihop wireless networks, and identifying groups of people in social networks. The algorithm can also be used to visualize the structure of a graph by identifying clusters or communities. In the context of Neo4j, the Strongly Connected Components algorithm can be used to identify strongly connected components in a graph and run other algorithms independently on identified clusters.

The Neo4j Online Meetup is a global event coordinated by Mark Needham and Karin Wolok, where individuals from around the world can participate in discussions about graph technology. Recently, an online meetup was hosted featuring a presentation by the International Salmon Data Laboratory (ISDL), which is using Neo4j to track "the canary in the coal mine" of climate change resilience. The ISDL is part of the Graphs4Good program at Neo4j and aims to leverage the power of Neo4j to develop tools that can have impacts throughout environmental sciences, with a focus on salmon resilience to climate change as the overarching goal.

Dr. Jim Webber described how to run Neo4j in a multi data center environment, while Max De Marzi showed how to find the shortest path on a rail network and Stefan Bieliauskas explained why graphs are a perfect fit for modeling data provenance. The community spotlighted David Stevens, Global Technology Transformation Lead at DXC, who is the author of an Enterprise knowledge graph built using Neo4j. Tom Geudens started a series on versioning graphs, focusing on time-based versioning in his first installment, and Tomaz Bratanic used the Pearson Similarity algorithm to analyze Kaggle's Young People Survey dataset. Additionally, Jennifer Reif continued her Marvel Series, building the controller and service classes for handling requests and shaping results. The week also featured a blog post by Stefan Bieliauskas on provenance with Neo4j and a tweet of the week from Thibault Chevrin highlighting OpenFoodFacts' data.

The Apache Spark community has completed a positive vote for a Project Improvement Proposal (SPIP) to add property graphs based on DataFrames to Spark, which will enable users to use the Cypher graph query language and access graph algorithms from the GraphFrames project. This is a significant step forward for standardized approach to graph analytics in Spark, reflecting collaboration by many contributors. The development of a single standard declarative query language called GQL (Graph Query Language), drawing on Cypher's ASCII-art representation, has also been proposed, with plans to create an international standard specified and maintained by the ISO working group. A W3C workshop on graph data management standards will bring together experts from RDF, labelled property graph, and SQL standards specialists to figure out ways of creating bridges between these disparate but related data models and languages. An openCypher Implementers Meeting (oCIM) will follow the workshop, discussing language improvement requests and proposals, including carrying out Cypher queries that project new graphs and incorporating those queries in parameterized views. The goal is to create a managed transition from Cypher to GQL while preserving familiar features and ensuring minimal disruption to existing customers and their applications.

The Global Graph Celebration Day is an initiative to encourage people around the world to participate in graph-related activities, particularly those related to Neo4j and graph databases. The event aims to help the world understand the value of graphs and celebrate the contributions of Leonhard Euler, a Swiss mathematician who made significant discoveries in graph theory. Participants can host local events or gatherings with friends and colleagues, which will be included in a community map and build upon a Neo4j community graph. Exclusive t-shirts and swag will be provided to event organizers with 12+ RSVPs. The event is scheduled for April 15, 2019, and registration is now closed, but interested individuals can sign up for next year's event.

LendingClub, a leading online marketplace lending platform in America, utilizes Neo4j to manage its infrastructure and operate its online platform. The company's DevOps team uses Neo4j to automate the deployment of microservices, monitor and alert on system performance, and provide visibility into components that were previously unrelated. By loading all infrastructure components into Neo4j, the team gained a single, central hub of information that they could query at any time, allowing them to answer important questions vital to ensuring site uptime and diagnose issues quickly. The graph model also enabled automated monitoring, alerting, and reporting, as well as datacenter deployments and cloud migrations. Additionally, LendingClub's graph model has allowed for the creation of a unified pipeline across multiple tools, enabling self-service and ownership among engineers, and treating third-party tools uniformly. This flexibility has been crucial in adapting to changing technologies and infrastructure needs.

The blog series discusses Centrality algorithms in graph databases, focusing on Closeness Centrality and its application in various fields such as organizational networks, telecommunications, and document analysis. Closeness Centrality measures the average distance to all other nodes in a graph, with higher scores indicating nodes that can reach all other nodes quickly. The algorithm is used to detect nodes that can spread information efficiently through a graph. It is also used in networks where information spreads through shortest paths simultaneously, such as infections spreading through a local community. Closeness Centrality works best on connected graphs and has limitations when applied to unconnected graphs. A variant called Harmonic Centrality was proposed to solve this issue, using the sum of the inverse of distances instead of the sum of distances. The blog concludes that Closeness Centrality is applicable in various resource, communication, and behavioral analyses where interaction speed is significant.

Graph databases and machine learning are being used to fight the spread of fake news by analyzing divergent data patterns. A two-part video series explores how a graph-based solution can aid human fact checkers in increasing the speed, efficiency and scalability of fake news detection. The News Graph is designed to separate fact from fiction, providing advantages over traditional solutions like NBC News' use of Neo4j to analyze Russian Troll Network data. By analyzing large datasets, researchers aim to make sense of the digital social landscape and prevent future fake news abuse.

The Neo4j community has been actively producing content around the use of graph technology on Google Kubernetes Engine (GKE), with a presentation explaining its benefits and a blog post showcasing how to deploy and manage Neo4j on GKE. Additionally, there are resources available for analyzing malware relationships using Neo4j, as well as tutorials on building PowerBI connectors, backing up databases, and identifying unique speakers from their voice prints. The community has also been exploring the graph technology landscape, with a blog post launching a map of the industry and a video showing how to edit nodes and relations in a grid. Finally, there are interviews with community members and a tweet of the week highlighting the evolving use of Neo4j for describing personal relationships.

Graph algorithms are becoming increasingly important as graph data is growing in size and complexity, making it essential to develop efficient methods for analyzing and processing these graphs. Dr. Steven Skiena's research on graph embeddings, particularly with his algorithm DeepWalk, aims to translate graphs into numerical representations that can be used in machine learning models, enabling powerful applications such as question answering, similarity detection, and clustering. The concept of graph embeddings is analogous to word embeddings in natural language processing, where words are represented by their roles in sentences, and similarly, vertices are represented as points in space. DeepWalk's power lies in its ability to generate these representations using unsupervised methods, similar to Word2vec for text data. Graph algorithms have the potential to become even more powerful with advancements in distributed algorithms, such as hierarchical graph embedding, which enables the processing of larger graphs. The increasing importance and ubiquity of graph technology make it an exciting area of research, with Dr. Skiena's work contributing significantly to its development.

This blog post focuses on how to leverage data science tools synergistically, particularly in the context of fraud investigation and information representation using graph databases. Big data presents numerous challenges, but data science provides a solution by extracting knowledge from data through various scientific methods. The post emphasizes that data science is an interdisciplinary field combining mathematics, statistics, information science, and computer science to extract insights from data. It also highlights the importance of converting unstructured data into structured data using tools like Elasticsearch, Solar, LESS, and open-source APIs, mostly in Python. The presentation covers various aspects of fraud investigation, including anomaly detection, predictive analytics, classification, and entity extraction, as well as information retrieval, graph databases, and real-world examples. The ultimate goal is to create a searchable database that can be used for multiple purposes, including uncovering fraud, conspiracies, and conflicts of interest. By leveraging data science tools and graph databases, investigators can gain valuable insights into complex cases and make more informed decisions.

This blog series aims to help developers better utilize graph analytics and graph algorithms, enabling them to develop intelligent solutions faster using a graph database like Neo4j. The focus is on Centrality algorithms, which measure the importance of nodes in a graph by analyzing their relationships. Betweenness Centrality measures the number of shortest paths that pass through a node, identifying influential nodes that serve as bridges between different clusters. It is used to detect network flow, identify influencers in organizations, and help microbloggers spread their reach on Twitter. The algorithm calculates scores based on the frequency of shortest paths passing through each node and can be affected by assumptions about communication patterns in real-life scenarios. An approximation algorithm called RA-Brandes is used for large graphs due to computational complexity, which considers a subset of nodes with strategies like random or degree-based selection.

This week in Neo4j has seen five releases across various projects, including the Neo4j Desktop, Graph Algorithms Library, and a new Python library. The Neo4j Desktop version 1.1.14 introduced security enhancements and a command bar for improved productivity. The Graph Algorithms release included improvements to algorithms such as ArticleRank and Pearson Similarity, as well as performance optimizations for Louvain. Additionally, Py2neo, Neo4j Versioner Core, and Neo4j Connector have been released, providing tools for data management, entity-state modeling, and ETL jobs. Community members Kunal Yadav, Paramveer Singh, and Raghu Madhava are featured for starting a local meetup group to advocate graph thinking in their community. The week also saw a master class on writing Neo4j stored procedures with Max De Marzi, Jennifer Reif, and Mark Heckler, as well as a blog post by Andrea Santurbano on building a just-in-time data warehouse using Neo4j Streams.

The Critical Threats Project uses Neo4j as a backbone for its intelligence analytics, leveraging its graph database structure to connect and analyze vast amounts of unstructured data from various sources. This allows the organization to easily bring together disparate data sources, visualize relationships, and derive insights into conflict areas globally. The team chose Neo4j due to its flexibility, ease of use, and reliability, which are crucial for their dynamic environment and ability to adapt to new ideas and data structures. Graph technology is valuable in the intelligence community as it supports network connections, traversal, and data munging, making it an ideal solution for analyzing complex networks and relationships. The project has found Neo4j to be a user-friendly interface and Cypher queries intuitive, even for novice programmers, and its reliability has been crucial for maintaining large datasets with minimal maintenance.