January 2019 Summaries

The creation of compliance laws by various authorities has led to a complex regulatory environment, with six federal agencies and state bank regulators contributing to the chaos. Financial risk management has become increasingly difficult due to siloed data, hindering efforts to produce accurate and on-time reports. However, graph databases offer a solution by unifying market and operational data, enabling end-to-end tracing of market change and risk, and providing precision and consistency in risk management through advanced analytics.

Prepr is a multi-channel engagement platform that streamlines content workflows and powers valuable audience interactions. They were using MySQL relational database to manage their data, but it was becoming increasingly slow due to the large amount of data points and properties they had to manage. By switching from MySQL to Neo4j, they were able to provide powerful real-time recommendations on both the front-end and back-end of the user experience. For example, Prepr created a dynamic queueing tool that could identify customers in a queue who would be most beneficial for ticket sales, and also provided users with a chatbot that offered real-time event recommendations based on factors like location, time of day, and weather. The fast, scalable, and highly-performant Neo4j graph database provided Prepr with the necessary tools to create these effective tools while requiring less storage and hardware than their previous database.

This blog series aims to help developers utilize graph analytics and algorithms to innovate and develop intelligent solutions faster using a graph database. Degree Centrality, one of the centrality algorithms, measures the number of incoming and outgoing relationships from a node, helping find popular nodes in a graph. Proposed by Linton C. Freeman in 1979, it is often used as part of global analysis to calculate minimum and maximum degrees across the whole graph. Degree Centrality can be useful for analyzing influence in social networks or detecting fraudsters in online auctions, where individuals with high weighted centrality tend to collude with each other. An example using a small dataset demonstrates how Degree Centrality works, highlighting its application in evaluating near-term risk and probability of information transmission. The algorithm is an important component of any attempt to analyze influence, making it a valuable tool for developers and graph enthusiasts.

The Neo4j community has been active in January 2019, with updates on the ETL Tool UI and new blog posts from Estefania Cassingena Navone on visual introductions to graphs. Featured community member Lju Lazarezic presented webinars and blogs about Neo4j, including a food graph based on the BBC goodfood dataset. Joe Depeau also presented a webinar on Intelligence-led Policing using Neo4j, while Michael announced the release of version 1.3.1 of the ETL Tool with improved UI and bug fixes. Estefania Cassingena Navone wrote a visual introduction to graphs for beginners, and David Barton graphed the Bath Christmas Market using Neo4j. Jennifer Reif continued her series on building a full stack application with Spring and Neo4j, while Tom Michiels shared his enthusiasm for the new Vectr training catalogue on Twitter.

Mason, the founder of Fast Forward Labs (now Cloudera Fast Forward Labs), is intrigued by problems that seem impossible and believes that clients often miss out on high-impact ideas if they only focus on obvious ones. She uses applied machine learning, graph technology, and specifically Neo4j to help customers accelerate and embrace machine learning and AI opportunities. Mason finds that the graph metaphor resonates with her work, particularly in building machine learning systems, and has been effective in turning complex problems into engineering challenges. Her team has worked on projects like a tool for banks to support better trading decisions, utilizing graph technology and Neo4j to connect entities and analyze relationships. Mason encourages exploring less obvious uses for graph technology and is excited about the future of developer experience for data science and machine learning applications, where she envisions tools that can support data scientists across various metaphors.

Adobe's Behance, a platform for showcasing and discovering creative work, was struggling with its legacy Cassandra-based system. The team decided to overhaul the system using Neo4j, a graph database. They replaced the fanout model with a tiered relationship structure, used three node clusters, causal clustering for robust backup solutions, and dark launching for testing before going live. Switching to Neo4j brought significant benefits, including reduced maintenance hours, storage requirements, and complexity, as well as improved user-facing functionality and faster performance. The team was able to measure KPIs and improvements, demonstrating the benefits of switching to a graph database. Lessons learned include comparing metrics between two systems with the same goal, having someone familiar with procurement processes, and not giving up when companies are hesitant to adopt new technologies. The team is now exploring additional benefits of graph databases for future projects.

PageRank is a centrality algorithm that measures the transitive influence of nodes in a graph, considering not only direct connections but also the importance of neighboring nodes. It calculates an estimation of how important a page is based on the number and quality of links to it, with higher-quality links indicating greater importance. PageRank can be applied across various domains, including social media platforms like Twitter, traffic flow prediction, and anomaly detection in healthcare and insurance industries. When using PageRank, it's essential to consider potential issues such as spider traps, rank sinks, and dead-ends to ensure accurate results. The algorithm has been used in real-world applications to personalize recommendations, predict traffic flow, and detect anomalies, demonstrating its effectiveness in understanding the influence of nodes within a graph.

The CEO and Co-Founder of Neo4j, Emil Eifrem, recently discussed the impact of graph technology on business and society in a podcast interview with Azeem Azhar. Graph technology is powering some of the largest companies today, according to Emil, who also coined the term "graph database." The International Consortium of Investigative Journalists used Neo4j to reveal hidden financial data relationships in the Panama Papers, one of the largest data journalism leaks ever. The interview provides insight into how graph technology is being applied in various industries and its potential for revealing complex relationships in data.

This week in Neo4j covers various topics including Full-Text Search, large-scale system use cases, and community member features. Christophe Willemsen has been exploring the Full-Text Search feature released in Neo4j 3.5, explaining how Apache Lucene works under the covers. Dr Jim Webber discusses using Neo4j for very large scale systems, while Lju Lazarevic provides a blog post on importing RDFS/OWL ontologies into Neo4j. The community member section highlights Michael Simons, who has been actively contributing to Neo4j through building examples, writing articles, and giving talks. Additionally, Azeem Azhar interviews Emil Eifrem on the Exponential View podcast, discussing graph databases and their applications. Jennifer Reif continues her Marvel series with a post on creating data-rich views using Spring Data Neo4j. The week's featured tweet is from Joe Depeau about the Graph Databases book by @OReillyMedia.

The Apache Spark community is collaborating with the growing graph data movement to extend the capabilities of Spark. Databricks contributor Xiangrui Meng is sponsoring a project to integrate Cypher queries and graph algorithms from the GraphFrames project into the core Spark project as part of Spark 3.0, slated for release mid-year 2019. This would bring Cypher queries into the core Spark project, aligning with the growing push for a standard property graph querying language, also known as Graph Query Language (GQL). The community is seeking feedback and support from users to move forward with this initiative, which has been compared to the previous push for GQL, emphasizing the importance of industry unity in graph data management.

Ivan Zoratti, the former CTO of MariaDB, has joined Neo4j as Director of Product Management, bringing his extensive experience in open source innovation and database management to the graph database platform. With over 35 years in the tech industry, Ivan has worked on various projects, including Rdb by Digital (now Oracle Rdb), MySQL AB, and SkySQL, before joining MariaDB. At Neo4j, he is excited to leverage his passion for relational databases and their limitations to improve the graph database platform, particularly in terms of enterprise and cloud features. Ivan believes that Neo4j has a significant potential to expand its power with specific features, and he aims to take the product to another level, making it an order of magnitude bigger. He invites the Neo4j community to connect with him on various platforms, including his personal blog, SlideShare, Twitter, GitHub, and more, and looks forward to sharing his ideas and expertise with the community.

The Minimum Weight Spanning Tree algorithm is a graph search technique used to find the most efficient path between nodes in a weighted graph. It starts with a single node and iteratively adds the minimum-weight edge that connects any new node to an existing one, effectively building a tree structure with the smallest total weight. This algorithm was first developed by Czech scientist Otakar Borůvka in 1926 and later improved upon by Jarnik in 1930. It is often used for network design, travel planning, and analyzing correlations between different nodes or relationships. The algorithm can tolerate negative-weight edges, making it useful for finding the minimum spanning tree of a graph. The example provided demonstrates how to apply this algorithm using Neo4j's Cypher query language, which creates a graph with places and links between them and then finds the Minimum Weight Spanning Tree starting from node D.

The Neo4j community has been active in January, with various updates and projects being shared. Jennifer Reif has started a new series on building full-stack applications with Spring and Neo4j, while Jay Myers showcased how to build an Alexa skill using Neo4j's knowledge graph capabilities. Greg Jordan, the founder of Graph Story, was featured as this week's community member, with his book "Practical Neo4j" being mentioned. The community has also seen updates on monitoring Neo4j databases with Halin and releasing new versions on Google Cloud Platform. Additionally, Frederico Braga explored open data from the Roche Floodlight Open Project using Neo4j, while İrfan Nuri Karaca worked on graph layouts using Neo4j's graph algorithms and machine learning techniques. The community is also active on social media, with a tweet of the week being shared by Artem Yarulin.

Adobe's Behance social network was struggling with a rigid and bloated data model using Cassandra for its activity feed feature, which is its home page. David Fox, a Neo4j community member, joined Adobe and brought the graph database to the company, making it work for their activity feed by reducing the dataset and operations burden, allowing for a much smaller and more manageable dataset with three Neo4j instances compared to 48 Cassandra instances. With Neo4j, Fox was able to prototype quickly, visualize data, and explore new use cases such as machine learning recommendations, and he sees the future of Neo4j including advancements in partitioning graphs off one machine, which would allow for faster querying and more flexible data management.

To address internal frustrations caused by disconnected data, organizations need to connect their data using graph technology. This enables the discovery of new opportunities, customer retention, growth, and compliance with regulations. A graph database serves as the foundation for a master data management system that ties together disparate systems to solve data challenges.

The presentation focuses on Amadeus' journey to mastering their highly distributed architecture using Neo4j and their visualizer tool "Your Architecture Cartography" (YAC). The company, which processes nearly four billion transactions daily, had to tackle challenges such as data complexity, security, and GDPR compliance. They built an ETL tool that allows for quick data importation and expansion of the model, while maintaining simplicity in their UI. Amadeus aims to leverage more Neo4j procedures for security and explore techniques for pulling information from multiple sources to improve understanding of their model. The presentation concludes by emphasizing the importance of keeping Neo4j accessible and fun for internal teams to enhance what they already have.

The All Pairs Shortest Path algorithm is used to calculate the shortest path between all pairs of nodes in a graph, which can be useful for urban service system problems such as location optimization or distribution of goods. It's also used in data center design algorithms like REWIRE to find networks with maximum bandwidth and minimal latency. The algorithm has optimizations that make it quicker than calling Single Source Shortest Path for every pair of nodes, but returns infinity if no shortest path exists between two nodes. A Cypher query example is provided to calculate All Pairs Shortest Path on a small dataset, showing the top 10 pairs of nodes with the furthest distance from each other. The algorithm has applications in various fields and can be used for tasks like traffic load evaluation or network design optimization.

Graph technology and its applications in movies like "Black Mirror: Bandersnatch" are fascinating topics that showcase the power of graph databases, such as Neo4j. The movie's complex decision trees have inspired data nerds to build flow charts detailing the story's parallel realities, highlighting the importance of managing complex workflows and business rules using dynamic decision trees in Neo4j. By exploring these concepts, viewers can gain insights into how their actions impact the future and appreciate the underlying math that underlies graph technology. The movie serves as a thought-provoking commentary on the consequences of our choices, while also showcasing the capabilities of graph databases like Neo4j.

The Neo4j community has released several new resources and content in the first week of 2019. Joe Depeau presented a webinar on using Neo4j for anti-money laundering investigations, which demonstrated how to use the database to help solve common problems in AML investigations. Will Lyon replicated GitHub's GraphQL API using neo4j-graphql.js and data exported via the download-all-of-your-GitHub-data feature. David Mack shared resources and approaches to get started with machine learning on graphs. Dan McCreary summarized the state of knowledge graphs in 2018 and looked forward to the coming year, covering topics such as Graph ML, custom graph hardware, and Google DeepMind's paper on Graph ML. The community also featured a member spotlight on Tim Hanssen, founder at Prepr & Co-founder Syncable, who has presented about his project at local meetups and recently presented Large-Scale Real-Time Recommendations with Neo4j at GraphConnect NYC 2018.