August 2019 Summaries

Greetings, Graph Gang! This week we have a look at using Neo4j with Kafka Streams, how to build a GRANDstack application to analyze football transfers, a beta release of Spring Data Neo4j RX, a guide for learning Cypher in 30 minutes, an overview of the new role based access control features coming in Neo4j 4.0, a look at NLP with graphs, a guide to knowledge graphs, a how-to on cloning subgraphs between Neo4j instances using RDF, a video example of using the Hume insights engine, and building an organization graph. Our featured community members this week are Nathan Smith and Erin Schuberth, who have started the Kansas City Graph Databases Meetup Group to spread the knowledge and understanding of graph databases. David Allen walks through an example of how to integrate graph analytics together with Kafka using Neo4j Streams Kafka integration and Graph Algorithms. Mark Needham takes us into the world of using GraphQL with Neo4j in his post Football Transfers Graph App with the GRANDstack Starter Kit. Gerrit Meier and Michael Simons announce the first beta release of Spring Data ⚡️ RX, a new implementation of Spring Data Neo4j that will leverage the new reactive capability coming to Neo4j but also support an imperative synchronous programming model. Paul O’Neill covers the new role based access control features included in the recent Neo4j 4.0 MR2 release in his post Neo4j 4.0 RBAC – an exploration (with examples). Vlad Batushkov published a tutorial on Learn Neo4j Cypher Basics In 30 Minutes, which shows how to use Neo4j Sandbox and Neo4j Browser, and covers graph data modeling, data import, and Cypher querying using graph patterns. Jesús Barrasa has published Cloning Subgraphs Between Neo4j Instances With Cypher & RDF, and Soham Dhodapkar wrote Accelerating Towards Natural Language Search with Graphs which shows how graphs can be leveraged for Natural Language Processing using tools such as NLTK, SpaCy, CoreNLP, and the GraphAware NLP plugin for Neo4j. Christophe Willemsen published a video demo of importing and analyzing research papers using Hume, a knowledge graph platform built on top of Neo4j. Dean Wilson published an update in his series about Building the Organization Graph showing how to enrich the graph with third-party systems. Akash Tandon wrote an overview of knowledge graphs: Reconciling Your Data and the World with Knowledge Graphs. Our tweet of the week is a sneak peek at a talk from our upcoming NODES online conference, where The Codex will deliver a presentation titled, "Building a Graph of History with The Codex."

The Panama Papers investigation was a massive data leak in 2015 revealing the illicit use of offshore bank accounts by wealthy individuals. The International Consortium of Investigative Journalists (ICIJ) used graph technology, specifically Neo4j and Linkurious, to quickly discover hidden relationships between entities in the leaked documents. This effort led to over $1.2 billion in tax revenue being recouped in 22 countries and earned the ICIJ a Pulitzer Prize for Explanatory Reporting in 2017. A movie about the investigation, "The Laundromat", starring Meryl Streep, Gary Oldman, and Antonio Banderas, is now available to watch on Netflix.

Natural language processing (NLP) is a domain of artificial intelligence that focuses on the processing of unstructured data, specifically textual data, to enable computers to understand and respond to human language. With 80-85% of business-relevant information originating from text format, computational linguistics and text analytics are essential for extracting meaningful information from large collections of textual data. Neo4j, a graph database platform, can connect bodies of text and establish context, making it suitable for NLP applications. By leveraging the power of graphs with Neo4j, elements of text can be stored as nodes, and connections between words are stored as relationships, allowing for efficient storage and analysis of textual data. The GraphAware library enables users to create a pipeline with various operations, such as tokenization, stop-words removal, and named-entity recognition, to process and annotate text data. With Neo4j, it is possible to build a knowledge graph by extracting information from raw text and tying the pieces together using links, enabling reasoners to derive new knowledge from the data. Additionally, NLP libraries in Python can be used to build a near-natural language querying feature on top of an existing graph database, such as Neo4j, by running an NLP pipeline on user input and utilizing the tokens to construct a Cypher query. This approach has limitations, but it demonstrates the potential for natural language search and knowledge graph construction using graph databases like Neo4j.

Eric Monk, a Neo4j consultant, presents a tool he's been working on that helps users create, visualize and parse Cypher. The first step in building any graph is to start with a question, which leads to business concepts, data import, and query building. Understanding tools like Arrows, db.schema, Neo4j ET, Bloom, and LOAD CSV is crucial for graphing success. Cypher plays a vital role in graph building and analysis, allowing users to construct analysis queries and visualize their data models. The tool allows consultants like Monk to see what customers are trying to do without touching the graph, making it easier to analyze and provide feedback. Eric uses the Hero's Journey example to demonstrate how visualization helps determine if Cypher is working correctly and identifies potential errors. He also showcases examples from NASA and the Tour de France, highlighting the tool's ability to parse and visualize complex Cypher queries. The presentation concludes that visualization is essential for understanding Cypher at a glance, finding graph disconnects, improving communication among team members, and seeing the context of queries and subgraphs within data models.

In the field of artificial intelligence (AI), understanding how an AI solution makes a particular decision is a significant challenge. Graphs have emerged as a promising area of research to address this issue, providing easier ways to trace and explain AI predictions. This ability is crucial for long-term AI adoption in various industries such as healthcare, credit risk scoring, and criminal justice, where explanations are necessary for credibility. There are three categories of explainability: explainable data, explainable predictions, and explainable algorithms, with graphs tackling the first two areas fairly easily using data lineage methods. Graphs can provide insight into features and weights used for a particular prediction by associating nodes in a neural network to a labeled knowledge graph. While significant progress is needed in explainable algorithms, research suggests that constructing tensors in graphs with weighted linear relationships may enable explanations and coefficients at each layer.

This week in Neo4j brings a mix of new data science and applied graph algorithms online trainings, including courses on using Neo4j with Python data science tools and applying graph algorithms to enhance applications. A blog series by Max De Marzi explores finding fraud with Neo4j, covering modeling credit card transactions as a graph and optimizing the model for scale and performance. Additionally, Keith Damiani presented on connecting the dots between graph databases and Laravel, and Jesús Barrasa released a new version of NSMNTX, a Neo4j plugin for working with RDF data. Graph algorithms were also covered in a talk by Amy Hodler and her colleague, introducing the Graph Algorithms Playground, a tool for using graph algorithms without writing code. Other topics include exploring League of Legends data in Neo4j, powering data discovery systems at Airbnb and Lyft, Cypher index hints, and migrating dimensional models into property graphs. The developer relations team also invites feedback on the new GraphQL integrations or GRANDstack via a short survey.

Digitate, a strategic business unit of Tata Consultancy Services (TCS), is using Neo4j as its graph database to improve performance and query speeds in its AI Ops product, ignio. Ignio is a cognitive automation tool that learns context, manages alerts, provides smart recommendations, and takes action autonomously. Digitate chose Neo4j due to its ease of use, features, scalability, and availability, which enabled the company to store and manage complex relationships and dependencies in IT infrastructure naturally as a graph. The adoption of Neo4j has allowed ignio to provide award-winning performance and deliver key benefits such as pinpointing probable causes of failure in batch jobs with millions of relationships and dependencies. Digitate believes that technologies like Neo4j will play a crucial role in creating a 360-degree view of the business environment, mapping every kind of relationship and component, and making IT smarter.

The newly launched Neo4j Online courses, Data Science with Neo4j and Applied Graph Algorithms, aim to introduce users to using Neo4j as part of their data science and machine learning workflows. The courses are designed for data scientists and application developers who want to leverage graph capabilities in their work. Users will learn various skills such as query a database for its schema, build and plot histograms, use link prediction functions, and apply graph algorithms like PageRank, community detection, and similarity metrics to enhance applications. The courses offer free training with exercises to test learning at the end of each section, taking approximately 2-3 hours to complete, and are accessible on the Graph Academy home page.

Graph technology platforms like Neo4j enhance AI by providing context and connections, enabling the extraction of connected features that improve accuracy in predictive models. Relationships within data are often stronger predictors of behavior than traditional input data built from tables, allowing for more efficient analysis and incorporation of relevant information. Connected feature extraction methods use graph algorithms to identify key patterns and structures, such as anomalies in tight communities, which can be used to detect complex behaviors like fraud and money laundering. These features can be engineered or discovered through various approaches, including considering labels or inferred relationships. By leveraging connected features, AI systems become more accurate and easier to explain, making them a valuable tool for various industries.

This week's Neo4j community update is a mixed bag of interesting topics and announcements. The new milestone release of Neo4j 4.0 is available for public testing, featuring exciting capabilities such as reactive support, multiple databases, additional security and role management, and optimizations for the database and algorithms. Meanwhile, Joe Depeau shows us how to find popular kids in high school using the PageRank algorithm, while Paul O'Neill shares his experience creating a custom procedure for paging through query results and representing Minecraft Craft Trees as a Graph. Other notable topics include navigating a social feed with pagination, interrogating a cancer graph with Neo4j Bloom, new releases of Neo4j OGM and yFiles visualization for Java, and a tutorial on installing Graphlytic as a graph app in Neo4j Desktop. The week also features a featured community member profile of Paul O'Neill and a tweet of the week from Melly Burns.

Neo4j is a graph database that enables data-driven decision making by focusing on relationships between data entities, allowing for significantly faster insights compared to traditional relational databases. This technology was utilized in the Paradise Papers investigation and can be applied to various industries such as retail, where it was demonstrated through an example of coupon recommendations. By leveraging Neo4j, organizations can improve their edge with better data analysis capabilities.

The author visualized This Week in Tech stories using machine learning and graph algorithms to explore connections between articles. They created a knowledge graph to represent entities, concepts, and topics from articles, using techniques like Named Entity Recognition (NER), external knowledge bases, and Doc2Vec embeddings. The resulting weighted graph was analyzed using the Louvain Community detection algorithm, revealing clusters of related stories on topics such as space exploration, security breaches, and banking startups. A web app was built to visualize these clusters, allowing users to explore related articles and discover new connections between stories. The visualization provided insights into what shaped this week's tech news, showcasing the potential of graphs in extracting actionable knowledge from unstructured data.

Graph technology platforms like Neo4j enhance AI by providing context for improved efficiency through the use of graphs, which offer greater processing power and allow for faster analysis of data connections. Traditional machine learning methods often rely on table-based data storage, leading to resource-intensive processing and challenges in iterative model training. Graphs enable relationships of numerous degrees of separation to be traversed and analyzed quickly at scale, accelerating processes such as filtering and collaborative filtering. This approach also helps address scalability issues by returning only the needed data through simple graph queries. By leveraging graphs, AI can provide context for decision support while reducing computational intensity and human involvement, ultimately enhancing the efficiency of machine learning algorithms.

It's a story about Mediaconnect, a company that uses Neo4j to tackle real-time data problems. They built a graph of customers and used it to drive recommendations for festival tickets, offering campsite options in real-time. Their experience with Neo4j has been positive, with blazing fast performance and the ability to see connections between nodes. They've seen new insights into their customer behavior and can run algorithms in real-time to suggest personalized advertising. The company's future plans include expanding on graph technology and overcoming initial fears about its use.

The Neo4j team has published the second milestone release of Neo4j Enterprise Edition 4.0, which is a pre-alpha version that includes some upcoming features for testing purposes. This release is not ready for production and may contain issues. It can be downloaded from the Neo4j Desktop or Download Center and is commercially licensed under the Neo4j Pre-Release Agreement. The new features include reactive drivers, back pressure and flow control, support for multiple databases, fine-grained security with a schema-based model, role and user management, system database, neo4j:// scheme, new Spring Boot starter, SDN/RX, index population algorithm, index key size, and improved space reclaim. The team invites users to try out the release, provide feedback, and contribute to its improvement.

The author uses a relatable analogy from the classic teen film "Heathers" to explain how PageRank works. The story revolves around the social hierarchy of Westerburg High, where different cliques have varying levels of popularity and relationships with each other. The author calculates the PageRank score for each student in the graph using a simpler query that calls the PageRank graph algorithms function. The results show that the Heathers are indeed popular, but their popularity is boosted by their relationships with other popular groups, such as the Jocks and Cheerleaders. Unpopular students like Betty Finn and J.D. also have higher PageRank scores due to their strong relationships with Veronica Sawyer. The story highlights how quality matters in determining one's popularity score, rather than just quantity.

Knowledge graphs are interlinked sets of facts that describe real-world entities, facts or things and their interrelations in a human understandable form. They offer a way to streamline workflows, automate responses and scale intelligent decisions by providing context for decision support. Knowledge graphs need to be connected around relevant attributes, dynamic, understandable and can combine and uncover connections across silos of information. There are three types of knowledge graphs: context-rich knowledge graphs that incorporate metadata tagging, external-sensing knowledge graphs that aggregate external data sources mapped to internal entities of interest, and natural language processing knowledge graphs that require understanding a company's specific technical terms and nuances of human language. Knowledge graphs provide context for decision support, help ensure answers are appropriate to the situation, and have applications in search, customer support, document classification, supply chain risk, market activity segregation, sales opportunities, improved search, chatbot implementation, and more.

This week, Anurag Tandon shared advanced search features in Neo4j Bloom and created Cypher queries via user-friendly natural language search phrases. Pablo Díaz built a project and library dependencies graph to help customers understand the relationships between different projects. Igor Rozani was featured as a community member for his Pokémon Graph project, which used Neo4j to track Mega Evolutions of Pokémon. Paul O'Neill wrote about graph data modeling, contrasting inferred vs explicit categories and labels. Robin Moffatt shared slides from his talk on building a streaming ETL solution with Rail Data, using Kafka and Neo4j. A new Haskell, GraphQL, and Neo4j library was released, along with tutorials and workshops organized by community members like Igor Rozani. Community members also shared their projects and experiences with Neo4j, including conditional WHERE clauses in Cypher queries and a project that uses the Haskell Bolt Driver.

This week, I'd like to turn everyone's attention back to the ICIJ Paradise Papers investigation, which is revealing connections and smoking guns in the secretive world of offshore financing. Fresh revelations have emerged about Jeffrey Epstein's offshore fortune, tracing it to the Paradise Papers. The investigation involves analyzing vast data sets using Neo4j graph database, a tool that provides an insider perspective into this major leak. Data journalists are now looking at enormous reams of data, much like Woodward and Bernstein initially uncovered secrets in the Watergate scandal. The discovery is ongoing, with new insights being provided through the use of graph technology.

Digital asset management (DAM) is a crucial aspect that affects various business aspects, including content availability and customer usage behavior analysis. It drives innovation and impacts companies' ability to gain insights into monetization potential. A graph platform like Neo4j offers advantages over traditional relational databases and metadata stores for DAM, as discussed in the webinar session featuring success stories of Scripps Networks and Adobe Behance.