June 2016 Summaries

The course updates aim to improve the learning experience for Neo4j users, focusing on Cypher, the most widely used graph query language. The revamped "Introduction to Graph Databases" class now covers not only graph querying but also creating and updating graph data using Cypher and importing CSV data from other sources. Additionally, the introductory class has been streamlined to be completed in about 60-90 minutes, allowing users to invest time savings in modeling and building their first graph application. The "Neo4j to Production" course was updated by field engineers to align with Neo4j 3.0 operational changes, and users can provide feedback through email or the public neo4j-users Slack channel.

The International Consortium of Investigative Journalists (ICIJ) used Neo4j, a graph database management system, to analyze and visualize the Panama Papers dataset. The data included offshore companies and tax havens in various countries, which were created by a Panamanian law firm called Mossack Fonseca. ICIJ used Linkurious, a software tool built on top of Neo4j, to explore and visualize the data in graph format. This allowed reporters to discover connections between individuals and companies, revealing complex networks of offshore companies and tax havens. The investigation involved 400 journalists from around 80 countries working together to analyze and report on the data. ICIJ's use of Neo4j and Linkurious enabled them to efficiently handle large amounts of data, including 11.5 million documents, and facilitated collaboration among reporters across different time zones. The project showcased the potential of graph databases in facilitating complex data analysis and visualization, and highlighted the importance of data-driven journalism in uncovering hidden stories and connections.

The Neo4j 3.0 language drivers provide a uniform API for easy access to Neo4j from any programming language, addressing inconsistent patterns of usage and different feature sets across existing drivers. The Bolt protocol is implemented by connectors on either side of the communication, sending Cypher messages to the server, which then sends streams of records back to the connectors. The client connectors implement a standardized API, allowing users to have the same set of classes, methods, and experiences regardless of language. The drivers are built around four key abstractions: the driver itself, which encapsulates interaction with Neo4j; sessions, which give the illusion of working with a database; transactions, which allow sending multiple Cypher statements as one logical unit of work; and statement results, which return handles to continuously pull out streams of records from the database. The drivers are hosted on GitHub and are Apache licensed, allowing commercial use without licensing issues. The project aims to complement community driver authors by providing a fundamental foundation and plumbing, while also enabling high-level API and language-idiomatic features. The Bolt protocol offers better performance than HTTP due to its smaller payload size and stateful setup. Built-in stored procedures allow users to write custom code that plugs directly into the database, and can be used for specialized algorithms or third-party systems. The drivers include a number of built-in procedures for listing and managing databases, as well as creating new procedures using a simple class-based approach. Finally, legacy stores can be plugged in to pull all data into a graph using a 10- or 20-line procedure.

The Docker + Neo4j combination has a long history, starting with the launch of the Community Edition Neo4j Docker image in beta about a year ago. After community feedback and iteration, the Official Repo was launched with support for both editions, including high availability capabilities. The image has received positive feedback from developers worldwide, with over 1 million pulls from the Docker repo. A notable example is the Graph Your Network application, which scaled to hundreds of simultaneous containers running Neo4j, allowing users to analyze their Twitter activity. The introduction of the Docker Store brings excitement as it enables users to rate and review images, receive faster updates, and provides access to high-quality images like Neo4j. This development also aligns with the need for developers to sharpen their skills in Neo4j, offering a free Learning Neo4j ebook to help them catch up.

The Neo4j team has announced a new program, the Data Journalism Accelerator Program, to support data journalists in analyzing connected relationships in Neo4j. The program aims to help journalists understand their data and uncover new stories by drawing connections between existing information and new sources. Participants will receive hands-on assistance from Neo4j engineers, while tools and integrations will be built to align with the needs of data journalists. Graph databases have a successful track record in investigative journalism, as seen in the Panama Papers and Swiss Leaks investigations. The program's details are announced at the Investigative Reporters and Editors (IRE) Conference, where the Neo4j team will also co-present on how graph databases can help uncover wrongdoing.

The Neo4j graph database has been used by big companies and small startups alike as a core part of their technology stacks, providing a quick introduction to its basics through step-by-step video tutorials designed for absolute beginners.

Linkurious is a graph visualization startup that helps organizations like NASA and the French Ministry of Finance analyze connections in their data to find hidden insights. The company's tool is used in various domains such as cybersecurity, anti-fraud, network management, and medical research where massive graphs are present, but business analysts struggle to extract valuable information from them. Linkurious was instrumental in the Panama Papers investigation, where a team of journalists used it to uncover suspicious connections in 11.5 million documents. The company's co-founder Jean Villedieu chose Neo4j as their primary graph database due to its leading position in the market and the expertise behind it. He has been impressed by the range of use cases for Linkurious, from anti-fraud and cybersecurity to knowledge management and medical research, where graph visualization can help find insights and solve complex data challenges. Jean Villedieu encourages Neo4j customers to reach out to Linkurious for assistance in turning their graph data into actionable insights.

The text discusses the development of a metadata generation system using Stack Overflow tags and topic keywords. The system uses graph theory to analyze relationships between topics, allowing for automated page rankings and personalized recommendations for customers. To improve accuracy, an ontology is added to provide context, and the TF-IDF algorithm is used to balance term frequencies with commonness. The system also incorporates parsing of ebooks to extract keyword data and subsections, enabling a more accurate picture of content. A spreading activation algorithm is applied to customers' profiles, adding weights to topics based on their interests and activities. This allows for dynamic personalization of recommendations, including automatic page rankings and customer profiling. The system has been successfully implemented in production using Neo4j and Rails API, enabling graph-based search, modularized content, and skill mapping.

The Neo4j Spark Connector is an open-source project that enables data transfer between Apache Spark and Neo4j, a graph database. The connector supports various Spark features, including Resilient Distributed Datasets (RDD), DataFrames, GraphX, and GraphFrames. It allows users to query and manipulate data in both systems using Cypher statements. A quickstart guide is provided, which includes downloading and installing Neo4j 3.0, creating a dataset, and running the Spark Shell with the connector and GraphFrames as packages. The connector supports various demos, including an RDD demo, a GraphX demo, and a GraphFrames demo. The source code is available on GitHub, and the connector is licensed under the Apache License 2.0. Users can provide feedback by raising GitHub Issues.

The Neo4j blog post discusses various articles, videos, slides, and other content related to graph databases. The blog highlights recent publications such as "Neo4j 3.0 Moves Graph Databases Closer to Mainstream" by Darryl K. Taft, which explores the advancements of graph databases in mainstream adoption. Articles like "Querying Graphs with Neo4j" and "Java Panama Papers Neo4J Network Generator" provide insights into querying and working with graph data. The blog also features videos such as "Building FlowModels with Graphileon Interactor" and slides from presentations by experts like Adrian Cockcroft on microservices application tracing standards. Additionally, the blog showcases various libraries, Gists, and code repositories, including APOC User Guide and bulk-neo4j-import-original.sh scripts. The content aims to educate developers about the benefits of using graph databases in conjunction with relational databases, as highlighted in the ebook "The Definitive Guide to Graph Databases for the RDBMS Developer".

Neo4j 3.0 introduces several new features that aim to improve data visualization and querying capabilities, particularly in conjunction with charting libraries like chart.js. The goal is to provide an auto-updating chart of graph data volume, including node counts by label and relationship counts by type, as well as support for free-form queries. To achieve this, Neo4j 3.0 leverages its binary protocol (Bolt), which enables faster entity counting, built-in procedures for querying labels, relationship types, and procedure metadata, and officially supported drivers for multiple programming languages, including JavaScript. The implementation involves integrating chart.js with the Neo4j JavaScript driver, utilizing database statistics to update charts in real-time, and allowing users to run free-form queries via a web-based interface.

The Financial Times has successfully implemented Neo4j in their data management system, leveraging its capabilities for inferencing and linking data together to provide a richer experience for readers. They experimented with using Neo4j after previously relying on an RDF triple store, finding that it offered significant productivity gains by allowing them to rewrite nearly 12 months' worth of work in just three months. The team enjoyed the ability to collaborate around whiteboards and discuss models amongst developers, making it a great experience for collaboration. In retrospect, they would have liked to be less scared about creating nodes and more willing to iteratively change their data model to fit queries. They are excited about the new Neo4j drivers, particularly Bolt, which they believe will offer improved performance in languages like Go. The project was made possible by taking control of the technology and demonstrating its capabilities, with support from management, allowing the team to take ownership and responsibility for the changes.