July 2016 Summaries

Neo4j has been successfully integrated into Kale Yazılım's software stack for nearly three years, resulting in significant improvements. The company uses Neo4j to analyze Turkish citizen data and identify seniors without nearby relatives, a task that was previously impossible with SQL. Cypher, Neo4j's graph query language, is cited as a key reason for the choice of Neo4j, along with its ease of use and speed. A notable project involves compiling ancestral trees from census data in real-time using Neo4j, which has been integrated with a relational database to ensure up-to-date data. The expert notes that graph databases are on the rise due to the growing amount of global data, and that understanding relationships between data points is crucial for managing valuable information. Ultimately, the use of graph databases will become a standard part of every system.

Transforming a data model from a Relational Database Management System (RDBMS) to a graph database requires significant changes, including replacing tables with node labels, rows with nodes, and columns with node properties. Foreign keys are replaced with relationships between nodes, and data with default values is removed. Additionally, denormalized and duplicated data may need to be pulled out into separate nodes for a cleaner model. Indexed column names can indicate array properties, and JOIN tables are transformed into relationships. The goal of these transformations is to achieve flexibility in arranging data in an easy-to-understand way.

Spring Data Neo4j is a framework that provides a convenient way to access databases, particularly NoSQL databases. It offers consistent APIs for typical crowd-like operations and executes queries with a very consistent API. The Object-Graph Mapper (OGM) in Spring Data Neo4j 4 allows users to map domain objects to the underlying database without reflection, reducing start-up times. OGM also provides smart object mapping, recognizing and sending only new domain model changes to Neo4j for persistence. This makes the application more performant. The framework offers features such as repositories, custom queries, transactional support, and variable-depth persistence. These features enable users to build applications quickly and efficiently using Spring Data Neo4j 4.1. The example demonstrates how to build a superhero graph with a simplified domain data model using Spring Data Neo4j 4.1. The code includes annotations for node entities, relationships, and rich relationships, as well as repositories and custom queries. The application uses auto-configuration or explicit configuration to set up the Spring Data Neo4j framework. The example showcases how to use Spring Data Neo4j 4.1 to build a graph-based application with a consistent API and efficient data persistence.

In this blog post, the author explores how to load XML data into a Neo4j graph database using APOC, a suite of procedural macros for Cypher. The author starts by preparing the data, including downloading the XML files and installing the necessary Python driver py2neo. They then use APOC to extract information from the XML files, creating nodes and relationships in the graph as they go. The author demonstrates how to create relations between nodes using internal node IDs and how to visualize the resulting graph. Finally, they explore how the healthcare industry is influenced by lobbying efforts, tracing connections between lobbyists, lobby firms, and legislators. Throughout the post, the author shares their Python code and Cypher queries, providing a step-by-step guide for readers to follow along.

I learned about Neo4j and its use in a proof of concept project by WayBlazer, an IBM business partner, which used it for their recommendation engine. The technology was implemented in a microservices architecture with Docker containers and a three-layer structure. I found the synergy between Neo4j and cognitive computing tool Watson to be remarkable, as Watson's ability to reason and learn complements the graph database's capabilities. Looking back, I wish we had access to more resources and performance available in Neo4j 3.0 at the time of our project, which would have streamlined our infrastructure investments.

The Internet of Things (IoT) is becoming increasingly complex, making it difficult to manage networks. Traditional methods such as Excel sheets and traditional databases are no longer sufficient due to the sheer number of devices on a network. Graph databases offer a more effective solution by storing data in a structured and connected manner, allowing for easier visualization and querying. This enables IT teams to better manage their networks and identify potential security risks, such as open ports that shouldn't be open. By leveraging graph databases, companies can catch discrepancies and make sense of their network data in a more natural state, reducing the need for manual translation and analysis.

The Financial Times, a newspaper that has been in operation since 1888, underwent significant changes in its digital strategy. The company transitioned from relying on paper subscriptions to digital subscriptions, which led to the development of new technologies and platforms. One of these projects was a semantic linked data platform built in Java, but it failed to deliver due to technical issues and slow loading times. The team recognized that they needed a better approach and decided to rewrite the project using Neo4j, a graph database, and Go, a programming language. This drastic change allowed them to overcome the limitations of their previous technology and create a more efficient and scalable system. With Neo4j, they were able to build relationships between entities, apply labels, and perform searches, ultimately resulting in a more robust and reliable platform. The team's willingness to take risks and adopt new technologies paid off, enabling them to produce multiple deliverables per week and deploy data into the cloud with resilience.

The Awesome Procedures on Cypher (APOC) library has been updated to version 1.1.0, introducing new features and fixing existing issues. The library provides a wide range of capabilities for Neo4j, including data import/export, integration with other databases like Cassandra, MongoDB, and RDBMS, and improved graph algorithms. New procedures have been added to support free text search, metadata management, and collection functions. APOC also provides controlled Cypher execution, schema/indexing, and graph representation capabilities. The library aims to achieve a more regular release cycle with new features and bug fixes every two weeks. Users are encouraged to try out the new functionality, check the documentation, provide feedback, and report issues on the GitHub issue tracker or Slack channel.

In this interview, Nicolas Rouyer from Orange Business Services shares their experience with Neo4j, a graph database software. They started using Neo4j for IT supervision use cases and found it useful for finding security issues in their information system. Nicolas attributes the choice of Neo4j to solving search problems such as shortest path, which was different from their other use cases. The team has seen surprising results from using Neo4j, including gaining insights into certain communities and refining data models. However, they would do things differently if given the chance, particularly in terms of data modeling and integrating with other software tools. The community around Neo4j is growing rapidly in Toulouse, France, and Nicolas encourages others to share their experiences and learn more about graphs.

The Neo4j community has published various articles, blog posts, podcasts, videos, slides, presentations, libraries, graphGists, and code repositories that showcase its capabilities. These resources cover topics such as graph databases for biological data models, neo4j backup via SSH, Brexit on Twitter, DAM Champ, importing RDF data into Neo4j, and more. The community also shares its experiences with using Neo4j in various industries, including finance and media. Additionally, the community is promoting GraphConnect San Francisco, a conference that brings together graph technology enthusiasts and experts for talks, demos, and hacking sessions.

The Royal Bank of Scotland (RBS) has implemented Neo4j to improve its software deployment process, addressing challenges with legacy deployment processes and integration in dependency management. The bank's goal was to achieve full flexibility and control over their deployment process while supporting a business model that includes cloud deployment. By using Neo4j, RBS can model their data as an entity-relationship diagram, making it easier to change the data model on a continuing basis. The tool also provides a true state query, allowing users to determine what was deployed where and when, and enables early warning systems for upgrading code libraries. This has resulted in increased developer productivity and is being rolled out to all RBS Group organizations, aiming to achieve hundreds of successful deployments per day.

The world of talent management is undergoing significant changes, driven by the shift towards more agile and networked organizational structures. The rise of the "network of teams" challenges traditional top-down management structures and leadership development, succession planning, and talent retention. To address these challenges, companies are increasingly relying on people analytics models that use data from existing HR, talent, and performance management applications to produce valuable insights into what makes people tick and how to attract top talent. However, most of these concepts are not new, and the key issue lies in deriving insights from the vast amount of available people data in a way that is efficient, cost-effective, and intuitive. The solution lies in recognizing the wealth of relationships that exist naturally within the data, organizing it as a graph, and making relationships first-class citizens. This approach enables companies to find hidden potential, retain talent, and differentiate themselves from competitors by extracting valuable insights from connected data.

Neo4j 3.0 introduces a new feature called user-defined procedures, which allow developers to extend the Cypher query language with custom functionality. These procedures can be used to add new capabilities and improve performance in graph algorithms. The APOC procedure library provides a collection of pre-built procedures that cover various areas such as graph algorithms, metadata, and data integration. With APOC, developers can easily create custom procedures using Java or other JVM-based languages, making it easier to integrate Neo4j with other databases and tools. The library also includes features like metadata capabilities, date and time functions, and export procedures for exporting database data in Cypher format.

The company behind Graph Everywhere, Jesus Llevadias, built a business around graph databases after noticing a gap between customer data and how it was being used. They chose Neo4j as the primary tool due to its ACID compliance and ability to handle complex relationships in customer data. By leveraging Neo4j, customers were able to simplify their data models and find connections more easily. The company carefully considered introducing new technology to avoid unnecessary complexity, ultimately concluding that Neo4j was the best fit for their customers' needs.

The py2neo Object-Graph Mapper (OGM) is a framework for building database-aware domain objects on top of graph databases like Neo4j, similar to how an Object-Relational Mapper (ORM) works with traditional relational databases. The OGM centres around the `GraphObject` class, which serves as both a base class and a manager for underlying node and relationships. Domain objects can be defined using this class, and properties are mapped to the corresponding nodes and relationships in the graph database. The framework also supports related objects through the use of `RelatedTo` and `RelatedFrom` attribute types, which define sets of related objects connected by specific relationship types. With py2neo OGM, developers can work with these objects using a set-like interface, adding and removing related objects, and pushing changes back into the graph database, where the framework automatically builds and runs necessary Cypher queries to make this happen. The framework also supports more complex selections through the `select` method, allowing for querying of nodes based on various conditions.

The GraphConnect San Francisco conference is a gathering of over 1,000 graph database enthusiasts, taking place on October 13-14. The event invites participants to share their experiences and expertise through lightning talks and full sessions, covering both business and technical aspects of graph technology. The Call for Papers remains open until July 31st, allowing individuals to submit their proposals at https://graphconnect.com/. Registration is now available for attendees who wish to connect with the world's leading graph professionals and participate in presentations, talks, and workshops.

The Neo4j-JDBC driver version 3.0 is now available, developed primarily by Italian partner Larus BA from Venice. This new driver aims to provide a standard library for integrating Neo4j with other data sources and tools, such as business intelligence suites, ETL tools, and specific system components. The JDBC standard has evolved beyond database access for Java developers to a general integration component that allows tools of all kinds to interact with trusted databases. The new driver supports authentication, connection strings, result metadata, and profiling, among other aspects. It also builds upon the HTTP endpoints and was initially started in December 2011 by Rickard Öberg. The development of this new JDBC driver from scratch using the latest APIs and technologies was driven by requirements from customers to integrate Neo4j with Pentaho, JasperReports, Talend, and QlikView/Qlik Sense. The driver is now available for download, and users can configure and use it with various tools such as Squirrel SQL, Pentaho, TIQ JDBC Connector, Tableau via jdbc2tde, Talend, JasperReports, BIRT, and APOC procedures library.

Yes, I'm talking about a digital road trip. This summer I will rock the world of healthcare using Neo4j to see how each group of stakeholders connect to each other. The healthcare industry involves a wide range of stakeholders from providers to drug manufacturers to legislators as well as patients, making it one of the most complicated industries in the U.S. The author's journey started with a whiteboard discussion about who plays a role in the industry and how they relate to each other. They drew a graph representing the procedures of a clinical encounter between a patient and a provider on one side and the relationships among different stakeholders on the other. The author is interested in tracking prescription drug abuse history by states, hospitals, and providers, as well as Medicare Part D costs by drugs, providers, and states. To model this data in a graph database, they drew out the general graph of the system, identified essential data related to drugs, manufacturers, providers, and lobbyists, and documented what was available through various public datasets. They then refined their model by answering two questions: whether two tables contain columns with shared values and which columns can be extracted as independent nodes. Finally, they created a graph schema in Neo4j, using the Arrows tool to visualize the data structure, and planned to ETL the data into Neo4j next week. The author has linked their project's GitHub repository for further information.

The graph database phenomenon has been around for a while and originated from wanting to use data in a different way, particularly in areas such as Google, social webs, CRM, and marketing. The growth of the Internet of Things (IoT) is expected to increase the need for graph databases due to the vast amount of data being generated by connected devices. Smart farming is one area where graphs can be applied to analyze relationships between crops, soil, and weather patterns. As IoT continues to grow, we're moving towards analytics of things (AoT), which relies on exploring relationships between everything, leading naturally to graph databases like Neo4j. For large-scale data sharing among multiple entities, open-source databases like Neo4j are becoming increasingly popular, as they offer the right characteristics for collaboration and data analysis.