March 2017 Summaries

Peter and his team at Cisco were struggling to find patterns in customer support case notes, which led them to explore graph databases like Neo4j. They loaded their daily case notes into Neo4j, including metadata such as team members and work groups, to uncover insights that helped reduce hardware return requests. The database's query capabilities allowed for faster data analysis than relational databases, and the team discovered that bug IDs often indicated software-related issues rather than hardware problems. By surfacing frequently mentioned bug IDs, Cisco was able to address avoidable Return Merchandise Authorization (RMA) problems and save money. Neo4j proved to be a valuable tool in their services organization.

Anders Ekström from NetConsult, a certified Neo4j solution partner in Sweden, shares the story of how Hästens, a family-run mattress company, optimized its business performance through digitalization using Neo4j. Hästens aimed to drive all traffic to stores and meet customers where they were, both online and offline. The company's data infrastructure was complex, with numerous silos containing disconnected data, which NetConsult integrated into a single graph using Neo4j. This enabled Hästens to leverage its data in new ways and add value. With Neo4j, Hästens created Stable, an administration interface for all the data, and Polo, a lightweight content management system integrated with Akamai's content delivery network. The technology also facilitated the digitization of the catalogue, marketing automation, and the rollout of a new web platform, ultimately improving customer experience and driving business growth.

Neo4j's latest release has revamped its security engine and surface, catering to demanding enterprises that value both productivity benefits of graph data models and robust security measures. The Enterprise Edition supports multiple users and introduces a role-based access control framework with four predefined roles: reader, publisher, architect, and admin, each granting specific privileges on the Neo4j data graph and schema. This release also includes pre-built support for three authentication and authorization options: native users repository, LDAP connector to Active Directory or openLDAP, and custom authentication/authorization provider plugins. By implementing these features, Neo4j Enterprise Edition aims to provide a secure environment for organizations to manage their data, meet compliance requirements, and ensure easy access to authorized users under their existing security regime.

This week in Neo4j covers various topics, including the latest community member featured, Johannes Unterstein, who showcased how to set up a Neo4j causal cluster on DC/OS. Graph visualization was also explored in posts about investigating corruption in Egypt and FIFA using tools like Linkurious and Tom Sawyer graph data visualization. Additionally, there were updates on visual graph modeling and importing, as well as new features and plugins for popular IDEs and programming languages. The podcast featured an interview with Alistair Jones about Neo4j's causal clustering feature, while upcoming events include a meetup in London and online sessions on building the Neo4j sandbox cloud trial environment and exploring the Bitcoin blockchain.

There are powerful advantages to using native graph databases, which are optimized to store and process graph data, allowing them to perform queries faster, scale bigger, and run more efficiently. FactGem chose to build their platform on top of Neo4j due to its power in graph technology and its ability to remove initial barriers of entry for enterprises. By leveraging Neo4j's native property graph capabilities, users can model their own data intuitively through the Neo4j Browser and use drag-and-drop features to build out their dashboard and queries without needing to write Cypher queries or involve engineers. The support team is fantastic, the staff are very accessible, and the documentation is great, making it easy to get started with Neo4j. By working with Neo4j, FactGem has discovered interesting results such as consolidating data from silos into a unified fabric, eliminating duplicative information, and performing calculations more accurately and efficiently. If starting over, Clark Richey would prioritize full training for customers sooner, emphasizing the importance of engaging with Neo4j training and onboarding to learn how to do things right.

The benefits of using a graph-powered software analytics tool include its ability to model software as a connected, extensible, and natural data structure. This allows for the exploration of existing structures through queries and the validation of applications on the fly using abstract concepts and rules defined from design and architecture languages. The tool can also provide a holistic view of the code structure and system architecture, enabling the identification of errors, inconsistencies, and potential issues before they become problems.

Graph databases can be used by companies to leverage business relationships and obtain new customers. Many companies struggle to gain insights from their customer relationship management (CRM) systems, but graph databases can help make this data more accessible. By using graph databases, companies can navigate complex business leader social networks to find new opportunities for sales and marketing. The company Kantwert has developed a European business and decision-maker network, which includes all companies and people within those companies, allowing them to show how these groups are interlinked and dependent on each other. This data is used to provide insights into the effectiveness of sales activities and to identify potential fraud patterns. Kantwert uses Neo4j as its graph database, combined with Cassandra for fast write operations and Apache Solr for searching personal and company data. The company has landed major clients such as Deutsche Bank and Commerzbank by providing them with valuable insights from their network data. Graph databases offer a new perspective on the same data, allowing companies to find new opportunities and gain a competitive edge in the market.

We are exploring how to manage and update large amounts of citizen data, which is estimated to be around 44 Zettabytes in size. To handle this massive volume of unstructured data, we have NoSQL solutions, big data computing platforms such as Hadoop and Spark, and advanced search engines like Neo4j, a graph database that can efficiently manage complex relationships between connected devices. We've been using citizen data for almost 20 years and found it challenging to meet analytic requirements with traditional databases, but graph databases helped us solve issues like ancestral trees online in real-time and system-wide searches of the entire database. Despite Neo4j's capabilities, there are still dependencies that prevent a full migration to the platform, such as character encoding problems and special characters. The report also highlights trends in database technology, with relational databases still dominant but graph databases gaining traction. To overcome data integration challenges, we need a complete solution that is declarative, built-in change capture mechanisms for popular RDBMS products, and a system that can recover from failure, scale for enterprise online transaction processing, and parallelize for high-throughput.

Neo4j is being used in various applications, including a WordPress recommendation engine built by Adam Cowley. The platform also hosts several interesting projects on GitHub, such as graphviz-config-template, which converts dot syntax into Cypher queries, and fec-2016-neo4j, which loads US Elections, campaign finance, and US Congress data into Neo4j. Upcoming events include presentations on Analyzing The TrumpWorld Graph at NYC Neo4j and an intro to graph databases at pyladies Seattle. The Neo4j community is active, with users sharing their projects and experiences on Twitter, including a tweet from Tobias Zander who encourages others to share their Neo4j-related fun projects.

The company Pitney Bowes has partnered with Neo4j to leverage graph technology for effective fraud detection analysis. The use of a graph database allows for the uncovering of hidden data relationships, enabling more accurate and efficient analysis. Pitney Bowes' Spectrum Technology Platform uses Neo4j to provide a single view of customer information, supporting multiple organizations in integrating, transforming, and consolidating their data. This approach brings value to market quickly and provides contextually rich insights. The partnership with Neo4j was chosen due to its flexibility in handling complex data connections, allowing for accurate patterns analysis and gaining valuable insights from connected data. A notable project involved pattern analysis of transactions to identify large-scale fraud structures, resulting in early detection and prevention. Looking back, Pitney Bowes wishes they had adopted Neo4j sooner, as it would have allowed them to recognize the full potential of graph technology earlier and drive more value for their customers.

The Graph Database Support plugin for JetBrains IDEs provides extensive support for Neo4j, a popular graph database. The plugin offers features such as syntax highlighting and error reporting for Cypher, the query language used in Neo4j. It also includes autocompletion, refactoring, reformatting, and user-defined function and procedure support. Additionally, the plugin provides documentation, language injection, data sources, and console output for improved development experience. With over 15 months of updates and more than 16,000 downloads, this plugin is a valuable tool for developers working with Neo4j.

Neo4j, an open-source graph database, is now available as a package on Mesosphere's DC/OS platform, allowing users to easily deploy and manage the database across large clusters. The joint effort between Johannes, a Neo4j user and community contributor, and Michael, who joined Mesosphere in Hamburg, made this possible by publishing Neo4j as a package for running on DC/OS using its Marathon scheduler. This simplifies the process of deploying Neo4j and allows users to take advantage of its scalability features without requiring custom scheduling. The release of Neo4j 3.1 introduced a new clustering infrastructure called Causal Clustering, which consists of core servers accepting quorum writes via a consensus commit protocol and read replicas serving read requests. DC/OS provides the necessary tools for deploying and managing Neo4j clusters, including Marathon's Resource Configuration, persistent data storage, and access to the database from outside the cluster using an HTTP proxy. The setup process involves installing a DC/OS environment, launching the Neo4j services either through the Universe UI or CLI, scaling the cluster, and deploying example applications such as a Twitter-like setup.

Neo4j has been busy with releases, including new versions of the Java and .NET drivers, a new release of APOC with many useful features such as date functions, path procedures, and improved periodic iterate functionality. The Neo4j Grails plugin also saw updates, while Microsoft released its Concept Graph library for Neo4j. In other news, Brock Tibert created a Docker image for Neo4j that includes APOC and JDBC drivers, Tomasz Bratanic showed how to connect Neo4j to Elastic using APOC, Markus Harrer demonstrated the use of jQAssistant and Pandas to analyse codebases, and Gabriel de Maeztu introduced neo4jupyter, a tool for visualizing Neo4j queries in Jupyter notebooks. Additionally, Rob Schoening released mercator, a tool for analysing physical, virtual and cloud infrastructure, while Samathy Barratt shared her slides from her talk on graph databases with Neo4j. Meanwhile, Cristina Escalante demonstrated how to use the Microsoft Concept Graph with Neo4j, and Michael Hunger created a collection of useful Cypher tips and queries. Various meetups and events are also scheduled for next week, including talks by Carlos Justiniano, Mike Morley, Dave Bennett, Johannes Unterstein, and others.

The company Cablevisión uses Neo4j to provide seamless cable services to customers by detecting and preventing system failures, leveraging its graph database capabilities. They use Neo4j with Docker, Apache Kafka, Spark, and other technologies to support their ecosystem, taking advantage of Cypher's powerful query language for fast data analysis. The company has implemented various Neo4j-based solutions, including fraud detection, rapid network failure detection, and graph recommendation engines, showcasing its design-centric approach to graph data modeling.

Neo4j, a native graph database, outperforms non-native graph databases in various benchmarks and scenarios. This is due to its efficient data structures and traversal algorithms, which are optimized for graph workloads. The database's storage engine and pointer chasing implementation also contribute to its performance advantages. Non-native graph databases, on the other hand, suffer from global indexes and expensive I/O operations, leading to slower traversals and higher costs. Recent advancements in non-volatile RAM and co-processors have opened up opportunities for Neo4j to further optimize its performance and scalability. Johan, Neo4j's CTO, has led benchmarking efforts that demonstrate the database's capabilities, including handling large graphs and high-performance queries. The author notes that Neo4j's native graph technology allows it to scale efficiently on modest hardware, making it an attractive choice for graph workloads. As hardware trends continue to evolve, Neo4j is well-positioned to take advantage of these advancements and ensure its efficiency and performance only improve over time.

The Microsoft Concept Graph is a research project that aims to build better search engines, spell-checkers, and recommendation systems by harnessing the power of web pages and search logs. The graph represents concepts (such as "fruit" or "company") and their instances (such as "apple" or "pie") using weighted IS_A relationships, which indicate the probability of an instance belonging to a concept. The dataset used in this project contains 5,376,526 unique concepts, 12,501,527 unique instances, and 85,101,174 IS_A relations. The graph can be imported into Neo4j using the `neo4j-import` tool and queried using Cypher queries. By analyzing the relationships between instances and concepts, researchers can gain insights into human knowledge of categories and develop more accurate models for natural language processing tasks.

The Neo4j community is actively contributing to the project, with a focus on improving the user experience and providing helpful resources. Florent Biville, a long-time community member, shared his experience building the Neo4j Procedure Compiler, which helps users identify common errors in their procedures. The Neo4j-Users Slack channel has reached 4,000 members, making it a valuable resource for users to get help with their Neo4j journey. Additionally, there are various meetups and presentations happening across different locations, covering topics such as data analysis, graph databases, and relational databases. The community is also using Neo4j to analyze social networks and explore its potential in various industries. Overall, the Neo4j community is thriving and actively contributing to the project's growth and success.

Choosing the right database for the right use case is critical, as it allows data to reach its highest potential. Miroculus, a molecular diagnostics company, uses Neo4j to manage large volumes of scientific data related to microRNAs. They chose Neo4j due to its performance and scalability, as well as its ability to handle complex relationships between data points. Since implementing Neo4j, the company has gained valuable insights into their data, demonstrating the platform's effectiveness in handling high-volume data storage and querying. If they could start over again, Miroculus would have integrated Neo4j earlier in their process. Overall, graph databases like Neo4j are well-suited for companies with complex relationship-based data, making it a recommended tool for similar use cases.

Neo4j 3.0 introduced user-defined procedures, while Neo4j 3.1 added user-defined functions, and Neo4j 3.2 will support custom aggregate functions, enabling direct extension of Cypher. These additions allow for easier interaction with other data stores, enhanced import/export functionalities, and implementation of graph algorithms. The Neo4j Procedure Compiler helps write safer code by detecting common errors before deployment, and it has been integrated into Neo4j 3.1.0. The compiler checks for various rule violations, such as proper parameter annotations, return types, and field access methods, providing detailed error feedback for invalid procedures or functions. Enabling the compiler is straightforward, requiring only adding it to the classpath, and it can be used with Maven or Gradle dependencies.

The food industry's lack of transparency in its supply chains has significant consequences, including high recall costs and damaged brand images. A survey found that only 12% of consumers trust the safety of their food, while 90% want to know what's in their food. To address this issue, Trace One developed a social network called Transparency-One, which allows brands to map their supply chains from farm to fork, track ingredients and suppliers, and ensure compliance with certifications. This technology is particularly useful for companies like pizza manufacturers who have complex supply chains with multiple levels of ingredients. In contrast to traditional SQL databases, Neo4j's graph database technology provides a more efficient way to query and analyze supply chain data, allowing brands to build strong consumer trust through transparency and sustainability.