November 2015 Summaries

Graph databases are being used by global firms across various industries to improve performance, simplify development cycles, and innovate in surprising ways. These organizations leverage data relationships around common use cases such as healthcare, financial services, and telecommunications. The Internet of Things (IoT) also presents a significant opportunity for graph databases to handle the complex data relationships that arise from connected devices communicating with each other. Two approaches to developing applications with connected data are creating new products and services around it, which helps businesses adapt quickly to changing market conditions, and reimagining existing applications with data relationships to lower costs and boost performance. Graph databases offer a powerful tool for discovering and making sense of complex interdependencies and relationships, allowing businesses to gain a sustainable competitive advantage by extracting maximum value from data relationships.

The ICIJ uses Neo4j and Linkurious for investigative journalism, particularly in offshore finance cases. They chose Neo4j due to its speed and simplicity compared to SQL relational databases, which also allowed for a single developer deployment. Hamish Boland-Rudder notes that the graph visualization provided by Linkurious is intuitive and valuable for journalists, allowing them to understand complex relationships between data points. The most interesting results from using Neo4j and Linkurious include being able to see and report on nuanced relationships between nodes, such as attorney-client relationships. If starting over, Hamish would use Neo4j and Linkurious earlier in the investigation process, prioritizing their setup and utilization for better reporting outcomes. Overall, the tools have been a valuable asset for ICIJ's investigative work, enabling more effective data analysis and reporting.

Graph databases are becoming increasingly mainstream, and they offer a flexible model that can tolerate many different domains and data. The author argues that graphs are the supermodel, and any data can be modeled as a graph - whether it's documents, trees, hierarchies, or spreadsheets. Graphs already do multi-model, but in a way that's more efficient and scalable than traditional approaches. When it comes to analytics, graphs offer a natural fit for reactive and predictive analytics, using graph theory to gain insight into data. The author also discusses the trade-off between reliability and availability in distributed systems, and how Neo4j is addressing this challenge with its core-edge architecture, which includes RAFT for ensuring data safety and durability on disk.

Monitoring the performance of a Node.js application built on top of a graph database like Neo4j can be challenging due to its complex service infrastructure and asynchronous JavaScript elements. Ruxit Agent can help by automatically discovering service dependencies, monitoring load, response times, and visualizing the topology of applications in real-time. By installing Ruxit Agent on the host machine, developers can gain insights into how their application performs under load, identify bottlenecks, and optimize their Neo4j database for better performance. The tool provides a smartscape view that shows the service communication between the Node.js application and the underlying Neo4j browser UI requests, as well as user requests to the application's webpage. By analyzing this topology, developers can gain valuable insights into how their application is performing and make data-driven decisions to improve its performance.

Graph databases offer a competitive edge by unlocking business value from data relationships, which is no longer possible with traditional relational databases due to their limitations in modeling and storing complex relationships. Non-graph NoSQL databases also fall short as they lack data structures to model or store relationships. In contrast, graph database technology allows for the natural storage, management, analysis, and use of data within the context of relationships, enabling companies to harvest new market opportunities and reimagine existing applications to innovate with data relationships. The leading enterprise-grade graph database, Neo4j, is well-suited for delivering faster performance, creating new products and services, and adapting to changing business needs, making it an ideal choice for organizations seeking sustainable competitive advantage in the market.

The 5-Minute Interview: David Colebatch, Co-Founder of xnlogic` David Colebatch, co-founder of xnlogic, discusses his first Neo4j project with Monsanto, a large accomplishment that helped them achieve their goals. The company chose Neo4j for the client's project, and they were called in to explore and validate its effectiveness. Looking back, David advises that starting with graph modeling expertise can save time and lead to better results. Graph databases are poised to revolutionize IT operations management, making it easier to manage complex networks. With his experience, David emphasizes the importance of understanding graph technologies for successful application development.

Graph Commons is a collaborative platform that enables users to transform their data into interactive graph maps, facilitating investigative journalism, civic data research, and organizational analysis. The platform aims to make network mapping and analysis accessible to non-experts by providing an intuitive visual editor and cartography interface. With over 2,000 members using the platform for professional and activist work, Graph Commons has already seen a momentum of interest from people with varying interests and skill sets. Members use the platform to create unique projects such as mapping NGOs helping Syrian refugees, social movements, and climate targets, which showcase its potential to generate new knowledge and crucial discussions. The project "Networks of Dispossession" demonstrates the platform's capacity to clearly display indirect links and organic clusters of power, raising questions and becoming a reference point for the general public and activist communities. To further develop the platform, Graph Commons is working on building new mechanisms that enable collaboration and encouraging users to build off each other's mapping efforts, with plans to release improved community tools and qualitative graph analysis methods.

The author has implemented a Markov chain-based autocompletion system for Cypher keywords in their CLI tool cycli. The system uses a dictionary of tuples, where each tuple contains a keyword and its probability of being used next given the current keyword. The probabilities are calculated based on the order of appearance of each keyword in a dataset of Cypher queries scraped from the GraphGist wiki. The author has demonstrated how to use this data structure to provide more intelligent autocompletion suggestions for Cypher keywords, such as suggesting `WHERE` and `WITH` after a `MATCH` keyword, rather than just listing all possible keywords alphabetically. This approach is expected to improve the user experience of the cycli tool by providing more relevant and context-dependent suggestions.

In a problem-solving approach, a context graph is used to describe the current state of a system, and changes are made using microservices that modify this graph. The goal is to reach a target context by filling in gaps or differences between the initial and target contexts. This process involves creating and linking nodes representing objects such as HTML files, code files, and natural language text files to a tree-like structure, which can be understood as a graph. A solution to a problem is represented by a path of microservices that modify the context graph until it fulfills all constraints of the problem. The process can be compared to evolving a Document Object Model (DOM) graph used in web development, where each step builds on the previous one to reach the desired outcome.

Graph databases are gaining traction as a technology to leverage insights from connected data, offering a flexible and powerful way to store relationship information. Legacy relational database management systems (RDBMS) struggle with handling relationships between data points due to their rigid schemas and tabular data models. Graph databases effectively store these relationships and can adapt to new business requirements, making them a key technology for generating competitive advantage and significant business insights from connected data. Enterprises that tap into the power of graph databases realize significant advantages over those who rely on traditional relational databases. By harnessing the potential of their data relationships, organizations can derive actionable insights and gain a competitive edge in today's big data landscape.

The Neo4j team has announced the availability of Neo4j on IBM POWER8, a joint effort to advance graph processing and storage capabilities. This partnership provides an ultra-high-end solution for large-scale data processing and real-time graph queries, leveraging the power of IBM's POWER8 hardware. Neo4j now supports the entire range of IBM POWER8 Linux servers, including commodity servers with powerful and cost-efficient performance. The combination of Neo4j's native graph processing and storage with POWER8's in-memory vertical scalability enables massive-scale graphs to be stored and processed in real-time, overcoming scalability challenges. With the union of these two technologies, enterprises can tap into real-time graph processing that scales higher than ever before, delivering meaningful insights from data relationships.

Below are some top picks from our stellar community members, who share their expertise on various topics related to Neo4j. To be featured in December's "From the Community" blog post, follow us on Twitter and use the #Neo4j hashtag for your chance to get picked. Our community has published articles such as "How to become a data hero in Neo4j?" by Jarosław Pałka, which provides insights into how to leverage Neo4j for data analysis. Additionally, there are blogs like "Master Data Management and Why Graphs Matter" by Aaron Wallace, which highlights the importance of graph databases in managing complex data structures. Community members have also created videos such as "Building Web Apps Using Flask and Neo4j", which offers guidance on building web applications using Neo4j. The community is growing rapidly, with many contributors sharing their knowledge and expertise on various topics related to Neo4j, including graph-based recommendation engines, congressional page rank, and low common ancestors in trees and directed acyclic graphs. Join the community by downloading a free copy of "Learning Neo4j" and master the world's leading graph database.

The text discusses the challenges of choosing a suitable database for a project, particularly in relation to relational and NoSQL databases. It highlights the complexity of selecting a database type and the need for guidance on when to use each. The webinar will focus on Neo4j graph databases, explaining their benefits and use cases through real-world examples from logistics. Joachim Arrasz and Christian Mennerich, experts at synyx, will present the webinar, bringing together their experience in software development, system architecture, and database implementation. Synyx aims to improve software and software development, optimizing existing solutions and developing high-quality new software, while supporting companies in modernizing their critical applications.

The concept of building a universal problem-solving machine inspired by Alan Turing's design for a universal computer, has been revisited in the context of modern technologies and concepts. A new approach considers a human problem-solver or programmer as the starting point, with actions aimed at solving specific problems, procedures with defined inputs and outputs, and an associated cost and duration of execution. The problem to be solved is defined by an initial context, target contexts, constraints (budget, timeframe, limited instructions), and a list of possible solutions. A graph database, Neo4j, and its query language Cypher are being used to describe these contexts and find sequences of actions or services that transform the initial context into one of the desired contexts, with the ultimate goal of creating an automated market of Cypher-annotated microservices.

Search powered by a graph database enables users to receive more proactive and targeted search results, allowing them to quickly identify relevant information. Graph-based search takes into account the entire structure of available connected data, providing richer and more precise results compared to traditional keyword-based searches. This approach is more akin to a "conversation" with the data, focusing on search and discovery rather than retrieval. The use of graph databases in enterprises offers numerous benefits, including fast, accurate search results in real-time, flexibility to easily change data structure, and the ability to make advanced search-and-discovery possible. Companies like Google and Facebook have successfully implemented graph-based search services, resulting in increased user engagement and commercial success. Graph databases provide a better way to make large volumes of data available to users, offering competitive advantages such as improved customer experience, targeted content, and revenue opportunities. Enterprises that tap into the power of graph-based search today will be well ahead of their peers tomorrow.

This interview with Andrew Royer, an independent Neo4j hacker, showcases the versatility of the Neo4j graph database platform. Andrew is using Neo4j to scrape articles for keywords, naturally process them, and serve content to users based on feedback from users. He also utilizes Neo4j's capabilities in daily fantasy sports to conduct player analysis, making it an ideal choice for his project due to its ability to handle unstructured data and quickly generate recommendations. Andrew highlights the benefits of using Neo4j, including its graphical interface and simplicity in generating complex algorithms through Cypher queries. Despite having some initial challenges with scaling, Andrew is excited about the new release of Neo4j 2.3 and plans to stress test it for processing large datasets, such as a Twitter firehose.

We have recently added a new feature that allows users to analyze their personal Twitter data in Neo4j, a graph database. To enable this, we set up a Neo4j Docker container for each user running on Amazon's Elastic Container Service (ECS), with each instance allocated 1/4th of a CPU core and 768MB of memory. The dataset includes various types of Twitter data such as followers, tweets, mentions, and hashtags, which can be queried using example queries provided on the web app or in the Neo4j browser. A custom built-in browser guide is also available to help users navigate their Twitter data in Neo4j.

The Tokyo Neo4j User Group is hosting a meeting on November 9th, featuring Jim Webber, Chief Scientist of Neo Technology and co-author of the O'Reilly Graph Databases book, who will present in English with Japanese translation provided. The group also reviews big announcements from GraphConnect San Francisco, including the Neo4j 2.3 release and openCypher introduction. Additionally, there is a growing library of Japanese Neo4j books available, such as a new book on advanced Cypher queries by Changhwan Lee, which helps developers translate their SQL skills into Cypher for more advanced graph database queries. Creationline, the official distributor of Neo4j in Japan, also provides resources and support for Japanese developers, including a free copy of the O'Reilly Graph Databases ebook in Japanese translation.

The article explores the concept of time travel in the Back to the Future universe, using Neo4j as a tool to model and analyze it. The "forking" conception of timelines is used, where each time travel event creates a new timeline. The article discusses how this affects the behavior of time travelers, including the creation of ripples that capture them and move them between timelines. It also explores how to use Neo4j to create nodes for time travelers, events, places, and relationships between them, as well as Cypher queries to find ripple events. The article concludes by referencing a free ebook on graph databases to learn more about using Neo4j for complex challenges.