May 2017 Summaries

To deploy Neo4j in the Azure Marketplace, you need to have an Azure account. At the time of this writing, there is a $200 promotion for signing up for a free account. Once logged in to your Microsoft Azure account, you can search for the Neo4j Enterprise Edition template by Neo Technology, Inc. and click on "Continue" to proceed. The deployment process involves configuring basic settings, such as creating an account to access provisioned compute instances and selecting the region where cloud resources will be deployed. Additionally, you need to configure Neo4j settings, including choosing the version of Neo4j to deploy, providing an initial password for the DBMS admin user, and selecting the Azure VM within which to run Neo4j. The deployment process also involves defining a virtual network, specifying the subnet for clustered VMs, and agreeing to the licensing agreement. Once the license agreement is accepted, the Azure Marketplace will provision all required resources defined in the template, including VMs, private and public IP addresses, load balancers, etc. After successful deployment, you can access Neo4j using its public IP address and connect to it using the Bolt URI or SSH.

Neo4j is being used by financial services enterprises to effectively solve emerging challenges in the industry, including data lineage and metadata management. Tracing data lineage across systems and time is a significant challenge, but Neo4j helps model data lineage and data flows as a metadata graph to get a complete understanding of data and systems. A Global 500 financial services firm built an integrated data distribution platform using Neo4j, which included a knowledge base that described the lineage of datasets and attributes from master systems to consumers. The platform addressed challenges such as coverage and flexibility, allowing it to accommodate new data sources and users. By leveraging Neo4j's flexible and responsive nature, these financial services enterprises can better respond to regulatory compliance and competitive advantages.

This week in Neo4j highlights the contributions of community members, new developments and updates to the graph database. Florin Pătraşcu is featured as a community member who has been creating drivers for the HTTP and Bolt protocols, and porting the movies starter kit. The CEO Emil Eifrem discusses graph databases and RDF, while Jesús Barrasa releases a new version of neosemantics, a set of experiments that bridge graphs and RDF. The Neo4j team attended the first GraphQL Europe conference, where they ran a GraphQL hackathon. Additionally, there are updates on various projects such as Max De Marzi's flight search model, Intermine's experience with GraphConnect Europe, and Greg Walker's bitcoin-to-neo4j project. The summary also includes news from the Neo4j Knowledge Base and upcoming events, including a workshop on using Neo4j for documentation projects.

Pitney Bowes, a software company, leveraged Neo4j to develop an enterprise master data management solution that differentiates them from other major vendors in the market. They use Neo4j as the primary repository for their master data management solution, Spectrum, which is part of a larger information management product. The company's decision to work with Neo4j was influenced by its compatibility with Java, support for multi-platform asset compliance, and being the market leader. By using Neo4j, Pitney Bowes' customers can benefit from improved marketing, sales, and customer support processes, as well as enhanced anti-money laundering and knowledge management capabilities. The company has seen exciting results, including integrating Neo4j into their entire platform and developing a graph-based solution for metadata management and data lineage analysis.

The Neo4j-graphql project is a Neo4j server extension that enables users to turn GraphQL queries into Cypher and run them against their graph data. The project allows for Cypher to be embedded in the GraphQL schema, enabling mapping between a GraphQL field and the results of a Cypher query. The Neo4j-graphql tool can quickly spin up a Neo4j-backed GraphQL API using a user-defined GraphQL schema, optionally annotated with Cypher queries. This integration enables powerful graph queries and exposes the power of Cypher through GraphQL. The project has been presented at the GraphQL Europe conference and organized a hackathon based on the GraphQL Community Graph, which exposes data from the GraphQL community, including GitHub projects, Stack Overflow questions and answers, tweets, and meetups related to GraphQL. The hackathon received many interesting submissions, and the first 15 valid submissions will receive a pair of Apple AirPods as a gift card.

Graph databases are helping financial services firms gain a competitive advantage by providing a clear understanding of relationships among different kinds of financial assets, enabling better risk management and regulatory compliance. The use of financial asset graphs with Neo4j allows firms to drill down to the root asset and understand the true exposure, rather than being veiled by layers above. This results in more efficient and accurate risk assessment, as well as improved customer confidence in data. Firms like Cerved are using financial asset graphs to identify real owners of businesses, track ownership links, and promote graph database uses in other areas of their company. The use of Neo4j has also improved the application's efficiency, reducing calculation time from 12 seconds to 67 milliseconds. By leveraging financial asset graphs, forward-looking firms can secure a competitive advantage in managing interdependent assets and maintaining granular risk management and regulatory compliance.

Machine learning and graph technology have revolutionized medical research, particularly in the field of cancer. MicroRNAs, small non-coding RNAs that play a crucial role in cell differentiation and regulation, have emerged as promising biomarkers for diagnosing diseases. By analyzing microRNA expression patterns, researchers can identify specific types of cancer, including stomach cancer, which is one of the deadliest forms of cancer globally. A machine learning model trained on large datasets of microRNA-expression profiles has shown high accuracy in predicting gastric cancer, with an area under the curve of 0.8. This technology has the potential to improve diagnosis and treatment outcomes by detecting diseases at early stages, reducing mortality rates, and alleviating congested endoscopy services. The approach leverages graph technology to analyze vast amounts of medical literature, extract relationships between microRNAs, genes, and diseases, and build predictive models for disease diagnosis.

This week in Neo4j highlights various community members, events, and developments in the world of graph databases. Ben Nussbaum, CTO of AtomRain, is featured as a community member who has been actively involved with Neo4j for five years. The Neo4j GraphQL Community Graph Hackathon has started, providing a platform for developers to build applications using the Apollo GraphQL client. Several individuals have shared their experiences and knowledge on using Neo4j for analyzing data from Salesforce, Git repositories, and creating automated menu planning algorithms. IBM released the results of their State of graph databases survey, which provides insights into why people are using graph databases and what they plan to use them for in the future. Additionally, Airbnb has developed a novel data resource search tool called Dataportal, and Neo4j is now available on both AWS and Azure Marketplaces. The Graphistania podcast featured an interview with Darko Križić, CTO of PRODYNA, discussing his experience with graph databases and Cypher query language.

Telegraph Hill Software, a San Francisco-based company that provides outsourced product development services and staff augmentation to SaaS startups, successfully deployed Neo4j to develop a Software-as-a-Service (SaaS) solution for their customers. They used Neo4j to manage complex interdependencies of components in cloud and virtual environments, including modeling social networks and IT cloud infrastructure at scale. The team found Neo4j's graph database to be the perfect repository match for this data structure, with a minimal training curve and high extensibility, allowing them to easily extend and enrich their entities without significant DevOps workforce. They attribute cost efficiency as one of the key benefits of using Neo4j for this application. Looking back, they would have made more effort to highlight Neo4j's unique features and adopted it without hesitation if they had the right repository data structure from the start.

The Hierarchy Management Platform (HMP) is a graph-based system developed by Cisco's enterprise data services organization to unify and simplify hierarchies across various business domains. The platform uses Neo4j as its underlying graph database, which allows for the creation of a "decoder ring" that maps one thing to another, enabling translation from cryptic to sensible information. HMP has been instrumental in automating processes, improving data sharing, and enhancing decision-making across teams by providing a unified view of hierarchies and relationships between them. The platform has been successfully applied in various domains, including product development, finance, sales, and customer service, leading to increased efficiency, reduced errors, and improved business outcomes. Key takeaways from the HMP implementation include the importance of organizational agnosticism, robust API layering, crowdsourcing mechanisms, and trust-based governance in unlocking the full potential of graph databases for the connected enterprise.

The text discusses knowledge architecture and its importance in extracting valuable insights from large datasets. It highlights NASA's experience with transforming their documents into a knowledge graph, which enables interactive visualizations and facilitates decision-making. The author emphasizes the benefits of knowledge architecture, including increased productivity, improved collaboration, and enhanced data-driven decision-making. They also share examples of how this approach can be applied in various industries to improve business outcomes. By combining knowledge management, informatics, and data science, organizations can unlock new insights from their data and make informed decisions.

Neo4j has been actively engaged in the graph database community, with notable achievements including hosting a data journalism focused hackathon and its 4th GraphConnect Europe conference at the QE2 Centre in Westminster, London. Luanne Misquitta, a long-time Neo4j community member, was featured as this week's community member, having organized Neo4j training in India and contributing to various projects including Spring Data Neo4j. Recent news includes the release of Neo4j version 3.2, with users advised to apply a configuration tweak for APOC users, and the creation of new tools such as neo4j-microservice and pheno4j, a graph-based HPO to NGS database. The community has also seen notable events like GraphQL-Europe, where Michael Hunger will showcase Neo4j as the backend of GraphQL applications, and the release of a 2014 interview with Dr Jim Webber and Ian Robinson about graph modeling on Twitter.

Neo4j's CEO, Emil Eifrem, has announced a data journalism fellowship program to complement the company's existing Data Journalism Accelerator Program. The inaugural year of the Neo4j Connected Data Fellowship will be piloted through the International Consortium of Investigative Journalists (ICIJ), with the ICIJ maintaining complete editorial independence and control over the fellow's work and topics covered. The six-month fellowship, sponsored by Neo4j, aims to support a data journalist or programmer in making sense of complex data and finding stories within networks, favoring candidates from diverse backgrounds. Applications for the program are currently open, with a deadline of June 1.

Neo4j 3.2 introduces multi-data center support for Enterprise Edition customers, enabling them to run their Internet-scale applications across continental data centers. This feature allows for locality-affined load balancing and minimizes the expense of time-consuming updates across WANs. The release also includes improved performance features such as a newly written native label index, composite exact indexes, compiled Cypher runtime, and optimized depth queries in Cypher. Additionally, Neo4j 3.2 introduces support for Kerberos authentication, Node Keys, query monitoring and administration, RPM packages, cloud deployment options, and improved Causal Cluster-Aware Drivers API functions. The release is designed to make Neo4j more enterprise-ready and globally capable, with significant boosts in speed and functionality.

Neo4j is a native graph database that stores data as graphs, enabling real-time traversal and execution of queries in milliseconds. Using Neo4j with Microsoft Azure provides an ideal solution for organizations seeking to leverage cross-domain intelligence and address evolving data challenges with speed and agility. This integrated solution enables the deployment of intelligent applications in areas such as graph-based search, fraud detection, and recommendation engines, among others. The Neo4j High Availability Cluster Template in the Azure Marketplace allows users to quickly deploy a graph environment for testing and evaluation purposes, while also providing access to various resources and support options, including a public Discord group, Stack Overflow channel, and YouTube content.

Fake news is spreading rapidly on social media, making it essential for social networking sites to detect and prevent its spread. To achieve this, graph analysis and visualization techniques can be employed to understand how fake news spreads online. A comprehensive fake news detection process involves understanding networks and utilizing Neo4j and the KeyLines graph visualization toolkit. The approach combines automated detection using a Neo4j graph database with manual investigation powered by a KeyLines graph visualization tool. This hybrid approach helps identify high-risk content and accounts, detects abnormal behavior, and uncovers new behaviors that may indicate fake news. By leveraging graph databases and visualization tools, social networking sites can efficiently detect and prevent the spread of fake news, protecting users from misinformation.

This year's GraphConnect Hackathon will take place on May 10th, 2017, at CodeNode London, focusing on data journalism with datasets from various open sources. The event will feature hackers competing in groups of up to five people to build the best Neo4j-powered application and winning prizes across different categories. The hackathon is open to Graphistas of all levels, offering a mix of hacking, food, drinks, and community interaction, with a specific agenda that includes welcome, demo nights, and prize distribution.

This week in Neo4j highlights the achievements of community members, showcasing projects such as Alessio De Angelis' graph-based entry in a competition and Christophe Willemsen's creation of a tool that executes Cypher queries using Amazon Alexa. The community also explores rare diseases research, APOC spatial, Twitter clone development, and online meetups for planning hikes with Neo4j. The Neo4j Knowledge Base provides an article on improving query performance, while the podcast features interviews with featured community members and new employees. Various projects are showcased on GitHub, including graph isomorphisms, visualization, natural language processing, and importing survey data into Neo4j. The week concludes with a reminder of GraphConnect Europe 2017 taking place in London on May 11th, featuring speakers who will give sneak peeks of their talks.

The development of search infrastructure that provides relevant information to users involves the creation of a knowledge graph, which is a multi-relational graph composed of entities as nodes and relationships as edges with different types. The knowledge graph model is designed to handle highly heterogeneous data in terms of sources, schema, volume, and speed of generation. Natural Language Processing (NLP) plays an important role in extracting "knowledge" from large datasets. The search architecture must be able to navigate this data in real-time, providing efficient ways for users to access the information they need. A relevant search application is built on top of a knowledge graph, which includes features such as text extraction and NLP, user modeling and recommendation engines, context information, and business goals. The infrastructure consists of a Neo4j database, an Elasticsearch cluster, and Apache Kafka, which work together to provide real-time data processing and storage. Neo4j stores the entire knowledge graph on which all searches and navigations are performed, while Elasticsearch provides advanced text search capabilities and faceting.

The Neo4j database, although not requiring a schema, often adheres to one, making it difficult to create efficient queries without understanding the underlying structure. The built-in Neo4j Browser can display node labels, relationship types, and property keys but lacks visualization capabilities for non-trivial schemata. A Cypher query called `db.schema()` provides virtual entities that do not exist in the database, allowing users to explore the schema manually. However, this method has limitations and is not suitable for complex databases. The yFiles library can be used to build a custom schema viewer that automates the manual tasks involved in understanding the Neo4j graph database structure. A demo application was created using yFiles for HTML, which includes features such as filters, master detail views, dynamic layouts, and interactive node creation, making it easier to explore and understand complex Neo4j instances.

The University of Washington (UW) is implementing a large Software-as-a-Service solution for HR and payroll, replacing its 30+ year-old system. To facilitate this change, the university created Knowledge Navigator (KN), a metadata repository that provides a web-based, interactive platform to understand data relationships and changes between old and new systems. KN offers self-service access to conceptual and technical descriptions, definitions, lineage, and impact analysis information, enabling users to stay informed and engaged throughout the system migration. The solution is designed to provide a common understanding and expectations across the university, managing metadata to oversee changes while delivering trusted, secure data in complex environments. By focusing on a specific problem and using a simple data model, KN stands out among metadata repositories with its unique ability to communicate about changes to end-users, illustrating parallels between old and new systems through interactive diagrams and explaining new concepts and definitions.

The developer community can now create a simple blogging app within a couple of hours using the new tutorial on how to create a Structr app, which showcases several new features. These include a deployment tool that allows exporting a complete application in HTML and JSON files for version control systems, a web-based configuration tool with individual service management, and improved productivity features such as an administration console and support for multiple scripting languages. The upcoming 2.1 release also includes improved test coverage, faster app development widgets, enhanced schema layout, and favourites for editable texts. Additionally, a Developer Support Program has been created to cover the most requested support services, and Structr is sponsoring GraphConnect Europe with a promo code offering discounts on tickets.

Neo4j was initially developed as a graph layer on top of a relational database to address the cognitive gap between how developers thought about data and how it was stored and queried. This helped improve productivity, but also led to ambition in querying complex data. However, this approach eventually resulted in the JOIN bomb problem due to the underlying relational database's limitations. To overcome this, Neo4j's founders built a native graph database that addressed scalability and consistency issues, leading to the development of Cypher, a powerful query language. Neo4j has since continued to evolve with advancements in clustering architectures, including causal clustering, which provides a graph-native solution for storing and querying large datasets at scale. The company aims to continue pushing the boundaries of graph technology and is excited about the future of its engineering evolution.