April 2017 Summaries

The Neo4j Developer Relations team has announced the opening of a dedicated Developer Zone at GraphConnect Europe 2017 in London on May 11th, where Neo4j engineers will be available to answer questions and discuss cool things about Neo4j. This week's featured community member is Michael Moussa, who has been active in the Neo4j community for quite some time, presenting introductions to Neo4j at multiple PHP conferences and contributing to PHP-related projects in the Neo4j community. The last few days of APOC Awareness Month are approaching, so if you haven't published your article yet, this is a good opportunity to do so. Nigel Small presented a Pythonic tour of Neo4j and the Cypher query language at the PyData conference, while Mostafa Moradian published gRest, a quickstart repository to build applications with Python, Flask, and NeoModel. The GraphConnect Europe 2017 schedule is now available as a graph, making it easy to view and navigate. Jesús Barrasa explained how to create the Wikipedia Knowledge Graph in Neo4j, while Rik van Bruggen published parts 2, 3, and 4 of his series on common questions about Neo4j. Igor Borojevic showed how to build a custom security plugin for Neo4j, while Chris Skardon set up a causal cluster with Neo4j on Microsoft Azure. Magnus Wallberg attended the PhUSE conference, where he compared RDF and graphs. Matt Andrews is hiring front-end/full-stack contractors to join Nikkei-FT project in London. Will Lyon analyzed local startup ecosystems based on their data using the Mattermark GraphQL API Graph. Fernando Izquierdo showed how to learn Chinese using Neo4j in this week's online meetup, while Christos Delivorias shared his experience with the Neo4j #Hackathon.

Graph technologies like Neo4j and Linkurious are ideal for managing change and complexity in complex environments, especially in enterprise architecture. Understanding and modeling interdependencies between entities is key to driving innovation and minimizing security and operational efficiency concerns. The traditional relational database approach has limitations when dealing with rapidly changing data, whereas graph technologies provide a natural advantage by using nodes and relationships to store data, offering a unified view of enterprise architecture, flexibility, and advanced analytics capabilities. Empowering enterprise architects with graph visualization and analysis tools can help them investigate data intuitively, monitor for anomalies, curate data in real-time, share reporting with decision-makers, and collaborate securely, ultimately building a scalable enterprise architecture.

Databridge, a bulk data import tool for Neo4j, has added the capability to import data into a running Neo4j instance, allowing users to perform fast bulk-load of initial large datasets while maintaining the database online. This new feature uses Neo4j's Bolt binary protocol and can be enabled by passing the `-s` option during import. The streaming endpoint is designed for small incremental updates and may take longer than the offline endpoint for large datasets. Additionally, Databridge now allows users to specify the output path to the database explicitly using the `-o` option, enabling the creation of new databases in different locations or co-existence of multiple datasets in a single physical database. New commands have been introduced, including `import`, `test`, `profile`, and `purge`, which offer improved simplicity and flexibility for users. These updates aim to support various use cases, such as importing large datasets into a running Neo4j instance, creating copies or deleting individual datasets, and managing the overall database structure.

Neo4j has been a key player in several high-profile data journalism investigations, including the Panama Papers and Brexit, due to its ability to handle complex relationships within large datasets. In response to these increased collaborations between journalists, GraphConnect Europe is hosting a Data Journalism Hackathon and a panel discussion on "Data Journalism in the Connected Age" featuring experts from major news organizations and investigative journalism groups. The event aims to provide insight into the challenges of modern data journalism, including issues of trust, collaboration, and handling large data leaks.

Neo4j Enterprise Edition provides tightly controlled access and permission to execute user-defined procedures, using custom security plugins and fine-grained access control. Authentication and authorization plugin interfaces support real-world deployment scenarios not covered by native users or built-in configuration-based LDAP connector. Custom-built plugins have access to the `neo4j.conf` configuration file and can write to the security event log, while a combined plugin provides both authentication and authorization in a single method. The feature allows mapping specific user-defined procedures with the list of roles that have explicit permission to execute it, enabling fine-grained access control. This is especially valuable if the procedure reads or modifies only a portion of the graph, allowing for sub-graph access based on custom roles. Authorizing roles to execute user-defined procedures is an evolving feature, and when deploying in production, it's recommended to follow standard database security best practices, including deploying Neo4j on a safe server, using secure authentication providers, protecting data-at-rest and in-transit, and managing access to logs and backups.

This week in Neo4j highlights the contributions of community members such as Dmitry Vrublevsky from Neueda Labs who has been actively involved in developing the Neo4j plugin for Jetbrains IDEs. The community is also exploring various use cases, including graph theory and call data records, with examples like the GMIT university timetable and analyzing CDRs. Additionally, new projects have emerged on GitHub, such as Memgraph and pcap4j, while jobs are being posted in the B2B cognitive music intelligence company Musimap. The upcoming events include a meetup on learning Chinese using Neo4j and presentations at NASA's JSC Data Science Day 2.0.

Neo4j is being used by multiple Fortune 100 financial services firms, major government institutions, and top-tier retailers to analyze complex fraud patterns in real-time. Traditional fraud detection methods using relational databases are no longer effective due to the increasing complexity of modern fraud schemes. Neo4j's graph database approach enables connected analysis of large amounts of data, allowing for the identification of fraudulent patterns quickly and accurately. This helps prevent bigger transactions from taking place, determines the origin of fraud rings, and operationalizes real-time pattern-matching events, making it a crucial component in fraud prevention architectures.

The GraphConnect Europe conference will feature keynotes by Neo4j's CEO, Emil Eifrem, and Chief Scientist, Jim Webber, covering the future of graph technology. The event includes three main learning tracks: Business Impact, Neo4j Deep Dive, and Solutions, which cater to executives, advanced developers, and those interested in common use cases of graph technology. Lightning talks will be hosted by speakers and sponsors showcasing the latest in Neo4j and graph technologies, while GraphClinics offer free consulting and troubleshooting services throughout the day. The event concludes with the GraphParty, an on-site networking event where attendees can relax and mingle with new contacts, old friends, and the Neo4j team over cocktails.

With the release of GORM for Neo4j 6.1, developers can now leverage the native features of Neo4j's Bolt Java Driver to build robust object mapping solutions. This tool allows users to model nodes, relationships, and paths, while also providing features such as dirty checking, batch writes, transaction management, and customizable ID generation. GORM for Neo4j is particularly well-suited for dynamic code, thanks to Groovy's ability to simplify and compile code on the fly, without requiring a statically defined schema. The tool also provides a rich query API and supports Cypher queries, allowing users to tap into the full power of Neo4j. With its focus on real-world challenges, GORM for Neo4j has been built in collaboration with Brinqa, a leading provider of unified risk management, to address their specific needs. A detailed tutorial is available to help developers get started with GORM for Neo4j and Grails.

The task delegation system being developed at Mativy uses Neo4j, a graph database that is well-suited for modeling and delegating tasks. The system is based on the "ant model", where tasks are split into smaller subtasks and delegated to individuals, with communication and synchronization between them. Each task has three mandatory elements: what, who, and when. The system allows for automatic notifications and tracking of task history, enabling managers to make informed decisions about user performance. A demo application called the "Forget Me Not" app is also being developed, which focuses on the synchronization cycle between two users. Neo4j's ability to store full histories of task lifecycles makes it an ideal choice for this type of process.

Neo4j 3.1 introduces increased control over database query management for administrators, allowing them to list and terminate currently running queries visible to users. This feature simplifies cluster topology management with the introduction of the Neo4j Browser widget, which lists all currently running queries in the cluster. Additionally, security event logging is introduced to record critical security and access control events, including login attempts, user management actions, and failed authentication attempts, providing a detailed log for administrators to review and customize log rotation settings.

Neo4j has been actively involved in several community projects and achievements this week. Featured community members include Ashley Sun and Rob Schoening from Lending Club's engineering team, who are working on open-source tools for managing service and operations information. The International Consortium of Investigative Journalists (ICIJ) won the Pulitzer Prize for explanatory reporting for their work on the Panama Papers, and Neo4j has released a sandbox to explore this data. Additionally, users have created interactive projects such as Star Wars Graph, which combines linked data sets from various APIs, and Twitter clone, Reddit Place Sensation, where participants collaborated to paint a virtual canvas. Other notable mentions include new guides, tutorials, and contributions to the Neo4j Knowledge Base, as well as user-submitted projects showcasing the power of Neo4j in various domains.

Neo4j is being used by Novartis' biomedical research team to mine huge volumes of biological data at record speeds. The team created a large graph combining heterogeneous biological data with text mining results, aiming to better understand biology and develop next-generation medicines. Neo4j's Cypher query language proved to be the key factor in adopting the technology, making it easy for users to formulate queries and get results. By bringing together vast amounts of biological data from various sources, Novartis can now navigate complex relationships between compounds, diseases, and annotations, unlocking new insights into medicine development. The team has already seen promising results and plans to scale up their database as they add more data, with the goal of testing Neo4j's scalability.

Neo4j has been used in the telecom industry for both churn reduction and event log data, but this application is at a business level that consumes services provided by the telecom provider. Call detail records (CDR) provide insights into telecommunications activity such as inbound calls, outbound calls, dropped calls, abandoned calls, and unanswered calls, helping organizations track details on extensions, toll fraud, and multiple locations, while also preventing productivity loss and controlling expenses. CDR data can reach big data proportions for some enterprise businesses, making it a challenge to update large tables in Microsoft SQL Server. A graph data model stores call detail records in nodes, relationships, and properties, with metadata setup to manage the data effectively. The solution utilizes Neo4j to process the data, providing orders of magnitude improvement in generating statistics on all call types compared to Microsoft SQL Server. This application can be used for various purposes such as employee utilization analysis, voicemail management, call distributions, and billing disputes verification. By modeling the CDR data with Neo4j, organizations can make better management and personnel decisions by analyzing patterns and trends, while also generating reports on money spent on calling minutes by specific departments or business units at different locations.

JCypher is a Java Domain Specific Language (DSL) for Cypher, a graph query language used in the Neo4j graph database. It provides a fluent API to formulate Cypher expressions while minimizing method nesting and ensuring completion proposals from modern Java IDs. The DSL creates Cypher expressions and introduces a database abstraction layer to execute these expressions against various types of graph databases. JCypher also includes domain mapping, which allows users to store complex graphs of Java objects in the graph database with minimal configuration. This feature enables querying domain models rather than underlying graph models. Additionally, JCypher provides non-functional features such as transactions and concurrency support. The project is continuously evolving, with new features being added regularly, including the ability to store domain queries for later use and a server-side implementation called JCypher-Server, which offers a RESTful API and web UI.

At GraphConnect Europe, a diverse lineup of speakers will present on various topics related to graph databases and data science. Notable speakers include Chris Williams and John Bodley from Airbnb, presenting "Democratizing Data at Airbnb"; David da Silva from Capgemini UK, discussing "Don't Choose One Database Choose Them All!"; Luanne Misquitta from GraphAware, focusing on Neo4j in Human Capital Management; Nicolas Rouyer from Orange, exploring Big Data Governance with Neo4j; and Jim Webber from Neo4j, delivering the closing keynote address. Other speakers include experts from Deutsche Bahn, Telia Company, Structr, and Neo4j, covering topics such as configuration management, scaling graph databases, and interpreting relational schema to graphs. The event promises to be informative and engaging for attendees interested in graph technology and data science.

To integrate Neo4j Enterprise Edition with an LDAP protocol connector, you need to define settings in the neo4j.conf configuration file, such as specifying the endpoint for the LDAP service and providing details on your user graph structure. The LDAP connector is used to manage federated users, while role-based access control is still performed in Neo4j. You can configure encrypted LDAPS and specify credentials for a special LDAP system account with read access to all users and groups. Proper configuration of the LDAP connector requires settings such as user_dn_template, user_search_filter, user_search_base, group_membership_attributes, and group_to_role_mapping. With the right settings, you can manage your existing directory of users outside of Neo4j while mapping them to pre-defined or custom roles for access to the graph database. The integration allows for easy management of federated users via the LDAP protocol, enabling secure access control and authorization.

This week in Neo4j has been quite eventful, with the community exploring various topics such as building a Twitter clone, automated static malware analysis using Neo4j, and learning Chinese. Featured community member Tomaz Bratanic has been making waves with his blog posts and projects, including the hospitals-neo4j project which analyzes a hospital dataset. Max De Marzi has written a series of posts on building a Twitter clone with Neo4j, covering topics such as designing an initial graph data model, creating a Twitter-esque HTTP API, and adding functionality to follow other users. Additionally, there have been blog articles on the Neo4j Data Science Stack, Open Data, and PageRank with APOC, including projects that use Jupyter, RStudio Server, and Neo4j, as well as examples of using Spring Data Neo4j 4.2. The community has also been exploring the Chinese language in Neo4j, creating a project to model the language's relationships. Furthermore, the Neo4j Knowledge Base has released articles on porting users/roles and constraints/indexes from staging to production. Next week will see presentations at PyData Amsterdam and APOC Office Hours as part of April APOC Awareness month.

Marriott, a large hospitality company, was facing performance issues with their pricing engine application due to its relational database. They needed faster and more accurate results for their 4,500 locations across the country. To address this issue, they implemented Neo4j, a graph database, which allowed them to process data much faster, reducing processing time from over four minutes to just 13 seconds. The implementation provided several benefits, including improved performance, increased adoption of the system due to better adoption rates, and reduced hardware requirements, resulting in a 50% reduction in infrastructure costs. Additionally, Marriott was able to maintain their existing relational database as a backup solution while leveraging Neo4j for faster data processing. The experience with Neo4j has allowed Marriott to explore other use cases within their company and adopt the technology without anxiety, enabling them to look for new opportunities to leverage its capabilities.

GraphConnect Europe is a two-day conference that offers unique offerings, new speakers, and familiar faces from the Neo4j community. The conference includes five major training classes on May 10th, featuring experts like Chris Williams from Airbnb and Marcel Donges from Deutsche Bahn. The event will be hosted at the QEII Centre in Central London, providing an ideal location for networking with graphistas. The conference also features Lightning Talks, a Hackathon, GraphClinics, and the GraphParty, offering attendees opportunities to share ideas, learn about new concepts, build projects, get expert advice, socialize, and have fun. Additionally, the event will provide the best product preview and walkthrough of Neo4j's latest releases, making it a must-attend event for graph experts.

Neo4j Enterprise Edition offers advanced features for administering native users, including managing user roles, suspending and activating users, changing passwords, and synchronizing changes across a cluster. Built-in System Procedures provide flexibility for scripting individual actions, while providing a range of views to examine the details of users and roles. Native users can view their assigned roles and change their passwords, but changes must be manually synchronized across a cluster or through automated processes. The system ensures that only authenticated users and applications can access the graph database.

The newly released Neo4j driver versions 1.2 offer improved code writing capabilities with reusable "transaction functions" and built-in retry functionality, simplifying the programming model for handling cluster server failures and role changes automatically. The drivers now handle routing bootstrap addresses, load balancers, and DNS resolution, allowing for automatic routing to multiple hosts. New methods `Session.read_transaction` and `Session.write_transaction` enable the execution of reusable units of work, while auto-commit transactions can run partially synchronous to the network. Some Neo4j drivers have also been updated with improved performance, handling hostname resolutions with multiple IP addresses, and compiled C modules for better performance on supported platforms. The developers invite users to try out the new releases, provide feedback, and report feature or bug requests on the driver repositories.

April is a month of celebration and awareness for APOC, a community-driven procedure and function library for Neo4j's Cypher. As part of this APOC Awareness Month, the Neo4j team will publish articles and demos on how to use APOC in innovative ways, with prizes including an Amazon Gift Certificate of $100 for the best submissions. To participate, simply tweet out your article link and picture using hashtags #neo4j #apoc, and follow @neo4j. The team will review and promote all submissions, even if they don't win a prize, and provide resources such as documentation, blog articles, and GitHub repository to help users get started with APOC. The goal is to encourage users to explore the possibilities of APOC and share their own experiences, making it a fun and engaging experience for everyone involved.

Since Neo4j version 3.0, users can extend its capabilities with user-defined procedures, functions, and aggregate functions. The APOC library was introduced during the 3.0 milestone phase and has since grown to feature over 300 procedures and functions. New features have been added in recent releases, including automated index updates, support for mixed content, and improved performance of the `apoc.periodic.iterate` procedure. Additionally, new contributions from developers such as Stefan Armbruster, Andrew Bowman, Florent Biville, Tomaz Bratanic, and Michael Hunger have expanded the library's capabilities, including new functions for data manipulation, graph algorithms, and JSON support. The APOC team has also partnered with Larus BA to improve the library's performance and address open issues. With the release of Neo4j 3.1, users can now enable automated index updates in their manual index definitions, and Michael Hunger has improved the documentation for the library, including independent versions for different Neo4j versions.

Neo4j has been actively involved in the graph database community this week, with various use cases and applications showcased. Christophe Willemsen, a Principal Consultant at Neo4j Solutions Partner GraphAware, was featured as the community member of the week for his contributions to the Neo4j ecosystem. The company's CEO, Emil Eifrem, was also interviewed by Forbes, discussing the creation of the graph databases category and early adoption of Neo4j. Additionally, there were several new releases, including Neo4j 3.2.0-alpha07 with composite indexes, GORM 6.1 with support for batch inserts and path queries, and a new version of Neo4j PHP OGM. The company's Bolt Driver was also updated to version 1.2. Meanwhile, users have been exploring the bitcoin blockchain in Neo4j, with over 1 billion nodes and 2 billion relationships stored. There were also presentations on real-time recommendations, GraphQL, and discovering taxonomies using Neo4j.

Neo4j's announcement of a name change to Neo5j on April 1, 2017, gleefully highlights the humorous spirit of an April Fool’s Day prank. CEO Emil Eifrem humorously suggests that the name change reflects the company's growth and evolution beyond its original Java-based roots, emphasizing the intuitive understanding that five is greater than four. The playful rationale for choosing the number five includes references to famous quintets like the Olympic rings and the Spice Girls, positioning Neo5j as a natural fit among these iconic groups. Various team members, including the VP of Product and VP of Global Marketing, express their excitement over the new, supposedly more intuitive brand name, while CTO Johan Svensson jokingly laments the change. The article ultimately serves as a lighthearted piece, inviting readers to enjoy the joke rather than take it seriously.