November 2022 Summaries

The annual NODES conference, held in 2022, was a success with thousands of attendees participating in over 100 technical sessions across multiple time zones. The conference featured renowned speakers, including Nicholas Christakis and Emil Eifrem, who discussed topics such as social networks, graph technology, and data science. All talks are now available online, along with code repositories from many speakers, allowing easy access to the content. A recommended list of top 20 talks by attendee rating was also provided, covering a range of subjects including cloud computing, machine learning, and graph analytics. The conference aimed to be an inclusive educational event for both developer and data science communities, and its success has been celebrated by attendees and organizers alike.

The Building Neo4j Application with TypeScript course is a new online training platform offered by Neo4j, providing a free, self-paced learning experience for developers. The course focuses on teaching users how to interact with Neo4j using TypeScript, covering topics such as installing dependencies, creating driver instances, executing Cypher statements, reading and writing data, and type checking. With a shorter duration of around two hours, the course features three online-only challenges that can be completed within an online IDE provided by GitPod. The platform aims to teach users how to use Neo4j & TypeScript together, providing a comprehensive learning experience for developers looking to build applications with these technologies.

Neo4j is revolutionizing the way users interact with graph databases by introducing a new interface called Workspace, which provides an intuitive and easy-to-use environment for developers, data scientists, and data analysts to get started with Neo4j. Workspace includes essential tools such as Import, Explore, and Query, allowing users to design their data model, visualize their data, and run Cypher queries without the need to navigate through multiple tabs or applications. For new users, Workspace offers self-guided tutorials and prompts to help them learn about graph databases and Neo4j. The platform is now available for free to anyone who signs up for a Neo4j Aura account during an early-access period, with plans to extend its features to Enterprise-level functionality in the future.

Neo4j 5 marks a significant shift in its continuous release and product support model. This departure from traditional branched product support allows for more frequent minor releases with new features and fixes, supported with hotfixes until the next minor release. The LTS version will now be supported for 42 months, compared to 36 months previously. Customers have an extended migration window of one year to move between Neo4j 4.4 and Neo4j 5.LTS. New features include any-to-any upgrade within Neo4j 5, reliable and safe database/server eviction, Helm Charts for Neo4j 5, near zero downtime migration from Neo4j 4.4 to Neo4j 5, and improvements in frequently asked questions. The new release model is accompanied by an extended migration window and support for customers who prefer to deploy versions without introducing new features.

At Ernst & Young LLP, the company recognizes the importance of Environmental, Social, and Governance (ESG) factors in investing, with 80% of their value composed of intangible assets. ESG stands for environment, social, and governance, encompassing issues such as brand awareness, intellectual property, human rights, working conditions, transparency, ethics, and data privacy. The ESG data market is highly polluted with numerous climate tech and FinTech data providers producing aggregated data on a daily basis, alongside hundreds of global ESG provisions. Adhering to these requirements is crucial for investors, as survey results show that every portfolio manager has started or plans to start using ESG attributes in their investment decision-making process. Graphs have been instrumental in reframing the ESG problem and making it clearer, enabling the company to untangle the complex data landscape by coordinating, connecting, and mobilizing ESG data through a graph structure. The approach involves creating nodes for each provision, mapping categories, and linking frameworks through similar questions, utilizing the universe of ESG data vendors. By chaining existing metadata relationships, the disclosure graph can translate requirements and find missing information to reduce reporting burdens when new frameworks arrive. The graph also enables expansion with real data and works across various examples, such as mortgage portfolios, stocks, baskets of companies, bonds, and indices. Future improvements include adding a semantic layer on top for easier understanding by business users and expanding the data fabric to integrate data and metadata in the same graph.

The author of a blog post explains how Java 17's record keyword has caused issues with the Neo4j-Java-Driver, specifically when importing records from the driver. The driver uses `org.neo4j.driver.Record`, which conflicts with Java 17's `java.lang.Record`. This leads to compilation errors and ambiguous references, making it difficult for users of the driver to process result sets. The author emphasizes that this is not a bug in the driver or code, but rather a necessary communication from the library provider. They provide a working example to illustrate the issue and caution against using star- or wildcard-imports.

Ufuk Kirik, Associate Director of Data Science at AstraZeneca, discussed the use of graph technology in drug discovery and highlighted Neo4j as a key tool for his work. He emphasized the importance of not being afraid to try new things and encouraged others to "dip their toes in the water" when getting started with Neo4j. Kirik praised Cypher, Neo4j's query language, for making it easy to translate difficult biological questions into queries. He also discussed the potential for future advancements in graph technology, including the integration of high-level programming languages and the ability to quickly adapt data models. As AstraZeneca works to identify new drug targets using graph data science, Kirik's insights provide a glimpse into the exciting possibilities emerging in this field.

Michela Ledwidge, Co-founder and CEO of Mod, showcased how her company uses Neo4j for spatial visualization in creative storytelling, project data management, and developing a prototype consumer productivity app called Grapho. She demonstrated 3D data that is intuitively explorable and manipulable in Extended Reality (XR), making complex data accessible to people without a technical background. Ledwidge emphasized the importance of building user-friendly data tools for non-technologists, citing her experience with creating interactive visualizations using game engines like Unreal. She also discussed her work on projects such as a real-time globe sync system celebrating K-Pop sensation BLACKPINK's success and an experiment to find if presenting spatial media adds value to the exploration experience of polarizing data sets. Ledwidge highlighted the potential of Neo4j to support collaborative research and improve user experience, with goals including releasing a prosumer tool that allows people to put their data into VR or AR-type experiences easily.

NODES 2022 is a 24-hour global event that brings together the brightest minds in graph technology, providing an opportunity for attendees to engage with experts and the global community of passionate graphistas. The event features keynote speakers such as Nicholas Christakis, known for his groundbreaking research on social networks, as well as interactive live talks and Q&A sessions with over 100 scheduled presentations from around the world. With beginner, intermediate, and advanced tracks available, there is something for everyone, regardless of their level of graph familiarity. The event offers multi-time zone coverage and is free to attend, with attendees also having the chance to win $500 by inviting friends to register.

Neo4j's AuraDB Enterprise is a fully managed graph database that allows developers to focus on application development while the database is handled in the cloud, with features such as zero administration, performance and reliability, and enterprise-grade security. The platform is now available on Microsoft Azure, providing customers with flexibility to choose their preferred cloud provider. With AuraDB Enterprise, users can expect automatic updates and patches, scalable on-demand, automated backups, and up to 80 percent cost savings by pausing instances, as well as a 99.95 percent service uptime guarantee and 24/7 premium support.

Neo4j 5 offers improved query performance, enhanced indexes, and support for Panda dataframes in the Python driver. The Community Edition features an upgraded runtime that accelerates queries by 30 percent. Autonomous Clustering and Fabric enable efficient scaling of large graphs. Neo4j 5 is showcased in various sessions at NODES 2022, including introductions to new features and best practices for graph ETL and data management. The platform is also highlighted in several articles and blog posts discussing its applications in sustainable development goals tracking, feature engineering with Amazon SageMaker, and API creation with NestJS. Various resources are available to learn more about Neo4j 5, including a contest for a chance to win $500 by inviting friends to NODES.

Neo4j's Tomaz Bratanic discusses knowledge graph embedding, a concept that pairs complex relationships with geometric representations to reduce storage and processing requirements. The process involves machine learning algorithms like Node2vec, which automatically construct vectors from graph adjacency matrices, allowing for efficient encoding of information in large graphs. This technique has applications beyond semantic functions, such as identifying stressed power lines in electrical grids, and can be used in conjunction with downstream machine learning models to predict outcomes.

Neo4j 5 has significantly improved query performance, particularly in K-Hop queries with aggregation functions, making it 1000 times faster than its predecessor. The new TEXT indexes implementation using trigrams enables Cypher clauses to filter results by text properties hundreds of times faster. Additionally, Neo4j 5 introduces syntax for label and relationship type expressions, allowing more complex pattern-matching queries. The database also supports sharding and Fabric, enabling instant creation of a Fabric database and federated queries across multiple clusters. Autonomous Clustering has been revamped to shift the administrative burden from operators to software, reducing manual effort required to scale out or in servers hosting databases. Neo4j Ops Manager provides a UI console for more administrative control and a bird's eye view of operational metrics across databases, instances, and clusters. The latest innovations around DevOps practices are also available in Neo4j 5, including simplified upgrade experiences and zero-downtime rolling upgrades.

A large bank in the European Union faced performance issues with its Neo4j cluster due to an increasingly large database, prompting a decision to scale out across multiple clusters using Neo4j Fabric. This approach allowed for sharding of data by time window, making it suitable for analytical workloads and batch operations. By distributing workload across smaller shards, the bank achieved significant performance boosts, easier maintenance, reduced infrastructure costs, and improved scalability. However, it's essential to consider optimal sharding strategies upfront, as different situations require tailored approaches, such as sharding by logical domain entity or geographical location.

Neo4j has introduced Autonomous Clustering in its version 5, which simplifies running and administering fault-tolerant and highly available Neo4j clusters compared to its predecessors. Autonomous Clustering automatically decides how to distribute primary and secondary database instances or copies across servers according to the requirements and constraints provided by the database administrator. This feature enables standard network load balancers and other cloud technologies to be used, making it easier to run highly available and scalable clusters with reduced manual configuration needed for setting up and maintaining the cluster. The new capability also automatically elects a new leader for primary leader instances that were being hosted on a failed DBMS server. With Autonomous Clustering, Neo4j 5 makes it easy to build reliable clusters that can handle large concurrent workloads by scaling horizontally to meet the needs of enterprise-class applications.

With Autumn in full swing, Neo4j Bloom has introduced new visualization features that make graph data exploration even easier and more intuitive. Users can now easily create a random subset of their graph by typing "Show me a graph" in the search field, allowing them to start exploring and investigating relationships between nodes. Additionally, perspectives have been updated to automatically synchronize with database modifications, providing an improved user experience. Minor updates include ordered scene actions, dismissible single nodes, and filtering of saved scenes and perspectives for Enterprise users. These features aim to provide a more tailored business view or domain found in the target Neo4j graph, allowing users to focus on specific categories and relationships that are relevant to their needs.

Graph analytics has become a valuable tool in economics to better understand patent data and the knowledge economy. Studying the knowledge economy helps us explore how inventors interact with each other over time to produce new ideas and innovation. Patent data is used as a graph, which allows for natural and contextual relationships to be understood, making it easier to study complex phenomena. The graph schema has 13 million nodes and 110 million relationships, representing patents, inventors, locations, assignees, and self-relationships. Citation trees are built using Neo4j Bloom, showing how knowledge develops over time, with an exponential increase in patents as the number of inventors increases. Collaboration networks, where collaboration leads to technological growth, are represented by a star-shaped graph, while frictions to collaboration, such as individual inventor behavior and success influencing collaboration, are also studied using patent data. The analysis reveals insights into the dynamics of innovation and the factors that influence it, highlighting the richness of patent data and the importance of graph analytics in understanding the knowledge economy.

Explore the world of graph technology with various topics covered in the NODES 2022 conference, including using Jupyter Notebooks to visualize graphs, sending forth microservices for a more efficient system, and discovering invisible patterns in data through Google Sheets tables and Neo4j. Graph machine learning is also discussed, as well as building Java applications with Quarkus and Neo4j, and creating knowledge graphs for target protein interaction exploration. The conference features several workshops, including a hands-on introduction to Neo4j, and showcases the strengths of native graph databases, such as index-free adjacency and network relationships. Additionally, there are sessions on TypeScript, GraphQL, REST, and app development, as well as a tweet of the week featuring Neo4j. Other notable mentions include new releases of Jhipster Lite and Apache Hop, as well as an upcoming appearance at AWS re:Invent.

We met up with Andy Emmonds, Chief Transport Analyst at Transport for London, to discuss his team's use of graph technology in creating a digital twin that enables quicker identification of incidents on the road and impacts other sections of the transportation network. Neo4j is used instead of relational databases due to its ability to handle interconnected data, enabling faster and more effective interventions. Andy shares surprising results from using Neo4j, such as being able to make relationships between data points, which has enabled his team to build a decision support system that incorporates this capability. He advises keeping it simple when getting started with Neo4j, citing a proof-of-concept project that elevated his organization's executive recommendation to take the technology further. Andy envisions a future where graph technology is used to develop smart cities, bringing together digital twins and making them seamless for users, enabling a wide range of applications such as transportation, shopping, and entertainment.

Graph embedding is a technique that enables the representation of complex systems as geometric patterns in a lower-dimensional space while preserving their essential characteristics. It allows for the efficient processing and analysis of large-scale graph-structured data, such as social networks, molecular structures, or financial transactions. By mapping high-dimensional graphs to lower-dimensional vectors, graph embeddings can facilitate machine learning algorithms to learn from these complex systems without requiring extensive manual feature engineering. This technique has numerous applications in fields like finance, healthcare, and climate science, where understanding systemic risks, patterns of behavior, and relationships between entities is crucial. Graph embedding offers a promising approach to tackle these challenges by providing a more efficient and scalable way to analyze and model complex systems, enabling organizations to make data-driven decisions and mitigate potential risks.

The Bolt protocol is a binary application protocol used for communication between Neo4j servers and clients. It was introduced in Neo4j 3.0 and has since become the standard protocol for interacting with Neo4j databases. The protocol specifies how clients and servers exchange data using specific Bolt messages following the PackStream format. Recently, issues were found with date and time-related tests in Neo4j's TestKit suite, specifically with the handling of DateTimeZoneId structures which can be ambiguous due to daylight saving time (DST) transitions. To resolve this issue, a new UTC-aware structure called UTCDateTimeZoneId was introduced in Neo4j 5 releases, which encodes the date and time in UTC time, eliminating the ambiguity caused by DST transitions. This update aims to provide a consistent developer experience across different programming languages and Neo4j deployment topologies.

The Neo4j Data Importer has undergone significant improvements since its beta debut in March 2022, making it easier for users to load flat file data into the graph database. The new feature, Preview, allows users to preview their data model and ensure accurate mapping details before importing, reducing potential errors. Additionally, the Data Importer can now automatically detect datatypes from flat file data, supports the DateTime datatype, and provides alerts when issues are detected with the imported file, such as missing column names or duplicated column names. These improvements aim to simplify the process of importing data into Neo4j, allowing users to focus on other aspects of their work without writing code. With these enhancements, users can start importing their data into Neo4j using a no-code approach, and a free Neo4j instance is provided upon sign-up for the fully-managed cloud service offering.

The author of this text worked on advanced analytics for IT risk stability in 2020 when Commonwealth Bank Australia decided to move its digital assets from on-premises servers to the cloud. The team soon realized that payments had a complex graph structure with many interdependencies, making it difficult for humans to manage these relationships. They used Neo4j to model this graph and eventually developed a document graph system that could handle large amounts of data. This system was later expanded upon to include knowledge graphs, network graphs, and flow policies. The team also worked on cybersecurity and observability, using tools like Qualys and Iris to monitor the network. After two years of work, they are still transitioning their workloads to the cloud and are exploring ways to make their graph database more accessible to a wider range of users.