October 2021 Summaries

This Halloween issue features articles on cybersecurity, hacking hacker news content, combining Neo4j with Power Bi for data visualization, and more. A featured community member is Michela Ledwidge, who used Neo4j to investigate anti-LGBTQI interactions between Australian politicians, organizations, and lobbyists in a documentary. The article also includes tutorials on creating graphs using Neosemantics, GraphKer, and Pytesseract, as well as tips on how to have a cybersecurity graph database on your PC and hacking hacker news for fun and profit. Additionally, there are articles on building the case for graph value and creating in-memory graphs with native projections.

More than 80 zettabytes of data will be created, captured, copied, and consumed in the world this year, according to IDC's Global DataSphere. Extracting value from data is becoming increasingly difficult due to the growing amount of data and its complexity. To become useful information for a decision maker, data requires context and relevance. Context describes the relationship between one piece of data and other pieces of data, while information is relevant data in the context that informs one's decision or situation. Data-rich but context-poor worlds inevitably create a poverty of attention for individuals and organizations, leading to a sense of powerlessness and an inability to separate signal from noise. Digital enterprises must transform data into actionable business insights to create value, which depends on various factors such as data-driven leadership, the knowledge and skills of employees, business processes, and organizational culture. The technology used to store data also plays a crucial role in extracting value from data. Three common types of database management systems are relational, non-relational, and dynamic data management systems. Graph databases have gained traction recently due to their ability to reveal hidden structures, provide greater efficiency, and offer wide-ranging capabilities. Graphs can be applied to various problems and use cases, including fraud detection, anti-money laundering, and supply chain visibility. Graph technology has important synergies with AI/ML, enabling the creation of knowledge graphs that capture enterprise data as a graph and use AI/ML to "understand" the context of the data. The use of graph databases is growing in lockstep with AI, driven by their ability to provide contextual searches, make smarter recommendations, and offer richer insights. Graphs can help master data management by building relationships between shared data objects, enabling companies to track and visualize data lineages more effectively and discover issues with data quickly. Various organizations, such as Allianz, Boston Scientific, Lockheed Martin Space, and Airbnb, have used graph technologies to improve their data management and analytics capabilities. IDC recommends that enterprises understand the core pathways to converting data into information, knowledge, and business value, identify how graph databases can help alleviate challenges, choose the right graph database partner for their ecosystem, and capitalize on graphs by assessing vendors offering graph analytics platforms that seamlessly integrate with their current ecosystem.

The new Twitch sandbox is a free online platform that allows users to analyze Twitch streamers and their audiences using Neo4j Graph Data Science. The sandbox provides an interactive browser guide that helps users get started with graph analysis and algorithms, including Cypher query language, PageRank, and Node Similarity algorithm. Users can experiment with the data and come up with new insights, and the sandbox is a great first step for those with experience with Neo4j and graphs or those looking to embark on the network analysis journey. The Twitch dataset was introduced into the Neo4j Sandbox fame, allowing users to dive into network analysis without having to download and configure a Neo4j environment.

The author of the article had been a software engineer for six years before looking to transition into a new phase of their career, seeking a company that embodied four key characteristics: interest and passion for work, innovative technology, a clear company roadmap, and strong cultural values. The company Neo4j aligns with these criteria, offering cutting-edge graph database technology with features such as ACID properties and user-facing problem-solving opportunities. The author was impressed by the company's commitment to its users and community, evident in its six core cultural values and CEO's goal of making Neo4j the best job available. After researching and interviewing, the author joined Neo4j, feeling that they had made a successful decision.

The Neo4j Browser has received several updates, including a new properties panel that displays property names and values in a dedicated side-panel, allowing for easier inspection of property-rich nodes or relationships. The browser also now supports copying property names and values, and provides type information for properties when hovered over. Additionally, the browser's UI has been improved with fixes and improvements, such as better cursor movements between wrapped lines in Cypher queries, wrapping less in table views, and exporting favorites to a zip or file of queries. Neo4j Desktop has also received several updates, including a new feature to view cached databases, a revised UI styling, and bug fixes and improvements, such as improved OS compatibility checks and reliability of offline bundled app installations.

GraphKer is a free and open-source tool that provides a detailed and updated cybersecurity graph database using Neo4j. It aims to find connections between huge amounts of data, providing connectivity and knowledge extraction from these data sets. GraphKer uses a new ontology approach for data feeds to reinforce the knowledge extraction ways that graph databases provide. To run GraphKer, one needs to download and install Neo4j, create and configure the database, install Python requirements, and install applications created for Neo4j. After running GraphKer, users can explore their data in Neo4j graphs, extract knowledge from connected graphs, and analyze cybersecurity incidents, threat actors, hardware, software, skills required, attack paths, etc., to update their cybersecurity level and operational readiness.

Neo4j Bloom is a powerful data visualization platform that allows developers and data scientists to easily visualize and interact with Neo4j graphs, making it simpler to translate complex data concepts. The latest release of Neo4j Bloom 1.9 introduces several improvements to the user experience, including tooltips for nodes and relationships, revealing relationships between nodes, and improved card functionality. These updates aim to simplify workflows and make graph exploration more accessible. Users can upgrade to the latest version through various channels, including automatic updates for Neo4j Desktop users or manual downloads for server plugin and self-hosted web application users. The development team encourages feedback and suggestions from users, with a chance to become a Bloom co-designer and receive rewards.

The featured community member is Mike Oaten, an entrepreneur working with his product Riskhunter, which uses graph technology to identify fraud and mis-selling in the UK financial services industry. He recently published blog posts on the topic of graph technology and entity resolution. The Neo4j community also includes a meetup group organized by Mike, covering topics such as graph databases reflecting reality and how they aid in entity resolution. Additionally, there's an article about diversifying stock portfolios using graph analytics, which uses similarity networks to reduce risk and potentially increase profits. Graphalue is featured with a new episode and practical resources on importing Wikidata into Neo4j, including a cool example of using Cypher to query the graph with voice input. Another resource explores how graph technology can aid in fraud detection, with an interview by Nik Vora from Neo4j explaining its value. There's also a podcast discussing Aura Free, a new gem in the graph database ecosystem that offers a novel way to build and maintain graph databases, and a stream showing how to build a graph database populated with data from Kickstarter projects.

This week I'd like to point you to a cool video on algorithmic bias in AI, as machine learning depends heavily on the data it's trained on and this data can contain biases that affect its performance. Bias can come from unrepresentative or flawed training data, such as Amazon's recruiting tool which learned to discriminate against female candidates due to an industry dominated by males. Tracking data lineage in a knowledge graph is one of the ways to mitigate bias and also helps with being more ethical, especially when you don't know where your data came from.

The new Cypher parser in Neo4j 4.2 has made data imports significantly faster, with improvements of up to 10 times faster than the previous version, Neo4j 3.5. The improved performance is due to the rewritten Cypher parser and the introduction of new optimization techniques such as UNWIND and UNWIND BATCH parameters. These optimizations allow for more efficient import operations, resulting in faster data loading times. In comparison, the export operation has seen a slight slowdown in Neo4j 4.2 compared to Neo4j 3.5, but overall, the improved performance of the new parser and optimization techniques has made imports significantly faster. The results show that with the new shell parameters, the import time has decreased from 10 minutes in Neo4j 4.1 to under two minutes in Neo4j 4.2, highlighting the efficiency gains of the new Cypher parser and optimization techniques.

Unleashing the power of knowledge is crucial for enterprises seeking a competitive edge, as turning data into knowledge remains an ongoing effort. Most organizations have scattered data across various systems, clouds, backups, and data lakes, leading to siloed data, data sprawl, and governance challenges. Cloud storage also contributes to data disconnection and compliance issues. Historical data is becoming obsolete due to disruptions like the COVID-19 pandemic, which highlights the importance of capturing connections and relationships in graph databases to increase predictive power and accuracy. As enterprises move toward connecting data in a graph database, they will gain valuable insights and knowledge to inform business processes.

The Financial Action Task Force (FATF) is a leading actor in drafting guidelines and regulations to combat criminal activity within financial systems. Its recommendations have driven global legislation concerning money laundering prevention, particularly regarding virtual assets, which represent value that can be traded or transferred digitally. New technologies, including blockchain technology, have the potential to make anti-money laundering measures faster, less expensive, and more efficient. However, this growth also creates regulatory demands, making it challenging for businesses to harness these technologies while meeting regulatory requirements. PwC Germany has developed a platform called BETA, which supports crypto asset service providers in complying with FATF-inspired flags and reducing financial crime risks. The platform uses Neo4j graph database technology to analyze transactions, link local data to on-chain data, and provide automated real-time scoring. By leveraging Neo4j's capabilities, PwC Germany can navigate the complexity of this field and offer a technological and analytical foundation for anti-money laundering transaction monitoring and risk-based flow of funds analytics for crypto assets.

The authors of the text created a free database called AuraDB Free using Neo4j, a graph database, to store a Kickstarter dataset. The dataset contains information about 375k Kickstarter projects from 2009 to 2017, including category, country, and pledge data. The authors imported the dataset into the database using Google Spreadsheets and then created nodes for categories, subcategories, countries, and projects. They also added labels to the project nodes to indicate their state (e.g., successful, failed, canceled). The authors demonstrated various queries to answer questions about the data, such as which categories exist, how many there are, and what is the most successful category or country. They also showed how to visualize the data using Neo4j's built-in visualization tools. The dataset can be used for further analysis, adding more data from other sources, or building a Kickstarter-like app or analytics dashboard.

The featured dataset this week includes a Twitch dataset for analyzing streamer data, a new Stack Overflow dataset to explore code-related data, and a beer recommendation engine created by Bart Simons using data from the Untappd app. The Neo4j Certified Professional certification is also available with a free t-shirt, and a masterclass on knowledge graphs is being held by Clair Sullivan. Additionally, there are resources for visualizing similarities between companies, exploring the Twitch Sandbox, and learning about graph value with Graphaule.

The text discusses the addition of a feature to produce backups from running sandboxes in Neo4j, allowing users to continue working locally or in Aura. The backup process involves requesting a backup, waiting for it to finish, and then downloading a pre-signed URL for 30 days. The dump file can be used in multiple ways, such as importing it into Aura Free, Neo4j Desktop, or other Neo4j instances. To produce the dump file, users need to make an online backup, restore the database locally, make a dump from the local database, and upload it to S3 using Amazon SQS and a Celery backup service. The feature is designed to be asynchronous, allowing users to work without waiting for the backup process to complete. The text also explains how to load the dump file into Aura and provides information on the newly added feature in Neo4j Sandbox.

Neo4j's Chief Scientist, Dr. Jim Webber, showcased stories from Neo4j community members and customers at NODES 2021 closing keynote. One such story is that of Julie Fisher, a Risk Modeling Scientist at Asurion, who uses graph technology to identify fraud patterns in cell phone insurance data. Julie came across graph technology while working with tabular data and saw connections between seemingly unrelated pieces of information. She applied graph technology to visualize her findings and identify patterns, which helped her expand the known bad actors into the unsure category. Julie's experience highlights the potential for graphs to combat fraud and suggests that the technology is maturing as it grows in various industries. The conversation also touches on the importance of dissemination of information about graphs and their expansion into new domains.

Graph analytics can be used to diversify a stock portfolio by analyzing the correlation between stocks. A graph model is created using Neo4j, where each stock ticker is represented as a node and the price and volume information for each stock ticker is stored as a linked list of stock trading days nodes. The Pearson similarity algorithm is used to calculate the correlation coefficient between most correlating stocks, and community detection algorithms are applied to identify clusters of correlating stocks. A simple linear regression model is then used to recommend top-performing stocks from each community. However, it's essential to note that this approach has limitations, such as only analyzing a 90-day window for NASDAQ-100 stocks and using a simplified correlation coefficient calculation. Further research and fine-tuning of the graph analysis may be necessary to improve the accuracy of the results.

The text discusses various methods for importing data into Aura Free, a free version of Neo4j's graph database. It highlights three main topics: loading a dump from a Neo4j Sandbox Backup, loading a dump from a Neo4j Graph Example Repository, and loading a dump from Neo4j Desktop. The first method involves taking a backup of the sandbox data, downloading it, and then uploading it to Aura Free using the Aura Import UI. The second method utilizes the Neo4j Graph Example Repository on GitHub, where users can find datasets such as the MovieLens user-ratings dataset that fit within the limits of an Aura Free instance. The third method allows users to create a dump in Neo4j Desktop and then upload it to Aura Free using the same import mechanism. These methods enable users to easily transfer data from other Neo4j instances into their Aura Free databases.

Neo4j AuraDB is a cloud-based graph database service that enables users to load dump files from various sources, including Neo4j desktop and Sandbox. The company has recently released GDS 1.7.0, which introduces machine learning pipelines for graph native link prediction, making it easier for users to define and apply link prediction pipelines. Additionally, a new course on Graph Data Modeling is available, covering concepts of graph modeling and how to refactor existing models. The company also provides tutorials on Neo4j performance architecture, tuning, and creating clinical knowledge graphs with Spark NLP & Neo4j. Furthermore, Michael Hunger will show users how to get their dump files loaded into AuraDB free, while a deep dive into Neo4j link prediction pipeline and FastRP embedding algorithm is available for those interested in learning more about link predictions.

The author discusses the use of knowledge graphs at NASA's people analytics branch to identify hidden skills and competencies within employees. The author introduces the concept of a knowledge graph, which is a mechanism for connecting different data sources together via common relationships, allowing for the combination of structured and unstructured data. The author uses Neo4j, a graph database, and various algorithms such as node similarity, centrality, and community detection to build and analyze the knowledge graph. The author demonstrates how the knowledge graph can be used to identify skills gaps, career paths, succession planning, and strategic workforce alignment. The author also discusses the use of external data sources, such as the PDW (personal data warehouse) and mission data, to enrich the knowledge graph. Additionally, the author highlights the potential for graph data science to support diversity, equity, and inclusion initiatives, as well as ranking skills by occupation and employees similar to an occupation based on abilities. The author concludes that knowledge graphs have the potential to answer many questions within people analytics of human capital and looks forward to seeing what new technologies can be developed in this field.

While summer may be over, webinars are always in season. In this month's blog, we recap two of our marquee digital events - Connections, which featured keynote speakers from NASA and industry experts discussing the transformative power of knowledge graphs and graph technology. Knowledge graphs are driving business transformation across various industries, including data governance, machine learning, and medical research. The August Connections event focused on accelerating innovation with graphs, showcasing examples of how to innovate using this technology. Both events provided valuable insights into the latest trends in graph technology and its applications.

Connecting to Neo4j from Microsoft Power BI using ODBC is made possible through OpenLink Software's ODBC-JDBC Bridge Driver, which extends the reach of Neo4j's BI Connector to ODBC-compliant applications such as Microsoft Power BI. To achieve this connection, users need to set up an ODBC DSN and create a data source, then use the ODBC driver with PowerBI to query their Neo4j instance. The process enables users to build visualizations using data from their Neo4j instance in PowerBI and also connects other ODBC-compliant applications to Neo4j's JDBC BI connector.

The International Consortium of Investigative Journalism (ICIJ) has published a massive dataset known as the Pandora Papers, which contains leaked information about shell companies, offshore accounts, and secret ultimate owners. The dataset is staggering in its scope, with 600 journalists from 150 media outlets in 117 countries working on it. The ICIJ used an open-source stack consisting of Neo4j and Linkurious to structure and analyze the data, which includes 11.9 million files and 14 offshore service providers. The data model for the investigation consists of entities (shell companies or offshore constructs), intermediaries (law firms or banks that helped create and manage these entities), officers (proxy or real owners/shareholders/directors of these entities), and addresses (registered addresses for these entities). The dataset is now being integrated into the Offshore Leaks database, which will be published in a few weeks. The ICIJ has also created an interactive document exploring the stories behind high-profile politicians' use of offshore companies, including 35 current or former country leaders and prominent politicians. A Neo4j graph database instance has been set up to query and visualize the data, with examples provided for data exploration.

The Neo4j community is featured in this issue, with several developers and users sharing their knowledge and experiences. Alexios Pantavos, a developer at Akelius, Residential Property AB, is highlighted as the featured community member of the week, having provided extensive help to other community members through his volunteer work. Graphlytic has published a video tutorial on style mapping rules for Neo4j graphs, while Graphileon has demonstrated how to create public and private API endpoints using their tool. Clair Sullivan from Neo4j will be teaching a four-hour course on Knowledge Graphs for ML on October 7 through O'Reilly Online Learning. Pankaj Kumar has shared his experience with analyzing Twitter data using the Neo4j graph database, while David Allen introduces the use of Kafka and Neo4j AuraDB in his recent article. Adrien Sales experimented with the Collatz Conjecture problem by creating a graph and analyzing some data using Neo4j's Graph Data Science Library. Jennifer Reif has published an article on migrating from Neo4j Spring Data 5 to 6, incorporating reactive programming into the application. The issue also includes news about new Developer Relations videos and a tweet of the week.

We're delighted to share with you a conversation between Stefan Wendin, Neo4j's Global Head of Business Design & Strategic Programs, and Stéphane Delysse, Head of IT Business and Data Analysts Team at AFNOR Group. These two have built a strong relationship through their work on all things graphs and had a particularly salient moment when Stephane and his team joined Stefan in the Innovation Lab. They discuss how connected data makes their worlds go 'round, with Stéphane explaining that graphs allow them to get a broad view of all standards, how they are connected between them, and how this categorization can evolve as the world evolves quickly. They also talk about the challenges of explaining or aligning this from a business perspective, and how the Innovation Lab helped bring people together from different parts of the company to work on this subject. The conversation highlights the importance of having a facilitator molding different mental models, avoiding the "expert" trap, and emphasizing user experience. Stéphane shares that the Lab was an astonishing journey, bringing new possibilities for future development and new tools for their customers. He also emphasizes the comfort and enthusiasm of working together despite the challenges posed by the pandemic lockdown.

The Approximate Maximum K-cut algorithm is a new feature in Neo4j's Graph Data Science Library that divides a graph into partitions based on the strength of relationships between nodes, aiming to maximize the sum of weights of relationships within each partition. This algorithm has applications in various domains such as dinner party planning, mobile wireless communication, and retail menu organization. By using this algorithm, users can identify clusters or communities within their data that share similar characteristics, enabling more effective decision-making and problem-solving. The algorithm's output provides a set of partitions, where nodes are assigned to the most compatible partition based on the strength of relationships between them. In the provided example, the algorithm was used to assign guests to tables at a dinner party, taking into account their mutual dislike or attraction, and to categorize items in a bakery menu based on how frequently they appear together in transactions.

The author of the text is a consulting engineer at PS who frequently receives questions from Neo4j customers about improving query performance on large graphs. They explain that achieving good performance depends on having an effective model for the data and efficiently designed queries. The article discusses the two main ways of querying a graph in Neo4j: through the Cypher query language and via a Java API. While a solid model and well-designed Cypher query are sufficient in most cases, the Java API provides more control and is useful when fine-grained control is needed. The author aims to show that writing a stored procedure is not an arcane process but rather a simple Java program. They provide a simplified example of building a stored procedure using the Neo4j Java API, which aggregates patterns in patient journeys after a certain condition. The article covers the design and implementation of the stored procedure, including the use of Java annotations to inject context objects, defining a custom data class for the result, and implementing methods to traverse the graph and aggregate results.