September 2019 Summaries

Couchbase's N1QL and Full Text Search (FTS) integration allows users to perform complex text searches within JSON documents stored in Couchbase, combining the strengths of both technologies. N1QL, Couchbase’s SQL offering, is used for manipulating JSON data and supports various operations with secondary indexes, while FTS enables natural language querying and scoring based on relevancy. The integration introduces a new SEARCH predicate in N1QL, enabling seamless execution of FTS queries within N1QL queries, thus extending FTS's usability beyond SDK, curl, or Couchbase's UI. Users can set up FTS indexes to optimize query performance and leverage N1QL's ability to apply SQL operations over FTS results for efficient development. The system also includes APIs to handle query sargability, paging of results, and verification of query matches. N1QL determines the most suitable index to use for a query, enhancing query execution efficiency, while future updates are anticipated to improve sargability and support for more complex index definitions.

FTS (Full-Text Search) indexes require careful configuration and maintenance to ensure optimal performance, especially when it comes to text analysis pipelines. A user new to the Information Retrieval domain may find configuring an FTS index tedious due to the numerous options available for character filters, tokenisers, and token filters. However, a detailed insight into the text analysis output is crucial as users often struggle with why search results are not returning hits despite proper index definition. The `analyzeDoc` endpoint introduced in 6.5.0 provides a safe way to explore and debug analyser outputs, allowing users to identify issues such as mismatched analysers between index and query time. Additionally, FTS cluster sizing is critical, and while sizing guidelines are recommended, easy indicators like slow indexing progress, rejected search queries with HTTP status code 429, and slow-queries in stats graphs can hint at under-provisioning. Suboptimal queries, such as complex compound queries with many sub-queries, can also lead to inefficient system resource use, highlighting the importance of carefully crafting targeted queries.

Couchbase Server 6.5 introduces new features to its NoSQL database, notably adding window functions to the N1QL query language, which enhance the database's ability to handle complex business queries efficiently. These window functions are executed in a distributed, partition-parallel manner in the Couchbase Analytics service, leveraging the MPP query engine that isolates workload from operational nodes and employs a parallel processing approach across multiple nodes. The execution plan of window functions involves data repartitioning, sorting, and computation, allowing for scalable and efficient data analysis. Additionally, N1QL for Analytics offers flexibility by allowing window functions in various clauses, unlike traditional SQL, and these functions can be directly applied to JSON data, simplifying the data processing workflow. The server's architecture supports memory management and resource-based load control to ensure efficient query execution, making it a powerful tool for real-time data analysis and reporting.

The text discusses the advantages of using Couchbase Full-Text Search (FTS) over the SQL "Like %" clause for applications at scale, highlighting the simplicity of integrating Couchbase FTS with existing database infrastructure. It emphasizes that Couchbase FTS allows for seamless combination with N1QL, Couchbase's query language, to enhance search capabilities. An example query demonstrates how Couchbase FTS can be used to search for data within a specific field, and it invites readers to explore a tutorial for building a basic movie search engine, illustrating the practical application of these tools.

Couchbase has partnered with Oracle to be a launch partner for the Oracle Cloud Infrastructure (OCI) Marketplace, offering customers the ability to run Couchbase alongside their Oracle investments, thereby simplifying the development and enhancement of applications. This collaboration marks the first time a NoSQL document database is available on Oracle's platform and allows customers to purchase Couchbase Server and Sync Gateway in a single marketplace listing. Oracle customers can deploy Couchbase through various methods, such as Paid Listing, Customer Bring Your Own License, and OCI Quick Start, all of which enhance scalability and integration with existing Oracle services. Notably, Couchbase distinguishes itself with its use of open standards, allowing seamless migration from relational databases and enabling the use of SQL skills through its SQL++ implementation. This partnership aims to support data-intensive applications with a scalable platform, leveraging Oracle's infrastructure and Couchbase's advanced features, including simplified query optimization and integration with Kubernetes.

Couchbase and Confluent Kafka are highlighted as highly complementary technologies that excel in disparate systems, with Couchbase being a scalable NoSQL database ideal for applications requiring low-latency data access, and Confluent Kafka serving as a distributed streaming platform capable of handling massive volumes of event data. The blog post details the seamless integration process between these two platforms, focusing on moving data from Couchbase to a Kafka topic using the Couchbase Kafka connector, which leverages Couchbase's replication protocol to transfer documents efficiently. The post provides a step-by-step guide on configuring Confluent Kafka and the Couchbase Kafka connector, including necessary prerequisites, setup commands, and configuration details. By following these instructions, users can quickly integrate the systems, enabling real-time data stream processing applications with tools like KSQL to meet business needs.

Couchbase has expanded its partnerships with industry leaders Infosys and other regional solution providers, allowing it to deliver its unique capabilities to a global footprint of enterprise customers. Through the recent OpenHack event featuring Couchbase, Infosys partners with Couchbase to deliver powerful digital transformation solutions in areas such as modernizing legacy applications and field service applications. The event drew over 400 participants from across various geographies and resulted in the creation of 74 solutions, including a navigation mobile app that leverages the entire Couchbase mobile stack, a data platform for microservices-based applications, and an SQL-N1QL converter to aid in modernizing legacy apps. Senior architects from both Infosys and Couchbase guided and mentored teams throughout the event, resulting in innovative solutions that showcased the power of Couchbase's capabilities. The OpenHack event was successful, enabling a large group of Infoscions with cutting-edge Couchbase data platform capabilities, which will be brought to joint customers.

In the realm of database management, query optimizers play a crucial role in enhancing performance by minimizing resource usage such as memory, CPU, and disk I/O. Couchbase 6.5 introduced a cost-based optimizer (CBO) for its N1QL query service, which marks a significant advancement over traditional rule-based optimizers. This innovation is akin to using a dynamic route optimizer in navigation apps, offering significant improvements in executing complex queries efficiently by employing statistical data for decision-making rather than static rules. CBOs are particularly adept at handling the flexibility and complexity inherent in SQL and NoSQL systems, significantly impacting system throughput and the ability to meet service level agreements (SLAs). They achieve this by rewriting queries, selecting optimal access paths, determining join orders and types, and creating execution trees, ultimately ensuring that queries run with maximum efficiency. The introduction of CBOs in NoSQL databases like Couchbase highlights the growing importance of statistical optimization to handle real-world workloads effectively, emphasizing the necessity for enterprises to prioritize databases with robust optimizers.

This hands-on tutorial demonstrates the setup and deployment of Couchbase Autonomous Operator, a Kubernetes operator that automates the management of Couchbase clusters. The tutorial covers various aspects of setting up and managing a Couchbase cluster on Minikube, including deploying custom TLS certificates, persistent volumes, and scaling the cluster. It also showcases backup and restore procedures using `cbbackupmgr` and running sample Python applications to interact with the Couchbase cluster. The tutorial provides a comprehensive overview of the features and capabilities of Couchbase Autonomous Operator, including self-healing capabilities and support for multiple nodes.

Couchbase Analytics, the latest addition to Couchbase Server's suite of services, is designed to provide rapid insights across various use cases by leveraging massively parallel processing (MPP) for complex analytical queries. This blog post, the first in a series, introduces the execution framework and explains a simple query plan, detailing how the query is processed through the Analytics engine. The process involves parsing and optimizing a query plan, which is a tree-like structure of operators capable of handling nested JSON data, before executing it across multiple nodes in a cluster for scalable and efficient data processing. This post aims to demystify the Analytics query plan, offering insights into query optimization and execution, and invites readers to explore further through Couchbase Server 6.5 and community engagement.