July 2020 Summaries

The document reviews the features of MongoDB 4.4 and Couchbase 6.5, emphasizing their advancements in query language capabilities. MongoDB 4.4 introduces enhancements to the MongoDB Query Language (MQL), including a new $UnionWith aggregation stage and custom aggregation expressions using Javascript, although these are considered verbose and potentially error-prone. Couchbase 6.5, released earlier in the year, supports a wide range of SQL set operations and has previewed SQL and Javascript functions across N1QL expressions, enhancing developer productivity with features like window functions. Both platforms have made notable improvements in search functionality, with MongoDB's $search based on Lucene available only on its Atlas service, while Couchbase offers a SEARCH() function integrated with its N1QL optimizer. Additionally, MongoDB now supports creating collections and indexes within non-sharded transactions, whereas Couchbase has introduced distributed transactions for Java SDKs, though not for N1QL statements.

This article demonstrates how to use the Couchbase SDK for Java to access data from a MongoDB dataset in a Couchbase cluster. The code samples show how to connect to the Couchbase cluster, perform key/value operations, execute secondary lookups via N1QL queries, and update an existing document using sub-document operations. The article also covers more advanced topics such as batching, reactive querying, and N1QL data access with secondary lookups. The goal is to provide a comprehensive guide on how to migrate MongoDB data to Couchbase using the Java SDK.

The article provides a comprehensive guide on migrating data from MongoDB to Couchbase, focusing on exporting data from MongoDB's BSON format, transforming it to suit Couchbase's JSON format, and importing it into Couchbase using its Eventing service for real-time transformation. The process involves using MongoDB Compass to export collections, configuring various Couchbase buckets for incoming, transformed, and error data, and deploying a JavaScript-based Eventing function to handle data transformation. The guide highlights the differences between MongoDB's Extended JSON and Couchbase's JSON, demonstrating how specific data types like ObjectId and Date are transformed during the migration. It also details using the cbimport utility for data import and provides instructions for setting up the necessary Couchbase infrastructure. The article concludes with suggestions for further exploration of Couchbase features and resources for additional learning and integration.

Couchbase Analytics is designed to handle ad hoc queries efficiently through parallel processing, but issues can arise when query modifications lead to performance drops, as demonstrated by a customer's experience. The underlying data consists of JSON documents detailing client charges, with fieldnames optimized for space efficiency and dates stored in epoch format. An initial query aimed at retrieving top revenue customer/service combinations within revenue code categories was complicated by a volume discount calculation, drastically increasing execution time. The slowdown was traced to the use of the "listify" function in the query execution plan, prompting a restructuring to compute sums outside a switch-case statement, which improved performance significantly. The article encourages users to report rare occurrences of the "listify" function in execution plans to Couchbase, as the optimizer continues to evolve, with improvements such as automatic rewriting of queries involving switch-case with aggregate functions planned for future releases. The piece concludes with resources for further learning about Couchbase's query language and acknowledges contributors to the discussion.

The Couchbase Connect conference in 2020 is transitioning to a virtual format, inviting developers and architects to submit technical presentation proposals through Sessionize. The event, scheduled for October 14th and 15th, will feature pre-recorded sessions ranging from 15 to 45 minutes, accommodating various expertise levels from introductory to expert. Topics of interest include a wide range of technologies such as ACID, Android, AWS, Kubernetes, and more. Prospective speakers are encouraged to create engaging titles and concise descriptions for their proposals, which can be submitted via the Call for Speakers form.

Online learning platforms like Udemy offer a variety of courses at affordable prices, including one on deploying microservices to the AWS Cloud using Kubernetes, which features a hands-on project called Fleetman. This application monitors a fleet of trucks by generating and storing geo-positioning data, initially using MongoDB for data persistence. The app is architected with microservices, including a Position Simulator, Position Tracker, and API Gateway, all developed with Spring Boot and Angular frameworks. The author explores migrating Fleetman’s database layer from MongoDB to Couchbase, leveraging Couchbase's features such as N1QL, Analytics, and Eventing to enhance the app's capabilities. The migration process involved updating the Spring Boot libraries, implementing key generation, and configuring Couchbase settings. The author successfully tested the migration locally, with the potential for containerization and deployment on Kubernetes, suggesting further extensions and improvements for the Fleetman application.

This post explores the use of Elasticsearch Connector 4.2 with Couchbase, a distributed JSON document database, to replicate and transform data from Couchbase to Elasticsearch using an Ingest node pipeline. The process involves installing and configuring Couchbase, Elasticsearch, Kibana, and the connector, building and deploying the pipeline, and verifying its functionality. The post assumes a basic understanding of these components and provides step-by-step instructions for replicating the travel-sample landmark dataset from Couchbase to Elasticsearch with transformations such as converting field values to uppercase and creating a new field with current date-time.

Couchbase Cloud is a managed NoSQL database-as-a-service offering that provides flexibility, control, and cost savings for organizations looking to move their data to the cloud. With its in-memory key-value system, JSON document schema, N1QL query language, and linear scalability, Couchbase Cloud offers performance and flexibility comparable to on-premises solutions. Unlike other DBaaS providers, Couchbase Cloud is installed inside a virtual private cloud account, allowing customers to own and control their data while seeing exactly what they're paying for. The service also features cross-data center replication, multidimensional scaling, and dynamic pricing options. Analysts and customers alike have praised Couchbase Cloud for its flexibility, control, and cost-effectiveness, with some reporting a 10x reduction in cost and a 100x increase in performance compared to other DBaaS offerings.

Couchbase Server uses two ejection methods for managing memory: Value-only ejection, which maintains document keys and metadata in memory while removing only the document values, and Full ejection, which removes keys, metadata, and values. Value-only ejection offers better performance by requiring more system memory, making it suitable for applications with high read latency demands, whereas Full ejection minimizes memory requirements, which is beneficial when working within budget constraints or limited hardware resources. The choice between these methods depends on factors such as memory availability, budget, and dataset size, with Full ejection being advantageous for large datasets that exceed typical memory configurations. Switching between ejection methods requires bucket restarts, leading to downtime, so it's essential to choose wisely before deployment. Performance can be improved through the use of Upserts and dedicated fast disks, particularly with Full ejection buckets, and monitoring bucket metadata overhead can indicate when a switch to Full ejection might be necessary due to limited memory.

The database industry has realized that full-text search and SQL are two sides of the same coin, with text search needing further query processing and query processing requiring efficient filtering for text patterns. As a result, various databases have added full-text search capabilities within themselves. Couchbase Full-Text Search (FTS) is designed to transparently search across multiple fields within a document, providing language-based stemming, fuzzy matching, and relevance-based search results. FTS achieves this on an inverted index and supports aggregation via search facets. In the NoSQL world, Lucene is a popular search index, and Elasticsearch and Solr are search servers based on Lucene that have added SQL support. Couchbase introduced FTS, which has been followed by support for search within N1QL in Couchbase Query service. This allows developers to use SQL-like syntax with full-text search capabilities, providing benefits such as predicates, operators, and functions that enhance query processing and filtering. The integration of FTS with N1QL enables features like join processing, common table expressions (CTEs), and the use of `SEARCH()` in DML statements, making it a powerful tool for querying and manipulating JSON data.