May 2019 Summaries

Couchbase Lite, an embedded NoSQL database initially available for iOS, Android, and .NET platforms, is expanding its reach with the introduction of Couchbase Lite For C, aimed at supporting emerging Internet of Things (IoT) applications and other platforms like embedded Linux boards. This new implementation features an ANSI C API, making it accessible to various languages via bindings and suitable for modest hardware such as the Raspberry Pi. It operates independently of high-level runtimes and requires minimal operating system dependencies, while offering full source code under an Apache 2 license. Though the project is in its alpha stage and not officially supported commercially yet, the development is open-source and welcomes contributions to enhance its functionality and platform support. The architecture builds on the existing Couchbase Lite 2.x architecture, and a Python 3 binding is provided as proof of concept, with an emphasis on ease of use and cross-platform compatibility.

Couchbase Mobile 2.x extends the capabilities of NoSQL JSON Document Store to mobile platforms such as iOS, Android, and .Net, enabling developers to create applications with offline-only and offline-first functionalities. It offers flexible deployment options, including using Couchbase Lite as a standalone embedded database in mobile apps, synchronizing data between mobile clients with a Sync Gateway, or syncing data to a Couchbase Server for cloud storage. The accompanying tutorial guides users on integrating Couchbase Lite 2.x in standalone mode within Android apps, focusing on database operations like creating, updating, and deleting documents. By following the tutorial, developers will gain insights into utilizing Couchbase Lite as a local data store, with further tutorials promised for expanding the app's functionality. For additional support, users are encouraged to reach out through various platforms such as Twitter, email, or the Couchbase Forums.

Business applications often rely on NoSQL databases, which have evolved from "No SQL" to "Not Only SQL" by incorporating SQL-like query capabilities to manage structured and semi-structured data. This evolution addresses various challenges through technologies like CQL for Cassandra, N1QL for Couchbase, and SQL extensions for CosmosDB, while MongoDB offers similar commands without direct SQL support. The article highlights the importance of evaluating SQL support in NoSQL systems based on factors like language support, indexing, query optimization, and performance. Key differences are noted in SQL-like functionalities such as joins, aggregations, and transactions, with Couchbase offering the most comprehensive SQL support among the discussed databases. Although NoSQL databases aim to support SQL, their capabilities and limitations vary significantly, necessitating thorough analysis during selection for business applications.

Luca Christille, a self-taught developer and Google certified Android developer, has created Fluttercouch, a package that integrates Couchbase Mobile functionality into Flutter apps, facilitating data synchronization for offline-first mobile applications. His experience with various NoSQL databases led him to choose Couchbase for its efficient data synchronization, which is crucial for many mobile apps. Originally focused on native app development, Luca transitioned to cross-platform solutions using Flutter, an open-source SDK for building high-performance mobile apps. Fluttercouch allows developers to avoid boilerplate code by managing Android and iOS native layers, enabling easy database operations and data replication through Dart. The package is open-source, with Luca actively maintaining it and inviting community contributions to enhance its features and capabilities, such as the N1QL Query abstraction layer and support for additional operations.

The blog post discusses the setup and functionality of a Couchbase Analytics cluster, demonstrating how it can be used to create real-time visualization dashboards with tools like Tableau. Couchbase Server, a hybrid NoSQL database, supports both operational and analytical workloads, and its version 6.0 introduces "NoETL for NoSQL," allowing ad-hoc analytical queries on JSON data without requiring data transformation. The blog presents a demo scenario involving a fictitious car fleet management company, showcasing how operational data from cars can be analyzed in real-time without affecting data transmission. The post details the process of setting up a Couchbase Analytics cluster in just five clicks, importing datasets, creating shadow datasets for analytics, and exploring the data using SQL-like queries. It further explains connecting Tableau with Couchbase Analytics to create a real-time dashboard, emphasizing the challenges enterprises face in operational analytics and how Couchbase's hybrid architecture addresses these by simplifying analytics without the need for data lakes and complex ETL processes.

The Full Text Search (FTS) indexing system in Couchbase has evolved with the introduction of Scorch, an advanced and optimized index type that addresses scalability concerns and improves performance. The previous upside_down indexing format was replaced by Scorch due to its limitations, including huge index size amplification, data deduplication potentials not being tapped, and less friendly representation for natural language queries. Scorch follows a segment-based index architecture, with each segment composed of term dictionary, postings lists, frequency norms/location details, stored fields, and docValues, which are optimized using Finite State Transducers, bitmap representations, varint encoding, compression techniques, and columnar representation. These optimizations have resulted in significant index size reduction (up to 4X) and query performance improvements (up to 20X) for many queries.

DreamFactory is a key topic of discussion in the latest episode of The NoSQL Database Podcast, where Jason Gilmore shares his expertise on using DreamFactory to generate and manage APIs over various platforms, including databases. Jason introduces listeners to DreamFactory, highlighting its ability to handle NoSQL data, and discusses several use cases, such as customer 360 integration and business logic implementation. The conversation also touches on additional non-database integrations, performance optimization, indexing, logging/monitoring, and profiling, providing a comprehensive overview of DreamFactory's capabilities.

The text discusses the role of Couchbase Sync Gateway in the Couchbase Mobile stack, which facilitates secure data synchronization for offline-first mobile apps using a websockets-based replication protocol. It highlights a tutorial that extends the capabilities of Couchbase Lite clients by incorporating data synchronization through the Sync Gateway, emphasizing the importance of offline data storage and cloud synchronization for multi-device accessibility. The tutorial guides users on setting up the Sync Gateway, configuring data routing and access control, and enabling data replication and real-time notifications using Xamarin as an example. The text concludes by noting that while the tutorial uses the Sync Gateway in "walrus mode" without cloud data persistence, future tutorials will address this limitation by integrating the Couchbase server for persistent cloud storage.

Spring Data offers a simplified programming model for accessing both relational and non-relational databases, gaining popularity among Java/JVM developers due to its minimal learning curve and reduced codebase requirements. Despite its ease of use, developers often face performance challenges when scaling, prompting the creation of a comprehensive tutorial that delves into the workings of Couchbase's implementation and provides recommendations to enhance performance. The full tutorial can be accessed for a detailed exploration of these insights.

Helm is a tool used for managing applications on Kubernetes platforms, allowing users to install and manage resources in an all-or-nothing approach. Helm streamlines the installation process by representing a group of resources as a single package that can be installed and upgraded as a unit. The Couchbase Autonomous Operator helm chart enables users to deploy a Couchbase Cluster into their Kubernetes environment. To start using Helm, users need to install the client tool and create an RBAC service account for the Tiller service. Once the Tiller is set up, users can install the Operator Chart and the Couchbase Cluster Chart using Helm commands. The Operator Chart deploys the dynamic admission controller, the Operator itself, and all necessary resources within the cluster. The Couchbase Cluster Chart provides a templated version of the Couchbase cluster type, with default values that can be customized for specific use cases. Users can view the list of installed charts using `helm list`, manage releases, and install additional chart dependencies as needed. Additionally, Helm supports template overriding to create customizations, such as installing a cluster with TLS and Persistent Volumes. The Autonomous Operator provides a deep dive into its features and functionality, including networking capabilities.

In the realm of application development, efficient query design and data integrity are paramount, with database performance often addressed later by specialists. To address this, Couchbase 6.5 introduces the N1QL Index Advisor, a feature that recommends GSI indexes to optimize query response times, enhancing developer productivity and allowing early performance evaluation. The Index Advisor focuses on the Indexing Service (GSI) without assessing Key-Value or Full Text Search retrieval techniques. It suggests indexes that align with best practices, even if existing indexes are already in place, and can identify inefficiencies in current indexing strategies. By providing actionable insights and recommendations, the Index Advisor facilitates improved query performance and productivity, supporting a proactive approach to database management throughout an application's lifecycle.

Couchbase Lite 2.x is a versatile NoSQL embedded data store for mobile platforms, including iOS, Android, and .Net, that offers powerful features such as SQL-like queries and Full-Text-Search capabilities. In a new tutorial, users are guided through utilizing multiple local instances of Couchbase Lite within Xamarin apps, allowing for one shared instance with pre-loaded data and another for user-specific data. This setup enhances user experience by minimizing data-fetching latency and startup times. The tutorial extends a "User Profile" app by incorporating a prebuilt "University" database to demonstrate the use of the QueryBuilder API for querying local data and persisting user profiles. By the end, participants will have learned how to integrate prebuilt databases and execute queries in a local context, setting the stage for future enhancements that leverage more features of Couchbase Mobile.

Couchbase Server's SDKs have evolved significantly over time, streamlining complex operations and improving consistency across versions. The transition from SDK 1.0 to SDK 3.0 reflects enhancements in handling data with Couchbase, from basic read/write operations using the Memcached protocol to supporting advanced features like N1QL for querying JSON documents and Role-Based Access Control. SDK 3.0 introduces Scopes and Collections, optimizing resources for multi-tenancy and microservice architectures, and aligns with Couchbase Server 6.5+ to offer a more intuitive programming model. All SDKs adhere to Semantic Versioning, which guides the release process with distinctions between major, minor, and patch updates. This version also emphasizes usability by refining overloaded interfaces and enhancing cross-SDK conformance, while retaining familiar structures like Cluster and Bucket objects. The new architecture facilitates more efficient Key/Value operations at the Collection level and supports global queries through the Cluster object. The release of SDK 3.0 includes alpha versions available via NuGet, enabling developers to leverage these improvements while acknowledging feedback for future iterations.

The Couchbase Autonomous Operator 1.2.0 introduces significant updates and new features aimed at enhancing functionality and user experience. This release includes major improvements such as cloud certification on major platforms like Amazon EKS, Google GKE, and Microsoft Azure AKS, addressing storage issues with Kubernetes' new volume binding mode, and supporting automated upgrades for the Couchbase data platform to ensure safe, online updates. TLS certificate rotation has also been improved, providing mechanisms for easy handling of certificate expiries or compromises. The addition of a dynamic admission controller integrates deep validation directly into the Kubernetes API, enabling better error reporting and backward compatibility with older configurations. Logging improvements now offer an interactive menu for log collection and extended logs for failed pod creation. The release also refines Kubernetes RBAC permissions, requiring explicit roles for the Operator's functions, enhancing security and maintainability. The update focuses on upgradability and ease of use, encouraging users to explore its advanced capabilities.

The Autonomous Operator version 1.2.0 introduces new network features to address the limitations of previous versions, allowing for wider audience support and diverse network architectures. The Kubernetes networking options include routed networks and overlay networks, which encapsulate pod-to-pod messages in protocols such as GRE or VXLAN. Node ports provide a generic mechanism to address a pod regardless of whether an overlay network is used. Establishing Cross-Datacenter Replication (XDCR) connections between two Couchbase clusters in different Kubernetes clusters relies on node ports and load balancer services. The new enhancements also include external client access, UI access, end-to-end encryption, and simplified network configuration through the use of load balancer services and DNS resource records.

Couchbase Autonomous Operator 1.2 introduces several customer-requested features, including automated upgrades of Couchbase clusters, integrated resource validation via an admission controller, Helm support, public connectivity for Couchbase clients, rolling upgrades of Kubernetes clusters, TLS x509 certificate rotation, and a unified log collection experience. The release also officially supports public Kubernetes services on GKE, AKS, and EKS, with a demonstration using GKE to set up a Kubernetes cluster and deploy the Autonomous Operator and Couchbase Cluster. The blog outlines detailed steps for deploying Kubernetes clusters with server group auto-failover, TLS certificates, and persistent volumes, emphasizing resilience and failover capabilities. It highlights the Autonomous Operator's ability to automatically manage cluster health and recovery, ensuring minimal downtime and robust disaster recovery through XDCR replication. The release aims to enhance operational efficiency and supportability, enabling users to quickly identify and resolve issues.

The NoSQL query optimization process is crucial for efficient execution of queries, and the optimizer selects an optimal index and access path to achieve this. However, in MongoDB, the query plan language is simplistic, and the optimizer only supports index selection, not query rewrite or join reordering. The restriction on creating multiple text indexes in MongoDB is due to the inability of the optimizer to validate these indexes against a text predicate, which can lead to performance issues. In contrast, Couchbase's N1QL has added support for text indexes, allowing users to create any number of text indexes and enabling the optimizer to choose a qualified index and use it. This highlights the importance of understanding the query optimization process in NoSQL databases and the need for robust optimization techniques to manage performance issues.

Couchbase Autonomous Operator 1.2.0 has been launched, offering significant advancements in managing Couchbase on Kubernetes with enhanced automation and manageability features. This release facilitates automated upgrades of both Couchbase and Kubernetes clusters, integrates resource validation through an admission controller, and supports Helm for deployment management, simplifying the installation and tracking of resources. It also introduces public connectivity for Couchbase clients, full production support for cloud-managed Kubernetes services like Amazon EKS, Azure AKS, and Google GKE, and ensures smooth rolling upgrades to prevent downtime. Additional features include TLS certificate rotation and improved log collection capabilities, all developed in close collaboration with customers and partners.

Couchbase Mobile 2.5 has been released, enhancing the NoSQL data platform for mobile applications with features aimed at improving data synchronization, machine learning integration, and logging capabilities. Building on the architecture of Couchbase Mobile 2.0, this version introduces a delta sync feature to reduce bandwidth by syncing only changed parts of a document, beneficial in bandwidth-constrained environments. Additionally, a new predictive Query API allows real-time machine learning predictions on the edge, leveraging mobile-optimized models for enhanced performance. Enhanced logging support includes continuous file-based, custom, and console logging, alongside fine-grained replication eventing, which allows applications to track the sync status of individual documents. Fine-grained filtered replication and the reintroduction of time-to-live support for document expiration further optimize data management. The Sync Gateway also sees improved monitoring capabilities with comprehensive statistics for better deployment insights. Overall, Couchbase Mobile 2.5 aims to provide more efficient, secure, and advanced functionalities for mobile data applications.

CURL is a utility function now available as part of Couchbase Eventing Functions with several enhancements and improvements coming to its usage, including automatic parsing and marshaling of data types, support for session cookies and additional authentication types. These changes are part of the upcoming Couchbase Server 6.5 release, which also includes improvements and enhancements to the Eventing framework. To use CURL in an event, a URL alias must be created in Function Settings, specifying the base URL for the desired API call. A simple example of using CURL is shown, demonstrating how to make an HTTP request to the Open Weather Map API and retrieve weather data, which will be saved to a document. The new CURL function offers more flexibility with its usage, including support for HTTPS connections and headers, and users are encouraged to update their code to take advantage of these improvements before the beta release in summer 2019.