March 2022 Summaries

In order to optimize performance when querying data from external storage in Couchbase Analytics, it is crucial to consider several factors including internet bandwidth and distance from the storage provider, number of Couchbase Analytics nodes and partitions, and prefixes and include/exclude filters. Optimal organization of data can significantly impact performance, and using prefixes and filters can help reduce the amount of data transferred, resulting in improved performance and resource utilization. By applying data organization best practices and corresponding external collection definitions, users can get the best possible performance from Couchbase Analytics when accessing external data.

The official Couchbase Discord server is now live, providing a space for developers to engage with each other and with Couchbase staff in real-time. The server complements existing community forums and support channels, allowing users to exchange ideas and connect on a new level. Users can ask questions, learn about upcoming events and product updates, and share their experiences with others. The Discord instance is still evolving, with some features and bots missing, but the community is encouraged to provide suggestions for improvements through the #suggestions channel.

This post introduces building microservices using Python and Couchbase, providing a fully scalable solution. Microservices architecture splits application components into small, independently scaled pieces that serve one functional area of the overall application. A good microservice should be lightweight, stateless, and designed for failure, with APIs such as RESTful interfaces working well in this environment. Couchbase is used to store user profile data due to its performance, scalability, and ease of use. Python is chosen as the programming language due to its accessibility, ease of learning, and scalability with multiprocessing. A sample application walkthrough demonstrates how to create a simple RESTful interface using Python and Couchbase, showcasing the concept of microservices architecture in practice.

The text discusses the challenges of aligning data governance with complex data management policies, highlighting how Couchbase Mobile can facilitate the implementation of data retention policies within enterprises. Data governance, as defined by NIST, involves managing data assets formally across an enterprise and includes aspects like data privacy, security, integrity, and consistency. With distributed data storage becoming more complex, Couchbase Mobile 3 aids in enforcing data retention policies by ensuring that unauthorized users cannot access specific documents after access is revoked. The Sync Gateway access control model plays a central role in this process by managing document access through channels and automatically purging documents from devices when access is revoked. This capability is exemplified in use cases like enterprise sales teams, cruise lines, and retail stores, where document access needs to be dynamically managed. Couchbase Mobile offers features like automatic document purging, fine-grained access control, and data encryption, supporting the broader objectives of data governance.

The blog series provides a comprehensive guide for developing an end-to-end mobile application utilizing Couchbase Lite, focusing on enabling offline functionality by embedding a local database to enhance user experience. It is segmented into three parts: the first part details the process of building a mobile app that uses Couchbase Lite as an embedded database, the second part will introduce Full Text Search capabilities on the device, and the third part will cover data synchronization with peer devices and a centralized server. The demo application, Rateit, allows users to send rating requests, receive ratings from others, and manage rated requests, showcasing functionalities such as creating and managing a database, writing and querying documents, and setting expirations. The project is built using Android Studio, leveraging Java programming, and includes practical examples and code snippets to guide developers through setting up the app, with future installments promised to explore advanced features like peer-to-peer synchronization.

Amid rising energy costs and geopolitical tensions, oil and gas companies face challenges in increasing production to meet demand, partly due to inefficient data management. The complexity of production and processing facilities, compounded by the vast amount of data generated, often leads to inefficiencies and underutilized capacity, with typical offshore platforms operating at only 77% of their potential. Leveraging data analytics, including predictive analytics and machine learning, can bridge these performance gaps by providing insights into equipment conditions and potential failures, thus enhancing efficiency and production. Edge computing emerges as a pivotal solution, enabling real-time data processing closer to the source, even in remote locations, thereby reducing latency, bandwidth costs, and dependency on internet connectivity while ensuring data privacy and reliability. A distributed cloud-to-edge database architecture is crucial for synchronizing and processing data effectively, allowing oil and gas operations to respond swiftly to dynamic conditions, ultimately contributing to increased production and potentially lower fuel prices.

Document databases offer schema flexibility, which can be advantageous but sometimes requires validation to ensure data integrity, particularly in applications relying on specific document fields. The article explores how to validate JSON documents using the Python library pydantic, emphasizing the necessity of schema validation in NoSQL databases, where data structure is often perceived as unnecessary. It demonstrates setting up a JSON schema for user profiles, identifying mandatory and optional fields, and employing the open-source Faker library to generate testing data. Using pydantic, developers define a model with expected data types and validate fields, such as phone numbers, to prevent data inconsistencies. The article includes an example of schema validation by loading data into Couchbase Capella and checking for errors using pydantic's parse_obj method. The conclusion highlights the utility of pydantic for JSON document validation and suggests further integration of schema verification during data operations, with additional resources and code available on GitHub.

Couchbase emphasizes its role as a high-performing, flexible, and scalable database solution for modern applications, adaptable across cloud and edge environments, as demonstrated in their recent fiscal 2022 earnings call. The call highlighted customer success stories, showcasing Couchbase's influence across various industries, including logistics, cloud contact centers, communications, social security, chemical manufacturing, and financial services. Notable examples include Anteraja's choice of Couchbase Capella over PostgreSQL and MongoDB for its superior performance, Five9's adoption for supporting global expansion, and CNAF's modernization efforts using Couchbase's integrated platform capabilities. Additionally, Couchbase's long-term partnership with a Fortune 500 shipping company underscores its commitment to delivering technical excellence and customer-driven outcomes. The company continues to motivate its team through a strong focus on solving challenging problems and enabling customer transformation, inviting potential users to explore their services through a free trial of Couchbase Capella.

With Couchbase Capella, companies can now provision an entire Couchbase cluster in minutes, eliminating the need for specific expertise in deploying and managing infrastructure. This is achieved through a fully hosted DBaaS that provides a user-friendly GUI, as well as an API that allows for automation of provisioning, configuration, and security tasks. The use of automated provisioning saves time and decreases toil, allowing developers to focus on delivering business objectives and competitive advantage. A Python script has been created to demonstrate the capabilities of Capella's API, enabling users to provision a cluster with just two arguments: project name and cluster name. This automation allows for faster deployment of clusters, reducing the time required to set up and manage Couchbase environments. The example configuration provided by the script demonstrates how to create a hosted cluster with a single AZ, node size, and all services, enabling users to connect their application to the cluster without having to go through the configuration GUI. As the Capella API continues to evolve, it will provide more functionality for automating tasks, allowing developers to focus on delivering business value rather than managing infrastructure.

The Couchbase Mobile 3.0 release introduces support for SQL++ queries, allowing developers to reuse code between Couchbase Server and Mobile applications. To test the query reusability, a sample mobile application was created in SwiftUI with Combine, which displays all ice cream shops in the United States. A simple query is written using the Query Editor from the Couchbase Server Web Console, and its performance is compared to running the same query on the mobile app. The results show that the query runs faster when an index is added to the database. However, reusing queries between Couchbase Server and Mobile may require modifications to optimize performance due to differences in storage and query engines. It's recommended to read through the excellent documentation on SQL++ for Mobile – Differences from SQL++ for Server and SQL++ Query Strings for valuable information on changes needed to use queries in mobile applications.

Couchbase has suspended all sales to Russia in response to the Russian military actions in Ukraine, which it states are contrary to its values. The company is also supporting the people affected in Ukraine by matching employee donations to organizations such as DirectRelief, the International Committee of the Red Cross, the International Rescue Committee, and UNICEF. Couchbase expresses solidarity with Ukraine and aligns itself with the international community's call for a swift resolution to the conflict, advocating for a return to peace.

In the second part of a series on monitoring Stellar assets using Couchbase and Python, this post explores loading and querying asset trades from the Stellar Decentralized Exchange, with a focus on Stellar Lumens (XLM) as the native asset. The process involves accessing an account's list of trades, loading them into Couchbase, and running queries to visualize trends and investments through the Couchbase web console. The post provides a step-by-step guide, including code examples for connecting to Couchbase, querying the Stellar Horizon service, parsing trades, and storing them as documents with unique keys. It highlights the trade details, such as asset types, counter assets, and transaction pairs, using REST API requests, and demonstrates how to create a basic index and perform queries using N1QL. The discussion aims to make these technical concepts accessible to beginners and hints at future posts that will delve deeper into document joins and calculations for evaluating trade profitability.

Couchbase is proud to support International Women’s Day 2022, with this year's theme being #BreakTheBias. The company spoke with women from various departments and locations around the globe, asking them about their experiences overcoming biases in the workplace and what advice they would give to women navigating careers in male-dominated industries. Michelle Smellie, Director of People – EMEA, London, shared how she has dealt with micro-aggressions as a black woman, while Mary Roth, VP Engineering Ops, California, emphasized the importance of listening more than speaking in order to build relationships and anticipate others' reactions. Rangoli Mathur, Manager of Software Engineering, Dallas, Texas, discussed overcoming biases by focusing on outcomes and treating people with respect. Molly Ng, Field Marketing Director APAC, Singapore, talked about challenges faced while working from home and the importance of future-proofing one's home workspace. The women shared their experiences and advice to empower other women in technology, highlighting that breaking bias is an ongoing process and Couchbase is committed to hiring more women in technology.

Web-based clients can now manage and interact with Couchbase Sync Gateway 3.0 through a RESTful interface using the Postman API client, which facilitates the configuration, administration, and monitoring of the database. The blog post provides a detailed guide on how to use Postman collections, which are organized sets of saved requests, to quickly set up and customize Sync Gateway environments for various administrative tasks. Available on GitHub, these collections include admin, public, and metrics APIs for handling database management, access controls, end-user interactions, and cluster-level statistics. The guide also describes the process of deploying Couchbase Server and Sync Gateway using Docker, along with instructions on setting up users, executing requests, and monitoring activities through Postman. While the collections are a helpful resource for administrators and developers, they are not officially supported by Couchbase, and contributions for updates are encouraged.

Couchbase Lite for C, introduced with the Couchbase Lite 3.0 release, extends the client-side document-database library’s capabilities to C and C++ applications through an ANSI C API, offering a single small native library with no language runtime dependencies and broad OS compatibility. It targets data-rich IoT devices running Linux, as well as non-embedded Linux distributions like Ubuntu and Debian, and supports macOS and Windows, making it an attractive option for lightweight servers and cross-platform apps. Although Couchbase Lite for C is not intended to match the scalability of Couchbase Server, it provides a JSON-document data model with a small footprint and zero configuration. The library's cross-platform API benefits applications with C/C++ core code and, through foreign function interfaces, can potentially be accessed by various programming languages. Couchbase has provided sample Python and Rust bindings to facilitate community-driven development of additional language bindings.

On December 14, 2021, the US Patent and Trademark Office granted Couchbase its first patent for a novel approach to cost-based optimization (CBO) for document-oriented database queries, marking a significant achievement for the company's engineering team. CBO is a process that selects the most efficient way to execute a database query by evaluating the costs associated with various resources such as memory, CPU, and disk usage, and has traditionally been used in SQL databases. Couchbase's implementation of CBO for NoSQL databases, specifically for their N1QL queries, is a pioneering advancement that distinguishes Couchbase from other NoSQL database providers, as it allows for more efficient query execution and resource management, leading to improved performance and reduced costs. This innovation has important implications for enterprises relying on Couchbase for mission-critical applications, as it enhances database performance and aids in meeting service-level agreements (SLAs).

The Sync Gateway 3.0 release introduces a new modular, cluster-aware configuration approach that replaces the previous node-level JSON file-based method. This updated system enhances the configuration workflow by allowing Sync Gateway nodes to be bootstrapped with a JSON-formatted configuration file, while subsequent configurations are managed via a RESTful interface. The centralized persistent configuration enables secure, remote administration and ensures that any database configuration changes made through the REST API are automatically propagated to relevant nodes within the cluster, surviving any restarts. This new model supports dynamic management of access control, user management, and database configurations, allowing for a more flexible and scalable setup. Additionally, the concept of database config groups is introduced to tailor database configurations to specific subsets of nodes, offering more granular control in multi-node setups. The legacy configuration model remains supported, providing continuity for existing setups.