November 2018 Summaries

Improving application performance in Couchbase can be achieved by creating indexes for common queries, although the process can be challenging when using Spring Data due to the lack of clarity in generated queries. This issue can be addressed by setting the DEBUG log level for the AbstractN1qlBasedQuery class, which allows N1QL queries to be printed in the log, similar to Hibernate's show_sql attribute. Users can configure this by modifying the application.yml or logback.xml files, with the latter requiring the addition of the logback-classic dependency. Once configured, the application will output detailed N1QL query logs, aiding in the creation of effective indexes. Some users have reported difficulties with logging in certain Spring Boot and Spring Data Couchbase versions, suggesting the use of specific packages from Spring Data 4 onwards to resolve such issues.

Couchbase and AWS are enhancing their long-standing partnership, with Couchbase becoming a featured launch partner for the AWS Marketplace for Containers. This marketplace offers a curated selection of container images within the AWS ecosystem, ensuring users can access official Couchbase Container Enterprise Edition images with ease. The marketplace also provides guidance on deploying the Couchbase Autonomous Operator on AWS's Elastic Kubernetes Service (EKS), a managed Kubernetes service. This collaboration aims to simplify infrastructure management by combining the advanced cluster management capabilities of Couchbase with the streamlined Kubernetes experience offered by AWS, allowing users to easily set up and manage their environments.

The Couchbase Data Platform now supports hybrid transaction/analytical processing (HTAP) workloads and its massively parallel processing (MPP) query engine allows for fast ad-hoc analytical queries without impacting performance. The CData drivers provide a SQL interface for NoSQL data sources like Couchbase Analytics, translating SQL queries into SQL++ to leverage the server-side capabilities of Couchbase Analytics for real-time data in BI and analytics tools. The drivers flatten objects using dot notation and array elements using 0-based indexing, allowing connectivity with tools that generate SQL queries internally. As Couchbase Analytics and its interfaces continue to evolve, so will the CData drivers, implementing new query features while leveraging powerful data processing capabilities to enable easy data visualization and reporting in SQL-enabled tools.

SQL++ is an emerging database query language designed to handle both structured and semi-structured data, particularly focusing on JSON databases, by extending the traditional SQL language. Initially developed at the University of California, San Diego, SQL++ has been adopted by systems like Apache AsterixDB and Couchbase, which utilize its capabilities in handling flexible data structures. Unlike standard SQL, which returns results in a tabular format, SQL++ delivers results in JSON format, making it well-suited for modern data applications. It retains the high-level declarative nature of SQL while introducing extensions to accommodate more complex and varied data types without requiring a predefined schema. Notably, Donald D. Chamberlin, one of the original co-designers of SQL, has published a tutorial book on SQL++ with practical examples for Couchbase Analytics. The language supports advanced features such as grouping, aggregation, and the UNNEST function, enabling comprehensive data manipulation and analysis in environments that deal with nested and diverse data structures.

The new window functions in SQL enable declarative and effective business reporting, improving query performance and developer/business-analyst efficiency dramatically, allowing for the creation of complex dashboards with various ranking and aggregation functions, reducing execution time from hours to seconds. The use of these functions simplifies query writing, reduces complexity, and improves overall TCO (Total Cost of Ownership).

In the context of Full-Text Search (FTS) indexing, creating efficient and effective indexes requires understanding and applying best practices, such as disabling default type mapping and specifying custom type mappings to include only necessary fields. Optimal field indexing is crucial, with options like indexing, storing, including in the _all field, and incorporating term vectors influencing index size and search capabilities. Numeric fields, often used for IDs, should be handled with caution; using text type and keyword analyzers instead of number types can reduce index size and improve performance. Future enhancements aim to provide more configurability in the index creation process, allowing for further fine-tuning at the storage level.

The Couchbase Autonomous Operator version 1.1.0 release improves the end-user experience by enhancing stateful services and persistent volumes, best practices, log collection, log retention, and cluster management. The new release allows servers to be backed by persistent storage, ensuring data is not lost in case of a server process crashing or accidental deletion. This feature is essential for many reasons, including business-critical information and the need to restore data in the event of a power outage. The Operator also introduces improvements to log collection, retention, and cluster management, making it easier for users to deploy supportable clusters and collect logs. These enhancements aim to prevent data loss scenarios and ensure that logs are available for collection, allowing diagnosis and solution to root causes of crashes.

Couchbase has released version 1.1.0 of its Autonomous Operator, building upon the initial 1.0 version which was the first NoSQL production-ready Kubernetes operator for open-source Kubernetes and Red Hat OpenShift. This new version addresses user feedback by introducing supportability improvements and bug fixes, particularly focusing on log management for ephemeral pods. The update allows for more efficient storage use by persisting only Couchbase logs, with options to control log size and retention. These changes enable specific stateless Couchbase node deployments in production, provided that persistent log volumes are attached. Additionally, logs collected via cbopinfo are now automatically redacted of sensitive information, and new functionalities have been added to accommodate a wider range of environments. Users are encouraged to review best practices and release notes for details on known and resolved issues.

The text discusses the complexities and considerations involved in selecting a database or database as a service (DBaaS) for modern applications, highlighting the differences between traditional relational databases and the diverse array of NoSQL databases designed to handle web-scale operational applications. It outlines the distinct characteristics and workloads of OLTP (online transaction processing) and OLAP (online analytical processing), emphasizing the need for application architects to understand and evaluate workloads to achieve optimal performance. The text also underscores the growing trend of mixed workloads and the necessity to integrate various database solutions to meet business requirements, while acknowledging the challenges posed by a wide variety of databases with different data models, query languages, and performance characteristics. It concludes by noting that, despite the availability of new database technologies, the fundamental business and application requirements remain unchanged, emphasizing the importance of understanding and translating workload requirements into effective database operations.

The latest episode of The NoSQL Database Podcast features a discussion on Liquibase, with Nathan Voxland as the special guest discussing database source control, and Denis Rosa co-hosting as he works on integrating Couchbase support into Liquibase. This episode, the 25th in the series, follows a previous discussion with Brant Burnett on Linq and is available on major podcast platforms like iTunes and Pocket Casts. Listeners can engage with the hosts via Twitter, and those interested in Couchbase can explore more through the Couchbase Developer Portal.

Couchbase was included in the Operational DBMS Magic Quadrant for 2018, but fell short on the license revenue requirement. The company's revenue grew 60% year over year and received several recognitions, including being voted best in-memory database by readers of DBTA. Couchbase also launched two major releases of its flagship product and earned "Top Rated" status on peer reviews site TrustRadius. Despite not making this year's MQ, the company is still visible through other Gartner reports, such as the State of the Operational DBMS Market, 2018, and the "Other Vendors to Consider" report. Additionally, Couchbase will be featured in Forrester Wave on NoSQL Databases, due out in March 2019.

Couchbase Eventing Service, introduced in Couchbase Data Platform 5.5 and enhanced with Timers in version 6.0, offers various best practices for effective use. It is essential to avoid using the Debugger in production environments due to potential thread allocation issues, and to manage application log files according to log rotation policies. Functions should not cause cascading writes between buckets, and JavaScript code should be encapsulated in try-catch blocks to handle runtime exceptions. Proper management of the Metadata bucket is crucial to prevent loss of checkpoint information and ensure data integrity, recommending against its use for other applications and advocating for sufficient replicas to prevent data loss. Infrastructure monitoring and provisioning should rely on statistics from the Administration console, with appropriate adjustments made to Eventing nodes and resources if function backlogs increase or timeouts occur. When using timers for data-archival, ample time should be allocated to ensure document availability. Understanding feed boundaries and prioritizing 'From Now' processing can optimize performance, while nodes running the Eventing Service should be core-heavy and not host other Couchbase services to maximize compute efficiency. Further insights into the service's capabilities are demonstrated in presentations from Connect Silicon Valley and Connect New York in 2018.