October 2017 Summaries

The latest enhancements to Couchbase Server's N1QL feature provide significant improvements, including built-in Role-Based Access Control (RBAC), new system keyspaces for monitoring and managing users, adaptive indexes that simplify the creation of complex queries, and index replicas that distribute query performance across a cluster. These updates aim to enhance the overall performance and usability of N1QL, making it easier for developers to create efficient and scalable queries. Additionally, Couchbase Server 5.0 introduces new features such as the CURL function, which allows querying external JSON data, and improved monitoring tools to help users optimize their queries. With these enhancements, Couchbase continues to strengthen its position as a leader in NoSQL databases.

Full Text Search (FTS) in Couchbase Server provides efficient and natural search capabilities by leveraging Bleve, a search and indexing library written in Golang. Unlike traditional SQL or N1QL queries with wildcards, FTS allows for fuzzy searching, ignoring minor spelling mistakes, and scoring results based on relevance. FTS supports various query types, including Match, Phrase, Fuzzy, Prefix, Regexp, Wildcard, Boolean Field, Conjunction, Disjunction, Boolean, Doc ID, Date Range, Numeric Range, Query String, and more. To utilize FTS, an index must be created within the Couchbase Administrative Dashboard, specifying document properties and mappings. The SDKs for various programming languages provide APIs to create queries, customize fields, and highlight search results with HTML markup. With FTS, developers can efficiently and naturally search through millions of records without sacrificing performance or accuracy.

Fast failover is now available in Couchbase Server 5.0, allowing for improved high availability in distributed databases. To demonstrate this feature, a console application was created using the .NET Core framework to connect to a three-node cluster of Couchbase nodes running inside a Docker host. The application continuously performs reads against the bucket, and at some point, one of the nodes is intentionally stopped to show automatic failover in action. The process involves detecting node failures, initiating fast failover to activate replicas, and ensuring data availability during the transition period. This feature can be useful for environments with solid networking in place, and its implementation demonstrates principles that can be applied to other ASP.NET or ASP.NET Core websites.

In an era of frequent data leaks, enhancing database and web application security is crucial, and Couchbase 5.0 introduces role-based access control (RBAC) to bolster this effort. Developers familiar with relational databases like MySQL and Oracle can now apply fine-grained access control to NoSQL databases by defining specific privileges for various roles on clusters and buckets within Couchbase. The process involves creating user accounts with unique combinations of roles such as Data Reader or Query Select, which dictate the tasks a user can perform. The Couchbase Shell (CBQ) allows users to interact with the database via a command line interface, highlighting the importance of granting only necessary permissions to minimize security risks. When building applications, developers must ensure that the Couchbase SDKs authenticate with RBAC credentials to establish a connection. This approach to access control reduces the likelihood of unauthorized data access and enhances the overall security of NoSQL database environments.

Couchbase Server 5.0, the latest release of the powerful NoSQL technology, has been launched, offering significant enhancements across three main areas: agility and flexibility, performance at scale, and manageability. This release introduces features such as full-text search, adaptive indexing, improved query performance, and role-based access control, along with a new storage engine called Plasma. The release aims to improve customer experiences by enabling faster, more efficient application development, and better management capabilities. Feedback from the beta program has been overwhelmingly positive, and users are encouraged to learn more through upcoming events and resources.

Couchbase Mobile 1.5 is now available for download and marks an important milestone in extending Couchbase Server deployments to remote edge devices, enabling seamless data replication between occasionally connected or disconnected edge devices and Cloud. Customers can mobilize entire buckets of data or specific subsets with flexibility, without changes to existing Sync Gateway configurations, and can take advantage of additional features such as multiple server URLs, secure connections, and a maximum depth limit for document revision trees.

Two years and two weeks ago, we launched N1QL in Couchbase 4.0 on a crisp fall day at Wall Street in New York, which solved the critical business problem of manipulating JSON data as a set using SQL-like queries. The product has grown rapidly with hundreds of customers using it in production, who have continued to innovate building modern scalable applications and contributing to the community on the Couchbase N1QL forum. Since the initial release, we've learned from use cases and enhanced query and indexing language, performance, and capabilities in all releases, including novel indexing technology that has improved performance by orders of magnitude. We're delighted to release the second edition of the N1QL book with updates to reflect the features in Couchbase 5.0, which is now available as a downloadable PDF and special edition guide.

The Couchbase Data Platform has been found to help surveyed enterprises achieve significant benefits, including a $5.41M increase in additional revenue per year, productivity benefits of $1.89M per year, and a five-year ROI of 274%. This is due to the platform's ability to provide scalable and performant NoSQL technology that enables effective interaction and engagement with customers, partners, and employees. By using Couchbase, organizations can improve scalability, performance, application development efforts, DBA operations, and help desk support, resulting in faster new development cycles, faster response times, and more efficient database management. The study also confirms the business value of Couchbase through customer testimonials, highlighting its ability to drive substantial revenue growth, increase productivity, and accelerate digital transformation initiatives.

Couchbase 5.0 introduced role-based access control (RBAC) to provide more granular security compared to the previous bucket password system in Couchbase 4.5, where each bucket was secured individually with a password. With RBAC, access to data is managed through specific roles assigned to users, such as query_select, query_insert, query_update, and query_delete, each parameterized by the bucket they apply to. This new system allows for more precise control over user permissions, addressing the limitations of the previous method where users had either full or no access to a bucket. The update also includes the ability to grant and revoke roles, and introduces new system keyspaces for querying user roles, enhancing security by restricting access based on the sensitivity of data. Despite these changes, legacy access patterns, particularly for passwordless buckets, are preserved during the upgrade process to ensure compatibility with existing applications.

Couchbase Server 5.0 introduces advanced profiling capabilities for analyzing N1QL query performance, providing detailed metrics on execution phases and operators, allowing database administrators to optimize slow queries. Profiling can be enabled at various levels, such as request, service, or on-the-fly, offering insights into the execution tree with statistics like time spent per operator, documents processed, and operator state changes. This detailed data assists in identifying and resolving bottlenecks, such as high kernel time or inadequate index selectivity, which can cause slow query performance. Profiling information is accessible through system keyspaces and can be visualized in the Query Workbench, aiding in performance monitoring and troubleshooting. While profiling offers valuable insights, it requires mindful management to prevent excessive memory usage, particularly when logging extensive execution timings for all queries.

Couchbase places a strong emphasis on customer engagement and feedback, as demonstrated during their user conferences like Connect, where they interact with customers to share best practices and gather insights. The company prioritizes listening to its customers through various channels, including social media and surveys, to understand their needs and challenges, and acknowledges the importance of acting on this feedback to enhance product quality and customer experience. This approach is exemplified by their development of the Engagement Database, a product created in response to customer input on applications aimed at improving customer experiences. By maintaining a cycle of listening and acting, Couchbase aims to foster continuous improvement and positive customer relationships, ensuring that feedback directly informs both their product development and customer service strategies.

Couchbase has released Developer Preview 3 (DP3) of Couchbase Analytics, featuring a rich query language called SQL++ designed for modern NoSQL environments, a common data model that supports flexible-schema documents, and workload isolation to ensure queries do not interfere with operational workloads. DP3 emphasizes unified administration by integrating Analytics as a managed service within Couchbase's Multi-Dimensional Services architecture, including enhancements like integrated cluster management and a consistent user experience through the Couchbase Console. The preview also introduces support for secondary indexes to boost the performance of smaller queries and offers multiple data source options, allowing data to be shadowed from local or remote Couchbase clusters. A practical application of DP3 is showcased in "Beer Analytics," demonstrating the use of SQL++ to analyze beer characteristics by city, highlighting the system's capabilities in real-world scenarios.

Couchbase 5.0 introduces a new feature in its N1QL query language by integrating the CURL function, allowing users to interact with external JSON endpoints through HTTP and HTTPS protocols. This addition enables querying data from various APIs, such as Google Maps, Yahoo Finance, and GitHub, directly within N1QL queries. The CURL function supports a subset of standard curl functionalities, with options for GET and POST requests and the ability to pass custom headers and user-agent settings. To ensure security, CURL requires certificates to be stored on each query node, supports role-based access control with specific roles like QUERY_EXTERNAL_ACCESS, and allows for whitelisting specific endpoints. The maximum result size is capped at 64MB to prevent memory overload, and a custom user-agent is added to all requests to enhance security.

Performance and load testing are crucial for ensuring that a Couchbase Server cluster is production-ready, and the open-source command line utility "cbc-pillowfight," part of libcouchbase, provides a means to do this. Users need to have Couchbase Server installed and can choose to run it locally or via a Docker container, and libcouchbase must be installed according to the operating system in use. The utility allows users to simulate a battle by creating, updating, and reading documents from a designated bucket, with various adjustable settings such as the number of documents, JSON payloads, document expiration, and document size to tailor the test to specific use cases. While the tool is designed for stress testing, users can monitor bucket statistics for performance insights, though the default settings may require adjustments to better fit specific applications. Additionally, there are some platform-specific limitations, such as the single-thread restriction on Windows, and the tool's flexibility allows users to address authentication and connection issues that may arise during setup.

The Developer Advocacy team is gearing up for a busy conference season, eager to engage with the community and showcase their work on Couchbase. Throughout October, they will participate in several events, including JavaOne in San Francisco, Silicon Valley Code Camp, Dev up, All Things Open, Swift Summit, and DEVintersection, with opportunities to interact with key advocates like Nic Raboy and Matthew Groves. The team will also host Meetups in London and Columbus, Ohio. As they transition into November, they’ll continue building connections at QCon San Francisco and conclude the year with a presence at AWS re:Invent. They're also planning the 2018 events calendar and are open to speaking or sponsoring at upcoming conferences.

Arkadiusz Borucki, a Site Reliability Engineer at Amadeus, uses various technologies including NoSQL databases like Couchbase and MongoDB, automation tools such as Ansible, and Python in his day-to-day work. He is interested in big data, distributed systems, analytics, and automation. As a self-proclaimed big data enthusiast, he speaks at conferences and user groups in the US and Europe. The motivation behind using Infrastructure as Code is to overcome problems caused by manual configuration, such as inconsistent environments, slow deployments, and errors, especially when dealing with large-scale server farms and growing data sizes. By centralizing infrastructure setup and treating it like software code, tools like Ansible can be used for automation, orchestration, and consistency. The use of Git version control helps track changes and roll back to earlier configurations, while Couchbase's REST API enables direct management and operations from Ansible playbooks or Python scripts. Overall, using Infrastructure as Code with automation and orchestration techniques is essential for managing large-scale data storage systems like Couchbase in a modern world where data growth rates are high and consistency is crucial.

Hapi is a framework for services that allows developers to create web services, and it's being explored as an alternative to Express for building RESTful APIs. Hapi provides a more modular and service-oriented approach, which can be beneficial for creating scalable and maintainable applications. In this example, a simple API is built using Hapi and Couchbase Server, with two endpoints: one for returning a list of documents and another for creating new data in the database. The API uses Joi for data validation, ensuring that incoming requests meet specific criteria before processing them. The example demonstrates how to create a RESTful API with Hapi and Couchbase Server, providing a foundation for building more complex applications.