January 2015 Summaries

The Query Team at Couchbase has released Developer Preview 4 of their N1QL feature, which brings the power of query languages like SQL to Couchbase. A new Java SDK compatible with N1QL DP4 is available for developers to use and test this exciting feature. The SDK introduces several changes, including a new `Statement` interface, a `Query` class hierarchy, parametrized statements, prepared statements, and changes in the result format. These updates improve query management, indexing, and performance. The Java SDK can be downloaded and used with the `-Dcom.couchbase.queryEnabled=true` parameter or through the CoreEnvironment when initializing the Cluster.

Couchbase Server is often described as a document database due to its effective handling of JSON documents, but when modeling data for Couchbase, considering it as a key-value store can be beneficial. The distinction between key-value and document stores primarily lies in data querying; key-value stores offer simplicity with a single index, ensuring fast, consistent access, whereas document stores provide multiple indexes for flexible querying at the cost of increased resource use. Couchbase supports both methods, allowing users to decide based on their needs. To manage user profiles effectively, especially when users change their email addresses, Couchbase can utilize atomic counters to create unique, stable keys and manual secondary indexes to maintain flexibility and speed. This approach allows for efficient data retrieval without significant performance drawbacks, although it requires additional application layer work. The potential for implementing various manual indexes, such as for Twitter handles or phone numbers, further enhances Couchbase's capability, demonstrating its adaptability in balancing speed, flexibility, and scalability.

The HackGenY event, hosted by the Couchbase team, brought together students from across the country to participate in a hackathon. The event provided in-person and online mentorship support throughout the weekend. Students attended lectures and workshops on cutting-edge technologies such as Machine Learning and Mobile development, before forming teams to design and develop their own projects over the course of two days. With the guidance of Couchbase mentors, including Hideki, a mobile engineer at Couchbase, students were able to utilize Couchbase Mobile to create innovative applications quickly, resulting in impressive projects that showcased the technology's capabilities.

With Couchbase Server active, vbucket takes writes and reads by default, allowing for read-your-own-write (RYOW) capabilities in steady state, thanks to its memory-first write approach and quick replication through DCP to replicas and disk persistence. Queries can be configured to either read what's been processed or wait until all updates are processed, with the latter providing RYOW but at higher latency. In Couchbase Server 3.0, getting RYOW no longer requires persistTo flags, resulting in lower latency compared to previous versions, making it an attractive upgrade for users who need "stale=false" queries and views.

Python 2.0 (beta) client introduces an updated API and additional features, building on the 1.2 series, and aligning more closely with other SDKs by implementing new verbs like upsert, insert, and counter to replace older methods while still supporting them with deprecation warnings. The Connection class has been renamed to Bucket, with a simplified constructor approach that utilizes a connection string consistent with other SDKs, supporting SSL connectivity by specifying the couchbases:// scheme and certpath parameter. Enhanced integration with the unofficial couchbase-python-cffi module allows the Python client to operate under PyPy, with internal restructuring facilitating the use of the txcouchbase module on PyPy, addressing user requests for improved support.

CouchbaseMock is a lightweight Java-based application designed for testing Couchbase client functionalities by simulating a Couchbase server, which is particularly useful in environments where installing a full Couchbase Cluster is not feasible. Originally developed when Couchbase was known as Membase, the mock server provides a resource-efficient alternative for smoke testing before code commits, supporting operations like failover and replication in a controlled manner. Recent enhancements include view support, allowing the mock to interpret design documents and execute custom map or reduce functions using the Mozilla Rhino JavaScript interpreter. Although primarily an internal project for SDK testing, CouchbaseMock offers a practical solution for developers to test applications without a full cluster setup, despite not being the official tool for Couchbase application testing. The mock supports key-value operations, view queries, and bucket management through a REST API, with a focus on mimicking Couchbase's behavior accurately. However, it is not guaranteed to remain stable as features may change, and while it does not currently support N1QL queries, its utility in testing basic Couchbase functionalities remains significant.

Database Change Protocol (DCP) is a fundamental replication protocol used in Couchbase Server 3.0, offering enhanced availability, performance, and scalability across geo-distributed data centers. DCP supports critical database functions such as protecting data from failures, maintaining fresh indexes through rapid data replication, and ensuring consistency and high performance through its memory-based streaming capabilities. It utilizes key components like vbucket UUIDs, sequence numbers, and failover logs to provide ordered, restartable, and consistent data replication. This framework allows for significant improvements in operations like incremental backups, low-latency view queries, delta recovery for faster node rebalancing, and durability guarantees through the "ReplicateTo" method. DCP also strengthens cross data center replication (XDCR) by minimizing data loss during regional disasters and efficiently recovering from network issues. Overall, DCP underpins over 200 features in Couchbase Server 3.0, driving its robust data handling and recovery capabilities.

The San Francisco meetup group is hosting an event on January 14th to discuss production deployment and future trends. Johann Lynch from YOU Technology will share their experience with deploying Couchbase Server, which they use for handling high transaction volumes, scaling easily, and managing data locality needs. They will showcase how they utilize nearly every part of the Couchbase Server platform. Following this, James Nocentini from Couchbase will introduce the Swift programming language, explaining its application in iOS app development with features such as user authentication, UI prototyping, model objects, and more. The event aims to provide insights into the latest technologies and trends in software development.

Hadoop is a de facto standard for storing and analyzing large amounts of data as batches, but this can be incomplete due to continuous data generation. Lambda Architecture proposes combining Hadoop's batch processing with stream processing to analyze all generated data in real-time, utilizing messaging systems, distributed stream processing systems like Storm or Spark Streaming, and databases that meet high throughput and low latency requirements. An application queries processed data from both Hadoop and the database to create a complete view of results, raising questions about storing processed data in either Hadoop or the database, with options including exporting from Hadoop to Couchbase Server or storing it directly in the database. The combination of batch and stream processing enables real-time analysis of all generated data, allowing for more granular insights and better predictive analytics.

The release of N1QL Developer Preview 4 (DP4) from Couchbase represents a significant advancement in the query language's development, marking its completion in terms of syntax and language elements for building rich data applications. This version introduces new features such as subqueries and Data Manipulation Language (DML) constructs including INSERT, UPDATE, DELETE, and UPSERT, enhancing its capabilities for application development. DP4 also includes an array of new functions and operators like UNION, INTERSECT, and EXCEPT, alongside an upgraded command line interface supporting multi-line shell input and robust REST APIs for seamless interaction. Enthusiastic about community engagement, the developers encourage users to explore the new features through hands-on tutorials and provide feedback, while also addressing user queries related to performance comparisons and compatibility issues.

A new Java SDK based on reactive and asynchronous components has been announced, offering benefits over traditional synchronous access patterns, including improved resource utilization without headaches. The purpose of this blog post is to convert common data access patterns from synchronous to reactive, showcasing the power of async/await and lambda expressions in transforming code into non-blocking, efficient, and scalable applications. Two examples are provided: a "lookup pattern" where loading comments for a blog post becomes more efficient with reactive programming, and a query execution example where fetching user records is transformed from a blocking operation to a streaming one, allowing for better performance and resource utilization. The new SDK provides a clear path away from blocking, resource hogging database drivers that slow down application servers and response times.

This is a bug fix and performance improvement release for the Couchbase .NET SDK 2.0, following up on the GA release 2.0.0 in early December last year. The transport layer has been rewritten to use SocketAsyncEventArgs instead of the older asynchronous Socket API, reducing garbage collection issues. A single contiguous fixed buffer is used for all SocketAsyncEventArgs instances, eliminating problems with garbage collection. New bulk methods have been added for removing keys in parallel, which can be useful in real-world scenarios. Several major bug fixes have also been included, addressing issues such as CAS changes and TTL follow rules. The release includes contributions from the community through Pull Requests, GitHub tickets, and forum discussions. The Couchbase .NET SDK 2.0 packages are available on NuGet, S3, or can be pulled directly from master using the "2.0.1" tag.

The announcement introduces a developer preview of a JCache specification implementation using Couchbase Server and the Java SDK 2.0, which aims to provide a standard API for caching that allows developers to easily swap implementations. Although the implementation is a work in progress, basic operations and configurations are functional and available through a Maven repository. Developers can experiment with the preview, which relies on the CouchbaseCluster and Bucket from the Java SDK, and configure caches using CouchbaseConfiguration. The preview encourages feedback and contributions from the community, with areas such as statistics gathering, locking, and support for additional features still under development. The implementation supports both shared and dedicated bucket configurations and expects users to create views for listing cache items.