February 2016 Summaries

Andy Kruth, a lead consultant at Avalon Consulting LLC based in Austin, Texas, specializes in the Hadoop Ecosystem and NoSQL systems, with a particular focus on integrating Couchbase with platforms like Elasticsearch, Spark, and Kafka. In his personal life, Andy enjoys spending time with his family and watching TV shows. He chose Couchbase for its usability compared to other NoSQL databases and has contributed to defining its role in larger systems. Andy highlights the ease of use of Couchbase's Java API and its compatibility with various programming languages. While he wishes for improvements in Couchbase's full-text search engine, he is also an avid reader, favoring authors like Isaac Asimov and Stephen King. The interview is part of a series to get to know Couchbase Champions and aims to encourage others to join the Couchbase Champion program by participating in activities such as blogging, webinars, and contributing to Couchbase community resources.

Couchbase Developer Advocates Matthew Revell and Laurent Doguin, along with engineers Marty Schoch, Dave Rigby, and Simon Baslé, discuss the new features in the Couchbase Server 4.5 Developer Preview. The session highlights the introduction of a sub-document API and a preview of Couchbase's free-text search capabilities. This interactive hangout provides insights into the advancements offered in the latest version of the Couchbase Server, engaging the expertise of the Couchbase team to explore these developments in detail.

The text discusses the use of Ottoman, an Object Data Modeling (ODM) library for Node.js and Couchbase, to manage complex data structures. It highlights the benefits of using Ottoman over traditional N1QL queries for fetching related documents. The example demonstrates how to define two models: Project and User, with a one-to-many relationship between them. It shows how to query for projects with their associated users using the `load` option in Ottoman. The text also explores how to handle changes to the data model, such as adding new fields or changing the structure of existing fields, without affecting the queries. Overall, the article emphasizes the flexibility and power of Ottoman's ODM approach for managing complex data relationships.

Couchbase Server's sub-document API enhances document databases by enabling more efficient operations on specific parts of a document without needing to retrieve, deserialize, and re-upload entire documents. This feature is particularly beneficial for applications that require frequent updates to documents, such as chat logs, as it reduces network bandwidth usage and shifts the responsibility for document updates from the application to the database layer. Sub-document operations are supported in all official Couchbase Server SDKs and utilize an extension of the memcached protocol for key-value operations. These operations require a document key and a path to the sub-document within the parent document, allowing for precise modifications or queries. The SDKs, including those for Python and Java, provide methods such as `lookupIn` and `mutateIn` to facilitate these operations, and the functionality is abstracted to simplify complex queries and mutations. To learn more about implementing these operations in different programming languages, users are encouraged to consult the updated SDK documentation.

Couchbase 4.5 introduces an experimental full-text search (FTS) feature in its Java SDK, leveraging the open-source Bleve project to enable sophisticated text indexing and searching capabilities. This feature, accessible via the 2.2.4 Java SDK client, simplifies the search process by integrating directly within Couchbase, eliminating the need for external connectors for basic needs. Through a variety of query types—including fuzzy, match phrase, regular expression, prefix, numeric range, and date queries—users can execute complex searches directly on Couchbase indices. The Java API for FTS follows a builder pattern, allowing developers to construct and execute queries with methods like `query(SearchQuery ftq)`, and provides options for combining queries using conjunctions, disjunctions, and boolean parameters. Additionally, developers can gain insights into query scoring and matching by using the explanation feature within search results. This release aims to streamline search processes for developers working with Couchbase, offering a preview of its capabilities ahead of further development.

Couchbase Server 4.5 introduces a preview of Full Text Search (FTS) as a feature integrated into its ecosystem, following its initial release as a standalone Developer Preview. FTS enables case-insensitive searches, supports typo tolerance, and uses advanced text analysis with various prebuilt analyzers for multiple languages. It provides multiple query types and scoring mechanisms, backed by the Bleve open-source project. While Couchbase FTS offers a seamless integration for full-text indexing of JSON documents stored in Couchbase, it remains in developer preview and is not yet suitable for production use, unlike established tools like ElasticSearch or SOLR, for which Couchbase maintains connectors. The feature allows users to run faceted searches and create aliases for indexes to facilitate app development without altering index names. Although FTS is not yet production-ready, Couchbase is working on improving its performance and multi-node support, with plans for future integration with N1QL and continued support for existing connectors.

Couchbase Server 4.5 Developer Preview introduces a range of new features aimed at enhancing querying efficiency, advanced data access, and administration capabilities. Key updates include the Query Workbench for executing ad-hoc queries directly from the admin console, extended join syntax for faster JOIN operations, and integrated full-text search based on the Bleve project. The release also offers partial document updates, memory-optimized global indexes, and array indexing to improve performance and flexibility. Additionally, it provides new tools for query profiling, high-performance backup and restore, and secure communications via X.509 certificates. The preview is intended for development use only and feedback is encouraged to refine the final release.

The second episode of the NoSQL Database Podcast, hosted by Couchbase, explores the application of NoSQL databases through a discussion with Brett Porter, the Head of Architecture at SafetyCulture. Titled "NoSQL and Where It’s Used," the episode delves into SafetyCulture's specific use case, examining the motivations behind their choice of NoSQL over alternative database technologies, the types and volume of data stored, and the company's overarching objectives. This episode is accessible on platforms like Pocket Casts and iTunes, and listeners are encouraged to send questions to be answered in future episodes.

The transition from an Oracle relational database to Couchbase's NoSQL document model can be less daunting than it seems, particularly because Couchbase offers a SQL-like query language that is familiar to those accustomed to RDBMS. While Oracle relies on structured tables with primary and foreign key relationships, Couchbase uses JSON objects and arrays, allowing for a more flexible and schema-less data modeling approach. The key differences lie in data modeling, where NoSQL allows for both referring and embedding documents, and the querying process, where Couchbase's N1QL provides a similar experience to Oracle's SQL. Despite these differences, both development and tooling for Couchbase are designed to ease the transition, with tools like the Couchbase Java SDK and CBQ offering comparable functionalities to Oracle's options. Migration tools, such as oracle2couchbase, facilitate the process of moving data from Oracle to Couchbase, highlighting that despite their differences, both platforms share enough similarities to make the switch feasible and beneficial for modern data requirements.

The guide provides a comprehensive overview for developers transitioning their data store from MongoDB to Couchbase Server, highlighting key differences and considerations for migration. Couchbase Server, unlike MongoDB, functions as both a document store and a key-value store, using JSON while MongoDB uses BSON, which requires handling additional type information. Couchbase employs automatic hash-based sharding, simplifying scaling, and favors consistency in its replication model, which can affect availability during node failures. Developers are advised to replace MongoDB's collection namespacing with key naming and document types, utilize Couchbase's N1QL for SQL-like querying with joins, and consider the advantages of key-value access for certain operations. The guide also touches on the allocation of resources to buckets, the differences in indexing, and the concurrency models, emphasizing the importance of understanding these aspects to effectively use Couchbase Server. Additionally, it provides insights into Couchbase's architecture, including its approach to sharding, replication, and indexing, while also offering tips on optimizing performance through bucket configuration and the use of semantic key naming for document differentiation.

Spring Data Couchbase 2 has been released and is now accessible through Spring Initializr, allowing developers to quickly set up a Spring/Couchbase project by selecting dependencies and generating a project archive. The tool is compatible with Spring Boot 1.4.0-SNAPSHOT and includes an auto-configuration feature that supports properties for Couchbase access, such as bucket name and password, and offers various consistency levels like read-your-own-writes and strongly-consistent. Developers can define these properties in the application.properties file or through a @Configuration Bean. The auto-configuration also provides automatic index creation, although it's recommended to disable this in production. Key annotations for index creation include @N1qlPrimaryIndexed and @ViewIndexed, while the framework offers various Beans like CouchbaseTemplate for low-level API access and ValidatingCouchbaseEventListener for entity validation. For more comprehensive details, users are encouraged to consult the official documentation or follow ongoing blog updates.

The development of Couchbase has benefited significantly from its open-source nature, which allows for close interaction with users. As the project expands, revisiting and adjusting core infrastructure becomes necessary to enhance efficiency, leading to the introduction of SDK-RFCs to gather community feedback on new features like the Sub-Document API and the Index Management interface. With the release of Couchbase Server 3.0, new SDKs were introduced that intentionally broke existing APIs, aiming for a more unified and idiomatic approach across platforms. This collaborative process, involving the creation of detailed API blueprints, has worked well but highlighted areas for improvement, particularly in ensuring interfaces are acceptable across co-projects and encouraging wider community input. The infrastructure for this process is inspired by IETF RFCs but utilizes modern tools like GitHub for open developer discussions. A new repository, SDK-RFCs, was established to document and track these discussions, and community input is sought on two significant proposals: Sub-Document operations, allowing for modifications without retrieving whole documents, and Index Management, facilitating ease of use in continuous integration scenarios. The entire SDK engineering team at Couchbase has been instrumental in developing and refining this process.

Spring Data Couchbase 2.0 is a comprehensive rewrite of its predecessor, enhancing the integration with Couchbase Java 2.2 SDK and supporting the N1QL query language introduced in Couchbase Server 4.0. This upgrade introduces features such as improved configuration elements that reflect Couchbase's structure, the ability to back custom repository methods with N1QL instead of views for more flexibility, and the inclusion of geospatial and multidimensional querying capabilities. It enhances query consistency options, allowing users to choose between multiple consistency models, and addresses compatibility issues with Couchbase Mobile by enabling type information field customization. Additionally, it supports Pageable and Sort parameters in N1QL queries, offers improved documentation, and includes numerous bug fixes and performance enhancements. The release aims to provide a more robust and flexible framework for developers working with Couchbase and Spring Data, with plans for further integration in future updates.

Join me at RWDevCon iOS Tutorial Conference in Alexandria, Virginia on March 11, 2016 to learn about iOS development with beginner, advanced, and inspiration-tracked training sessions. I will be hosting a hands-on training session titled "Couchbase Mobile Lab: Taking Your Mobile Apps Offline" on March 12 at 11:30 a.m., where you'll learn how to create a local database for your mobile apps to persist data on the device and sync with a remote cloud instance when available. The conference also features sessions on custom controls, Xcode UI testing, and learning from Ryanair's iOS mobile app on Couchbase Mobile.

Couchbase Mobile is presented as an ideal NoSQL database solution for mobile applications, featuring an embedded database (Couchbase Lite), built-in synchronization (Sync Gateway), and backed by the scalable Couchbase Server. Following the announcement of Parse's indefinite shutdown, developers are advised to migrate their applications to Couchbase Mobile, which offers a similar data storage approach using key-value pairs, making the transition straightforward. Couchbase provides a comprehensive full-stack database solution with features such as conflict resolution, REST API, and synchronization capabilities, which surpass the limited features of Parse. The text includes a code comparison between Parse and Couchbase Lite to illustrate their similarities and outlines a guide to set up Couchbase with Digital Ocean for migration purposes. Couchbase Mobile enables secure local and cloud data storage, seamless data synchronization, and scalable system capacity, while also supporting integration with other systems and data sharing between web and mobile platforms.

Parse, a prominent mobile backend as a service (MBaaS), recently announced its closure, prompting its 600,000 developers to explore alternative solutions for their mobile applications. Developers are presented with three primary options: installing and using the open-source Parse Server independently, migrating to another service provider like Google Firebase or Amazon Cognito, or adopting an installed database software such as Couchbase Mobile. Each option carries its own set of challenges and risks, including the cost and complexity of self-management, the potential instability of relying on third-party vendors, and the increased responsibility of managing hardware. Developers must carefully consider these factors to ensure the long-term viability and stability of their applications, particularly for mission-critical operations. The text emphasizes the importance of selecting technologies and strategies that mitigate the risk of service shutdowns and facilitate quick market entry.

The Couchbase Developer Advocacy team is participating in Jfokus, the largest developer conference in Sweden and one of Europe's biggest Java developer events, with 1,600 attendees and over 50 presentations. Couchbase representatives, including SDK Software Engineer Simon Basle, VP of Developer Advocacy Arun Gupta, and Director of Developer Advocacy Matthew Revell, are presenting talks on topics like Practical RxJava, Docker and Kubernetes recipes, and the capabilities of NoSQL. The team invites attendees to visit their booth for giveaways, games, and a chance to win a GoPro during the Tuesday night social event.

The blog post outlines a method to integrate Apache Kafka with Couchbase using Java applications as a producer and consumer to handle data transfer. The setup consists of four key components: a Kafka producer, an Apache Kafka queue, a Kafka consumer, and a Couchbase server. The producer sends messages to the Kafka queue using sample JSON data generated with Mockaroo, which are then consumed by the Kafka consumer. The consumer retrieves these messages and inserts them into Couchbase using the Couchbase Java SDK, with an emphasis on simplicity by employing the synchronous SDK. The blog emphasizes that while the demo is run on a single node setup, production environments would typically distribute these components across multiple machines for scalability and reliability. The complete code, including Maven dependencies, is available on GitHub for those interested in replicating or expanding upon this example.

Couchbase Mobile 1.2 introduces significant enhancements in performance for mobile devices by integrating ForestDB as the new storage engine, replacing the previous SQLite option. This upgrade offers faster speeds, improved concurrency, a lower RAM footprint, and automatic database compaction, enhancing the efficiency of mobile applications. ForestDB serves as a persistent key-value storage library and is compatible with various server and mobile operating systems, including iOS and Android. Developers can enable ForestDB in their projects by adjusting settings in their code, such as setting the `storageType` property to ForestDB in iOS projects and using the `setStorageType` function in Android projects. While SQLite remains the default engine, ForestDB provides up to a 5x performance improvement in certain tasks, making it a valuable optional upgrade for developers seeking enhanced performance and responsiveness in their applications. Future blogs will delve into additional features of Couchbase Mobile 1.2, including encryption and further performance optimizations.

Full Text Search (FTS) is a crucial feature of Content Management Systems that facilitates searching both content and associated metadata. The article explores a scalable integration of FTS into a Content Management architecture using Couchbase, specifically through Couchbase Full Text Search (CBFT), which was announced in 2015 and covers a majority of application features without requiring additional tiers like ElasticSearch. The article provides a step-by-step guide on setting up CBFT with Apache Chemistry to enable metadata full-text search, including creating a CBFT index on a Couchbase bucket and testing the index. It also details how to activate full-text query capabilities in the CMIS Couchbase repository and outlines using a REST API with a Jersey client to query the CBFT index, allowing users to retrieve content efficiently from the Couchbase repository.

The text discusses the advantages and mechanics of using prepared statements in N1QL, particularly focusing on how they can enhance query performance in Couchbase by reducing the overhead associated with parsing and executing frequently repeated queries. By setting the adhoc property to false, users can instruct the SDK to cache a compressed execution plan after the first execution, enabling subsequent queries to bypass initial parsing and analysis steps and proceed directly to execution. This process utilizes an LRU cache to manage memory efficiently by discarding the oldest cached queries once the cache reaches 5000 entries. While prepared statements can significantly improve performance, especially for repeated queries with different parameters, they are not suited for highly adhoc applications due to the initial overhead of preparing the execution plan. Additionally, they are specific to each SDK instance, and care must be taken with topology changes that could invalidate cached plans. For further exploration of Couchbase's functionalities, the text suggests consulting the Developer Guide and the GitHub repository.

This blog post describes how to set up a Couchbase Cluster using Azure Resource Manager (ARM) templates. To do this, you need an Azure Subscription, Azure CLI installed on your system, and a Github Account for optional experimentation with ARM templates. The cluster size is configurable but starts with a three-node cluster setup with replication to one node. You can deploy the template using Azure Portal, Azure PowerShell, or Azure CLI. The template uses variables and parameters to configure resources such as virtual networks, network interfaces, and virtual machines. It also includes scripts for installing Couchbase Server on each VM and setting up the cluster. The ARM template is publicly available on GitHub, and you can use it to deploy a Couchbase Cluster in Microsoft Azure.

The text discusses the advantages of using prepared statements in Couchbase, building on a previous conversation about their performance benefits. The author shares a practical analysis using a control loop to simulate load on Couchbase with the travel-sample dataset, examining various query types including parameterized queries, different predicate strategies, and a mix of indexed and non-indexed fields. The findings indicate significant performance improvements with prepared statements, although limitations remain when full primary index scans are necessary. Overall, the author advocates for the use of prepared statements to enhance performance in most situations.

N1QL is a powerful query language developed by Couchbase for use with Couchbase Server, providing a common query language and JSON-based data model for distributed document-oriented databases. It allows developers to interact with JSON in Go when the schema/structure of the document is known in advance, but struggles with dynamic queries built at runtime or unknown JSON structures. The Go SDK provides tools to handle these scenarios, enabling developers to issue N1QL queries with Couchbase and retrieve results in a flexible and efficient manner. By using N1QL with Couchbase and Go, developers can build scalable and high-performance applications that take advantage of the power of distributed document-oriented databases.

Couchbase has launched Couchbase Mobile 1.2, focusing on enhancements in administration, security, and performance at scale. The release simplifies administration through new management controls and RPM capabilities for in-place upgrades while introducing enterprise-level 256-bit AES encryption for improved security. Performance is bolstered by integrating the ForestDB storage engine in Couchbase Lite, offering 2 to 5 times faster performance, alongside optimizations in memory, connection, and resource management to handle high loads more efficiently. Users are encouraged to download the new release and explore the Couchbase Mobile Developer Portal for further information.

The webinar series covers the topic of full stack development, focusing on building a full stack application with NoSQL, Node.js, and Angular 2. The first two episodes cover bootstrapping a Node.js application with NoSQL, back-end application logic and data modeling. The third episode focuses on building a responsive front-end with Bootstrap and Angular 2, while the fourth episode adds permalinking capabilities. The final episode extends the stack to mobile devices using NativeScript, building cross-platform Android and iOS apps that communicate with the back-end API. The series is designed for those looking to learn about full stack development and bleeding-edge technologies.

In a continuation of the Coffee on Couchbase Sync Gateway series, the article delves into the security and data orchestration capabilities of Sync Gateway, focusing specifically on using Channels to manage access to mobile applications. Chris Anderson discusses how administrators can control data visibility by specifying channels through the admin port, secured by a firewall, while end users connect via a public port. Access to specific data is managed through the Administrator API, allowing for dynamic channel assignment using JSON configurations, which can be updated via REST API commands. This setup enables precise control over user access to data, ensuring users only see permitted information, and allows for adjustments to channel access in real time. Future topics in the series will explore data encryption and the use of ForestDB as a storage engine in Couchbase Lite, with further community engagement encouraged via forums and Stackoverflow.