February 2017 Summaries

Graphql is a query language that allows to retrieve only the information defined by the user, avoiding underfetching or overfetching and providing certainty about having all the required data. In a Graphql server, clients use one single endpoint, hiding complexity and logic of the backend, allowing it to connect with different backends or databases, without changing the client's logic when switching between them.

The article introduces a tool called the CBM Changes Explorer, designed to facilitate understanding and manipulation of the Couchbase Sync Gateway changes feed, which is integral to data synchronization in the Couchbase Mobile stack. This stack, comprising Couchbase Server, Sync Gateway, and Couchbase Lite, employs Multiversion Concurrency Control (MVCC) to ensure reliable data handling and conflict resolution without relying on constant network connectivity. The CBM Changes Explorer allows users to monitor the changes feed, manipulate data via Couchbase Lite, and observe interactions between clients and the Sync Gateway, including handling document conflicts. The tool's user interface features panes for document listing, content editing, and changes feed output, and provides functionalities for basic authentication and synchronization toggling. The article also hints at a follow-up post that will delve into the tool's code, which is available on GitHub, encouraging contributions to enhance its functionality.

The author of the text was browsing the Couchbase forums and came across a question about querying array data in Couchbase, which they also struggled with when coming from a relational database background. The user wanted to know how to query for objects that were nested in an array within a single document, using a proposed document model as an example. The author provided two N1QL queries to accomplish this task: one using the `UNNEST` keyword and another using a collection operator to find array items that meet the criteria. These examples should help developers get started with querying arrays in Couchbase using N1QL. Additionally, the author mentions that there are other ways to achieve this goal, but these two queries should provide a good starting point.

At the CodeMash conference in Ohio, the speaker participated in three distinct sessions, showcasing both technical and community-building topics. The first session, "Are You Ready for Chaos? Horizontal Scaling in a Briefcase," featured a demonstration of the CouchCase project and discussed the CAP Theorem and Couchbase, despite technical challenges that necessitated a switch to Docker. In "Have Your Best Season Yet: Becoming a (Microsoft) MVP," the speaker joined a panel to discuss the Microsoft MVP program, sharing insights on community building for developers. The third session, "Lego Bucket Races," was part of the KidzMash event, where children participated in a hands-on activity designed to teach basic concepts of database indexing and performance testing, although the varied ages led to mixed results. The speaker emphasized the value of CodeMash as a learning experience and encouraged engagement with the community through comments or questions on platforms like Twitter and the Couchbase Forums.

Brant Burnett, a Couchbase expert, has been working with CenterEdge Software for 12 years, transitioning their cloud infrastructure from Microsoft SQL to pure Couchbase NoSQL platform. He recently shared his experience on using .NET Core and Docker for building applications that interact with Couchbase. The transition involves configuring environments, dependency injection, service discovery, and deployment. Brant uses the Couchbase.Extensions.DependencyInjection package to bridge the gap between Couchbase SDK bucket objects and the dependency injection system. He also uses DNS SRV records for service discovery and Docker Compose for managing containers. The company has successfully deployed their application using EC2 Container Service and follows a continuous integration process that involves publishing, building, tagging, and pushing images to a Docker registry.

The text explores the process of joining data from multiple NoSQL documents, focusing on MongoDB and Couchbase, two popular NoSQL databases. It compares MongoDB's use of the $lookup operator within aggregation queries to perform left outer joins, noting that additional steps such as unwinding arrays can complicate the process, especially for complex queries. In contrast, Couchbase's N1QL query language is presented as more intuitive and efficient, allowing for straightforward SQL-like JOIN operations and ease in managing complex queries. The text suggests that while MongoDB's $lookup is manageable for simple joins, Couchbase's N1QL provides a more user-friendly approach for handling intricate data joining tasks.

The State of Database DevOps survey highlights the challenges of integrating database changes into a DevOps process, with synchronizing application and database changes being the biggest challenge. The relational database model is not well-suited for Database DevOps due to its impedance mismatch between domain classes and database structure, requiring frequent updates to ORM classes and coordination with DBAs to update the production database schema. NoSQL document databases simplify Database DevOps by offering schema flexibility, no impedance mismatch, and scalability, allowing developers to operate directly on JSON data and derive meaning out of it, without the need for an ORM or coordination with DBAs. However, NoSQL is not a panacea, and relational databases may be a better fit for systems that require complex transaction logic or real-time data warehousing.

Microsoft Azure and Couchbase organized a free "Lunch & Learn" hands-on lab on March 8, 2017, at the Microsoft Technology Center in Southfield, MI, to introduce participants to the fundamentals of NoSQL technology and its integration with Docker Containers. The event aimed to demonstrate the ease of incorporating Couchbase into Azure Cloud or Hybrid Cloud environments, highlighting how leading companies utilize Couchbase’s NoSQL solutions for mission-critical applications on Azure. Participants learned about the basics of using Couchbase NoSQL for cloud environments, the deployment of Couchbase Server using Containers on Azure, provisioning virtual machines, and best practices for Couchbase Server on Azure. The session was designed for both newcomers and those with prior experience in NoSQL technologies, offering insights into why many organizations choose Azure and Couchbase for their modern web and mobile applications. Attendees were encouraged to register and explore Couchbase on Azure beforehand, as the latest Couchbase Server release was available in the Azure marketplace.

The new META().plan functionality and the new Plan UI combine in Couchbase Server 5.0 to improve the N1QL writing and profiling process, allowing developers to better understand how their queries are being executed and identify potential performance issues. The plan includes a tree of operators that combine to execute the N1QL query, providing detailed information about the query cost without going into detail of each operator. This feature is available in the Query Workbench and can be enabled by setting the profile feature for the entire node or on a per-request basis. Developers can also use this feature to identify potential performance issues such as slow indexing or high servTime values, which may indicate that the indexer or key/value service cannot keep up with the load. By adding more nodes to the cluster if necessary, developers can address these issues and improve overall query performance. The new functionality is available in the February 5.0.0 developer release of Couchbase Server and provides valuable feedback opportunities for developers to share their experiences and suggestions.

Node.js developers often favor NoSQL databases, such as MongoDB and Couchbase, due to their compatibility with JavaScript and JSON. This article explores transitioning applications from MongoDB, using Mongoose as an Object Document Model (ODM), to Couchbase with its ODM, Ottoman, highlighting their similar APIs for ease of migration. The guide outlines building a REST API in Node.js with Mongoose and migrating it to Couchbase, detailing the creation of data models, API endpoints, and server connections for both databases. It emphasizes the flexibility of NoSQL for data modeling, allowing for numerous configurations of student-course relationships, and demonstrates the simplicity of this transition process. The article concludes by underscoring the performance benefits of Couchbase, which integrates caching and scales effectively, making it an appealing option for large-scale applications.

Couchbase's Cross Datacenter Replication (XDCR) feature aids in disaster recovery and data locality by providing two conflict resolution strategies: revision ID-based and timestamp-based. The revision ID-based strategy, suitable for applications following a "Most updates is best" policy, uses a document's revision ID to resolve conflicts. Conversely, timestamp-based conflict resolution, introduced in version 4.6, is tailored for applications that prioritize the "Most recent update is best" approach, utilizing a hybrid logical clock to maintain consistent mutation ordering. Choosing the appropriate strategy is crucial, as it is a bucket-level parameter that cannot be changed post-creation, and both source and destination buckets in an XDCR setup must share the same strategy. Timestamp-based resolution requires synchronized clocks, typically achieved using NTP, to function accurately, as time skew can lead to incorrect outcomes. Testing the timestamp-based approach involves validating its effectiveness against revision ID-based resolution and ensuring the monotonicity and correctness of the hybrid logical clock.

Couchbase has introduced a new feature in version 4.6 to manage secrets, which are stored encrypted and can be rotated easily, without impacting SDK clients or performance. The system uses a two-level key hierarchy for rotating master passwords and supporting multiple master passwords, making it easier to integrate with other systems like KMIP servers. A strong Key Derivation Function is used to generate keys, and AES 256-bit algorithm in GCM mode is employed for encryption. Secret management allows rotation of secrets at different levels of the key hierarchy periodically or after a breach, and can be done using REST API or CLI commands. The system provides auditing capabilities for all rotations and setting of master passwords. A sample setup example is provided to demonstrate how to set up a server with a master password using CLI on Ubuntu 14.

The blog post provides a detailed walkthrough of deploying a simple Docker container using Oracle Container Cloud Service, specifically focusing on running a Couchbase service. It begins by explaining the foundational concepts of the Oracle Container Cloud Service, such as services, stacks, deployments, and resource pools, which are crucial for managing Docker containers effectively. The author guides readers through the process of creating a new service using a Compose file, deploying the Couchbase container, and accessing it via the Couchbase Web Console. The post also highlights key configurations, such as exposing common networking ports, and provides insights into inspecting container details and logs. The narrative concludes by suggesting future explorations into creating a Couchbase cluster and running applications, encouraging the use of Couchbase Starter Kits for further learning.

Couchbase Server 4.6 is now compatible with macOS Sierra, offering both Enterprise and Community Editions to suit different user needs. The announcement encourages users to explore the capabilities of Couchbase through a recent article titled "Couchbase and the Document-Oriented NoSQL Database" and provides guidance on utilizing Couchbase via the Developer Portal.

The latest Couchbase Server 4.6 release introduces several new SDK features aimed at enhancing development efficiency, including significant updates to big data connectors for Spark and Kafka. Native Collection bindings in the Couchbase SDK simplify mapping document data into language-specific structures such as List, Map, Set, and Queue, with .NET and Java having additional support from their Collections Frameworks. The integration with .NET Core supports cross-platform development, enabling applications to run across different operating systems without code changes. The Kafka 3.x updates improve integration with Sink and Source connectors, facilitating easier data streaming between Couchbase and Kafka through Kafka Streams technology, with simplified deployment via Kafka Connect and Confluent Control Center. Similarly, the Spark 2.x updates offer enhanced support for Spark 2.0, including Structured Streaming and Dynamic Topology for better failover and rebalancing. Overall, these updates aim to streamline development and deployment across various platforms and enhance the performance and scalability of applications using Couchbase.

The latest release of Couchbase 4.6 introduces Data Structures as a new time-saving SDK feature that allows client applications to easily map JSON array-based data into language-specific structures, reducing the need for manual management and saving developers time and hassle. This feature supports various languages including Java, .NET, Node.js, Go, PHP, Python, and C, with specific examples provided in Java (using the Map type) and Python (using List and Queue types). The new data structure types include Map, List, Set, and Queue, which are variants of JSON arrays presented as native types to client applications. These structures can be used for various use cases such as IoT applications where sensors record timestamped activities and related data values. By leveraging native collections support in Couchbase, developers can offload management of these structures to the server, simplifying communications and reducing development time.

The recent release of Couchbase .NET SDK 2.4.0 signifies the official General Availability (GA) support for .NET Core, marking a significant shift for developers as it allows Microsoft .NET applications to run on Windows, Linux, and macOS. This release includes substantial updates, such as packaging changes that provide binaries for both .NET Full Framework and .NET Core, with a transition to MS Logging for .NET Core from Common.Logging due to limited third-party support. A notable update is the introduction of Datastructures, which enable Couchbase documents to be manipulated as common data structures like lists and queues, enhancing usability. Furthermore, the SDK now defaults to a Multiplexing IO model for improved resource utilization, and supports streaming N1QL and views for performance optimization during large data retrievals. The new features also include the ability to cancel long-running N1QL queries using task cancellation tokens, along with an important note on upgrading libcurl on Linux to avoid SSL issues. Overall, this release expands the versatility and performance of Couchbase for developers across different platforms.

Couchbase Server 4.6 introduces enhancements designed to improve query performance, replication, security, and data management for its data platform, which supports web, mobile, and IoT applications. The new release enhances N1QL with additional string, date, array, and JSON object functions to simplify data transformations and improve query efficiency. Cross datacenter replication now features timestamp-based conflict resolution, facilitating Last Write Wins conflict management across clusters. Security is bolstered with Pluggable Authentication Modules for centralized password management and a new server secret management feature. Developer previews include flexible data import/export tools and improved full-text search capabilities with MossStore, allowing custom index mappings and more flexible search result sorting. These updates aim to enhance Couchbase's enterprise-level availability, scalability, and security requirements.

In August 2016, Couchbase Server was initially incompatible with the Windows 10 Anniversary Update, but this issue has since been resolved with the release of Couchbase Server 4.6. Initially, the fix was experimentally introduced in the 4.5.1 release, which included two versions: a standard release for Windows and a Developer Preview specifically for the Anniversary Update. Over time, Couchbase Server 4.6, which also had a Developer Preview, has been officially released and now fully supports the Windows 10 Anniversary Update. Users are encouraged to download the latest version and refer to the Couchbase Developer Portal or forums for further information and support.

The Couchbase QA team thoroughly tests DCP Rollbacks, a critical feature for ensuring data integrity in Couchbase products. To test rollbacks, the team uses a custom DCP client to simulate various scenarios, including failovers and crashes with non-persisted data. They verify that producers request rollbacks when necessary and that consumers properly undo later mutations and apply new ones correctly. System tests also focus on hard failovers at high mutation rates, checking for stability and consistency issues such as data loss or incorrect client behavior during rollback.

The text discusses the use of Docker and Portworx to manage stateful containers, such as Couchbase, highlighting the challenges and solutions for incorporating persistence in containerized environments. It emphasizes that while containers are typically ephemeral and best suited for stateless applications, stateful containers require special treatment to ensure data persistence, replication, and high availability across different nodes and availability zones. Portworx is presented as a robust solution offering features like container granular volumes, cross availability zone high availability, and ease of deployment. The text provides a detailed guide on setting up an AWS EC2 instance, creating EBS volumes, and deploying Portworx and Couchbase containers to maintain data integrity and availability across restarts. It also touches on the debate about whether all databases should be containerized, suggesting that while not all databases need to be Dockerized, there are viable options for those that do.

The document discusses the integration of Couchbase NoSQL databases with Tibco ActiveMatrix Business Works 6.3 using third-party JDBC drivers provided by Simba Technologies and CData. It highlights the rising adoption of NoSQL databases in enterprise environments due to their developer-friendly nature and the challenges posed by their integration into existing systems. Couchbase addresses these challenges with N1QL, a SQL-based query language that facilitates complex queries on JSON data. The document provides a detailed guide on setting up a custom JDBC connection within Tibco Business Works, enabling the use of Couchbase data through standard SQL queries. It also touches on possible issues, such as a licensing error encountered during the setup process, and suggests consulting partner websites for additional documentation and compatibility information.

This blog post is part of a series that explores the transition from relational databases like Microsoft SQL Server to document databases such as Couchbase Server, focusing on data modeling, data types, and the migration process. It highlights the differences between SQL Server and Couchbase in terms of data types, with SQL Server offering a variety of specific types while Couchbase relies on JSON's flexible structure. The post provides guidance on planning and executing data migrations, emphasizing the need for collaboration among different team roles such as DBAs and DevOps. It also discusses the use of tools like Dapper and Entity Framework in migration processes and explains that while Couchbase lacks some SQL Server features, it offers a scalable, fast alternative with ongoing development. The article concludes by acknowledging the trade-offs between relational and non-relational databases and hints at further discussions on application coding changes when using Couchbase.

Couchbase can be run using Docker Compose on a single host or as part of a multi-host Swarm-mode cluster. To start Couchbase on a single host, use the command `docker-compose up -d` and then connect to the server at http://localhost:8091 using credentials Administrator/password. N1QL queries can be executed using the `cbq` shell. The service can be terminated using the command `docker-compose down`. In a Swarm-mode cluster, Docker allows multiple hosts to be configured in a cluster, and the Compose file can be used to start the cluster by running the command `docker deploy --compose-file=docker-compose.yml couchbase`. The status of the service can be seen using the command `docker service ls`, and tasks/containers running inside the service can be checked using the command `docker service ps couchbase_db`.

Oracle Container Cloud Service is an entry into the managed container service market, competing with established options such as Docker for AWS or Azure, Amazon Elastic Container Service, Google Container Engine, and others. It offers a free trial period, during which users can get started by registering for a cloud trial, logging in to the main Oracle Cloud dashboard, creating a Container Cloud Service instance, and accessing the service console. The initial setup process involves creating a service instance with a single worker node, which takes around 30 minutes to provision. Once the service is created, users can access the Container Console, where they can deploy containers, including a simple Docker container, using the service. The blog aims to provide a step-by-step guide on getting started with Oracle Container Cloud Service and deploying a simple Docker container within it.

Ratnopam Chakrabarti, a software developer at Ericsson Inc., has been working on IoT and machine-to-machine technologies for several years. He is currently exploring ways to generate unique keys in an orderly sequential fashion using database sequences or UUIDs. In the context of Couchbase, a popular NoSQL database, Ratnopam discusses how to set up a bucket to store data with auto-generated sequence numbers, which can be used as primary keys for documents. He explains that Couchbase uses "counter" documents to handle sequence generation and demonstrates how to initialize a counter document with an initial value and use it to generate sequential IDs for inserting product data into a bucket. The code is written in Java using Spring Boot and Spring Data Couchbase dependencies, and the same concepts can be applied to other Couchbase client SDKs.

This summary is about a REST API developed using Node.js, Hapi.js, and Couchbase ODM Ottoman. The API allows for CRUD operations on users and posts, with validation and error handling implemented using Joi and Hapi.js. The application uses a database connector to connect to the Couchbase database and interact with it using Ottoman. The code is written in JavaScript and follows best practices for readability and maintainability. The API can be tested using tools like Postman or cURL. The key points of this text are that the API was developed using Node.js, Hapi.js, and Couchbase ODM Ottoman, and that it allows for CRUD operations on users and posts with validation and error handling implemented. The application uses a database connector to connect to the Couchbase database and interact with it using Ottoman. The code is written in JavaScript and follows best practices for readability and maintainability.

cURL is a widely-used command line tool for transferring data with URL syntax, and it can be integrated with the Couchbase N1QL query language to access external JSON data over HTTP(S) endpoints. This integration allows users to execute N1QL queries that incorporate external data sources, such as Google Maps API for geocoding or Fulltext Search (FTS) for querying specific datasets. By using cURL within N1QL queries, users can enhance their data manipulation capabilities by joining JSON endpoints with Couchbase buckets, effectively mapping external JSON data to a database structure. While not all cURL features are supported, basic functionality like GET and POST requests, along with some transfer-related options, are available. This approach offers an innovative way to handle large JSON files and external API data, enabling advanced operations such as updating documents with geolocation data or retrieving popular name statistics from public datasets. The flexibility of cURL within N1QL opens up possibilities for more dynamic and integrated data workflows, and feedback is encouraged to improve and expand this feature.

Couchbase is a robust and versatile NoSQL document database that is widely used by prominent companies such as LinkedIn, PayPal, and eBay to manage their data layers. It offers the flexibility of using JSON-formatted data, allowing complex and nested structures, which can be stored in unique Buckets. Couchbase provides multiple tools for querying data, including the Query Workbench and Couchbase Shell (CBQ), which support SQL-like N1QL queries, making it accessible for developers transitioning from relational databases. Indexing in Couchbase can be tailored to application needs with options for local and global secondary indexes, including memory-optimized versions for enhanced performance. The database is complemented by extensive language support for SDKs and is production-ready, as evidenced by its adoption by major organizations and its strong community support.

This blog post, part of a series on Couchbase Mobile, provides an in-depth exploration of Sync Gateway from the command line, aimed at enhancing understanding and facilitating testing and debugging. It emphasizes the ease of installing and running the entire Couchbase Mobile stack on a single machine and outlines minimal configuration steps for Sync Gateway, which serves as a secure web gateway application with various APIs for data synchronization. The article demonstrates how to configure and run Sync Gateway using command-line commands, including creating and updating documents, monitoring changes, and managing user access and channels. It explains the significance of Multiversion Concurrency Control in updating documents and highlights the role of the changes feed in tracking database activities. Additionally, the post shows how to add users, manage channel access, and verify document accessibility, encouraging readers to explore Sync Gateway's REST API and built-in web admin utility for further insights.

Docker simplifies the creation and shutdown of a microservice by introducing a `deploy` attribute in Docker Compose version 3, which allows for specifying replicas, rolling updates, and restart policies for containers. A microservice is defined using two services: `db` and `web`, with the `db` service starting a Couchbase server and the `web` service starting a WildFly application server. The `docker-compose.yml` file defines the services and their dependencies, including port forwarding and environment variables. To deploy the microservice, the command `docker stack deploy --compose-file=docker-compose.yml webapp` is used. Once deployed, the microservice can be accessed by sending HTTP requests to its exposed ports. The service can also be managed using Docker commands such as `docker service ls` and `docker service logs`. Additionally, the microservice can be created, updated, or deleted using RESTful API endpoints.

When moving from a relational database like Microsoft SQL Server to a document database like Couchbase, it's essential to consider data modeling. A key aspect of this process is understanding how to model data using denormalization or referencing, which can be used to differentiate documents within a bucket. The choice between these approaches depends on the application's needs and how data reads and writes are handled. Document databases offer flexibility in retrieving data through N1QL queries, but indexing is crucial for performance. Meaningful keys and discriminator fields can also be used to differentiate documents and improve readability and referenceability. Ultimately, modeling data in a document database requires a thoughtful approach, considering factors such as scalability, speed, and application interactions.

The addition of the sub-document API to Couchbase 4.5 has enabled efficient data structure support in Couchbase, allowing developers to store complex data structures such as lists, queues, and dictionaries within a single document, reducing the need for loading and saving entire documents. The Couchbase .NET SDK provides implementations for these data structures, including List, Queue, and Dictionary, which can be used to efficiently manage data by only sending specific parts of the document over the wire, rather than the entire document. These data structures are designed to be more performant than traditional whole-document approaches, making them a valuable tool for developers managing large datasets in Couchbase.