October 2015 Summaries

Couchbase Server 4.0 introduces significant new features, such as N1QL, a query language based on SQL designed for querying JSON data with a flexible schema, enabling rich queries against Couchbase documents. N1QL allows users to perform complex queries that resemble SQL while accommodating JSON's flexible structure, offering operators to enhance data manipulation. The Couchbase .NET SDK supports N1QL, providing seamless integration with the ability to use it alongside LINQ for type safety, code completion, and compile-time checking. Developers can explore these functionalities through open-source resources available on GitHub and NuGet, with "try-cb-dotnet" offering step-by-step tutorials to facilitate learning. Other notable features in this release include filtered XDCR for selective data replication and LDAP security for improved access management.

The upcoming talk on data modeling for unstructured or semi-structured data will explore the promises and challenges of using NoSQL databases, which offer agility and schema-less data storage. While NoSQL databases allow for easy storage of unstructured data, structuring is necessary for data utilization. The discussion will cover the differences between structured, unstructured, and semi-structured data, highlighting that while structured data requires predefined schemas, schemaless systems like key/value stores allow for flexible data storage. Semi-structured data, such as JSON and XML, provides implicit schemas with flexible field addition. The talk will address the practical aspects of writing, reading, and mapping data in these systems, using Couchbase as a case study to illustrate how data is structured and queried in distributed environments.

The author attended their first OSCON Europe conference, which had around 500 attendees compared to the US version's 4K attendees. They spent two days at a booth, engaging with attendees and participating in the "SWAG war" competition among exhibitors. The authors discussed various topics with attendees, including Couchbase vs. other NoSQL vendors like MongoDB and Cassandra. They also highlighted key features of Couchbase, such as its high performance cache, Key-Value store, and Document store. Additionally, they emphasized the importance of team building and networking opportunities at conferences, mentioning conversations with colleagues and interesting people like Leslie Hawthorn from RedHat. Overall, the author enjoyed their experience and looks forward to attending future OSCON events.

In the Couchbase LIVE New York session, the implementation of a peer-to-peer syncing feature using Couchbase Lite was demonstrated through a photo-sharing application called P2P PhotoDrop. This application allows seamless exchange of pictures between devices by generating a sync URL for database connections and employing QR codes to authenticate and initiate transfers. The process involves setting up listeners, observing database changes, creating documents for each photo, and replicating documents between devices. Authentication is secured by generating unique username and password combinations, and the overall code structure remains concise, focusing on core functionalities without directly handling network communications.

At the DEVIntersections 2015 Conference, Couchbase Server will be highlighted with presentations and demonstrations, featuring experts in .Net and Couchbase Server at the booth. A session is scheduled for Tuesday at 2:00 PM, where Couchbase Server 4.0 and N1QL will be discussed, focusing on data modeling, app development with .net & LINQ, and deployment on Azure through a series of demonstrations. Attendees are encouraged to visit both the booth and the talk to gain insights into these technologies.

Couchbase has announced the general availability of its official LINQ provider for Couchbase Server, which integrates with the N1QL query language for JSON documents. The Linq2Couchbase provider aims to offer a simpler ORM/ODM experience akin to Linq2SQL, differing from more complex frameworks like EntityFramework or NHibernate. It features a fully functional LINQ implementation with comprehensive support for N1QL features, and utilizes Re-linq to parse queries and generate N1QL statements. The provider is available on GitHub and NuGet, with dependencies managed through the Couchbase.NET SDK. To utilize Linq2Couchbase, users must set up a Couchbase Server or cluster, create a primary index, and work with the BucketContext for building and submitting queries. Integration with ASP.NET and Owin/Katana is facilitated through specific patterns, ensuring efficient resource management. Future updates will include features like Change Tracking, with community contributions welcomed to enhance this open-source project.

Designing and developing a web application involves breaking down the idea into parts, including a backend API layer, client-facing UI layer, and data layer. The data layer uses a NoSQL database like Couchbase to store and manage data, which can be queried using SQL or key-value operations. The Java SDK for Couchbase provides a convenient way to interact with the database, allowing developers to create RESTful APIs that serve JSON data. The backend API layer serves as a bridge between the data and UI layers, while the client-facing UI layer provides a user-friendly interface. To develop the application, developers can focus on creating endpoints that perform different queries or lookups against Couchbase, using options like Spring Boot for simplicity.

Montreal is not only known for its Canadiens hockey team and famous bagels but also as a hub for technology, with Couchbase Day Montreal offering a free educational event on November 17th. The event features workshops and presentations on Couchbase Server and Couchbase Mobile, covering topics such as architecture design, high availability, disaster recovery, programming, and modeling. Attendees will also explore the new features of Couchbase Server 4.0, including N1QL for querying and Couchbase Mobile for offline data access. The event promises to be a comprehensive introduction to Couchbase, providing insights into data-document modeling and syncing.

The session at Couchbase LIVE New York introduced developers to integrating Couchbase Lite into iOS and Android projects, using the Grocery Sync sample app as a reference. Key concepts discussed included the Manager class for database creation, handling documents as JSON objects with unique IDs, and the use of 'type' properties for indexing. Attachments were highlighted as a method for optimizing replication, allowing documents to attach binary data independently. The session also covered Views and MapReduce functions for creating indices and Queries for retrieving and organizing data, with LiveQuery providing real-time database updates. Practical examples demonstrated how to set up and use Couchbase Lite features in both iOS and Android, focusing on tasks such as querying data, updating UI components, and responding to user interactions. Future sessions promise deeper dives into Couchbase Sync Gateway and peer-to-peer capabilities to enhance mobile app functionality.

Sync Gateway is a component that ties together Couchbase Lite and Couchbase Server, providing key functions for securing mobile applications. It addresses data replication, authentication, and data access control, allowing users to define fine-grain security policies for read side policies at the document level and write side policies down to the field level. The Sync Function is a JavaScript function that gets executed whenever any JSON document gets written to Sync Gateway, where it can be used to manage read-write access and implement complex security rules. By configuring the Sync Function, users can control what data is visible to whom, ensuring that sensitive information remains private. Sync Gateway also supports SSL and TLS on the transport for additional data security when sending data over the wire. Additionally, file system encryption on the client-side is necessary for secure storage of local Couchbase Lite databases on mobile devices. By utilizing Sync Gateway, developers can create fully featured multi-user synchronizing experiences while maintaining control over sensitive data.

Building a social network begins with the crucial step of user registration, which involves setting up a backend using Node.js and Express.js, with modules like Couchbase Node.js SDK/N1QL, body-parser, uuid, and node-forge. The process involves creating a comprehensive file structure for the application, including key files such as app.js for setup, routes/routes.js for defining API endpoints, and models/usermodel.js for handling user data. The '/api/registerUser' endpoint processes user registration by validating user information such as name, email, and password both on the front-end and back-end to ensure security and prevent manipulation. Successful registrations are stored as JSON documents with unique identifiers (UUIDs), and further secured with hashed passwords. The document structure is designed for flexibility and efficiency in querying, using Couchbase’s N1QL language. Additionally, user email verification is a recommended step, to be discussed in subsequent posts.

The Elasticsearch Plugin for Couchbase version 2.1 is now generally available, providing new features such as a filter capability to prevent unwanted documents from being indexed and the ability to ignore deletes from XDCR in Elasticsearch, allowing customers to keep deleted documents searchable. This release expands compatibility with several additional Elasticsearch releases and is compatible with previous Couchbase Server releases up to 3.1, but may require special consideration for use with Couchbase Server 4.0. The plugin can be downloaded and used by following step-by-step installation instructions and consulting the release notes for further information.

Couchbase is participating in the "All Things Open" conference in Raleigh, NC, focusing on open source, open technology, and the open web in enterprise settings. The conference includes a wide range of topics such as Machine Learning and Apache NiFi. Couchbase will host two sessions, including "NoSQL, The Vital Open Source Ingredient For Modern Success," which discusses the impact of open source on the NoSQL database market, and "No Bars, No Problem! Always Available App," which highlights the importance of offline experiences in mobile applications. Attendees are encouraged to engage with Couchbase representatives throughout the event and connect via Twitter.

The Couchbase team attended the Big Data Spain event in Madrid, where they were well-received by attendees from across Europe. They launched their new Spanish developer community efforts and received interest in running developer workshops in the future. The team's talk on the evolution of NoSQL query methods, specifically N1QL, was a hit with the audience. It is likely that the Couchbase team will return to Madrid next year.

Couchbase has released the developer preview of the Couchbase Query Workbench with the launch of Couchbase 4.0 GA, addressing customer requests for a graphical user interface (GUI) to create and run N1QL queries. The Query Workbench, which can be downloaded separately, offers features like a metadata explorer, N1QL editor, and flexible query results presentation, enhancing the traditional command-line tool by allowing easier editing, syntax coloring, and auto-completion. A notable feature is the describe command, which samples documents to provide a schema description, aiding in writing better queries and identifying data structure differences. The tool is accessible via a web browser, requires Couchbase 4.0 GA or later, and invites user feedback for further improvements.

The Java SDK version 2.2 is now Generally Available (GA), offering enhancements, bug fixes, and compatibility with Couchbase Server 4.0's new features. The release includes improvements to N1QL and Multi-Dimensional Scaling (MDS) support, enhanced durability, metrics for better monitoring, a simplified Object-Document Mapping (ODM) system, and API changes such as the switching from hot observables to cold ones, along with various naming changes and dependency upgrades. To get started with this release, users can update their pom.xml files or download the archive, which includes the com.couchbase.client/java-client/2.2.0 package. The javadocs and SDK documentation provide detailed information on the new features and changes.

Koji, a Solutions Engineer at Couchbase, shares insights on integrating Couchbase Server with Apache NiFi, an open-source data flow automation tool, in his blog post. He explains the key components of NiFi, such as FlowFiles, Processors, Relationships, and Process Groups, which are essential for organizing complex data flows. Koji highlights his contribution to NiFi by adding Couchbase processors, enhancing its capability to handle Couchbase data through central configuration with ControllerService. He illustrates data flow examples like storing tweets in Couchbase and downloading specific documents as a Zip file, emphasizing the ease of automating data tasks without programming. Koji expresses his enthusiasm about expanding the functionality with additional processors for more sophisticated queries, showcasing the potential of NiFi and Couchbase integration for efficient data processing.

Couchbase 4.0 introduces several new features and enhancements, including the N1QL query language for JSON documents, Multidimensional Scaling for independent scaling of services, and a Global Secondary Index for faster data access. It also offers enhanced security with LDAP integration and auditing, geospatial indexes for location-aware applications, and a new database engine called ForestDB. Improvements in Cross Datacenter Replication filtering are included, and the release boasts a codebase with 1.7 times more lines of code and 2.25 times more tests than previous versions, significantly reducing test execution time. Users are encouraged to explore these features through various training sessions and tutorials, while keeping updated with the public release train and seeking assistance via the Couchbase Forums.

Couchbase will be participating in ConnectJS, a community developer conference focusing on modern web technologies, held at the Cobb Galleria Centre in Atlanta, Georgia, from October 16 to 17. The event, known for its affordable access to discussions on platforms like AngularJS, ReactJS, and Node.js, will feature numerous speakers, including one giving two presentations on topics such as offline-first mobile application development and full stack Couchbase, ExpressJS, AngularJS, and Node.js (CEAN) development. The talks, titled "Quick and Easy Development with Node.js and Couchbase Server" and "Developing for Offline Mobile Experiences," are scheduled for Saturday afternoon. Attendees can engage with the speaker via Twitter for inquiries about the event or the functionalities of Couchbase.

The developers of Touchbase started by defining the structure for storing user data, which includes three main components: User Document, Posts, and Images. The User Document stores all information about a user, including their name, expertise, job title, and login credentials, and is stored under a unique UUID to prevent duplication. The Posts document stores information about each post created by users, including the author ID, blurb, hyperlink, image path, publication type, and time posted. The Images document stores images uploaded by users and associates them with the user's document using a unique ID. The data model is designed to be flexible and customizable, allowing developers to modify the attributes used in the User Document and Posts documents through a config.json file.

Transitioning to a big data environment with Couchbase Server is facilitated by integrating existing tools, aided by N1QL and Simba's ODBC and JDBC drivers, which enable connections with relational tools. Simba provides a guide on transferring data between Couchbase Server and SQL Server, illustrating how to perform cross-product queries using Linked Servers in SQL Server alongside the Simba Couchbase Server ODBC driver. For more detailed instructions and to download the necessary drivers for Couchbase Server 4.0, users are directed to visit Simba's website.

This release of the Couchbase .NET SDK brings significant features with version 2.2.0, including feature compatibility with Couchbase Server 4.0 GA, N1QL support, multidimensional scaling (MDS) support, enhanced durability, and GEO Spatial Views. The new SQL-like query language for JSON, N1QL, allows users to query JSON documents in a more efficient and scalable way. The QueryRequest class provides a type-safe interface for executing queries, handling retry logic, uniform distribution of queries, and mapping query results to POCOs. The QueryResult class contains notable properties and methods such as Success, Message, Exception, Rows, Status, Errors, Warnings, Metrics, and HttpStatusCode. Prepared statements are supported to improve overall performance and server health. Enhanced durability allows enforcing durability constraints with a replica sequence number to determine if a key has achieved the desired ability. GEO Spatial Views provide support for querying geographic ranges using Map functions in JavaScript. The SDK is available for download directly, through NuGet, or by cloning and pulling the Github repo.

It’s a busy month for the Couchbase developer advocacy team as they attend several conferences in October, including Tableau, Devintersection, OSCON EU, Spark Summit, and Droidcon London. The team will be at each event's booth, available to answer questions and provide demos of new features in Couchbase 4.0. Attendees can expect prize giveaways and a chance to meet the developer advocacy team. Additionally, special promotions are available for attendees who use specific codes, such as couchbase25 for OSCON EU or code Couchbase20EU for Spark Summit registration.

Last week at WebCamp Zagreb, a conference for developers and designers, a presentation titled "Let your devices talk to each other" explored the potential of IoT, M2M, and decentralization to enable devices to communicate directly on a home network, rather than through cloud-based servers. The presentation highlighted concerns about data privacy in the cloud and proposed using Couchbase Lite for peer-to-peer (P2P) sync on local networks to maintain control over data. Demonstrations included using network protocols like Bonjour and JmDNS for automatic service discovery to facilitate device communication. An experimental project was showcased, employing Java and the JmDNS library to enable devices to automatically sync without credentials, though security remains a concern. The goal is to encourage more secure and efficient device interaction within local networks, inviting feedback and further exploration of enhanced synchronization methods.

Relational databases revolutionized data retrieval by allowing descriptions of required data instead of relying on link-walking, thanks to SQL's declarative nature. The emergence of NoSQL databases prioritized scalability, uptime, and redundancy, often sacrificing query flexibility. However, Couchbase 4.0's introduction of N1QL offers a balance by providing a scalable, clustered document database with a generally applicable declarative query language for JSON documents. N1QL leverages familiar SQL syntax with enhancements for handling semi-structured, nested data, retaining the ad-hoc query capabilities, including JOINs, without extensive new learning. This development marks a significant advancement by combining NoSQL scalability with SQL's query flexibility.

Couchbase Server 4.0, a significant release for the company, is now available for download following multiple previews, thanks to the contributions from the developer community and beta customers. This release introduces several innovations, including the new N1QL query language which combines SQL simplicity with JSON flexibility, enhanced BI integration via ODBC/JDBC drivers, and spatial queries for location-aware applications. Performance improvements include new indexing for responsive queries, multi-dimensional scaling, and the advanced ForestDB storage engine. The release also focuses on simplified security compliance, improved high availability and disaster recovery through enhanced data distribution and protection, and a connected big data experience. Additionally, it offers improved provisioning with high-performance container deployment, expanded cloud support, and new enterprise platforms. Enhanced developer documentation and resources are available through a new developer portal, and further learning opportunities are offered via events and online training sessions.

Couchbase Live New York 2015 kicked off with a series of presentations and demos from Couchbase and its customers, focusing on the latest developments in Couchbase Server 4.0. Key topics included data modeling practices, highlighting the simplicity of evolving data models with JSON-based documents, and the new multi-dimensional scaling architecture featuring a service-based model. There will be a deep dive into querying using N1QL and the new Global Secondary Indexes, covering indexing, index maintenance, and query best practices. The event promises many exciting sessions, with opportunities for attendees to engage and learn more about Couchbase’s innovations.

The Silicon Valley Code Camp (SVCC) is a community-driven event held at Evergreen Valley College in San Jose, California, where developers can learn from each other across a wide range of topics, including NoSQL and software engineering interview tips. With 192 sessions over the weekend, it is a bustling event that draws participation from various organizations, including Couchbase. Members from Couchbase will present on various subjects such as cross-platform storage, quick development with Node.js, and Docker and Kubernetes for Java applications, among others. Notably, Couchbase is a sponsor of the event, and Arun Gupta will lead a Docker and Kubernetes workshop the Friday before the event. Attendees are encouraged to visit the Couchbase table to ask questions and connect over social media during the event, which is free, making it an excellent opportunity to explore new technologies and best practices.

A speaker is preparing to travel to Vilnius to present at the Big Data Strategy conference, where they will discuss the evolution of NoSQL query methods, tracing their development from 1960s hierarchical databases to the modern N1QL, which integrates SQL with NoSQL's flexible JSON format. The speaker enjoys introducing N1QL to developers due to its combination of SQL's familiarity and JSON's adaptability, and they are looking forward to experiencing Lithuania for the first time. They invite attendees of the event to connect with them during the conference.

An Android application allows users to save reviews with a rating and name to a Couchbase Lite database, utilizing TabLayout and ViewPager components to present three scenarios: Unique, Conflict, and P2P. In the Unique tab, a randomly generated _id field is created for each new document, while the Conflict tab enables users to set the _id field manually, allowing documents to be updated if an existing ID is used. The P2P tab facilitates continuous replication with a remote database by inputting its URL, which can be customized in the StorageManager.java file. A table displays review counts and conflicting revisions across tabs, with live queries updating the Recycler View adapter as changes occur. To deploy the application, a Sync Gateway and Couchbase Server instance can be set up using Tutum, requiring specific configurations for the Sync Gateway to function correctly. Additionally, a script is available for deleting all documents from a target database.