May 2022 Summaries

In exploring the architecture of multilanguage microservices, the focus is on tailoring each microservice to its specific function using the most suitable technology stack, which can lead to greater efficiency and flexibility compared to a uniform language approach. While critics argue that using multiple languages can complicate operations and increase overhead, the article outlines scenarios where this architecture is beneficial, such as leveraging specific languages for tasks, retaining technical flexibility, and aligning with developer skills. Although multilanguage microservices can introduce challenges in communication and maintenance due to increased APIs and the need for consistent updates across different languages, tools like Protocol Buffers, gRPC, and Swagger Codegen are highlighted as solutions to facilitate these processes. The article also discusses the importance of careful orchestration in large applications, using tools like Netflix Conductor to manage complex workflows effectively. Ultimately, the choice of architecture should be based on the specific needs of the application, balancing the advantages of language flexibility against potential operational challenges.

A Conductor workflow is implemented to automate the ETL process for Orkes, focusing on extracting data from GitHub, transforming it, and loading it into Orbit to efficiently manage community interactions and data. This workflow involves using GitHub's API to extract user information, transforming the data into a JSON format suitable for Orbit using JQ and JavaScript tasks, and then uploading it to Orbit with the help of API keys. In addition to streamlining this process, the workflow is scheduled to run every 24 hours using Orkes’ new scheduler tool, ensuring data remains current by capturing new stars and forks in the Conductor repository. This approach not only saves time but also allows for extending the workflow to analyze multiple repositories concurrently, contributing to a comprehensive understanding of user engagement across various platforms.

Netflix and Orkes are co-managing a new community contribution repository for Conductor, an open-source workflow orchestration engine developed by Netflix. Conductor is widely used at Netflix to manage workflows at scale, and its extensible design allows for easy integration of new features without altering the core system. Since its open-sourcing, Conductor has seen significant community engagement, leading to contributions such as support for Postgres and MySQL backends, Elasticsearch 7, AMQP and NATS queues, GRPC, Azure blob stores, and more. The volume of contributions has outpaced the integration capabilities of the core team, prompting the creation of this new repository to better manage and integrate community contributions. Partnering with Orkes will facilitate timely code reviews and releases, maintaining Conductor's usability while fostering a more autonomous community involvement. The new structure ensures continuity for users with no changes to installation or updates, aiming to enhance Conductor's standing as a leading open-source orchestration engine.

Microservice architecture, widely adopted for its flexibility in adding or removing features and updating specific application parts, benefits significantly from workflow orchestration, which manages the implementation, deployment, and maintenance of microservices. This orchestration simplifies complex processes and enhances efficiency through automation, despite challenges such as centralized control and potential bottlenecks. Tools like Kubernetes, Amazon EC2 Container Service, and Orkes streamline the management of these distributed systems, allowing developers to focus on business logic rather than intricate system management. The use of multiple programming languages in microservice architecture, driven by specific application needs and the limitations of legacy languages, is common, allowing for faster and more cost-effective development. While monolithic architectures may face constraints with multiple languages, microservices offer developers the freedom to choose the best tools for each task, although compatibility issues can arise when using third-party orchestrators. Orkes, an enterprise-grade orchestration platform, is noted for its language-agnostic capabilities that integrate seamlessly with diverse technical stacks, facilitating serverless operations and streamlined scaling.