July 2022 Summaries

Counting records in a database table seems like it should be a fast task. However, with tools like MySQL and PostgreSQL, that's not the case. In these databases, counting rows is slow because they use Multiversion Concurrency Control, which means each unique transaction may have slightly different data from other transactions processed at the same time. This makes it impossible to get an accurate count of rows in a database table like there is with MySQL/MyISAM tables. To overcome this challenge, one can leverage indexes or use triggers and functions to maintain a MySQL/MyISAM-like count that is kept up-to-date at all times. PostgreSQL and MySQL/InnoDB engines also have estimated row counts stored in their respective tables, which can be used for fast but inaccurate estimates of table size. Additionally, using the EXPLAIN command and scraping data from it can help build a count estimator for queries. The best approach depends on whether an exact or estimated count is needed, with accurate counts requiring more efficient queries, while estimated counts offer faster performance at the cost of accuracy.

Counting records in database tables, particularly in MySQL and PostgreSQL, can be unexpectedly slow due to the use of Multiversion Concurrency Control, which necessitates sequential scanning of tables to determine row counts. While MyISAM in MySQL maintains a count of rows for fast retrieval, InnoDB and PostgreSQL do not, resulting in slower count operations. Techniques to speed up counting include leveraging indexes and creating triggers with functions to maintain a running count of rows, though both have trade-offs in terms of accuracy and operational overhead. For those willing to compromise on precision, estimated counts based on database metadata can be used, providing a faster but less accurate solution. Tools like New Relic offer observability solutions to monitor database performance and optimize counting strategies further.

The green fintech industry is rapidly growing as consumers increasingly demand sustainable financial products and services. The UN Environment Program has proposed a global digital ecosystem for greater collaboration between stakeholders regarding climate and biodiversity impacts. Green fintech products include carbon calculators, green loan products, sustainable investment and wealth management products, and enterprise dashboards that report on environmental social governance metrics of supply chains. Design thinking can speed up green product innovation by identifying customer needs and pain points, focusing on businesses, enterprises, and industry sectors rather than individual consumers. The three design-thinking steps to ideate products for sustainability are widening the target market, considering high-impact categories, and creating features with action in mind. Metrics such as enabling activities that raise awareness and contributing activities that directly improve environmental status can be used to measure product impact. App developers can apply their skills in monitoring and using data to design green fintech products, including hypothesis testing and A/B testing practices. By tracking user actions and their expected climate impact, businesses can start to have a holistic picture of the total positive climate impact generated by their app or service.

Green fintech represents a growing category that merges business value with environmental benefits, as consumers increasingly adopt ethical financial practices and regulations emphasize sustainability. Products like carbon calculators, green loans, and sustainable investment options are designed to help individuals and businesses reduce their environmental footprint. However, the focus is shifting toward encouraging enterprises to adapt their operations through innovative design thinking and actionable product features. Developers are urged to employ design thinking strategies to identify high-impact areas and create solutions that enable users to make sustainable choices, using metrics and monitoring to measure their impact. By leveraging skills in data analysis and hypothesis testing, app developers can enhance green fintech product offerings and contribute to a broader positive climate impact.

Alerts help identify and address issues before they impact customers, but too many alerts can be overwhelming and slow down incident response. An optimized alert strategy is crucial for observability, focusing the team on the right things at the right time to increase uptime and performance. Alert quality management (AQM) helps create fewer, more valuable alerts that pinpoint incidents and minimize alert fatigue. To optimize the strategy, follow best practices for alerts and notifications, such as creating alerts that matter to your business, taking advantage of automatic anomaly detection, configuring notification workflows, setting up and tracking alert metrics, and reviewing AQM data to improve alert policies. By implementing these strategies, teams can streamline their workflow, reduce noise, and increase the value of their alerts.

Effective alert management is crucial for maintaining system performance and customer satisfaction, as it helps identify and address issues before they impact users. However, an excess of alerts can overwhelm teams, leading to desensitization and slower incident response. An optimized alert strategy, as discussed in the context of New Relic's tools, focuses on creating fewer, high-value alerts that minimize fatigue and highlight critical incidents. Key strategies include setting alerts that align with business priorities, leveraging anomaly detection for automatic identification of unusual trends, and configuring notifications to reach the appropriate team members at the right time. Additionally, tracking alert metrics is essential to ensure the quality of alerts and adjust policies to maintain system reliability and stability. This approach is part of a broader observability framework that encompasses operational efficiency and customer experience.

With SAP applications increasingly powering critical business processes, ITOps teams need a quick and intuitive way to understand system health and troubleshoot performance issues within their overall ecosystem. New Relic Monitoring for SAP solutions provides a centralized view of all SAP and non-SAP systems through one UI, enabling teams to inspect application performance, infrastructure, and business processes in real-time. With SAP certification, it works seamlessly with SAP technology, allowing teams to spot problems quickly with visualizations, prevent issues before they occur, and gain end-to-end visibility into their entire ecosystem. By monitoring both SAP and non-SAP systems from a single view, teams can identify root causes, resolve issues efficiently, and drive business innovation without hindering their foundational business functions.

New Relic Monitoring for SAP solutions provides IT operations teams with a centralized platform to monitor both SAP and non-SAP systems, offering key insights into system health and performance issues. This SAP-certified tool allows users to view application performance, health data, and telemetry metrics in a single interface, facilitating quick identification and resolution of problems through customizable visualizations and advanced alerting functionalities. By integrating business processes like order-to-cash and procure-to-pay into the monitoring system, New Relic enhances communication and collaboration between business and IT teams. The solution includes features such as SAP IDoc explorer for error tracking, dynamic baseline alerts, and the ability to monitor systems running on cloud platforms like AWS and Microsoft Azure. This integrated approach helps organizations preemptively address potential disruptions, ensuring smooth and efficient business operations.

The usage-based pricing model used by New Relic takes into account various factors that can affect data ingest costs for organizations. To estimate these costs, users can set up a test New Relic account and extrapolate their usage from it or use the cost estimator spreadsheet provided by the company. The spreadsheet allows users to input estimated data volumes for different types of data sources, such as APM agents, infrastructure agents, and logs, and provides an estimated total cost based on these inputs. Additionally, users can adjust retention settings to account for longer-term storage of data, which affects the overall cost. While the cost estimator is designed to provide directional guidance, it's essential to note that actual costs may vary depending on individual organization architectures and New Relic setups.

To estimate New Relic data ingest costs under a usage-based pricing model, it's essential to understand various influencing factors and utilize available tools like a test New Relic account or a cost estimator spreadsheet. Users receive 100 GB of free data ingestion per month, and the amount of data ingested can vary based on monitoring needs and application behaviors. A test account allows users to observe actual data usage, facilitating accurate cost predictions, while the spreadsheet provides rough estimates based on predefined assumptions. Data costs range from $0.35 to $0.55 per GB, and additional costs may arise from extended data retention, which can be adjusted across all data types. The blog emphasizes the necessity of assessing application and infrastructure characteristics to predict data ingest accurately, noting that factors like the number of agents or integrations and application complexity can significantly influence data volumes and costs.

PHP is a widely used server-side development platform, known for its flexibility and ease of coding. Several popular PHP frameworks are available to meet different development needs, including Laravel, Symfony, CodeIgniter, Zend, Yii, Phalcon, CakePHP, Slim, PHPixie, FuelPHP, Aura, and others. When choosing a framework, it's essential to consider factors such as performance, scalability, ease of use, security features, and compatibility with MVC architecture. Some frameworks, like Laravel and Symfony, are well-suited for large-scale development, while others, like Slim and Phalcon, may be better suited for smaller applications or prototyping. Ultimately, the choice of framework depends on individual project requirements and developer preferences.

This summary highlights the importance of monitoring Node.js applications in ensuring performance, identifying issues, and maintaining system health. It explains that monitoring is not a one-time task but an ongoing process as new challenges and possibilities emerge. The article discusses common challenges associated with Node.js monitoring, including asynchronous execution, event loop insights, memory leaks, real-time performance monitoring, scalability, and distributed tracing. To overcome these challenges, it recommends using specialized monitoring tools, collecting key metrics, implementing asynchronous tracing, setting up alerts, monitoring dependencies, log aggregation, and security monitoring. The article concludes by encouraging developers to start monitoring their Node.js applications with New Relic, an observability platform that helps build better software.

Node.js monitoring is crucial for ensuring optimal performance, system availability, and user satisfaction in applications built with this event-driven, non-blocking architecture. Since applications are never truly finished and can encounter various issues post-deployment, monitoring becomes essential to identify and address anomalies, reduce latency, and optimize user experiences. Effective monitoring involves tracing code execution, visualizing application health, and managing key metrics like CPU and memory usage, latency, and error rates. Specialized tools like New Relic offer prebuilt dashboards and alerts to help developers monitor these metrics, enabling them to quickly diagnose performance problems, especially in asynchronous and high-traffic environments. By employing best practices such as real-time monitoring, asynchronous tracing, and log aggregation, developers can maintain the health of Node.js applications, ensuring scalability and reliability while addressing common challenges like memory leaks and distributed tracing in microservices architectures.

Accessing application logs in the context of application performance monitoring (APM) metrics has been a long-standing request for developers due to its importance in efficiently troubleshooting performance issues. The new updated APM agents from New Relic now automatically collect and forward logs without requiring manual configuration or setup, making it easier for developers to troubleshoot applications and diagnose performance issues. This feature addresses the need for log collection by providing granular configurations tailored for billing and security needs, minimizing context-switching, and enhancing the UI to surface logs alongside APM metrics and events. The update also includes robust support for security, compliance, and control, including opt-out anytime options, ingest control, compliance, and data security features. With this new feature, developers can collect logs via the APM agent, view application logs in context, use logs where needed in the enhanced UI, ensure data privacy by managing their logs, and get started with automatic logs in context.

When Simply Business started, they sought to incorporate best practices in tech and software development into their operations. However, they found that these best practices didn't always translate into effective cultural processes for a scale-up. They shifted from an Elasticsearch, Logstash, and Kibana (ELK) stack to using New Relic log management across all software engineering teams. This transition taught them three key lessons: creating pre-filtered saved views, using tagging to make accessible dashboards, and avoiding best-in-breed tools that breed complexity. By simplifying their tech stack and focusing on a single tool, they were able to improve developer productivity, reduce mental strain, and save money. They now use New Relic as their primary observability tool, providing all the data their developers need in one place, eliminating the need for multiple interfaces and query languages.

Simply Business transitioned from using an ELK stack to New Relic for log management to streamline operations and improve digital insurance services. Initially, the company found that the complexity and underutilization of the ELK stack features impeded efficiency, especially for non-technical teams. By adopting New Relic, Simply Business gained a more intuitive, centralized platform that facilitated easier access to logs and data across different departments, enhancing collaboration and customer experience without the need for extensive technical knowledge. This shift also reduced costs and mental strain for developers, who no longer needed to juggle multiple tools and interfaces, thus enabling the company to maintain a leaner, more effective observability toolkit.

To achieve observability across your stack, it's essential to collect and analyze telemetry data from multiple sources, including metrics, events, traces, and logs. New Relic Query Language (NRQL) allows you to connect and explore your data with subqueries, enabling more sophisticated queries across different data sources and time ranges. Subqueries can be used to associate app errors and performance, find trends in multiple time ranges, and connect transactions and spans. To use subqueries effectively, it's recommended to test them separately, understand their limitations, and upgrade your query experience with Data Plus. With the new subquery feature in NRQL, you can unlock more powerful queries and gain deeper insights into the health and performance of your apps and systems.

New Relic Query Language (NRQL) has been enhanced with the ability to include subqueries, allowing users to perform more complex and insightful data analysis across multiple sources and time ranges. Subqueries, which are queries nested within other queries, enable the combination of data from various sources, enhancing the observability of application performance and system health. Examples include connecting app errors with performance metrics, analyzing trends over different time frames, and associating log data with transactions. These advancements facilitate easier data exploration and reveal deeper insights into system behaviors, overcoming previous limitations where such correlations required manual input. As users become more adept at using NRQL subqueries, they can explore a wider range of data insights, leveraging the increased query limits and performance enhancements offered by the Data Plus service.

A resilient testing suite for your application is crucial to ensure reliability and confidence when deploying code changes. However, it's equally important to simulate the behavior of your users to anticipate their interactions with your platform. Synthetic user testing, also known as synthetics, allows you to test how your users interact with your application in a controlled environment, simulating real-world scenarios. This type of testing can help identify unhappy paths and ensure robustness in your application. With tools like New Relic's synthetic monitoring options, developers can create and manage their own synthetic tests, gaining a deeper technical understanding of how these tests work, which enhances their appreciation for the capabilities offered by New Relic. Ultimately, incorporating a robust synthetic testing suite into your testing framework is essential for creating confidence in your code as you continue refactoring and introducing new features.

The blog post discusses the importance of incorporating synthetic user testing into a testing suite, alongside unit, integration, and acceptance tests, to ensure the reliability and performance of applications such as e-commerce platforms. It explains how synthetic testing, or synthetics, involves simulating user behavior to identify potential issues by using tools like New Relic synthetic monitoring and manual setups with Node.js, Chromedriver, and Selenium Webdriver. The post provides a tutorial on building a synthetic monitoring test for a website form, emphasizing the importance of testing both "happy" and "unhappy" user paths to anticipate possible interactions and responses. It also highlights the option of using New Relic to manage and create synthetic tests efficiently, encouraging developers to integrate robust synthetic testing into their existing frameworks to maintain confidence when deploying code changes.