September 2018 Summaries

Prolific Interactive is an award-winning, mobile-focused agency that operates with lean, cross-functional teams and measures success through growth outcomes. They build innovative apps using user-centric design, with Instabug as a key tool for feedback collection and analysis. The agency's Director of Product Development emphasizes the importance of fast feedback loops, user research, and partnerships in driving successful app development. With multiple offices across the US, Prolific Interactive focuses on digital products, including retail, e-commerce, lifestyle, and health apps. Their team structure is lean, with 90 people in their Brooklyn office and a growth team that works on marketing and acquiring users. They prioritize 'Show, don't tell' approach to development, partnering closely with clients throughout the process. Instabug plays a crucial role in their feedback workflow, enabling easy reporting of bugs and feedback, and facilitating collaboration across teams.

Prolific Interactive is a mobile-focused agency known for its innovative, user-centric app development process, which has earned numerous industry accolades. Operating like a tech startup, the agency utilizes lean, cross-functional teams to design and develop digital products, primarily focusing on mobile and related applications for various sectors including retail, e-commerce, and lifestyle. With a strong emphasis on partnerships rather than traditional client relationships, Prolific Interactive involves partners in the development process from the outset, using tools like Luciq and HockeyApp to facilitate feedback and iterative improvements. The agency does not have a dedicated QA team; instead, quality assurance responsibilities are shared across product managers and team members, with occasional outsourcing. Feedback is collected through Luciq, which integrates with Pivotal Tracker for streamlined issue tracking and resolution, enhancing the agency's ability to quickly address and iterate on user feedback.

Beta testing is considered a low-cost tactic to test an app and collect user feedback. It involves identifying the target market, creating a landing page, defining goals and planning, choosing the right tools, making sure the app is stable, using guerrilla usability testing, beta testing with friends and family, reaching out to existing users, using online recruitment channels, attending offline events, offering incentives, onboard users, segmenting testers, engaging with testers, gathering feedback, analyzing qualitative and quantitative results, and holding a post-mortem analysis. To run an efficient beta test, it's recommended to start with a small number of testers and gradually increase the number as needed, aiming for at least double the initial number. A minimum of 4-6 weeks should be allocated for a beta test cycle, and the size of the team should be considered when planning the test to avoid stretching them too thin. After the end of the beta test, a post-mortem analysis should be held to evaluate its success and identify areas for improvement.

Beta testing, a low-cost tactic for app development, involves gathering user feedback to enhance product quality and market readiness. Key steps include identifying the target audience and engaging with user communities, starting a blog and creating a landing page to build a following, and defining goals and choosing appropriate tools for testing. Recruiting beta testers can be done through guerrilla usability testing, using friends and family, reaching out to existing users, and leveraging online and offline recruitment channels. Incentives can motivate testers, who should be onboarded and segmented for effective feedback collection. Gathering qualitative feedback is crucial, and post-mortem evaluations help refine future beta testing processes. Luciq offers tools for bug reporting, performance monitoring, in-app surveys, and visual session replay to support beta testing, providing comprehensive data and insights to improve app development.

The practice of beta testing has gained traction and popularity, especially with the emergence of SaaS tools that facilitate the process. However, many beta programs commit common mistakes, such as not defining a clear plan or recruiting testers who don't represent the target audience. Rushing QA testing, recruiting too few or too many testers, ignoring usability bugs, and using the wrong tools for the job are also pitfalls to avoid. Additionally, it's essential to define and track metrics, educate beta testers, engage with them, and use the right tool for the job to ensure a successful beta test. Tools like Instabug can help streamline the process and provide valuable insights into app performance and user feedback.

Beta testing has become increasingly popular with the advent of SaaS tools that simplify the process, yet common mistakes can hinder its effectiveness. Key pitfalls include failing to define clear goals and a plan, inadequate internal QA prior to beta testing, and recruiting an inappropriate number or type of testers, which can skew results. Additionally, neglecting to educate and engage testers, overlooking usability issues, limiting testing environments, using unsuitable tools, and not tracking key metrics can all impair the success of a beta test. Luciq offers a comprehensive solution for beta testing mobile apps, providing tools for bug reporting, app performance monitoring, in-app surveys, and visual session replay, allowing developers to gather valuable user feedback and insights while maintaining user privacy.

At Instabug, they build an SDK for mobile apps and have developed a bespoke continuous integration workflow using CircleCI. Their workflow consists of four jobs: installing gems, running unit tests, generating a binary, and releasing the SDK. The first two jobs run in parallel and check for code quality, while the last two jobs are sequential and involve generating a binary and releasing the SDK. Instabug has automated their release process using CircleCI 2.0, which allows them to ship with confidence and save time by releasing once a week.

Luciq has developed a streamlined continuous integration workflow for their mobile app SDK using CircleCI 2.0, which enhances productivity by automating key processes. Their workflow consists of four primary jobs: running unit and integration tests with tools like Danger for code review automation; executing UI tests and static analysis; generating binaries upon approval; and releasing the SDK using a custom CLI app, Unleash, to manage tagging, GitHub releases, and distribution via CocoaPods and their own API. This system allows for parallel job execution, with UI tests running on specific branches and binaries generated post-testing, ensuring thorough checks while maintaining efficiency. By employing this automated CI process, Luciq can confidently ship weekly updates, significantly reducing manual effort and improving release reliability.

Technical beta tests focus on identifying and fixing bugs, crashes, and performance issues in the app's code and infrastructure to ensure a smooth user experience across various devices and environments. Marketing beta tests aim to gather user feedback, understand audience behavior, and optimize marketing strategies to improve user acquisition and engagement. A balanced approach that combines both types of testing can maximize benefits and provide valuable insights for product development and marketing teams. Conducting the test in stages, starting with internal dogfooding, followed by small closed beta tests with technical testers, and eventually opening it up to a broader audience, helps ensure that technical and marketing goals are met without interference. Tools like Instabug can facilitate this process by providing comprehensive bug reporting, mobile performance monitoring, in-app surveys, and visual session replay capabilities.

Understanding the distinction between technical and marketing beta tests is crucial for planning a successful beta test, as each serves different purposes and requires specific considerations. Technical beta tests focus on identifying and fixing bugs, optimizing app performance, ensuring compatibility with various devices and environments, evaluating infrastructure, and collecting feature requests from tech-savvy testers. In contrast, marketing beta tests aim to gather reviews, cultivate early adopters, test marketing messages and channels, and segment audiences for tailored campaigns, with less emphasis on bug detection and more on user interaction and feedback. Combining both types of beta tests can maximize benefits, with a recommended approach involving several stages—starting with internal testing, followed by a closed beta with selected testers, and eventually expanding to an open beta. Luciq offers a comprehensive beta testing tool that facilitates bug reporting, performance monitoring, in-app surveys, and session replays, providing valuable insights and enhancing user feedback through features like screen recordings, visual replays, and customizable privacy settings.