Introduction to Testing

Testing Basics

Testing is an essential quality assurance step in the software development process. It serves to verify that your code functions correctly and reliably.

Proficiency in testing distinguishes adept developers and successful companies from their counterparts. An effective testing strategy should encompass a wide range of scenarios while remaining efficient to optimize resource allocation.

In this session, we will delve into the significance of testing, elucidate the consequences of neglecting it, and explore the various categories of software tests at your disposal. Please join us in examining the critical role that testing plays in software development!

Understanding the risk of not testing

The impact of skipping automated tests

Let's explore two fundamental tasks commonly encountered by software developers: fixing bugs and adding features.

When the codebase remains modest, such as a small application started on a given morning, adding a feature or correcting a bug is relatively quick. Testing these alterations, often performed manually by simulating user interactions, appears efficient. This practice, called manual testing, proves effective within small-scale codebases.

However, as the codebase expands in size and complexity, issues inevitably emerge. Any modification within the application could introduce bugs elsewhere, a phenomenon aggravated in monolithic codebases. Even seemingly minor changes, like rectifying a typo, can yield unexpectedly widespread repercussions — the "blast radius" — all while the original code authors may no longer be available for consultation.

How does manual testing perform in this scenario?

For substantial applications, manual testing becomes a lengthy effort, requiring an individual several days or even weeks to complete. This meticulous process involves scrutinizing a thorough checklist of numerous tasks, each demanding precise verification: "Register a new user, perform a series of specific actions, log out, log back in, execute particular operations, and inspect the inbox to ensure the correct arrival and formatting of specific emails." Each task is just one entry on this extensive list. It's a demanding and error-prone undertaking that no one wishes upon their worst adversary.

Owing to the time-intensive nature of manual testing, it becomes unfeasible to comprehensively test individual work units in isolation. The cost associated with manual testing gets exponentially higher due to the increased communication needed, the number of people involved, and the tight timeline testing is typically constrained to (the tight timeline is usually brought about by a misunderstanding of how long and crucial good testing takes).

One of the most costly issues of manual testing is the inability to test everything after each change. Imagine a developer effectively addresses a bug. The bug fix is deployed, and time passes until a prominent customer directly reports a problem to the CEO — an embarrassing predicament. Astonishingly, the prior "fix" introduced a new bug elsewhere in the application, a connection that remained unnoticed.

A new ticket is generated and assigned to a different developer, who identifies and rectifies the issue. However, this new "fix" inadvertently reverts the original bug, leading to a regression detected only later.

Externally, significant codebase enhancements appear to be underway. In reality, an undisclosed number of defects resurrect, incurring hidden costs of tens of thousands of dollars.

After years of such practices, incorporating new features into the application becomes exceedingly daunting. Developers hesitate to make alterations due to uncertainties about potential repercussions. Productivity diminishes, customer satisfaction weakens, and management grapples with frustration. Simply increasing the number of developers on the team does not mitigate the relentless influx of defects (in fact, increasing developer sizes can perpetuate the issue further if the issue is not properly identified and managed).

Ultimately, manual testing accrues millions of dollars in expenses, all factors considered.

The Impact of Embracing Automated Testing

If the challenges above seem daunting (which they indeed are), let's explore a vision of an alternative reality that automated testing can bring — an aspirational scenario.

Automated tests are code artifacts framed to verify that specific sections of your code function as intended. Various testing philosophies, approaches, and frameworks abound. Still, the fundamental principle remains consistent: if you create a function that returns "Hello World," you'll also compose an automated test that executes the function and verifies that the outcome equals "Hello World." In the event of this correspondence, the test succeeds; otherwise, it fails.

Indeed, producing one or more automated tests for each feature or bug fix can seem time-consuming, especially for those with limited experience. This lures many junior developers to forgo test writing, often with the implied approval of misguided managers. However, it's vital to recognize that the perceived time savings in skipping testing are unreal.

As tests accumulate, the codebase becomes increased with thousands of individual validations. Each test executes swiftly, with the complete suite taking only seconds. Consequently, it becomes entirely practical to assess the entirety of the codebase each time an alteration is made, regardless of its scale. It's akin to having a magical button that, when pressed, instantly informs us of any application issues introduced during development. This button remains endless and is available for frequent use at critical milestones, such as merging commits into the main branch or deploying the latest app version to production.

A growing repertoire of automated tests significantly reduces the occurrence of bugs, preventing the need for a dedicated manual testing team. The product becomes substantially more robust and predictable, facilitating iterative development. Developers can confidently make code changes, knowing that automated tests will detect any unexpected disruptions in distant corners of the application.

A virtuous cycle emerges as well. Developers who consistently craft tests for their code recognize that the same functionality can be implemented in two ways: one that is easy to test and another that is challenging. Consequently, they gravitate toward coding practices that enhance testability to save time. Significantly, code that is easy to test is invariably more robust than code that is not.

If a codebase proves challenging to test automatically, it often signifies suboptimal code quality. It could indicate entangled problems within a complex code web or huge functions tasked with various responsibilities. Regardless of the specific issues, refactoring to enhance testability invariably results in better, more understandable, and maintainable code (editor's note: I cannot express how true everything on this page is.).

Moreover, tests function as living documentation that remains consistently accurate. Sometimes, understanding a code segment's intended behavior is simpler by studying its tests rather than relying on its potentially lacking documentation.

Frequently Asked Questions

Is a program shielded from defects when extensively tested? Absolutely not, but testing substantially reduces bugs at a modest cost. The benefits are so significant that testing is an unquestionable necessity.

Isn't it tedious to create tests for every feature or bug fix? Writing tests is akin to delaying a purchase for a few minutes to pay with cash instead of immediately resorting to a high-interest credit card. Over time, the interest cost far exceeds the expense of the items purchased.

Do software development teams create tests for their clients? Occasionally, they do not. However, this is equivalent to transmitting a ticking time bomb to every individual who will work on the project in the future. Skipping tests is as unprofessional for a developer as disregarding hygiene practices for a doctor or a chef. Moreover, if the product endures without tests, it inevitably becomes rigid, fragile, and challenging, as discussed above.

How do we ensure the tests themselves are accurate? Although tests are not subject to testing, poorly constructed tests may yield false positives as the application evolves. This is considerably less detrimental than having no tests.

Shouldn't a testing specialist be responsible for creating all tests? Given that the author of the feature or bug fix possesses the most context, they are the best-equipped individuals to incorporate tests as part of the task. With modern tools like chatGPT, tests can be written expeditiously if the code adheres to sound practices. There is simply no justification for neglecting testing.

Differentiating the types of testing

Software testing is typically categorized into three distinct scopes, each serving unique purposes and frequently conducted during different phases of the development process. Moreover, these testing activities are often assigned to different developers or teams.

Unit Testing

Imagine you're building a robot, and you want to make sure that each tiny part of the robot works perfectly before assembling it. Unit testing is a lot like that. It focuses on testing individual pieces or units of your software, like small functions or methods. The goal is to ensure that each part works correctly on its own. If you're writing a calculator app, unit tests might check if addition, subtraction, multiplication, and division functions all give the right answers. Unit tests are like the building blocks of testing, helping you catch bugs early and making it easier to fix them.

• Purpose: To test the smallest pieces of code in isolation (e.g., individual functions, methods).
• Scope: Limited to a single "unit" like a function or method within a class.
• Isolation: External dependencies are generally mocked or stubbed out.
• Speed: Typically very fast.
• Examples: Testing if a sorting algorithm returns a sorted array, checking if a utility function correctly validates email formats.
• Geeky Detail: Unit tests should be deterministic, meaning given the same input, they'll produce the same output every time.

Integration Testing

Now, think about putting your robot together. You want to be sure that all the parts work well together. Integration testing is similar – it checks if different units or components of your software can collaborate effectively. For example, in a video game, integration tests might ensure that characters, items, and the game world interact smoothly. This type of testing helps you catch issues that might arise when different parts of your software communicate or connect. It's like making sure all the gears in your robot fit together perfectly.

• Purpose: To test the interaction between multiple pieces of code or system components.
• Scope: Covers multiple units, but not necessarily the whole system.
• Isolation: May or may not mock external services, depending on what is being integrated.
• Speed: Generally slower than unit tests, but faster than end-to-end tests.
• Examples: Testing if a React component correctly fetches and displays data from an API, verifying if a database layer correctly saves data to an actual database.
• Geeky Detail: These tests can help catch issues like incorrect data flow between functions, timing issues, or inconsistencies between different parts of a system.

End-to-End Testing

Imagine your robot is now ready to perform a real task, like cleaning a room. End-to-end testing is like observing your robot in action to see if it completes the entire job correctly. In software, this means testing the entire application from start to finish, just like a user would. For a website, it could involve checking if a user can sign up, log in, browse content, and make a purchase seamlessly. End-to-end testing ensures that your software functions as a whole and provides a great user experience. It's like watching your robot clean an entire room without missing a spot.

• Purpose: To test a flow from start to finish in a system as a user would experience it.
• Scope: The entire application and its associated components, often including third-party services.
• Isolation: Usually no mocking; tests interact with the real system.
• Speed: Typically slow due to complex setups and teardowns.
• Examples: Testing an entire login flow, including filling out fields in a web browser, clicking buttons, and verifying outcomes.
• Geeky Detail: E2E tests can be "flaky" (i.e., prone to random failures) due to their complexity and the number of moving parts.

In summary, unit testing focuses on testing individual parts of your software, integration testing checks how those parts work together, and end-to-end testing evaluates the entire application from a user's perspective. Each type of testing plays a crucial role in ensuring your software is reliable and works as intended. Just like building a robot, these testing approaches help you create software that performs its tasks flawlessly and delivers a great user experience.

Function add(a, b):
    Return a + b

Test "Adding two numbers":
    Result = add(5, 3)
    Assert(Result, 8)  // Ensures that the result of the add function is 8 when passing 5 and 3

Function connectToDatabase():
    // Logic to connect to the database
    Return connection

Function fetchData(connection):
    // Logic to fetch data from the connected database
    Return data

Test "Fetching data from the database":
    Connection = connectToDatabase()
    Data = fetchData(Connection)
    Assert(Data, ExpectedData)  // Ensures that the fetched data matches the expected data

Function OnPageLoad():
    // get data needed for this page
    userData = getUserData()
    menuData = getMenuData()

Function getUserData():
    // makes a call to the backend database for the user's data
    return fetchData("users/1")

Function getPageData():
    //make a call to the backend database for the page's data
    return fetchData("menu")

Function fetchData(path):
    // Logic to fetch data from the connected database
    Return backend.fetchData(path)

Test "Fetching data from the backend server":
    OnPageLoad()
    waitForFetchesToComplete()
    Assert(userData, ExpectedUserData)  // Ensures that the fetched data matches the expected data
    Assert(menuData, ExpectedMenuData)

Conclusion

In recognizing the perils of an inadequate or absent testing strategy, it becomes evident that testing is a cornerstone of the software development process. Embracing this practice not only ensures robust software but also distinguishes you as a diligent team member.

Software testing encompasses a diverse range of categories and tools. It's essential to grasp that coding transcends mere logic formulation. As applications undergo evolution and maintenance—sometimes by various teams over extended periods—it's invaluable to document and test your code thoroughly during its creation. This proactive approach ensures longevity and ease of future adaptations.

Tests in Action

One of the most popular testing libraries for JavaScript is JEST. Created by Facebook, JEST is versatile, fast, and comes with everything you need out of the box. Whether you're testing simple JavaScript functions or complex interactions in your applications, JEST provides an intuitive syntax and a robust set of features to help you.

While JEST focuses on the general JavaScript testing ecosystem, the React Testing Library (often just called RTL) narrows its sights on React components. Rather than dealing with component instances or trying to manipulate the state directly, RTL encourages you to test your components in a way your users would use them. This leads to more reliable, maintainable tests that ensure your UI works as expected.

Together, JEST and the React Testing Library form a powerful duo, enabling you to write tests that are both comprehensive and user-centric. As you embark on this Core Competency, remember that while writing tests might seem like extra work initially, they often save time and stress in the long run by catching issues before they become problems.

Let's embark on this exciting journey to ensure our applications not only function but thrive in the real world. Happy testing!

Unit testing with Jest

Unit testing is a software testing method that focuses on individual components or functions within a program. These components are isolated and tested independently to verify their correctness. Unit testing involves creating small, specific tests that assess the behavior of these components, ensuring they perform as intended and catch any bugs early in the development process.

The primary goal of unit testing is to confirm that each piece of code functions correctly in isolation before integrating it into the more extensive system. This approach provides rapid feedback and improves code maintainability, ultimately enhancing overall software quality and reducing the likelihood of defects in complex applications.

Jest is a widely used testing framework that provides all the tools to create unit tests easily. Here is a table that outlines some of the most frequently used (conditionals called "matchers") in the JestJS testing library:

npx @bloomtools/react@0.1.16 unit-tests
cd unit-tests
npm install
npm run dev

test("sanity", () => {
  // a single unit test
  let expected = 5; // setting up our expectation
  let actual = 2 + 2; // computing the actual value, which might meet expectations or not
  expect(actual).toBe(expected); // a single assertion inside the test (there can be multiple assertions)
});

npx jest unit.test.js

function mathematize(num1, num2) {
  // normally this is imported from another module
  num1 = parseInt(num1);
  num2 = parseInt(num2);
  if (isNaN(num1) || isNaN(num2)) throw new Error("cannot sum those numbers");
  return {
    addition: num1 + num2,
    subtraction: num1 - num2,
    divisible: num1 % num2 === 0,
  };
}

describe("mathematize function", () => {
  // the describe is used to organize groups of tests
  // Test case 1: Any input numbers
  test("returns an object with addition, subtraction and divisible keys", () => {
    const result = mathematize(1, 2);
    expect(result).toHaveProperty("addition");
    expect(result).toHaveProperty("subtraction");
    expect(result).toHaveProperty("divisible");
  });
  // Test case 2: Valid input numbers
  test("correctly calculates addition, subtraction, and divisibility", () => {
    let result = mathematize(10, 3);
    expect(result.addition).toBe(13);
    expect(result.subtraction).toBe(7);
    expect(result.divisible).toBe(false);
    // IMPORTANT: to assert that objects have the same shape, use `toEqual` instead of `toBe`:
    expect(result).toEqual({ addition: 13, subtraction: 7, divisible: false });
    // trying different numbers
    result = mathematize(6, -2);
    expect(result.addition).toBe(4);
    expect(result.subtraction).toBe(8);
    expect(result.divisible).toBe(true);
  });
  // Test case 3: Input numbers that are not integers
  test("throws an error for non-integer input", () => {
    const message = "cannot sum those numbers";
    // the code we expect to throw must be wrapped inside a callback:
    expect(() => mathematize("abc", 5)).toThrowError(message);
    expect(() => mathematize(10, "xyz")).toThrowError(message);
    expect(() => mathematize("yes", "no")).toThrowError(message);
    expect(() => mathematize(10, NaN)).toThrowError(message);
    expect(() => mathematize(15, undefined)).toThrowError(message);
    expect(() => mathematize(null, 7)).toThrowError(message);
  });
  // Test case 4: Division by zero scenario
  test("checks divisibility with divisor being zero", () => {
    const result = mathematize(20, 0);
    expect(result.addition).toBe(20);
    expect(result.subtraction).toBe(20);
    expect(result.divisible).toBe(false); // Division by zero is not possible
  });
});

npx jest unit.test.js # runs the one file and exits
npm test # uses the "test" script in the package.json (check it!) and runs all test files in the project, watching for changes in the code (You can edit the JS file and trigger an automatic re-test, pretty cool!)
npm test -- unit.tests.js # uses the "test" script in the package.json and runs the one file, watching for changes in the code

Integration testing with RTL

Conducting tests on components using React Testing Library (RTL) is generally classified as UI (User Interface) testing, and more precisely, it falls under the category of integration testing. This distinction arises because components are not as straightforward and isolated as typical functions subjected to unit testing.

UI testing places its focus on examining the interplay among various components and how they collaborate to construct the user interface. React Testing Library is intentionally crafted to evaluate how components behave within a real-world context. It promotes the creation of tests that closely mimic a user's interaction with the application.

However, it's important to note that these tests do not qualify as end-to-end tests. This is because they scrutinize only portions of the application within a simulated DOM, rather than launching a full-fledged web browser to assess the entire application.

Given that React Testing Library assesses component interactions and their contribution to the overall user interface, it falls under the realm of integration testing.

It's worth mentioning that React Testing Library does not delve into inspecting the internal implementation details of components. Consequently, developers may still need to resort to unit testing for evaluating specific functions within the application before proceeding with RTL testing.

npx @bloomtools/react@0.1.16 rtl-tests
cd rtl-tests
npm install
npm run dev

import React, { useState } from "react";

export default function App() {
  const [friends] = useState(["Cynthia", "Fish", "Alex"]);
  return (
    <div data-testid="friends">
      <h2>Friends App</h2>
      <div>
        {friends.map((friend, idx) => (
          <div key={idx}>My friend {friend}</div>
        ))}
      </div>
    </div>
  );
}

import React from "react";
import { render, screen } from "@testing-library/react";
import "@testing-library/jest-dom";
import App from "../App";

test("the container div is present", () => {
  render(<App />); // we need to render App for each test
  const div = screen.queryByTestId("friends"); // note the <div data-testid="friends"> in the JSX above
  expect(div).toBeInTheDocument(); // here is an assertion
});

npm test -- App.test.js

test('the text "Friends App" renders', () => {
  render(<App />);
  const heading = screen.queryByText("Friends App", { exact: false });
  expect(heading).toBeInTheDocument();
  expect(heading).toBeVisible(); // a node could be in the DOM but not be visible
});

test("inspecting the rendered DOM", () => {
  render(<App />);
  screen.debug();
});

test("using a loop", () => {
  render(<App />);
  ["Cynthia", "Fish", "Alex"].forEach((name) => {
    const friend = screen.queryByText(name, { exact: false });
    expect(friend).toBeVisible();
  });
});

test("using queryAllByText", () => {
  render(<App />);
  // friends is a collection of HTML nodes
  const friends = screen.queryAllByText("My friend", { exact: false });
  expect(friends).toHaveLength(3);
});

<form>
  <img alt="cute cat" src="./cat.jpg" data-testid="catImg" />
  <input type="text" placeholder="Type cat name" role="textbox" />
</form>

test("capturing nodes in different ways besides text content", () => {
  render(<App />);
  let img = screen.queryByAltText("cute cat"); // by alt attribute
  expect(img).toBeInTheDocument();
  img = screen.queryByTestId("catImg"); // by a data attribute "testid"
  expect(img).toBeInTheDocument();
  let input = screen.queryByPlaceholderText("Type cat name"); // by placeholder
  expect(input).toBeInTheDocument();
  input = screen.queryByRole("textbox"); // by role
  expect(input).toBeInTheDocument();
});

Conclusion

React Testing Library (RTL) emphasizes the importance of testing your components in a manner that resembles how users interact with them. One of the crucial aspects to bear in mind is the careful design of your document tree and associated tests. Minor alterations in the HTML structure or class names can lead to test failures. By being diligent in your test design, you shield yourself from these potential pitfalls, ensuring that your tests remain resilient against superficial changes in your codebase.

Furthermore, adopting a test-first or test-driven mindset fosters the creation of high-quality, maintainable code. RTL serves as a powerful tool in a developer's arsenal!

Guided Project

Intro to Testing Starter Code

Intro to Testing Solution

Module 4 Project: Introduction to Testing

This project will have you implement internationalization on a site! In particular you will make a form display in Spanish and in English. You will also use Jest to test some logic, and React Testing Library to test that the form renders correctly in both languages. By the end, you will have a fully functioning form that can switch languages, and unit tests and integration tests passing in your terminal.

The module project contains advanced problems that will challenge and stretch your understanding of the module's content. The project has built-in tests for you to check your work, and the solution video is available in case you need help or want to see how we solved each challenge, but remember, there is always more than one way to solve a problem. Before reviewing the solution video, be sure to attempt the project and try solving the challenges yourself.

Instructions

The link below takes you to Bloom's code repository of the assignment. You'll need to fork the repo to your own GitHub account, and clone it down to your computer:

Starter Repo: Introduction to Testing

• Fork the repository,
• clone it to your machine, and
• open the README.md file in VSCode, where you will find instructions on completing this Project.
• submit your completed project to the BloomTech Portal

Solution