Mutation Testing

Focus on one file at a time

If you have a large number of surviving mutations, it’s best to focus on one at a time.

In the frontend, you can run stryker on one file at a time like this:

npx stryker run -m /full/path/to/the/file/being/mutated.jsx

You can copy this full path by right clicking on the top of the tab in VSCode. Remember that you want:

• the path of the implementation of the component (e.g. FooTable.js) ✅
• not the path for the test of the component (e.g. FooTable.test.js) ❌

For the backend instructions for running pitest on only a portion of the code can typically be found in the README.md, for example

• ucsb-cs156-f25/STARTER-team02
• proj-courses
• proj-dining
• proj-frontiers
• proj-rec

Ask: is the code really needed?

Typically, when there’s a mutation that survives, the code is essential code for which you need to write a test.

But sometimes, the fact that the test suite passsed without that line of code is a sign that you might not need that code.

This is often the case in the frontend with data-testid values.

Suppose all other tests are passing, you have 100% test coverage, and the last surviving mutation is something like this:

• You could write a test to detect that this data-testid value shows up in the code.
• But it woudl be better to just eliminate this data-testid value!

Here’s why: the sole purpose of data-testid values is to help you locate specific elements on the page when writing tests. It’s always better to find those elements by text, or label text, or something visible to the user; but occasionally, that’s not feasible (e.g. when there could be multiple elements with the same text.

If you’ve already written all the tests you need, and you never had a need for this particular data-testid value, you can just eliminate it.

This is not the only case where just eliminating the code may be the best choice to eliminate a surviving mutation.

A few other examples:

• Default values in components that are never actually used or needed in the application.
  • Sometimes we add default values when first setting up a component, and they may be helpful for storybook or testing, but are never used in the application code.
  • Rather than writing tests to cover the defaults, just fill in the default value in the tests and storybook, and then eliminate the default value.
• The {" "} constructor in React Components
  • Sometimes this is needed to preserve space between words (in which case you can write a test to ensure that the space between words is there.)
  • But often, you can just eliminate these spaces (which are inserted by prettier when you do npm run format) and there is no impact on the displayed text.

• Sorting/Filtering in the frontend (if it can be moved to the backend)

  Here’s some code that sorts data in a table:

  By moving the sort to the backend as explained here, we were able to eliminate this code, and there the need to write a frontend test for it.

Stryker was here mutations

If you are having difficulty killing a Stryker was here mutation, here’s a strategy:

1. Make the mutation your self in the code. For example, actually change:

     const [courseNumber, setCourseNumber] = useState(");
   

   to

      const [courseNumber, setCourseNumber] = useState("Stryker was here);
   

2. Then, load the component on Storybook and see if there is a circumstance where you can actually see the impact of the mutation on the component.

3. Then, try to write a test that fails because of the impact of the mutation.

Note that this is actually a strategy for any kind of mutation, but the Stryker was here is often the easiest to see in the component.

For example, when I made the change above, I got this:

The problem with my test is that I was checking for the contents of this field like this:

expect(screen.getByTestId("CourseOverTimeSearchForm.SearchString")).toHaveTextContent("ANTH");

when I needed to use a regular expression to be sure that I wasn’t just finding a substring. The ^ means “start of string”, and the $ means end of string.

expect(screen.getByTestId("CourseOverTimeSearchForm.SearchString")).toHaveTextContent(/^ANTH$/);