Java edition

Returns and errors

Operators that change what a method gives back: the value it returns, its whole body, or the exception it throws. Together they ask whether your tests look at a method’s answer, and at the errors it’s meant to raise.

Operator	Changes	Set
`mutastic/return-value`	a returned value to the other `boolean`, `0`, an empty one or `null`	defaults
`mutastic/replace-body`	a method’s whole body, to returning one value or nothing	extended
`mutastic/remove-throw`	a `throw`, removed, and the code carries on	extended

Each example is a class in samples/operator-docs with a test that misses its mutant, and each card is what mutastic run samples/operator-docs --operators=all prints for it.

Return value

mutastic/return-value, in the defaults. A method returns the value its kind of method most plausibly gets wrong, instead of what it computed:

The method returns	The mutant returns
`boolean`	the other one: `true` becomes `false`, `false` becomes `true`, and a computed one becomes each
a number	`0`, or `1` where it returns `0`
`Optional`	`Optional.empty()`
`String`	`""`
`List`, `Collection` or `Iterable`	`List.of()`
`Set`, `Map`	`Set.of()`, `Map.of()`
`Stream`	`Stream.empty()`
`Boolean`, `Integer`, `Long` and the other boxed numbers	as their primitive does
anything else	`null`

The bug it stands for: a method that doesn’t return what it worked out. This is less a typo than a sign: tests that call the method but never look at its answer. If returning the other boolean, 0, an empty list or null passes every test, the tests run the method without checking it, and the coverage report says it’s tested when it isn’t. The value is the one a method of that kind most plausibly gets wrong: a check that answers the wrong way, a count of nothing, a search that finds nothing. That’s why a list becomes an empty list rather than null: code that forgot to add the results returns one, and a null would fail on its first use, which any test notices.

  public int points(int spent) {
    int points = spent / perPoint;
    return points;
  }

The test only tries a spend too small to earn anything, so its answer is the 0 the mutant returns:

  @Test
  void smallSpendsEarnNothing() {
    assertEquals(0, new Loyalty(10).points(5));
  }

  ✗ MISSED  no test checks what points() returns here
    ┌─ src/main/java/com/example/returns/Loyalty.java:13
    │
 12 │     int points = spent / perPoint;
 13 │     return points;
    │     ^^^^^^^^^^^^^^ now returns 0
 14 │   }
    │
    = mutant  - return points;
              + return 0;
    = note: tests reach this return, but none fails when it returns 0
            inputs seen: spent = 5
            covered by LoyaltyTest#smallSpendsEarnNothing (its 1 covering test still passes)
    = help: assert the value points() returns on this path, with inputs where it isn't 0

            // in LoyaltyTest
            @Test
            void pointsWithSpent10ForPerPoint10() {
              var loyalty = new Loyalty(10); // as LoyaltyTest#smallSpendsEarnNothing builds it
              assertEquals(1, loyalty.points(10)); // today 1 · with this mutant 0
            }
            check today's value is the one you intend: the test pins what the code does now

    done when  mutastic verify m9e9ca7ae samples/operator-docs passes
    reproduce  mutastic run --only=m9e9ca7ae samples/operator-docs
    or ignore  mutastic ignore m9e9ca7ae --reason="<why>" samples/operator-docs

The test that catches it: one that asserts a result other than the mutant’s. A computed boolean gets both mutants, so it needs a test where the answer is true and one where it’s false; a number needs an answer that isn’t 0; a list needs an answer with something in it.

  @Test
  void aPointForEvery10() {
    assertEquals(3, new Loyalty(10).points(35));
  }

Edition notes: a return that already gives that value, return List.of(); say, can’t be changed by it, and is skipped as equivalent. A test that calls a method and asserts nothing about what comes back is the commonest gap this operator finds, and often the easiest to fix. The Rust edition has this operator too, mutastic/return-value, with Rust’s own empty values (String::new(), None, Vec::new(), Ok(..) of the inner value), and with --operators=extended adds Default::default() and an Err(..) the function returns elsewhere.

Replace body

mutastic/replace-body, in the extended set. A method’s whole body is replaced: it returns the value return value would choose, or nothing, if it returns nothing.

The bug it stands for: a method that’s wrong from top to bottom, or does nothing, and nobody would know: not a slip in one line, but code no test depends on. It costs in the code everyone assumes is tested because coverage says it runs: a method that tests reach but never check can be wrong in any way at all.

The same method and test as return value’s: the whole body returning 0 passes too.

  public int points(int spent) {
    int points = spent / perPoint;
    return points;
  }

  ✗ MISSED  no test notices points() doing nothing but returning 0
    ┌─ src/main/java/com/example/returns/Loyalty.java:12
    │
 11 │   public int points(int spent) {
 12 │     int points = spent / perPoint;
    │     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ now only returns 0
 13 │     return points;
    │
    = change  points() does nothing but return 0 (not shown in the source)
    = note: inputs seen: spent = 5
            covered by LoyaltyTest#smallSpendsEarnNothing (its 1 covering test still passes)
    = help: add a test that calls points() and asserts a result other than 0

            // in LoyaltyTest
            @Test
            void pointsWithSpent10ForPerPoint10() {
              var loyalty = new Loyalty(10); // as LoyaltyTest#smallSpendsEarnNothing builds it
              assertEquals(1, loyalty.points(10)); // today 1 · with this mutant 0
            }
            check today's value is the one you intend: the test pins what the code does now

    done when  mutastic verify m829aa2d2 samples/operator-docs passes
    reproduce  mutastic run --only=m829aa2d2 samples/operator-docs
    or ignore  mutastic ignore m829aa2d2 --reason="<why>" samples/operator-docs

The test that catches it: one that calls the method and asserts a result other than that value. The test that catches return value’s mutant above catches this one too.

  @Test
  void tenSpentIsAPoint() {
    assertEquals(1, new Loyalty(10).points(10));
  }

Edition notes: use it as a fast first pass, alone, with --operators=mutastic/replace-body: it has few mutants, and each miss is a whole method to look at. It isn’t a measure of how well the rest is tested, as any test that looks at the result catches these. A method gets one mutant, except one returning boolean, which gets both values: always true and always false are different gaps, and neither shows the other. A value the method’s one return already gives isn’t used. Constructors and static initialisers are left alone, and trivial getters and setters, toString() and hashCode() are skipped as arid. A body that’s only the return of that value is skipped as equivalent, as with return null;. The Rust edition has this operator too, mutastic/replace-body, and leaves alone new, fmt, hash and a getter of one field.

Remove throw

mutastic/remove-throw, in the extended set, and experimental. A throw is removed, and the code carries on with what comes next.

The bug it stands for: a check that doesn’t stop anything, or an exception swallowed where it should have gone up. It costs in validation: bad input carries on into the code that assumed it had been refused, and what comes out is wrong, or a different error from the one the caller handles.

  public static int parse(String text) {
    if (text.isEmpty()) throw new IllegalArgumentException("no quantity given");
    return Integer.parseInt(text);
  }

Without the throw, parseInt("") throws a NumberFormatException, which is an IllegalArgumentException too, so the test passes either way:

  @Test
  void noQuantityIsAnError() {
    assertThrows(IllegalArgumentException.class, () -> Quantities.parse(""));
  }

The condition never true leaves the throw out just the same, so the card holds both:

  ✗ MISSED  no test needs text.isEmpty() to be true in parse() · m495af6bc
   ┌─ src/main/java/com/example/returns/Quantities.java:5
   │
 4 │   public static int parse(String text) {
 5 │     if (text.isEmpty()) throw new IllegalArgumentException("no quantity given");
   │     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ now always false
 6 │     return Integer.parseInt(text);
   │
   = mutant  - if (text.isEmpty()) throw new IllegalArgumentException("no quantity given");
             + if (false) throw new IllegalArgumentException("no quantity given");
   = also    ✗ no test needs parse() to throw IllegalArgumentException here · mf69bfac0
   = change  the throw of IllegalArgumentException is removed, so the code carries on (not shown in the source)
   = note: covered by QuantitiesTest#aQuantity, QuantitiesTest#noQuantityIsAnError (all 2 covering tests still pass)
   = help: add a test where text.isEmpty() is true, and assert what parse() does then

   done when  mutastic verify m495af6bc,mf69bfac0 samples/operator-docs passes
   reproduce  mutastic run --only=m495af6bc,mf69bfac0 samples/operator-docs
   or ignore  mutastic ignore <id> --reason="<why>" samples/operator-docs for each of m495af6bc, mf69bfac0

The test that catches it: one where it throws, asserting that it’s this exception, not one that happens to be there anyway.

  @Test
  void noQuantityIsRefusedAsSuch() {
    var refused = assertThrows(IllegalArgumentException.class, () -> Quantities.parse(""));
    assertEquals("no quantity given", refused.getMessage());
  }

Edition notes: run it with --operators=defaults,extended. The code after it has to be able to run: with nothing after it, as at the end of a method, the mutant is skipped as unviable:. javac’s own throws aren’t changed: the rethrow it writes for a finally, a synchronized or a try-with-resources, and the default it writes for a switch with none. The throw of a failed assert is skipped as arid. The Rust edition has this operator too, mutastic/remove-throw, for return Err(e);, panic!(..) and bail!(..), and adds mutastic/unwrap-try, which turns f()? into f().unwrap().