Plugins mutastic ships

Some operators aren’t for every project. They’re about a library or a language it may not use, or they cost more than a run should pay unasked. mutastic ships these as plugins, in the plugins directory of its distribution:

File	Adds
mutastic-plugins.jar	Annotation removal, Kotlin’s empty returns, proposed rewrites (experimental)
bigdecimal.yaml	BigDecimal and BigInteger swaps
reactive.yaml	Reactor and RxJava stages

The jar is also on Maven Central, as io.mutastic:mutastic-plugins, for a build that fetches its tools rather than keeping them. List what you want under plugins: in mutastic.yaml, wherever you keep it, by its path from the project’s directory:

plugins:
  - tools/mutastic-plugins.jar
  - tools/bigdecimal.yaml

Every operator here is in the extended set, so listing a plugin runs nothing until you choose it, by its namespace or by the set:

mutastic run --operators=defaults,annotations

mutastic run --operators=defaults,extended

Annotation removal

annotations/remove-validation and annotations/remove-security take one annotation away from what it’s on: a class, a field, a method, a parameter or a record’s component. The first removes Bean Validation’s constraints (jakarta.validation.constraints, the older javax ones, Hibernate Validator’s), @Valid and Spring’s @Validated. The second removes @PreAuthorize, @PostAuthorize, @PreFilter, @PostFilter, @Secured, @RolesAllowed and @DenyAll.

The bug it stands for. A constraint or a guard that’s missing, or on the wrong thing. Nothing fails until a value it should refuse gets through, or a caller it should refuse does.

Where it costs. More than an operator on code. An annotation has no line a test runs, and a test that a @PreAuthorize refuses never reaches the method at all, so coverage can’t say which tests matter: each mutant runs against every test, those that reach the class first, until one fails. What reads an annotation keeps what it read (a Validator in a field, a Spring context), so each also runs with every class loaded anew, and that context starts again. So a mutant no test catches costs a run of every test, and a context’s start for each context they use.

Where a framework loads the app itself (@QuarkusTest), the framework has read its annotations before a test starts, and a mutant can’t reach it: these mutants are skipped as unviable:. An annotation nested in a type, List<@NotBlank String>, isn’t one of these changes.

@Size(max = 3) private final String code;
// becomes
private final String code;

The test that catches it. One with a value the annotation refuses, or a caller it refuses, and an assertion that it’s refused:

@Test
void aCodeHasThreeLettersAtMost() {
  assertThat(validator.validate(new Order("a", "abcd", 1))).hasSize(1);
}

@Test
@WithMockUser(roles = "USER")
void onlyAnAdminAudits() {
  assertThatThrownBy(payroll::audit).isInstanceOf(AccessDeniedException.class);
}

BigDecimal and BigInteger

Money and other exact arithmetic go through methods rather than + and *, so mutastic’s own arithmetic operators never see them. bigdecimal.yaml has three operators, for BigDecimal and BigInteger: bigdecimal/swap-arithmetic swaps add and subtract, and min and max; bigdecimal/swap-scaling swaps multiply and divide; bigdecimal/remove-step removes abs, negate, setScale, round and stripTrailingZeros, and the value goes on as it was.

The bug it stands for. A refund that adds, a rate applied the wrong way round, a total that’s never rounded.

Where it costs. Next to nothing: a call becomes another. divide can throw on a value with no exact result, and a test that fails that way has caught it, whatever the card would have asked for. Swapping one RoundingMode for another isn’t among these: it’s an argument, and a rule changes calls.

return paid.subtract(kept);
// becomes
return paid.add(kept);

The test that catches it. One with values where the two differ, and an assertion on the result:

@Test
void aRefundTakesOffWhatsKept() {
  assertEquals(new BigDecimal("8"), Invoice.refund(new BigDecimal("10"), new BigDecimal("2")));
}

Reactor and RxJava

reactive.yaml has four operators, for Reactor’s Flux and Mono and RxJava 3’s types. reactive/remove-stage removes filter, distinct, distinctUntilChanged, take and skip. reactive/remove-recovery removes onErrorReturn, onErrorResume and the like, so the failure goes on. reactive/swap-match swaps any and all. reactive/swap-order swaps flatMap and concatMap.

The bug it stands for. The same as mutastic’s own stream stages: an item a stage should have held back, a failure nothing has tried. Swapping flatMap and concatMap stands for assuming an order that only holds by luck.

Where it costs. Next to nothing, though a rule matches the type a call is on as the code declares it, so a call on a subclass of Flux isn’t one. Where the inner streams can’t overlap, flatMap and concatMap do the same: nothing tells them apart, and it’s fine to ignore that mutant, saying so.

return amounts.take(2).collectList().block();
// becomes
return amounts.collectList().block();

The test that catches it. One with an item the stage must hold back, and for a recovery one where the upstream fails:

@Test
void onlyTheFirstTwo() {
  assertEquals(List.of(1, 2), Payments.firstTwo(Flux.just(1, 2, 3)));
}

@Test
void aFailureBecomesZero() {
  assertEquals(0, Payments.orZero(Mono.error(new IllegalStateException())));
}

Kotlin’s empty returns

kotlin/empty-return makes a Kotlin method that returns an IntRange, a LongRange, a CharRange or a Sequence return an empty one. mutastic’s own return operator makes those null, which Kotlin’s types say can’t happen, and which the first use of the value throws on, so a test that only calls the method and looks at nothing can catch it.

The bug it stands for. A range worked out wrongly, or a sequence that’s never filled: code that runs, and hands back nothing.

Where it costs. Only what kotlinc compiled, as @kotlin.Metadata shows. A method that returns the empty value already is skipped as equivalent:.

fun letters(name: String): CharRange = name.first()..name.last()
// becomes
fun letters(name: String): CharRange = CharRange.EMPTY

The test that catches it. One that looks at what’s in the result:

@Test
fun lettersRunFromFirstToLast() {
    assertEquals(listOf('a', 'b'), shelf.letters("ab").toList())
}

Kotlin projects

Kotlin support is experimental.

In any Kotlin project, listed or not, mutastic skips what kotlinc wrote that isn’t in the source, as generated:: its null checks on parameters, a data class’s copy, componentN and equals, a lateinit’s check and getter, a coroutine’s state machine, and code copied in from an inline function in a library, such as the loop in filter.

x!! and x as String are yours, so their checks stay: without one, a null carries on where you said it shouldn’t. x!!.size is the exception, skipped as equivalent:, as reading the value throws the same NullPointerException.

Code copied in from your own inline function is on that function’s line, and a test of the call catches it. The function’s own body is skipped, as nothing runs it, unless Java calls it. One that only tests call is skipped too, and it’s a blind spot: what’s copied into a test isn’t changed, so nothing shows whether the tests would notice.

Proposed rewrites

Proposed rewrites are experimental: the layout of .mutastic/proposed/, and the ids of its mutants, may change in a minor release (see What’s experimental).

proposed/rewrite lets you write a bug yourself and find out whether your tests would notice. Built-in operators make mechanical changes, one place at a time. They can’t write the bug a person would really make, and a person or a model can.

Here’s a real one, from Apache Commons Lang (Lang-21 in the Defects4J bug database), cut down. This method is right:

public static boolean sameTime(Calendar a, Calendar b) {
  return a.get(Calendar.HOUR_OF_DAY) == b.get(Calendar.HOUR_OF_DAY)
      && a.get(Calendar.MINUTE) == b.get(Calendar.MINUTE);
}

The bug was Calendar.HOUR in both calls, the 12-hour clock, so 8am and 8pm were the same time. Changing one call makes a different bug, so no built-in operator makes this one.

1. Copy the whole source file, and put the bug in the copy. The folder is the class’s binary name, and the file’s name is a label for the bug:

.mutastic/proposed/com.example.time.Times/twelve-hour-clock.java

// the whole of Times.java, with this method changed
public static boolean sameTime(Calendar a, Calendar b) {
  return a.get(Calendar.HOUR) == b.get(Calendar.HOUR)
      && a.get(Calendar.MINUTE) == b.get(Calendar.MINUTE);
}

2. Run it. With mutastic-plugins.jar under plugins:, choose the operator:

mutastic run --operators=defaults,proposed

Tests comparing 9:30 with 9:30, and with 10:30, pass either way, so the rewrite is missed:

✗ src/main/java/com/example/time/Times.java:9 no test tells sameTime() from the rewrite in .mutastic/proposed/com.example.time.Times/twelve-hour-clock.java

3. Add the test, or delete the file. Read the rewrite. If it behaves differently, as this one does, write a test that tells them apart, and mutastic verify with the mutant’s id confirms it’s caught. If it really does the same thing, delete the file.

Commit the files in .mutastic/proposed/: the label is part of the mutant’s id, so an id lasts as long as its file does.

Where proposals come from

Mutastic never asks a model for anything: it needs no network and no key. Whatever writes a proposal, a person, a script or a model told to write the bug a test would miss, only has to read the source file and write another. Research on models used this way has found they catch faults rule-based operators don’t, and that many of what they write don’t compile, or change nothing. So a proposal only runs if it passes each of these, and otherwise it’s skipped, saying why:

Check	Skipped as
It compiles, against the project’s classes and libraries, for the class’s Java release	unviable:, with javac’s message
It has the class’s fields and methods, and makes no more classes	unviable:
It changes one method. A lambda’s change counts as the method it’s written in	unviable:
It compiles to different code from the class, comments and layout aside	equivalent:
It compiles to different code from the proposals before it, in file name order	equivalent:

The bug it stands for. Whatever the rewrite does: that’s the point of having one.

Where it costs. A javac run for each proposal when mutants are found, on every command that finds them, and a load of every class for each one that runs. It runs against the tests that reach the method. No check can show that what’s left isn’t the same program in other words, so a miss may be one: the card says to read it.

return a.get(Calendar.HOUR_OF_DAY) == b.get(Calendar.HOUR_OF_DAY)
// becomes, in twelve-hour-clock.java
return a.get(Calendar.HOUR) == b.get(Calendar.HOUR)

The test that catches it. One that a real difference shows up in:

@Test
void morningAndEveningAreDifferentTimes() {
  assertFalse(Times.sameTime(at(8, 0), at(20, 0)));
}