Skip to content
mutastic

Output

mutastic prints its result on stdout and its progress on stderr. How the result looks is --format’s choice.

Formats

Section titled “Formats”

--format chooses how much each missed mutant says, and for whom:

FormatForEach missed mutant
fullPeople at a terminal, the default thereA card: the gap, the code with the change marked, the evidence, the test to add, and how to prove it
briefPeople who know the cardsThe gap, the changed line and the evidence
conciseLogs and CI, the default off a terminalOne line
groupedReading many misses at onceOne line, under its file and method
agentCoding agents, the default with MUTASTIC_AGENT=1Plain text: the verdict first, then the change, the fix and how to prove it
jsonlTools and scriptsOne JSON event per line, as each mutant finishes

The same run of the pricing sample, in each:

Terminal window
 Missed ────────────────────────────────────────────────────────────


 ❯ src/main/java/com/example/pricing/Pricing.java  1 missed in 1 method


  ✗ MISSED  no test checks discount() at total == 100
    ┌─ src/main/java/com/example/pricing/Pricing.java:17
    │
 16 │   public int discount(int total) {
 17 │     if (total >= 100) {
    │               ^^ changed to >
 18 │       return total / 10;
    │
    = mutant  - if (total >= 100) {
              + if (total > 100) {
    = note: every test calls discount() away from the boundary
            inputs seen: 50, 250 (none is 100)
            covered by PricingTest#bigOrdersGetTenPercentOff, PricingTest#smallOrdersPayFullPrice (all 2 covering tests still pass)
    = help: test discount() with total = 100 and assert the result


            // in PricingTest
            @Test
            void discountWithTotal100() {
              assertEquals(10, new Pricing().discount(100)); // today 10 · with this mutant 0
            }
            check today's value is the one you intend: the test pins what the code does now


    done when  mutastic verify m855c8e3c samples/pricing passes
    reproduce  mutastic run --only=m855c8e3c samples/pricing
    or ignore  mutastic ignore m855c8e3c --reason="<why>" samples/pricing


   Classes  0 clean | 1 with misses (1)
   Mutants  ✓ 8 caught · ✗ 1 missed · 2 skipped (2 equivalent) (11)
  Worst at  Pricing.discount() 1 ✗
 Test runs  11 (27 without coverage selection)
  Start at  17:19:23
  Duration  2.1s (setup 1.5s, coverage 206ms, mutants 382ms)
     Tests  3 on JDK 17 · 4 workers
   Reports  build/mutastic/mutation.json · build/mutastic/mutation.html
     Score  88.9%  threshold 80


  PASS  1 missed, above the threshold.
 Next: mutastic verify <id> samples/pricing · mutastic run --only=<id> samples/pricing · mutastic explain <id> samples/pricing
  • Cards are grouped by file, and in each file the method with the most misses comes first. A file shows at most seven cards, and counts the rest (--each-mutant shows them all): more than a few at once go unread. On a terminal, file locations are links (--hyperlinks) that terminals supporting OSC 8 open; others show the text.
  • The agent format’s ten entries are those most worth a test first: a condition or a removal no test checks before arithmetic, and among those, code more tests run without checking.
  • The agent format puts the verdict first, so a truncated read still has it, and the rule that a fix changes tests, never production code. It also says when not to write one: a mutant whose case can’t happen in practice, or whose change no caller can see, is declined with its reason and ignored, not given a contrived test.
  • MUTASTIC_FORMAT sets the format when --format isn’t given. While it runs, each source file gets a line as it finishes, on stderr, with its misses under it.
  • --vocabulary=classic (or MUTASTIC_VOCABULARY=classic) says killed and survived instead of caught and missed. JSON and the report files always use the standard statuses.
  • Colour follows the usual rules: only on a terminal, never with NO_COLOR or TERM=dumb, always with FORCE_COLOR. Every colour has a symbol beside it (✓ caught, ✗ missed, ○ not covered, ⏱ timed out), so nothing is lost without it. Labels such as MISSED and PASS are coloured blocks, [MISSED] without colour, and have rounded ends with MUTASTIC_PILLS=rounded for terminals with a Nerd Font. MUTASTIC_PILLS=off draws no blocks: the label’s words in its colour.
  • jsonl writes mutastic.events/v1 events. Statuses follow the mutation-testing report schema: Killed, Survived, Timeout, NoCoverage.
  • A mutant’s id comes from its class, method, position and change, not its line number, so it stays the same while code around it moves.

Shared cards

Section titled “Shared cards”

The mutants of one line share a card, and an entry in the agent format: one test there usually catches them all, the edge of a condition its boundary and its removal alike. The card is written about one of them, with its id after the headline, and lists each of the others under also, with its id and the line as it changes. A suggested test says which mutant it was found for. The card’s done when checks them all at once, so the test proves the whole fix; each is still counted, and ignored, on its own.

In the suggestions sample, inCredit() misses both its boundary and the condition always being true. A test at balance = 0 catches both:

Terminal window
  ✗ MISSED  no test checks inCredit() at balance == 0 · md4db94d8
    ┌─ src/main/java/com/example/rates/Rates.java:18
    │
 17 │   public static boolean inCredit(int balance) {
 18 │     return balance > 0;
    │                    ^ changed to >=
 19 │   }
    │
    = mutant  - return balance > 0;
              + return balance >= 0;
    = also    ✗ no test needs balance > 0 to be false in inCredit() · m05382fc9
              + return true;
    = note: every test calls inCredit() away from the boundary
            inputs seen: 50 (none is 0)
            covered by RatesTest#aPositiveBalanceIsInCredit (its 1 covering test still passes)
    = help: test inCredit() with balance = 0 and assert the result


            // in RatesTest
            @Test
            void inCreditWithBalance0() {
              assertThat(Rates.inCredit(0)).isFalse(); // today false · with md4db94d8 true
            }
            check today's value is the one you intend: the test pins what the code does now


    done when  mutastic verify md4db94d8,m05382fc9 samples/suggestions passes
    reproduce  mutastic run --only=md4db94d8,m05382fc9 samples/suggestions
    or ignore  mutastic ignore <id> --reason="<why>" samples/suggestions for each of md4db94d8, m05382fc9

--each-mutant gives every missed mutant a card of its own.

Streams

Section titled “Streams”

The result goes to stdout and progress to stderr, so mutastic run > result.txt keeps only the result.

The commands it offers

Section titled “The commands it offers”

The commands mutastic prints for a mutant name the project it ran on (mutastic verify m855c8e3c samples/pricing), so they work as copied. A mutant’s id belongs to its project: run in a directory with no Java code, or one that isn’t a Gradle or Maven build, mutastic says so (for Gradle, it names the modules that have Java code). A Maven module is named by its directory, as -pl takes it: mutastic run . :services/api.

One mutant, explained

Section titled “One mutant, explained”

mutastic explain ID [dir] [:module] runs that one mutant and shows its card, whatever became of it, then what it is. The summary’s Next: line offers it, for the one-line formats. On the pricing sample, after a run:

Terminal window
  ✗ MISSED  no test checks discount() at total == 100
    ┌─ src/main/java/com/example/pricing/Pricing.java:17
    │
 16 │   public int discount(int total) {
 17 │     if (total >= 100) {
    │               ^^ changed to >
 18 │       return total / 10;
    │
    = mutant  - if (total >= 100) {
              + if (total > 100) {
    = note: every test calls discount() away from the boundary
            inputs seen: 50, 250 (none is 100)
            covered by PricingTest#bigOrdersGetTenPercentOff, PricingTest#smallOrdersPayFullPrice (all 2 covering tests still pass)
    = help: test discount() with total = 100 and assert the result


            // in PricingTest
            @Test
            void discountWithTotal100() {
              assertEquals(10, new Pricing().discount(100)); // today 10 · with this mutant 0
            }
            check today's value is the one you intend: the test pins what the code does now


    done when  mutastic verify m855c8e3c samples/pricing passes
    reproduce  mutastic run --only=m855c8e3c samples/pricing
    or ignore  mutastic ignore m855c8e3c --reason="<why>" samples/pricing


 About this mutant ───────────────────────────────────────────────
    = bytecode  IF_ICMPLT → IF_ICMPLE
                IF_ICMPLT: jump if the first int is less than the second
                IF_ICMPLE: jump if the first int is less than or equal to the second
                javac compiles `total >= 100` to IF_ICMPLT, a jump past the body when `total < 100`:
                the opposite of the source. With IF_ICMPLE it jumps when `total <= 100`, so the body
                runs only when `total > 100`
    = operator  mutastic/boundary: `<` becomes `<=` and the other way round: the off-by-one at the
                edge of a range, which tests that only check values far from the edge never notice.
    = id        m855c8e3c: `m` and the first 8 hex digits of its fingerprint, 855c8e3ca8a39847, a
                SHA-256 of the class, method, site and change
                no line number goes in, so the id stays the same when code above it moves
    = site      instruction site 1 of discount(I)I, counting from 1 (arithmetic, branches and
                returns are counted in bytecode order; calls apart)
    = last run  missed: every covering test still passed

Under About this mutant:

  • bytecode: the change, opcode by opcode (IF_ICMPLT → IF_ICMPLE), and how the jump relates to the source. javac compiles if (total >= 100) to a jump taken when total < 100, the opposite of what the source says, which is why a boundary mutant can look backwards in the bytecode.
  • operator: what its operator does, as on Operators.
  • id: how the id is made, and why it stays the same when code above it moves.
  • site: which of the method’s sites it is.
  • last run: how the last full run went, or none yet.