FAQ

FAQ

Questions that come up when writing a first specification, with the short answer and where the long one is. The guide teaches each of these in order; this page is for finding one by its keyword. Every statement here is checked against the README of Fix 1.0, which is itself run by the test suite.

Writing

Why does it MUST eq 42 fail for a subject that returns 42.0? eq is eql?: same value and same class, for every value, not only numbers. 1.eql?(1.0) is false, and so is "a".eql?(:a). Compare with be_within(delta).of(42) for an Integer, Float, Rational or BigDecimal near 42, or be_a_kind_of Numeric for the class alone.

Why can't I write it MUST eq name when name is a let? A let is a parameter of the subject, visible from the block of test and from the other lets only. Named in an expectation, it would be read while the file loads, before any subject exists, so Fix raises NameError at load with that very remedy. Write the value in the expectation, and let the subject block read the let: with name: "Alice" do it MUST eq "Hello, Alice!" end against test { "Hello, #{name}!" }. See Design Decisions.

let(:format) raises ArgumentError. Why? A let becomes a method of the object the subject block runs on, and that object sees Kernel and the methods of the top level. A let named like one of them, public or private (format, p, test, open, select, method, class, hash…), or like a word of Fix (let, with, on, it, and its with fix-its), would redefine it in silence. The message names the owner: format is already a method of Kernel. Pick another name.

What is the difference between let(:items) { [] } and with items: []? A let is a block, called anew for each expectation: every expectation gets a new array. A with is a value, the very object given, shared by the expectations of its context: with items: [] gives them all the same array. Two contexts are two child processes, so the sharing never crosses a context.

Can a with name a let that was never declared? Yes. A with declares the lets it names; a key no let declared is a new parameter, not an error. So a misspelt key is a new let, silently: check the spelling against the subject block.

How do I test a method that takes a block? on sends its method without a block, since the do … end of on is the context. Either call the method in the subject block, test { list.map { |x| x * 2 } }, or use satisfy { |value| value.map { … } == … }.

Is there a be_empty, have_key, be_valid? No. The idiom is on(:empty?) { it MUST be_true }: the method is sent by on, and its name appears in the result as .empty?. See Design Decisions.

Can change observe a let? No. The object of change(object, :method) must exist when the file loads, because the matcher is built then; a let lives with the subject, later. Observe a variable of the file or a global.

it MUST fix(:Other) raises KeyError. fix(:Name) finds its specification when the document is read, so Fix :Other must come earlier in the file (or in a file required earlier). Did you mean? is shown for a near miss.

How do I test a class method, or a constructor? Return the class from the subject block and send the method with on: Fix[:Calculator].test { Calculator } with on :new, 2 do … end. An on inside that one is sent to the new instance.

In Ruby 3.4, its(:size) { MUST eq it } builds eq nil. In Ruby 3.4 and later, a bare it in a block is the implicit parameter of the block; it with a requirement is still the word of Fix. On 3.3, the bare it is the word of Fix and raises for want of a requirement. Write the value. A satisfy meant for both Rubies names its parameter: satisfy { |value| … }.

Outcomes

My raise_exception ArgumentError is a Failure, not an Error. The subject raised TypeError. The exception is what raise_exception judges: another class is a match that is false, so a failure under MUST and MAY, a warning under SHOULD. An error is an exception that reached an expectation whose matcher was not there to judge it.

Why is a missing method an Info under MAY but an Error under MUST? Two questions tell the levels apart: does the method exist? (MUST and SHOULD require it, MAY does not) and does it do what is specified? (MUST and MAY require it, SHOULD recommends it). A duck that does not speak meets on :speak do it MAY … end. The table is in the README under Outcomes, and the reasoning in Design Decisions.

Does a warning make the run fail? No. A success, a warning and an info pass; a failure and an error do not. test returns true for a run with warnings, and so the exit status is 0.

I see undefined method 'speak' on 3.4 but undefined method `speak' on 3.3. Ruby changed the quote in 3.4, and names the class of a method in a backtrace frame. The texts are those of Ruby; Fix does not rewrite them.

Running

Why exit(Fix[:Duck].test { Duck.new }) with parentheses? Without them, exit Fix[:Duck].test do … end gives the do … end block to exit, not to test, and test raises ArgumentError for want of a block. Keep the parentheses, or use braces.

A file with two specifications: how does it exit? Test them all, then exit once: exit([Fix[:A].test { … }, Fix[:B].test { … }].all?).

Nothing failed, but the process exits 1. / Something failed, but it exits 0. Check the last line of the file: it must pass the result of test to exit, and nothing else. test never calls exit, so a file that forgets it exits 0 whatever happened. The command fix reads that status and nothing else.

How do I run one file? All of them? ruby fix/duck_fix.rb runs one, as any Ruby file. fix from the fix-command gem runs the *_fix.rb files of fix/, each in a new process, or the files and directories given: fix fix/duck_fix.rb lib/.

My subject needs lib/ on the load path. The command passes its environment to every file: RUBYOPT=-Ilib fix, or bundle exec fix for the gems of a Gemfile. bundle exec puts the gems on the load path; it does not require them, the file does.

Does fix run files in parallel? Does it have a timeout? No and no, in 1.0. Files run one after the other, each in its own process; Ctrl-C ends the command and, through the terminal, the file being run.

Is the order random? Within a file, no: contexts run in document order, every time. Between files, only on request: fix --random shuffles the files and writes first the command that replays the same order, fix --random=3.

What does the subject's own output do to the lines? It comes out as it is, on the same standard output, before the lines of its context. A text left without a line break joins the first of them. A runner reads the exit status, not the lines, for that reason.

SimpleCov reports nothing for the code my subjects ran. Each context runs in a child that ends with exit!, which runs no at_exit handler, so SimpleCov never writes the child's result. Either store the result before the child ends, as the tests of Fix do, or run with test(fork: false) for the measurement.

Does Fix work on Windows? On JRuby? No. Fix forks a child per context and needs fork(2); a context with expectations raises NotImplementedError on a Ruby without it. Linux and macOS are tested in CI, on Ruby 3.3, 3.4, 4.0 and head.

Using a specification elsewhere

Can I use a specification of Fix from RSpec or Minitest? Yes, as a matcher: Matchi::Fix.new(:Duck).match? { Duck.new } from the matchi-fix gem answers true or false, forking the contexts as Fix does. Inside Fix itself, it MUST fix(:Duck) is the same matcher. A specification meant for a matcher is written without with, since the value is given once and the contexts cannot build it anew.

A failed fix(:Other) doesn't say which expectation failed. By design: a matcher answers true or false. Test the inner specification on its own with Fix[:Other].test { … } for the lines.

Can I get the results as data instead of lines? Fix[:Name].run { … } returns an Array of Fix::Result, frozen data with location, chain, outcome, passed?, summary, and the Spectus::Result underneath. test is run plus the printing.

How do I add a word to the language, as fix-its does? Add a method to Fix::Dsl that composes on and it, and pass it the location: of your own caller (caller_locations(1, 1).first) so that the result points at the line of the specification. fix-its is about twenty lines and is the reference implementation. See Architecture.