Compares exported positions and drives the diff cases

README.md

@@ -345,11 +345,11 @@ claims to pass, and the gate fails if one of them does not.
 | Claim | Cases claimed | Of |
 |---|---|---|
 | `scion` | 119 | 119 cases in the scion suite |
-| `statifier` | 22 | 36 cases in the statifier suite |
+| `statifier` | 36 | 36 cases in the statifier suite |
 | `w3c-mandatory` | 152 | 154 mandatory cases in the w3c suite |
 | `w3c-optional` | 2 | 14 optional cases in the w3c suite |
 
-The registry holds 295 entries, which is the sum of the four rows. A claim is
+The registry holds 309 entries, which is the sum of the four rows. A claim is
 the set of its entries, not a statement that every case of its suite passes.
 
 What is not claimed yet:
@@ -358,9 +358,6 @@ What is not claimed yet:
   the reference's registry does not claim them either.
 - Twelve optional w3c cases whose host needs the Basic HTTP Event I/O
   Processor, which this package does not provide.
-- Fourteen cases of the `statifier` suite, the reference's own: the ten diff
-  cases, whose host keys the conformance runner does not read yet; and the four
-  cases that state an exported position, which it does not compare yet.
 
 Running the gate prints the gap list: every unclaimed case with the features it
 needs and whether it passes or why it fails.

conformance/README.md

@@ -103,27 +103,43 @@ once every configuration agrees, the sends handed over the whole drive, in
 order and each with its outcome, must be exactly `expect_sends`. A
 `<cancel>` that reaches the host marks the delayed send it names
 `"outcome": "cancelled"` when the expected item at that send's position
-asks for it, and a step of a case with a host object sends its event's
-`data`, when it gives one. A send whose expected item asks the host to
+asks for it, and a step of a case with a host object, or of a case stating
+an exported position, sends its event's `data`, when it gives one. A send
+whose expected item asks the host to
 report it failed (`"outcome": "fail"`) is marked so and reported through
 `Statifier.report_send_failed` once the driver call that handed it returns,
 before the configuration is read, as the reference's harness reports it
 before it reads the next configuration; a report the driver refuses because
-the chart has stopped is dropped. Any other host key (the accepts check's
-`declared_events` and `expect_accepts`, the diff pair's `to_source`,
-`expect_diff`, `mapping` and `expect_compatible_at`), and a step stating an
-exported position each fail the case naming what it asks for. This
+the chart has stopped is dropped. A case's `host.declared_events` and
+`host.expect_accepts` are read as the reference's harness reads them:
+`Statifier.check_accepts` answers for the compiled chart before it starts,
+and its two lists must be exactly `expect_accepts`. A host object's diff
+pair, `to_source` and `expect_diff` with an optional `mapping` and
+`expect_compatible_at`, is compared by nothing here, as the reference's
+harness compares none of the four: a case carrying it is driven like any
+other host case, and agrees on its configurations and its sends. A step's
+`expect_position` is compared, once that step's configuration agrees, with
+the state's position exported through `Statifier.export_position` and
+rendered as the reference's harness renders it
+(`Conformance::PositionExpectation` in
+`test/conformance/position_expectation.rb`): the configuration, the entered
+states and the states to invoke, each sorted; each history state's recorded
+states, sorted; each active invocation as its state and index, sorted; the
+running flag; and the datamodel without `_event`, `_ioprocessors` and
+`_sessionid`, and without `_name` unless the `<scxml>` element carries a
+`name` attribute, each value in its JSON form. A date, a datetime or a
+duration has no JSON form and fails the step naming its variable, and a
+case stating a position on a document with a state that has no id fails
+before it is driven. Any other host key fails the case naming it. This
 registry claims the scion suite, every one of its cases, the w3c cases the
-run passes and the statifier cases the run passes: the `w3c-mandatory`
+run passes and every case of the statifier suite: the `w3c-mandatory`
 claim covers its mandatory entries, the `w3c-optional` claim its optional
 ones and the `statifier` claim its statifier entries, since a claim is the
 set of its entries (the copy's `RATCHET.md`, "Claims"), not a statement
 that every case of the suite passes. The w3c cases left out are the two
 mandatory cases the run fails and the optional cases whose host object
-names the BasicHTTP processor, which the drive does not provide. The
-statifier cases left out are the ones that fail for what the drive does
-not do yet (the gap list names each with its reason); each joins the claim
-when the ratchet sees it pass. Beside the drive, the position round-trip
+names the BasicHTTP processor, which the drive does not provide. Beside the
+drive, the position round-trip
 property (`test/conformance/position_property.rb`), a test of the gate,
 takes a point after the start and after every step of each scion case,
 exports the position there, carries it through JSON, imports it into a fresh

conformance/registry.json

@@ -123,13 +123,26 @@
     {"case_id":"scion/targetless-transition/test2","suite":"scion"},
     {"case_id":"scion/targetless-transition/test3","suite":"scion"},
     {"case_id":"statifier/accepts/loan_declares_an_unreachable_event","suite":"statifier"},
+    {"case_id":"statifier/diff/hold_queue_data_added","suite":"statifier"},
+    {"case_id":"statifier/diff/hold_queue_identical","suite":"statifier"},
+    {"case_id":"statifier/diff/hold_queue_renamed_with_mapping","suite":"statifier"},
+    {"case_id":"statifier/diff/hold_queue_renamed_without_mapping","suite":"statifier"},
+    {"case_id":"statifier/diff/hold_waiting_at_an_edited_state","suite":"statifier"},
+    {"case_id":"statifier/diff/hold_waiting_through_an_idle_edit","suite":"statifier"},
+    {"case_id":"statifier/diff/hold_waiting_under_an_edited_ancestor","suite":"statifier"},
+    {"case_id":"statifier/diff/loan_waiting_across_a_damaged_arm","suite":"statifier"},
+    {"case_id":"statifier/diff/loan_waiting_across_a_mapped_rename","suite":"statifier"},
+    {"case_id":"statifier/diff/loan_waiting_across_a_regrouping","suite":"statifier"},
     {"case_id":"statifier/library/hold_queue_available_with_no_holds","suite":"statifier"},
     {"case_id":"statifier/library/hold_queue_collected_cancels_timer","suite":"statifier"},
     {"case_id":"statifier/library/hold_queue_expiry_moves_to_next","suite":"statifier"},
     {"case_id":"statifier/library/hold_queue_head_gets_pickup_timer","suite":"statifier"},
     {"case_id":"statifier/library/loan_dispute_restores_late_shallowly","suite":"statifier"},
     {"case_id":"statifier/library/loan_dispute_returns_to_history","suite":"statifier"},
     {"case_id":"statifier/library/loan_due_soon_then_overdue","suite":"statifier"},
+    {"case_id":"statifier/library/loan_lost_after_timer","suite":"statifier"},
+    {"case_id":"statifier/library/loan_position_counts_renewals","suite":"statifier"},
+    {"case_id":"statifier/library/loan_position_records_history","suite":"statifier"},
     {"case_id":"statifier/library/loan_renew_refused_at_max","suite":"statifier"},
     {"case_id":"statifier/library/loan_renew_refused_by_hold","suite":"statifier"},
     {"case_id":"statifier/library/loan_renew_refused_by_standing","suite":"statifier"},
@@ -138,6 +151,7 @@
     {"case_id":"statifier/library/patron_checkout_allowed","suite":"statifier"},
     {"case_id":"statifier/library/patron_checkout_refused_while_owed","suite":"statifier"},
     {"case_id":"statifier/library/patron_initial_both_regions","suite":"statifier"},
+    {"case_id":"statifier/library/patron_position_in_every_region","suite":"statifier"},
     {"case_id":"statifier/library/patron_reinstated_after_block","suite":"statifier"},
     {"case_id":"statifier/send/registered_delayed_cancel","suite":"statifier"},
     {"case_id":"statifier/send/registered_immediate","suite":"statifier"},

test/conformance/drive_test.rb

@@ -46,10 +46,10 @@ def test_the_drive_starts_the_chart_and_compares_the_active_leaves_after_each_st
   # sabotage: answer nil for a case with a host key (drop the `other` line in
   # Drive.unhonoured); the first assert_equal on the reason fails.
   def test_a_case_asking_what_the_runner_does_not_honour_fails_naming_it
-    item = chart_case(%(<state id="a"/>), %w[a]).merge("host" => { "send_types" => [], "expect_diff" => {},
+    item = chart_case(%(<state id="a"/>), %w[a]).merge("host" => { "send_types" => [], "x_other" => {},
                                                                    "x_unknown" => 1 })
 
-    assert_equal "the case's host object (expect_diff, x_unknown) is not honoured here",
+    assert_equal "the case's host object (x_other, x_unknown) is not honoured here",
                  Conformance::Drive.call(item)
     assert_match(/does not compile/, drive("<state", []))
   end
@@ -97,13 +97,14 @@ def test_a_case_naming_a_processor_that_is_not_provided_is_refused_naming_it
                                                  provided: { PROCESSOR => -> { {} } })
   end
 
-  # sabotage: add `to_source` to HONOURED_HOST_KEYS; the assert_equal on the
-  # refusal fails (the key is then read and passes unnamed).
+  # sabotage: answer nil from Drive.unhonoured whenever a processor is
+  # provided (`return if provided.any?` before the `other` line); the
+  # assert_equal on the refusal fails (the case is then driven).
   def test_a_host_key_the_drive_does_not_read_is_refused_beside_a_provided_processor
     item = host_case([PROCESSOR])
-    item["host"]["to_source"] = "<scxml/>"
+    item["host"]["x_unknown"] = "<scxml/>"
 
-    assert_equal "the case's host object (to_source) is not honoured here",
+    assert_equal "the case's host object (x_unknown) is not honoured here",
                  Conformance::Drive.call(item, provided: { PROCESSOR => -> { {} } })
   end
 

test/conformance/gap_list_test.rb

@@ -23,7 +23,7 @@ def test_names_every_unclaimed_case_in_corpus_order_and_no_claimed_one
     ids = lines.map { |line| line.delete_prefix("conformance: gap: ").split.first }
 
     assert_equal UNCLAIMED.map { |item| item["id"] }, ids
-    assert_equal %w[w3c statifier], ids.map { |id| id.split("/").first }.uniq
+    assert_equal %w[w3c], ids.map { |id| id.split("/").first }.uniq
   end
 
   # sabotage: leave the features out of the line in gap_lines; the

test/conformance/position_expectation.rb

@@ -0,0 +1,177 @@
+# frozen_string_literal: true
+
+require "json"
+require "predicator/tagged"
+require_relative "../../lib/statifier"
+
+module Conformance
+  # The position a statifier case's step states in its `expect_position`,
+  # compared as the reference's harness compares it
+  # (`Mix.Statifier.Corpus.PositionExpectation` in
+  # lib/mix/statifier/corpus/position_expectation.ex at the tag the copy
+  # names): the state's position exported with {Statifier.export_position}
+  # and rendered as the JSON value a case writes, with exactly these seven
+  # members:
+  #
+  # - `configuration`, `entered_states` and `states_to_invoke`: each set of
+  #   state ids as an array sorted ascending;
+  # - `history_values`: each history state's id to its recorded state ids,
+  #   an array sorted ascending;
+  # - `active_invocations`: one object per live invocation, `state` (the id
+  #   of the state that owns the `<invoke>`) and `index` (its ordinal among
+  #   that state's own), sorted by state and then by index, the invocation's
+  #   id left out;
+  # - `running`: the position's boolean as it is;
+  # - `datamodel`: every variable but `_event`, `_ioprocessors` and
+  #   `_sessionid`, and but `_name` too unless the document's `<scxml>`
+  #   element carries a `name` attribute, each value in its JSON form: a
+  #   string, a number, a boolean and nil as they are, the undefined value
+  #   as nil, a list as an array and a map as an object, member by member.
+  #   A date, a datetime or a duration, or a list or map holding one, has no
+  #   JSON form, and the step fails naming the variable.
+  #
+  # The position's other members (the three sequences, `status`, the
+  # counters, `trace` and the round budget) are not rendered, as the
+  # reference renders none of them. Values compare as JSON values: two
+  # numbers agree when they are equal in value, arrays element by element in
+  # order, objects member by member.
+  module PositionExpectation
+    # The system variables the rendering leaves out whatever the document.
+    LEFT_OUT_VARIABLES = %w[_event _ioprocessors _sessionid].freeze
+
+    module_function
+
+    # Whether any of item's steps carries an `expect_position`.
+    def expected?(item) = item.fetch("steps").any? { _1.key?("expect_position") }
+
+    # nil when every state of machine but the root carries an id, else the
+    # reason naming the count of states without one: a case that expects a
+    # position needs every state named, since the export refuses a position
+    # holding a state it cannot name.
+    def named(machine)
+      unnamed = machine.states.drop(1).count { _1.id.nil? }
+      return if unnamed.zero?
+
+      "a case that expects a position needs every state to carry an id, " \
+        "and #{unnamed} state(s) of this document have none"
+    end
+
+    # The rendering of state's exported position: `[rendering, nil]`, or
+    # `[nil, reason]` for the export's refusal or a datamodel value with no
+    # JSON form.
+    def render(state)
+      exported = Statifier.export_position(state)
+      return [nil, "the export refused the position: #{exported.error.reason} #{exported.error.detail}"] \
+        unless exported.ok?
+
+      position = exported.value
+      datamodel, reason = datamodel(position["datamodel"], state.machine_state.machine.name)
+      return [nil, reason] if reason
+
+      [members(position, datamodel), nil]
+    end
+
+    # The seven members, the datamodel already rendered.
+    def members(position, datamodel)
+      { "configuration" => position["configuration"].sort,
+        "entered_states" => position["entered_states"].sort,
+        "states_to_invoke" => position["states_to_invoke"].sort,
+        "history_values" => position["history_values"].transform_values(&:sort),
+        "active_invocations" => invocations(position["active_invocations"]),
+        "running" => position["running"],
+        "datamodel" => datamodel }
+    end
+
+    # Each live invocation as its state and index, sorted.
+    def invocations(active)
+      active.map { { "state" => _1["state"], "index" => _1["invoke_index"] } }.sort_by { [_1["state"], _1["index"]] }
+    end
+
+    # The datamodel's texts decoded and rendered: `[datamodel, nil]`, or
+    # `[nil, reason]` naming the first variable, by name, with no JSON form.
+    # `document_name` is the `<scxml>` element's `name` attribute, nil
+    # without one.
+    def datamodel(texts, document_name)
+      left_out = document_name.is_a?(String) ? LEFT_OUT_VARIABLES : [*LEFT_OUT_VARIABLES, "_name"]
+      rendered = {}
+      (texts.keys - left_out).sort.each do |name|
+        value = Predicator::Tagged.decode!(texts[name])
+        json = json_form(value)
+        return [nil, "the datamodel's #{name} holds a #{kind(value)} value, which has no JSON form here"] \
+          if json.equal?(NONE)
+
+        rendered[name] = json
+      end
+      [rendered, nil]
+    end
+
+    # What json_form answers for a value with no JSON form.
+    NONE = Object.new.freeze
+
+    # value's JSON form, or NONE.
+    def json_form(value)
+      case value
+      when Predicator::UNDEFINED, nil then nil
+      when String, Integer, Float, true, false then value
+      when Array then collect(value.map { json_form(_1) })
+      when Hash then object(value)
+      else NONE
+      end
+    end
+
+    # A map's JSON form, member by member in key order, or NONE.
+    def object(map)
+      keys = map.keys.sort
+      values = collect(keys.map { json_form(map[_1]) })
+      values.equal?(NONE) ? NONE : keys.zip(values).to_h
+    end
+
+    # items, or NONE when one of them is NONE.
+    def collect(items) = items.any? { _1.equal?(NONE) } ? NONE : items
+
+    # The kind of a value with no JSON form, as a person reads it.
+    def kind(value)
+      case value
+      when Predicator::Date then "date"
+      when Predicator::DateTime then "datetime"
+      when Predicator::Duration then "duration"
+      when Array then "list"
+      when Hash then "map"
+      else value.class.name
+      end
+    end
+
+    # nil when `step`, the one-based step `number`, states no position or
+    # state's agrees with it, else the reason.
+    def at_step(state, step, number)
+      return unless step.key?("expect_position")
+
+      compare(step["expect_position"], state, number, step.fetch("event").fetch("name"))
+    end
+
+    # nil when the rendering of state's position is exactly `expected`, a
+    # step's `expect_position`, else the reason: the step by its one-based
+    # `number` and its event's name, then each member that differs with both
+    # values as JSON, or why the position cannot be rendered.
+    def compare(expected, state, number, event)
+      label = "after step #{number} (#{event}), expect_position"
+      actual, reason = render(state)
+      return "#{label} cannot be compared: #{reason}" if reason
+
+      differing = differing(expected.is_a?(Hash) ? expected : {}, actual)
+      "#{label} differs: #{differing}" unless differing.nil?
+    end
+
+    # Each member of the two that differs, with both values, in member
+    # order; nil when none does.
+    def differing(wanted, actual)
+      keys = (wanted.keys | actual.keys).sort.reject { wanted[_1] == actual[_1] }
+      return if keys.empty?
+
+      keys.map { "#{_1}: expected #{encode(wanted[_1])}, but got #{encode(actual[_1])}" }.join("; ")
+    end
+
+    # A member's value as the reason writes it: nothing for an absent one.
+    def encode(value) = value.nil? ? "nothing" : JSON.generate(value)
+  end
+end