[Gus-prog]

Summary

Sparse UDLF is the successor of UDLF-CSR: sparse implementations of CPRR,
LHRR, RDPAC, RFE and BFSTREE for datasets where the dense N x N matrices of UDLF do not fit in memory. This PR makes
it the second backend of pyUDLF, in place of UDLF-CSR:

from pyUDLF import UDLF

sparse = UDLF(bin_path=".udlf/Sparse_UDLF/bin", backend="sparse")   # or UDLF_SPARSE_BIN
res = sparse.run("RDPAC", ranked_lists=rks, params={"l": 50, "k_end": 10}, classes=labels)

On the synthetic dataset, CPRR, LHRR, RDPAC and BFSTREE give ranked lists byte-identical to UDLF 1.70, so the
tests now compare the two backends directly.

Changes

• scripts/build_sparse.sh: clones Sparse UDLF at a pinned commit (de3c14c) and builds both of its binaries:

  • udlf (fast-math): CPRR and LHRR;
  • udlf_strict: every method.

• bin_path / UDLF_SPARSE_BIN is the directory of the binaries (or one of them). Each method runs in the binary
  that supports it.

• Methods accept UDLF or Sparse UDLF names (BFSTREE / BFSTree, CPRR / CPRR_SPARSE).

• Parameters keep the UDLF 1.70 defaults and validation. Sparse UDLF adds:

  • threads for LHRR (defaults to UDLF(threads=));
  • b and memory_budget_mb for BFSTREE.

• Each run gets its own UDLF_TMPDIR, so simultaneous runs do not share scratch files.

• Fixed: the log parser read the "RELATIVE GAINS" block as measures.

• Removed with UDLF-CSR:

  • engine= and the dense/MKL/BLAS/Eigen engines;
  • the retry of its intermittent startup error (0 failures in 300 Sparse UDLF runs);
  • scripts/build_csr.sh.

  backend="csr" and UDLF_CSR_BIN stay as aliases. A UDLF-CSR binary is detected and rejected with a pointer
  to the build script.

• examples/csr_engines.py → examples/sparse_methods.py. Updated the README (EN/PT), the examples table and
  the CHANGELOG.

• CI builds Sparse UDLF (libtbb-dev, libjemalloc-dev).

Known issues of Sparse UDLF de3c14c (reported upstream)

• RFE cannot run: its PERFORM_CCS, RERANK_BY_EMB and EXPORT_EMBEDDINGS parameters are commented out in
  Validator.hpp, so the config parser drops them. The error message explains it, and a strict-xfail test flags
  when it is fixed. RFE with perform_ccs=True is not ported yet, so pyUDLF runs RFE without connected components
  and warns.
• The matrix output is all zeros: output="matrix" is refused, and run_config warns.

Tests

• 283 passed, 1 skipped, 1 xfailed locally (Linux, UDLF 1.70 and Sparse UDLF de3c14c).
• New suites:
  • tests/test_sparse_binary.py: the binaries run directly, without the wrapper;
  • tests/test_sparse.py: the backend.