Skip to content

imp: harden signal handling - #245

Merged
mergify[bot] merged 7 commits into
flux-framework:masterfrom
grondo:imp-signal-hardening
Nov 15, 2026
Merged

mergify[bot] merged 7 commits into
flux-framework:masterfrom
grondo:imp-signal-hardening

Conversation

@grondo

@grondo grondo commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

This PR builds on #244 (only the last 6 commits are new here) and addresses issues found while reviewing the IMP's signal handling:

  • Fix a hang when the job shell is terminated by a signal the IMP does not forward (e.g. SIGSEGV, SIGABRT). imp_raise() re-raised the signal while it was still blocked, so the IMP waited in pause(2) forever.
  • Disable core dumps in the privileged IMP with PR_SET_DUMPABLE. Otherwise, a job shell that dumps core could make the IMP dump core as well when fs.suid_dumpable is nonzero.
  • Keep signals blocked in the IMP parent and only unblock them atomically while waiting in ppoll(2).
  • The SIGUSR1 handler previously called fopen(3) and malloc(3); it now only sets a flag, and the cgroup SIGKILL is done outside the handler.

A new sharness test covers the blocked-signal hang.

@codecov

codecov Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.13559% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.59%. Comparing base (2d33af0) to head (2d42fbf).
⚠️ Report is 9 commits behind head on master.

Files with missing lines Patch % Lines
src/imp/signals.c 90.38% 5 Missing ⚠️
src/imp/exec/exec.c 75.00% 1 Missing ⚠️
src/imp/run.c 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #245      +/-   ##
==========================================
+ Coverage   84.23%   84.59%   +0.36%     
==========================================
  Files          41       41              
  Lines        5563     5596      +33     
==========================================
+ Hits         4686     4734      +48     
+ Misses        877      862      -15     
Files with missing lines Coverage Δ
src/imp/cgroup.c 73.21% <100.00%> (ø)
src/imp/exec/exec.c 77.92% <75.00%> (+2.14%) ⬆️
src/imp/run.c 83.22% <50.00%> (+2.86%) ⬆️
src/imp/signals.c 89.24% <90.38%> (+15.56%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@grondo
grondo force-pushed the imp-signal-hardening branch from 218861a to 47c2fc0 Compare November 14, 2026 15:18
@grondo

grondo commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

This has been rebased after #244

@grondo
grondo force-pushed the imp-signal-hardening branch 2 times, most recently from 08ad564 to d7358a0 Compare November 14, 2026 16:09
Problem: If the IMP child is terminated by a signal that is not
forwarded by the IMP, such as SIGSEGV or SIGABRT, the IMP calls
imp_raise() to exit with the same signal. However, the signal is
blocked by imp_setup_signal_forwarding(), so raise(3) leaves it
pending and the IMP hangs forever in pause(2).

Unblock the signal in imp_raise() before raising it.

Assisted-by: Claude:opus-5.5
Problem: When the job shell is terminated by a signal, imp_raise()
terminates the IMP with the same signal. If the signal dumps core,
the job user can cause the privileged IMP to dump core at will.

Clear the dumpable attribute with PR_SET_DUMPABLE before raising the
signal. The IMP still exits with the same signal, so the job shell
status is reported correctly. Other IMP crashes are not affected.

Assisted-by: Claude:opus-5.5
Problem: imp_raise() calls pause(2) after raising the signal, so if the
signal does not terminate the IMP, for example because its default
action is to ignore it, the IMP hangs forever.

Since the signal is unblocked before raise(3), it is delivered before
raise(3) returns. Drop the call to pause(2) and always fall back to
exiting with 128+sig if the IMP is still running.

Assisted-by: Claude:opus-5.5
Problem: imp_raise() terminates the IMP with a signal, so coverage data
is never written and imp_raise() appears uncovered in CI, even though
it is exercised by tests.

When built with code coverage enabled, call __gcov_dump() before
raising the signal, as is already done before execve(2) in run.c.

Assisted-by: Claude:opus-5.5
Problem: No test ensures the IMP exits when its child is terminated
by a signal that the IMP does not forward.

Add a test in which the job shell kills itself with SIGUSR2 and
ensure the IMP exits with the same signal.

Assisted-by: Claude:opus-5.5
Problem: The IMP's signal handlers may interrupt the main loop at any
point, so they are restricted to async-signal-safe functions. This
makes it difficult to handle signals that require more work, such
as SIGUSR1.

Keep all signals blocked in the IMP parent, and only unblock forwarded
signals and SIGCHLD while waiting with ppoll(2). Since signals are
unblocked atomically by ppoll(2), a signal that arrives before the IMP
begins waiting is not lost.

Add imp_signal_wait() and imp_waitpid(), and use them in place of
usleep(3) and waitpid(2). Have imp_waitpid() disable signal forwarding
once the child is reaped, so callers do not need to.

Add imp_sigunblock_forwarded() and call it before pam_finish(), so that
helpers run by PAM inherit the same signal mask as before.

Assisted-by: Claude:opus-5.5
Problem: The IMP SIGUSR1 handler calls functions that are not
async-signal-safe, such as fopen(3) and malloc(3), and does not save
errno.

Set a flag in the handler, and send SIGKILL to the job's container in
imp_signal_wait() after the handler returns. Save and restore errno
in the handler.

Assisted-by: Claude:opus-5.5
@grondo
grondo force-pushed the imp-signal-hardening branch from d7358a0 to 2d42fbf Compare November 14, 2026 16:47

@garlick garlick left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mergify mergify Bot added the queued label Nov 15, 2026
@mergify
mergify Bot merged commit 3765b13 into flux-framework:master Nov 15, 2026
25 checks passed
@mergify

mergify Bot commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

✅ Merged into master as 3765b13 at 16:35 UTC

Less than a minute in the queue, with no time running CI.

So far

  • ✅ Entered the queue Sun 15 Nov, 16:34 · rule default · triggered by rule refactored queue action rule
  • ✅ Checks skipped · PR is already up-to-date
  • ✅ Merged 16:35 · 3765b13 · merge
17 merge conditions met
  • #approved-reviews-by>0
  • #changes-requested-reviews-by=0
  • -title~=^\[*[Ww][Ii][Pp]
  • approved-reviews-by=@flux-framework/core
  • base=master
  • github-review-approved · required by GitHub branch protection
  • label!="work-in-progress"
  • label="merge-when-passing"
  • any of · required by GitHub branch protection:
    • check-success = validate commits
    • check-neutral = validate commits
    • check-skipped = validate commits
  • any of · required by GitHub branch protection:
    • check-success = spelling
    • check-neutral = spelling
    • check-skipped = spelling
  • any of · required by GitHub branch protection:
    • check-success = coverage
    • check-neutral = coverage
    • check-skipped = coverage
  • any of · required by GitHub branch protection:
    • check-success = bookworm
    • check-neutral = bookworm
    • check-skipped = bookworm
  • any of · required by GitHub branch protection:
    • check-success = bookworm - 32 bit
    • check-neutral = bookworm - 32 bit
    • check-skipped = bookworm - 32 bit
  • any of · required by GitHub branch protection:
    • check-success = bookworm - clang-15,chain-lint
    • check-neutral = bookworm - clang-15,chain-lint
    • check-skipped = bookworm - clang-15,chain-lint
  • any of · required by GitHub branch protection:
    • check-success = bookworm - gcc-12,distcheck
    • check-neutral = bookworm - gcc-12,distcheck
    • check-skipped = bookworm - gcc-12,distcheck
  • any of · required by GitHub branch protection:
    • check-success = el8
    • check-neutral = el8
    • check-skipped = el8
  • any of · required by GitHub branch protection:
    • check-success = focal
    • check-neutral = focal
    • check-skipped = focal

Times in UTC · dashboard · how the merge queue works

@mergify mergify Bot removed the queued label Nov 15, 2026
@grondo
grondo deleted the imp-signal-hardening branch November 15, 2026 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants