Repository navigation
Conversation
8f72bc4 to
47b0580
Compare
|
@mkaruza the problem of non-responsive backends running wrapper queries needs to be solved at an architectural level. For example by making the tokio runtime wait on the process latch (in addition to I/O) or to offload all I/O to a separate thread (which ultimately will still mean the main thread of the process waiting on the proc latch). A user should be able to cancel a query by calling For the above reasons I don't think we should merge this PR and instead work on the architectural level issues plaguing the wrappers framework with regards to blocking I/O. |
47b0580 to
c17f5d7
Compare
|
@imor you are right, previous implementation was limited and gave each fixe timeout, which was not complete I've replaced it with I think this is a much simpler way to solve the problem. We would need change wrapper wide block_on calls with this approach function but we can do it in another PR. |
Add `block_on_with_check_interrupts()` that checks Postgres interrupts after each timeout interval. The new function polls the future in with 250ms timeout. After each timeout it calls `check_for_interrupts!()`, outside the runtime, to check if an interrupt is pending, Postgres raises its usual error and the future is dropped during unwinding; otherwise it keeps waiting. It has no deadline of its own and only interrupts futures that yield; blocking code inside the future is not interrupted. Switch iceberg_fdw to the new function for all remote calls.
c17f5d7 to
5a32f15
Compare
|
Hey @mkaruza, what I meant by fixing this at the architectural level is that we should fix this in a fundamental way such that we drastically reduce the chances of backends getting stuck. This means not only in iceberg but all wrappers. This also meant to write tests which would reproduce the backends getting stuck and then fixing it should pass those tests. I've attempted one approach with AI's help here: https://github.com/supabase/wrappers/pull/673/changes to give an idea about the scope of work this fix needs. The PR is not ready to merge and it's not necessarily the final approach but it acts as a catalyst to think more deeply through the structural issues in the code. Please go through it and let me know what you think. |
Add
block_on_with_check_interrupts()that checks Postgres interruptsafter each timeout interval.
The new function polls the future in with 250ms timeout. After each timeout
it calls
check_for_interrupts!(), outside the runtime, to check if aninterrupt is pending, Postgres raises its usual error and the future is dropped
during unwinding; otherwise it keeps waiting.
It has no deadline of its own and only interrupts futures that yield;
blocking code inside the future is not interrupted.
Switch iceberg_fdw to the new function for all remote calls.