Chances of two concurrent runs are much lower since we switched to an internal scheduler. Let's remove some complexity here. Implements #20681.
