The decision concludes an experiment by combining statistical significance, effect size, confidence interval, guardrail metrics, and implementation cost. It can be to ship, to abandon, or to continue the test. A significant result does not always imply an effect large enough to justify shipping.
Example
Significant result but effect below the minimum useful effect: do not ship.