It’ll remain on the Bill Pay dashboard until the expiration date and time set by the biller. After that, you’ll no longer see it listed.
Please be aware this might be different from the payment due date and time.