Tried all the steps above and nothing worked, all network items looked fine. Found a blog somewhere stating a Xerox tech at one point had a customer simply change the from email address and then retry to scan to email. Apparently the email sending will sometimes get stuck in some sort of relay with the from email address.
Had the end user change the from email address to a completely different address and domain.
TEST scan to email was SUCCESSFUL.
No more "The job was deleted, destination server IP address not configured." error