Contractually this is an important topic and it has gotten a little more difficult with the wide-spread adoption of cloud services as your vendor may not physically poses or own the hard drives storing your data. Upon contract termination it's common to require a "certificate of destruction" and this is mentioned in financial regulations. I'll post exact references later today. You'll have to rely on your vendor's word that the data is destroyed or no longer accessible.
Another challenge comes in with cloud services storage as you cannot expect the vendor to physically destroy the hard drives your data may have touched throughout it's life cycle. A solution to this is to ask for or perform yourself, depending on the service, a process called cryptographic erasure. Essentially this is encrypting data at rest and throwing away the key.
Here is a contractual statement that we see in many vendor contracts where data is shared:
"Return of Information. Your Vendor will, at the request of the Client, during the Term or thereafter (a) promptly return all Confidential Information held or used by Your Vendor in whatever form, or (b) at the discretion of the Client, promptly destroy all such Confidential Information, including all copies thereof, and those portions of all documents that incorporate such Confidential Information and provide a certificate of destruction."
Edit -
Section III.C.4 of 12 CFR Appendix B to Part 30 Interagency Guidelines Establishing Information Security Standards states:
Develop, implement, and maintain, as part of its information security program, appropriate measures to properly dispose of customer information and consumer information in accordance with each of the requirements of this paragraph III. Link to Regulation Text
Edit 2 - Additional Common Contractual Statement
"At Client's direction at any time during the term of the Agreement, and in any event upon termination or expiration of this Agreement, Your Vendor shall, and shall cause its representatives to, immediately delete Client’s Data and/or return to Client all Client Data and then (except in the event Client requests preservation) destroy and certify the destruction of any and all residual copies of Client Data."