Go to customer module and open the "Billing and Mailing Address" section for the client you want to modify. From here you can edit a current address by clicking on the small blue icon (copy).
This copy icon will copy the same address underneath, and it is from here that you can modify the address. Once done click save at the bottom of the page. You can delete the copied address before saving if you no longer want that record, just click on the delete icon.
Make sure to select your current default mailing and billing address.