By default the address book is not able to be changed at the device, you can somewhat enable it, but it will then only allow you to add an entry, or edit a current one, but not remove any.
You have to enable it for All Users here in EWS
But, as stated, you will only gain add and edit as functions