Hi Sankaran,
You can add a custom button on SAP UI's if those are enabled for extensible through PDI.
If yes, then in the Extensible Explorer you should be able to see the possibility of adding a button using OBN Navigation which basically your new added button is configured with the navigation to other screen where you have to carry on your desired custom action activities.
Please check this and I even hope there should be a section for doing the same mentioned in the SDK help documents.
If I had any document with me I will try to upload it here. Thanks
Regards
Hanu