Threat given by cousin brother

I have taken 1 lac rs in 2011 from my cousin brother for my son wedding however my son wedding failed due to my brother intervention as well as other reasons as well . Till that time i have returned 40000 rs back to my brother. The divorce other case of my son runs for 4 years and finally settlement done now my brother is asking for money balance amount and i have already spent money to protect my brother as he lives in maharastra and we in delhi. I dont want to return 60 k to him because he is the reason for divorce and i have already spent money on his bail apll and lawyer fees. Today he came to my house and said that i will take money from the way i know and all the wrong stuff. What shuld i do now please help ??? Regards