Property ownership issue

My father who passed away few years back did not keep a will. His children including me, my elder brother, sister and my mother are his heirs. He owned a hotel (2 shops combined) in which his younger brother had also invested and persuaded him to forcefully to give 50% share in the property and the business income when we were small children. My father being uneducated had agreed to it and signed blindly on a notarized agreement for the 50% share. Afterwards his another younger brother gave him some money for which he also asked a share in the business income. Both of them are getting a fixed amount as part of the business income share.
After my father's death both the brothers came to our home and demanded that the other brother also get a share in the property and that otherwise we cough out an unreasonably high amount to buy them out. That argument ended when my brother refused to include the second brothers name and we agreed to pay them their fixed share as before. Now the second brother doesn't have any documentary proof that he paid my father any money but still my father and after his death we continued to give him his fixed share which has exceeded the amount he gave as principal by 3 times. The only reason why my father continued to pay him and also give him a high amount later is because he was his brother and he helped in times of distress and the same money would have fetched him higher returns if he had invested in a flat.

Now I have a question as to whether we can challenge them in court and finally settle this dispute in our favour where we give the first brother a share of the property and say bye bye to the second brother. The society's share certificate of both the shops has my Father's name as primary member and his brother as the associate member. The secretary of the society said that associate membership gets dissolved when my father passed away. So this leaves the property in the name of his heirs for the first shop and just half share in the second shop. Please advise if the understanding is correct and how we can resolve this issue once and for all.