Notary agreement is acceptable in court for flat sale ?

Only Notary agreement of 12.5 lakhs (full cash payment on 2015) for under construction flat sale. Due to 5 years project delay they are now demanding to return of 50 lacs rupees with their ongoing severe harassment. they warn us that they will file FIR against us. We already refunded 5 lacs in 2019. Required proper guidance. Pls suggest. Can we first file NC against them. ?? Are notary agreement are valid in court ??