Easement act

If he was permitted to use your land for pathway towards ingress and egress,  he cannot use your land for any other purpose. 

In that situation you can file a suit for mandatory injunction restricting him from utilising your land for any other purpose other than pathway. 

Did you make boundries (limitation) in the agreement, if so, just settle the matter amicably before exploring legal remedy of encroachment. ?

Prima facie it is a hypothetical moot court topic which should be placed before your professor / tutor /coach.

To use your land for access to his land is under  easementary  rights and since you have agreed too means you cannot stop him from using the road. But if he has exceeded the limits then simply you need to get the way demarcated by submitting application to Tehsildar for on spot verification and demarcation. You must not agree to any changes on Patwari map, just verbally should allow him to use the path till his access to land

apply for mandatory injunction order in the local courts.A mandatory injunction is issued when a court directs a person to perform certain acts, as opposed to prohibitory injunction, which seeks to preserve the status quo. The defendant named in a mandatory injunction must undo the wrong or injury that one has caused.

Remedy lies in approaching a civil court seeking mandatory injuction and eviction of the property alresdy encroached.