If you have proof that he threw you and your daughter out, then you are absolutely right that the question of you deserting him simply does not rise and he cannot claim that. However, what concerns me is your statement "moved it to another of our house and told me come there and stay with him." So he did give you an option to stay with him in another house. Instead you chose to stay with your parents. This can work against you for sure unless you can claim and prove that the other house was totally inferior to the one you were staying in originally and/or that he had an ulterior motive in shifting you to the new house.
If he has provided you alternate arrangements under Right to Residence and was willing to stay with you, you may not prevail by filing a DV case to seek residence or maintenance.
One needs to know more facts on why you chose not to go to the alternate residence and/or why he was asking you to go there, before giving you proper advice. Advocates will encourage you to file DV and other cases but anyone asking you to do so, without having all the facts before them, are shooting in the dark. You will be wasting your time and monies for no reason.
Your petition for maintenance (whether filed under DV, 125 CrPC or any other Act) is also dependent on the facts of why you are refusing to stay in the alternate residence.
On the other hand... if he proves that he wanted to stay with you and you have chosen not to, he can certainly claim desertion by you AFTER TWO YEARS of continuous desertion.
Lady, I answered your post because you have a young daughter. You should keep HER INTEREST above yours and hopefully your husband will do the same. Please do not go for filing any DV or 498 or other such cases, unless you absolutely must and have exhausted all other choices and are sure that the facts of your case will enable you to get some relief. Filing such cases will be the end of any hope that you may have of salvaging the family environment for your daughter. You and your husband owe it to your daughter to sacrifice your egos and personal interests for your daughter's sake. Keep that in mind at all times... Be assured that that is rarely any advocate's priority.