Certified copies of affidavit will not mean anything. People can give in writing anything to court and can back track. That is why all necessary things agreed upon by both husband and wife should come in black and white on paper in the order of the family court.
If it is not mentioned about child custody in order sheet, though physically the kid is with you, wife can always file peition in court seeking vistiation and also custody. You have been duped by your advocate.
If your wife files for custody of visitatoin rights though you can show this certified copy of her affidavit, she is natural mother and she can always backtrack and Court will not object. Blood Parental relations are more important to court than anything else especially in case of children and parents. Till then,
You will simply have to hope that your wife wont file custody case or vistitation rights case.