Listen carefully. If you have child custody with you, don't file any case right now for custody or sole custody, which you would like to call it. If you file any custody case, he will definitely object it and also file visitation rights application, so that geopardizing the uninterrupted coddling and cohabitation, you have with your child. Enjoy the custody dispelling any fears that he would snatch your son from you. His continuous non-attention and failure to attend the child's needs, all would go in your favour, once he stakes claim of custody over your child in future (I am quite sure he will not do that), because the courts will grant custody only keeping in view of paramount interest of the child, which is with you.