When I went through the visitation process in CT, our pick ups were in a public place, so we needed to be clear on the times.
I was told that if my ex did not pick up my child at the designated time of say 4:00, that I could bring the child home, and the ex would have to make arrangements to pick her up, if he wanted. I would still have to have the child available to him from 4:00pm on fri to 4:00 pm on sunday, he would just have to call and make reasonable arrangements.
so if you keep the child from him just because he missed the designated official time, you may still be considered in contempt, because in my stae at least, the ncp "becomes" the cp during thier visitation time. This is also spelled out in Ma. papers also.