How do I fix my 17 month old's MAJOR stranger anxiety??

My son cries hysterically when people try to hold him or even talk to him.This includes my father and other members of my family who he should be farmiliar with. The only people allowed to hold him,talk to him or play with him are myself, My husband,My sister, and my mother. We have had the same babysitter for nearly 3 months and he is just now starting to warm up to her. Everyone loves him but I am at my wits end trying to make people believe that he is the most wonderful baby.....