How do I get rid of wasps? PLEASE HELP ME NOW!!!!?

Today, we've found 12 WASPS in our family room. We've killed them all so far and investigated. THey seem to be coming from the fire place. So, we're thinking that maybe there is a nest in the chimney. As of right now, all we've done is tape off all the openings in the fire place with painters tape? What do I do now? Who do I call? Has this happened to you? PLEASE ANSWER!! THEY NEVER STOP!!!