razor blade and rubbing.
Ace Hardware has a product that removes silicone, but it should stay completely away(1/2 inch or more) from the silicone that needs to stay. Let me know if you want the name of the stuff. I'll have to go into the other room to look at it.