Hi, Donna: I know that deer do not like rosemary. Not sure how cold it can get to cause them harm, but check with a local nursery. Rosemary has some delicate pale blue or lavender flowers, too. Good luck - those deer are really hard to train!
Here are some suggestions: https://www.provenwinners.com/learn/top-ten-lists/12-hardy-deer-resistant-proven-winners-perennials
Where do you live? how cold does it get? Hot Tropical? Wet or dry? this is world wide site so if we know where you are we can beetter suggest plants that will grow