Can you post a picture?
Other than scale, tea olives generally have very few insect pests. We really need to see a photo.
Douglas, my tea olive doesn't look good this year. I wonder if it is lack of water or cold from the winter. There are no apparent pests, just bad overall look on two tea olive plants.
Mine have lost a lot of leaves, Sherrie. I hope it's the drought and that when it starts raining more regularly they will be happier.
I have tea olives also here in Tn. Yep- I found when I watered them more, they look so much better!
Something attacking the roots? I have problems with voles and even though I planted new bushes with hardware cloth surrounding the root ball (hardware cloth is like window screen only with bigger holes and sturdier wire) I still found one plant that is in vicinity of a favored vole location losing leaves on about 1/3 of the plant. (And found a tunnel leading to the wire surround.) My solution is crushed basalt mixed with the soil further away from the plant.