Carpet border lilies can be a nice choice. I also just bought some perennial, large bloom hibiscus and they can be quite a show stopper!
Not knowing what Zone you are in, sun or shade, I would visit a reputable local nursery and chat with the master gardener on staff for suggestions. Another resource would be your local Ag Extension. Here are some suggestions to get you started!
bleeding hearts, astilbe and columbine come to mind. I am sure there are many.