If you pluck the saffron from your crocus plant is it safe to use?

To harvest, cut off the open flowers at midmorning on a sunny day. Allow all foliage to grow through winter and nourish the plants for next year.

Pile the flowers on a table where you can work comfortably. Remove the stigmas. Take only the red-orange parts; the stronger the color, the stronger the flavor. Dry the saffron without exposing it to high heat. Small amounts can be placed on a plate. A dehydrator at the lowest setting or a screen on blocks in a warm dry room can be used for larger quantities.

but is it safe to consume?

If your plants are on the ground outside you have to be sure critters don't sprinkle on them I guess.