I've used Magic Erasers. If I don't have any of those on hand, I use baking soda mixed with water to form an paste and a soft cloth to wipe it away.
Mix a few tablespoons of baking soda with warm water to make a paste. Rub into the scuff marks with a soft cloth.
Have you tried a Mister Clean Eraser?
Magic Eraser will do the trick.
Use magic eraser.
Pencil eraser or Mr. Clean magic eraser.
Try a Mr. Clean magic eraser, they are awesome!