You need tobeire brush off the tennicals first.
Tooth brush & Exacto knife off.
You can scrape them off until flat then try Stain Blocker paint from the DIY store then decorate over it.
We had the same problem with ivy on bricks. Just left it alone after the ivy was removed.
You could scrape off the brown marks, yes this will be a lot of work, and then use stain block paint.