I've been doing it the hard way, any spray cleaner, toothbrush and rags. Then my hubby said "why don't you use the steamer. " Wipes right off and no chemicals. Wish he'd told me earlier, would have saved a lot of time and elbow grease.
Here's a good tutorial on how to clean window sills and window tracks: https://www.theintentionalmom.com/how-to-clean-window-tracks-like-a-pro-in-no-time-flat/
Baking soda and vingar. It will bubble up out of the corners when you pour the vinegar on top of the baking soda. Neither soda or vinegar will hurt the woodwork.
Tooth brush and detergent works well. I have also seen where people have used baking soda and vinegar.