How can I decorate these holes in my bathroom?

neutral colored baskets

I think the ones above and next to the fireplace would look great with some plants. The ones below, maybe some baskets for a more uniform look.

I lightened your picture because I have bad eyes. Looks to me like you've got plenty of closed storage for practical but not pretty stuff, so you can use these lovely places for the more attractive things. They can be practical things or something just for whimsy. Stacks of decorative towels. Attractive bottles, shells, boxes, pots. Plants that thrive in bathrooms with less light, flower arrangements, seashore items, boats, anchors, if you want to go nautical. Vary it with something that doesn't completely fit within the cubical spaces, like a vining plant that trails downward toward the shelf. Hand a plant from the ceiling to contrast. Put an antique toy car or washboard, or something unexpected on the floor for balance. Don't over-clutter just because you can. Fewer things, with varying size, color and texture would probably work best.

the first thing that came to mind is the fabric squared organizers or woven baskets to hold bath supplies and cleaning products.

I just wanted to say that you are one lucky woman! This is absolutely gorgeous! I could only dream of such beauty.

I just want to say THANK YOU ALL for your remarkable ideas to help me complete something nice in this area!!! Gonna put them into use and will put my photo online when completed for you all to see what you helped me accomplish. 😉🙋

storage baskets are expensive and since all the storage holes are empty Im guessing you dont have any..I made some for my laundry room out of diaper boxes, covered them in cream colored material and then a burlap edging...or you could cover the empty spaces up with a curtain and display some matching baskets or plants or whatever you like

I'd roll up some towels and put in them. It is a bathroom afterall. Maybe put some battery operated candles in one, a small metal basket of bath bombs, decorative soap, & or travel size shampoos/conditioners. Keep it simple.

The cubbies look like shoe holders to me. But if I had your bathroom, Florence, I would buy a few luxurious bath towels, handtowels, and washcloths, and roll them up in set and put one set in each of four cubbies. Then I would like to see Her really fluffy, furry slippers in one, and maybe His, even if they were never worn. Jfe’s suggestions sound wonderful, especially if guests use this bathroom. But with this, you will be giving bathroom tours forever. Enjoy! ☺️