Hi Linda, here's how one blogger dressed up boxes. So simple and cute! https://createandbabble.com/how-to-make-decorative-storage-boxes/
How about something like this: https://www.hometalk.com/3711785/twine-wrapped-decorative-box
Cover boxes with contact paper in baby themes
You could paint them with the nursery theme colors
Linda...You can use material to cover the box using mod podge, lace, ribbon, beads, etc.