How do you make use of a 4 x 2ft window that has drywall behind it?

You don't. Either the install was never completed, or it was drywalled up for a reason. Is it on an exterior wall? if you want the window to work, remove the window, cut the drywall out, and then use a sawzaw to cut the exterior siding off, frame it, and then install the window. 2 parts.... https://www.youtube.com/watch?v=P21tW6XfUwQ

https://www.youtube.com/watch?v=-wK-sYcfphw

What about a hidden compartment that's hidden by a picture attached to a hinged door? Or put shelves inside to use for storage.

Open the window, place your favourite picture on the back of the wall inside and use LED lighting for effect (can be battery operated if you don't want to run electrical wiring) change the pictures as you wish....

Without a photo I am taking a wild guess. I would put LED lights behind the window and then hang curtains on the front. Put the lights on and it looks as if the window open to the outdoors.