What can I do get rid of rust stains on my stainless steel fridge?

Try making a paste with lemon juice and baking soda or cream of tartar ,baking soda and water. Rub on the stains and leave on long enough to break the rust down. Rub off with a soft cloth.

Try Bar Keepers Friend cleaner

If it is actually rust, I would contact the manufacturer and make a complaint.

Bar Keeper's friend is the way to go!

Here are some other suggestions for you as well: http://www.hometalk.com/diy/q-how-can-i-remove-rust-from-stainless-steel-doors-2068445

and http://www.hometalk.com/diy/clean/appliances/q-stainless-steel-that-s-stained--18487203?expand_all_questions=1

Whink. Usually found in the kitchen cleaner section of the grocery store or hardware store. just wipe and gone !

Try Bar Keepers Friend with a non abrasive scrubbing pad. Sold near the Comet and Ajax. Does not scratch. Also Mr. Clean Magic scrubbers.