How old is your furnace? If it's old and hasn't been serviced recently, then yes.
We always do!
I leave mine on and turn my thermostat down all the way.
If you turn it off, make sure the gas is turned off to the furnace also.
We always do....just turn the thermostat from heat to cool and works fine. We have our furnace/air conditioner inspected every spring and fall to insure all are working as they should.