We collected bottles from friends, restaurants, etc. and used a bulb planter to make holes to stick the 'neck' of the bottle into. We have another bottle border in another area of our yard that we actually dug a small trench and planted the bottles much deeper. Both ways work very well.
The bottles are beautiful when the sun shines through the colors at different times of the day.