We review our recent work on tunable, ultra-high quality factor whispering-gallery-mode bottle microresonators and highlight their applications in non-linear optics and in quantum optics experiments. Our resonators combine ultra-high quality factors of up to Q=3.6×108, a small mode volume, and near-lossless fiber coupling, with a simple and customizable mode structure enabling full tunability. We study, theoretically and experimentally, non-linear all-optical switching via the Kerr effect when the resonator is operated in an add–drop configuration. This allows us to optically route a single-wavelength cw optical signal between two fiber ports with high efficiency. Finally, we report on progress toward strong coupling of single rubidium atoms to an ultra-high Q mode of an actively stabilized bottle microresonator.