Responsive hybrid colloids containing both organic and inorganic components have been the subject of many investigations in recent years. These new materials combine the stimuli-responsiveness of some polymer-based colloids with the unique properties of inorganic nanoparticles. This article will review the different possible variations of such hybrid colloids. Moreover, synthetic approaches, methods of characterisation, and a few applications will be discussed. Due to the rather high number of recent publications dealing with hybrid materials based on a variety of polymers, we will mainly focus on responsive hybrid microgels made of poly(N-isopropyl-acrylamide) or poly(N-vinylcaprolactam) in aqueous media.