Thermoresponsive polymers in water, for example, poly(N‐isopropylacrylamide) (PNIPAM) are investigated extensively, due to a wide range of biomedical applications. However, the attempts to control thermoresponsiveness are still rare in less or nonpolar media. Herein, the three thermoresponsive homopolymers tethering an N‐butylurea group in the side chain with a different polymer backbone are reported. They exhibit lower critical solution temperature (LCST)‐type and upper critical solution temperature (UCST)‐type thermoresponsiveness depending on association of the urea group in the polymer chain and hydrogen bonding small molecules (effectors) in ternary systems (polymer/effector/organic solvent). The difference of polymer backbone appears as their change of solvophobicity in organic solvents. Poly(methacrylate) backbone needs more amount of the effector in nonpolar organic solvents, and poly(vinyl ether) backbone needs more amount in polar organic solvents. However, the influence of polymer backbone is too little to change the phase transition behavior, and the thermoresponsiveness is dominated by association and dissociation of hydrogen bondings between polymers and effectors. The supramolecular design of the thermoresponsive polymers is strong and extensible for the design of their phase transitions.