Silicone caulking is an excellent solution to seal the gaps in your windows' frames.

A set of used caulking materials and tools.