I would suspect mold growth around the window frames on the SE side of the home from rain leakage. When the odor is present, sniff the windows on side of the home that the wind is coming from. If so, correct the trim defect to keep the window frame dry.