The Orioles scored all their runs in the fourth inning and Gausman (4-2) got three straight outs in the fifth. After Steve Pearce drew a walk in the bottom half, the rain came.
The game was finally called after a delay of 2 hours, 22 minutes.
Gausman allowed four hits, struck out two and walked none in his first major league complete game. The right-hander retired 12 of the final 13 players he faced to improve to 3-1 lifetime against the Yankees.
Brett Gardner hit a leadoff homer for New York, but Chase Whitley (4-3) gave up three runs and five hits in 3 2-3 innings.