See improv like a game of tennis, not golf.
In both tennis and golf players each get a turn to hit the ball. In golf you're doing your own thing in your own world and only try to do better than your opponent. In tennis however, everything you do is in direct response to what your opponent just did. You cannot play tennis without paying very close attention to your opponent all the time.
Perhaps a more accurate comparison would be 'a friendly game of tennis', just to leave out the competitive element altogether.