Robert Downey Jr. has finished most of the heavy lifting needed for his role in “ The Avengers ” and last week, over lunch at his office in Venice, the two-time Oscar nominee felt ready to talk about a different heroic project: “ Iron Man 3 .” “My sense of it is that we need to leave it all on the field — whatever that means in the end,” Downey said of the May 2013 film. “You can pick several different points of departure for that.” Speaking of departures, director Jon Favreau won’t be back for the third installment of a franchise that propelled his career to new heights — even as it established upstart Marvel Studios as a serious Hollywood player and signaled the popcorn redemption of Downey after years of addiction and legal entanglements. ” Iron Man 2 ,” however, was an especially grueling shoot and by the end of it the relationship between the director and the star was pretty frosty. Downey and Favreau are pals again now and even appeared on stage together earlier this summer