Summary:

It took a lot for Tony Stark to let his outward appearance slide.
They all said he was nothing but show, and in the end it was true to a certain degree, because he really was all show for most people. It took time and persistence to bore through that mask, to worm out worries and sorrows from this always troubled mind - showing vulnerability was something the billionaire simply refused to do.
So, for Tony Stark to have stopped caring about his performance and allowed people to actually see that he felt miserable and broken, some terrible things must have happened. 

After Pepper's funeral, he had retreated to Malibu, alone.

Six months after the battle of New York, the Chitauri return to earth for another attack, sucessfully destroying populated areas and leading to millions of deaths. The Avengers have split up, each of them going their own way after the devastating assault.
But after having locked himself away in his Malibu Mansion for months, Tony gets a visit from Loki, who makes him an unexpected offer that changes everything.

Post-Avengers, Frostiron