'Based on my experience from being in Downing Street at the time, I think the state that we have is too passive, too relaxed, too siloed and too slow.'