Tom Swarbrick claims that with men like Richard Sharp and Boris Johnson, the 'rules don't apply'