Confidence in the UK food and drink sector is declining, with Brexit a major concern, a survey by advisory firm BDO has found.