Bakkavor has announced plans to close a fresh fruit site in the UK after it lost a contract to supply a "major customer".