A survey of growers by the farming union found that this year’s harvest had been ‘significantly impacted’ by the sector’s ongoing labour crisis