Farmers were at risk of companies switching sourcing arrangements while preferential tariffs offered by the EU could come to an end, according to Fairtrade report