Alan Murdoch was granted planning permission for conversion of the barn into residential units in April 2016.
But that was subject to a condition that required him to install acoustic fencing to meet the concerns of neighbour David Smith-Ryland, of Plestowes Farm, Barford.
Smith-Ryland fears residents of the new homes would hound him with complaints about "noise nuisance".
The farmer's judicial review challenge to the planning permission was previously rejected by a judge.
But Smith-Ryland was back in court after the council discharged a planning condition in February this year which stipulated that acoustic fencing had to be sound-proof enough to ensure that new residents of the barn would not suffer "unacceptable disturbance."
Council planners discharged the condition after Murdoch said he would install a fence recommended by acoustic experts. Officers found that the proposed fence would deliver "an acceptable acoustic environment".
At the High Court, Smith-Ryland's lawyers argued that the fence proposed by Murdoch would have been "non-compliant" with the condition had it remained in place.
The lifting of the condition meant he would still be exposed to noise nuisance complaints, said his barrister, Paul Cairnes QC.
But dismissing the farmer's challenge last week, Mr Justice Jay said a council environmental health officer who approved the fence clearly took noise levels into account.
The issue of what would or would not be an "unacceptable disturbance" could not viewed in "purely mechanistic or numerical terms", he added.
The council's decision to lift the condition had not been attacked as irrational and the judge could detect no legal flaw in its conclusions.
Last week, an Oxfordshire council failed in a High Court challenge to block plans for a housing development on the edge of a village after a judge rejected its arguments that an inspector had ignored critical parts of its emerging local plan.
Also last week, a Leicestershire man failed in a High Court challenge against a planning consent to convert a local shop into a Domino's pizza outlet, after a judge rejected his arguments that the decision had not taken account of relevant local planning policies and had breached equality legislation.