The nine-flat block in Theobald Road, in the London Borough of Croydon, was built in 2007, but deemed not to conform to planning permission obtained in 2005.
Council officers said the building did not comply with the permission "in form, shape, height, size or extent".
An enforcement notice requiring demolition was issued by the council in 2010, but not complied with and, in January 2012, the block was sold.
The purchaser, Susan Stansbury, then granted 99-year leases on six of the flats to her husband, Derek.
Despite attempts to ensure the block was demolished, it never occurred and a criminal case proceeded.
In November 2015, Susan Stansbury pleaded guilty to failing to comply with the enforcement notice.
And in December last year, her husband was convicted of the same offence at Croydon Crown Court.
They are both yet to be sentenced, but awaiting his punishment, Derek Stansbury yesterday took his case to the Court of Appeal to challenge his conviction.
Stansbury claimed that, because he was not the freehold owner of the property, he could not be expected to bring about demolition.
He said the leases were granted to him to facilitate the obtaining of mortgages, not to frustrate the attempt to enforce the demolition order.
And he said his statutory defence to the charge - having done all he reasonably could to comply - should have succeeded.
During his wife's prosecution, he had become ill with a brain tumour and was under too much stress to do more than he did.
However, he accepted that during the same period he was well enough to negotiate with the council, evict tenants and liaise with an architect and builders.
Dismissing his appeal, Mrs Justice McGowan said it was the prosecution case that Stansbury accepted he was an owner of the property for the purposes of the Town and Country Planning Act 1990.
The judge continued: "Where one or more persons are in joint ownership of a property, whether as freeholders or leaseholders, each is required to take reasonable steps to secure compliance.
"There is nothing arguably unsafe about this conviction."
The appeal application was refused.
R v Stansbury. Case Number: 201800174 C1