Kemi Badenoch said setting a date to settle the cases was "not the priority".
But she said the government was moving "as quickly as we can" to rewrite the wrongs of the IT fiasco.
Hundreds of subpostmasters were prosecuted by the Post Office for theft and false accounting because of a faulty computer system made by Fujitsu, Horizon.
Rishi Sunak has described it as “one of the greatest miscarriages of justice in our nation’s history”.
However, Mr Bates himself, Alan Bates, has since warned that subpostmasters are dying before they receive proper compensation in the long-running saga.
Asked on the BBC's Sunday with Laura Kuenssberg programme if she would set a deadline to deliver full compensation for those affected, Ms Badenoch said: "The Prime Minister has said we're not setting a deadline.
"We're going to move as quickly as possible. There are always issues with setting deadlines… you put a date on, people rush, they get things wrong.
"We are moving as quickly as we can. I promise, we couldn't move any faster than we already have been.”
She added: “Setting a deadline is not the priority, getting the money out, getting fair compensation, sorting out the governance of the Post Office, is the critical thing."
She also claimed she was forced to intervene over "difficulties" with Post Office governance after she sacked its former chairman on Saturday.
And she would not be drawn on whether other departures from the board of directors might follow.
Ms Badenoch told Sky's Sunday Morning with Trevor Phillips the “issues” went “well beyond the Horizon scandal”.
Mr Staunton stepped down as chairman of the state-owned business after just a year in the job.
Ms Badenoch said they had "parted ways with mutual consent", but later added that she had asked him to go.
The Post Office prosecuted the subpostmasters for more than a decade and a half, between 1999 and 2015, after the software made it look as though money was missing from their branches.
Hundreds are still awaiting compensation. Those who have had convictions quashed are eligible for at least £600,000.
In the wake of the ITV drama Mr Sunak announced blanket legislation to swiftly exonerate all those wrongly prosecuted. Ministers hope their names will be cleared by the end of the year.
Labour questioned the move to oust Mr Staunton at a weekend and called on ministers to explain his departure.
Shadow business secretary Jonathan Reynolds said Mr Staunton had not worked for the Post Office during the scandal “so there must be specific reasons why they don't have confidence in (him)”.
"I think the public will want to know this is not just about one person, one chair being changed. The overall approach and the entire organisation is going to come to terms with the scale of this and put it right, and also fundamentally people want to see the subpostmasters exonerated and compensation got to them as soon as possible.”