Children's services at the council which was heavily criticised for failings over the death of four-year-old Daniel Pelka have been criticised for not improving quickly enough after being branded inadequate by Ofsted.
The Department for Education (DfE) said that while it recognised Coventry City Council's improvements since last year's publication of a damning serious case review criticising children's services's handling of the boy's case, the pace of change was too slow.
It said the measures already carried out at Coventry were "simply not good enough".
The DfE added it was now considering what "further actions" were needed to improve the situation, but did not specify what form those measures might take.
Daniel died of a head injury in March 2012 after a campaign of abuse by his mother, Magdelena Luczak, and his stepfather, former soldier Mariusz Krezolek, both of whom were jailed for a minimum of 30 years for his murder following a trial last year.
George Duggins, the council's cabinet member for children and young people, vowed there would be "rapid improvement" following publication of Ofsted's report into the service.
He added that the children's services department had seen an "unprecedented" volume of cases, which was still increasing.
Amy Weir, chair of the Coventry Local Safeguarding Children Board (LSCB), said she was "disappointed" with the inspection's conclusion but accepted that more still needed to be done, particularly in strengthening the way agencies such as councils, schools and police work together on keeping youngsters safe.
The report, which has found children's services to be "inadequate overall", comes after a three-week inspection earlier this year.
Coventry City Council said: "The report published today found that case loads for social workers are too high.
"Although social work teams have been increased, a dramatic increase in workloads - 46% over the last two years - mean that case loads remain too high."
By the end of last month, social workers in Coventry were working with 4,529 children in the city - up from 3,085 in March 2013, the council said.
An additional £5.6 million is being invested in the service, which will be mainly spent on creating more social work teams to deal with the huge increase of referrals, he said.
Mr Duggins said: "There is no hiding from the fact that the report's overall findings are disappointing, but many of the problems we face, particularly in the front line of children's social care, are down to the fact of the unprecedented volume of work, which continues to increase.
"As the Ofsted report reflects, the findings are not a verdict on the staff, who are working extremely hard in the face of ever-increasing workloads, and their passion, commitment and dedication cannot be questioned.
"We have already responded to this challenge by adding another team of social workers and we have funding to add more as part of the additional £5.6 million because, as a council, safeguarding is one our highest priorities."
He added that the council had also set up its own improvement board to implement Ofsted's recommendations in driving standards up, although its work is still in the very early stages.
Ms Weir added: "There is more to be done, and the partners accept this and that we must all exercise our single and joint responsibilities for safeguarding more effectively as members of the LSCB.
"Safeguarding board partners are absolutely committed to delivering and monitoring improvements and to hold each other to account over the next few months so we can deliver change for the better and demonstrate the impact on safeguarding in the city."
A DfE spokesman said: "We have been concerned about the arrangements for the protection of children in Coventry since the publication of the serious case review into the death of Daniel Pelka in September 2013.
"We were clear at the time that we expected children's services to dramatically improve without delay, which was why we required the LSCB to undertake a deeper analysis of the wider issues so that Coventry was able to learn lessons from Daniel's death.
"While we accept securing sustainable improvement takes time, today's report shows that the pace of change in Coventry has simply not been good enough.
"We will now consider what further actions are needed to ensure all vulnerable children in Coventry are sufficiently protected."
In their report, Ofsted inspectors concluded that the children's services department's management, leadership and governance were inadequate.
In a damning assessment of the work of the safeguarding board - also branded inadequate - it concluded that the LSCB was "not demonstrating it has effective arrangements in place to discharge its statutory duties".
The report highlighted that children were not being seen quickly enough and were "left at risk of harm".
On the issue of heavy workload, it found that the council had already had time to tackle the problem because a separate review by the Local Government Association identified the issue back in March last year.
The problem of too much case work also meant social workers "cannot do their jobs properly".
Inspectors added that more work was needed on timely information-sharing between the council and police, particularly on reports of domestic abuse - an issue which was one of the key criticisms in the serious case review into Daniel's death.
However, Ofsted did welcome the appointment of new children's services director Brian Walsh, concluding: "T here is clear evidence of decisive action now being taken to address areas of concern."
Daniel died after months of physical and emotional abuse at the family home in Coventry, amid an atmosphere of domestic violence between his mother and her various male partners.
The little boy was beaten, suffering a broken arm which needed hospital treatment, and arrived at school one day with two black eyes.
He was also kept locked in a boxroom at home, fed salt and starved to the point where he stole food from other pupils' lunchboxes and raided school bins for scraps at playtime.
Despite these signs, last year's serious case review report found that social workers had never spoken to Daniel independently about his home life or acted decisively to intervene in his care.
It also found that the child protection agencies were taken in by the "web of lies" spun by Daniel's mother and stepfather and should instead have been "thinking the unthinkable".