Councils, the NHS and other public bodies have saved an estimated £1 billion with the loss of 26,600 full-time workers over the last four years, according to a report.
But the public sector will "need to make further workforce changes" because of financial pressures, public spending watchdog Audit Scotland said, although without improving the way services are provided it is unlikely the required savings will be made.
Audit Scotland has examined changes to the public sector workforce which is an "important source of employment in Scotland".
Its report focused on local authorities, the health service, the Scottish Government and other central government bodies, and did not include colleges, the police, and fire services as these areas were "undergoing major reform" at the time of the audit.
Between March 2009 and March 2013, the public sector lost the equivalent of 26,600 full-time workers, or 7%, to make a total of 373,400, the report said.
The equivalent of 15,816 full-time workers left through early departure schemes such as voluntary redundancy or early retirement.
The equivalent of 9,664 full-time jobs were transferred to arms length external organisations (Aleos) such as the bodies set up by some councils to run sports and leisure services, or other non-public sector bodies.
"These changes helped public bodies to reduce their staff costs by an estimated £1 billion (8%) between 2009-10 and 2011-12," the report said.
But "some of this reduction in staff costs will be offset by other payments to Aleos and other non-public sector bodies to deliver public services."
Staff costs across councils, the NHS, the Scottish Government and other central government bodies amounted to about £12.7 billion in 2011-12, although Audit Scotland said it was difficult to get reliable information because "there is no single source of data for staff costs".
Over the four-year period, councils lost the equivalent of 23,800 full-time workers, 10% of their staff, while the NHS saw its workforce shrink by 1%, losing the equivalent of 1,400 full-time workers, Audit Scotland said.
Public bodies have to "manage their workforces strategically" to provide services at the same time as dealing dealing with financial pressures and the need to make savings.
"Many bodies are currently reducing the size of their workforces to help meet budget cuts," Audit Scotland said.
"Public sector finances will be under pressure for the foreseeable future as budgets decrease and the ageing population increases demand for many public services. Given the challenges ahead, public bodies, and especially councils, will need to make further workforce changes. Without service reform, these are unlikely to deliver the savings needed.
"Public bodies need to think differently about how they deliver services, for example by prioritising and redesigning services and increasing joint working and collaboration."
Audit Scotland recommends that the Scottish Government and local government body Cosla provide "strategic guidance on how bodies can redesign services and work better together to jointly plan and deliver services".
It also wants the Government and Cosla to improve the "consistency and accessibility" of figures detailing public sector employment and staff costs.
Auditor General for Scotland Caroline Gardner said: "While it is difficult to get reliable information in this area, we estimate that staff costs in the public sector have reduced by around £1 billion in the three years up to 2011-12. Public bodies record these costs in different ways and an element of the money paid to Aleos to deliver services will also be spent on staffing.
"One of our recommendations is that that the Scottish Government and the Convention of Scottish Local Authorities work with public bodies to help them improve the data they collect on staff costs."
Bill Hendry, human resources spokesman for Cosla, said the report " reinforces the crucial work already being undertaken by councils in redesigning services in a period of major demographic change".
He said: "Planning now for the local government of the future in terms of right people, right place, right time and at the right cost is a key activity for HR professionals in all councils. We cannot be complacent, however, and we will consider carefully the recommendations of the Audit Scotland report and act on these where we feel they add value to our existing work in this area."
Dr Brian Keighley, chairman of the British Medical Association in Scotland, said: " Staff costs make up a significant proportion of the NHS budget and, therefore, those who work in the NHS can be an easy target for NHS boards to make financial savings.
"NHS staff have had to endure 'pay restraint', recruitment freezes and cuts to staffing numbers and while this approach might deliver some savings in the short term, it will in the longer term cause damage to patient care."
There is "a n obstinate reluctance of the Scottish Government to consider significant service redesign", he said.
"Without a realistic debate on this issue, the NHS will be under huge pressure to make sure it has sufficient staff to deliver services as they are now. Trying to get even more work out of existing staff is not a sustainable solution."
A Scottish Government spokeswoman said: " The Scottish Government welcomes Audit Scotland's report which illustrates the progress Scotland's public bodies have made in reshaping their workforce in the face of severe budgetary challenges resulting from the current constitutional arrangements within the UK.
"We note Audit Scotland's recommendations and will continue to drive forward reform in the delivery of public services in Scotland, building on our solid reputation for effectiveness and efficiency."