The British Medical Association annual conference heard how a proper system of regulation needed to be introduced for managers so they could be held to account.
This idea has already been put forward by the Francis Inquiry into the Stafford Hospital scandal.
But so far ministers have resisted such a move.
However, the government has said managers responsible for failures will be barred from working in the health service.
Addressing the meeting of doctors in Edinburgh, which voted in favour of a motion calling for a regulation system for managers, GP Dr Peter Holden criticised the way managers were operating in the health service.
He said there was a “culture of secrecy, covert bullying and gagging clauses”.
“The result is the perfect toxic professional working environment for this explosive mixture to generate disaster such as Mid Staffordshire which did so much harm to patients.”
He added: “We all know of managers who abuse Care Quality Commission and General Medical Council procedures to bully doctors.”