Mr Bremer met with Secretary of State Colin Powell, Defense Secretary Donald Rumsfeld and national security adviser Condoleezza Rice at the White House on Tuesday.
The coalition has refused to explain the reasons behind Mr Bremer’s hasty departure – or the cancellation of his meeting with Polish Prime Minister Leszek Miller.
But the move is said to reflect Washington’s frustration with the increasingly lawless situation in Iraq.
Despite Mr Bremer outlining a seven-step plan for Iraqi sovereignty in September, nothing has been put in place to assure the ceding of political and military control.
“It’s beginning to be realized that it’s not going to follow that path,” one administration official involved in Iraq policy in Washington said.
There are also concerns about the body set up to take over control of Iraq’s political and military affairs.
The Iraqi interim council has done “nothing of substance” since it was appointed in late August, a US official in Baghdad recently asserted.
It has failed to set policy or communicate well, said the official, who described the council as being “inept” at reaching Iraqis.
Mr Bremer warned the council last week that the December 15 deadline for drawing up the constitution and elections was non-negotiable.
Interim foreign minister Hoshyar Zebari said that Iraq would meet the looming deadlines.
“The ball lands in our court and we must deliver,” Mr Zebari said.
But he also warned: “A great deal will depend on the security situation in the country.”