The reason for his decision is complicated. Stratfor believes Iran has been a large factor in Al-Sadr's decision, and that there may be a greater pact for joint progress on Iraqi Security. Bush called this fight a "defining moment" for the future of Iraqi Security, but the ultimate conclusion may have been a defining moment in a way that he could have never guessed--and may mean an end to the adversarial positioning that the US and Iran have been going through for years.
After six days of fighting, Al-Sadr called his forces to leave the streets. He refused to give up arms, as the Iraqi government demanded, but kept vehemently to the cease-fire he called 9 months ago, and has claimed “Anyone carrying a weapon and targeting government institutions will not be one of us.” He has not only pledged not to fight, but to work together with the Iraqi government: "We have decided to withdraw from the streets of Basra and all other provinces... [and to] cooperate with the government to achieve security.” This is a long stretch from his previous anti-government sentiment, vowing to fight them for their cooperation with American imperialists.
Stratfor believes the renewed calls of cooperation and peace come in part due to Iranian pressure on Al-Sadr. Al-Sadr and the Iranian government likely depend each other for mutual security; Iran is looking for a Shiite-dominated Iraq that will never pose a security risk to it again (given the terror of the Iran-Iraq war of the late 1980's), and Al-Sadr needs outside support to secure his base in Iraq.
But the agreement seems very pro-American, and we must consider why each party seems to be agreeing:
Iran: President Ahmadinejad visited Iraq last month, proclaiming friendliness and cooperation between both governments. Certainly, Ahmadinejad had security on his mind; Iran wants neither an adversarial Iraq nor a refugee or terror problem. How much military meddling Iran is doing to hedge its bets is unclear, but Stratfor believes there may have been a secret US-Iran meeting in Baghdad to talk security concessions--Iran may have given a commitment to helping stabilize the Iraqi government for US concessions in both ceasing action against rival Shiite factions (which have remained fractured for years) and making military and UN resolution moves against the Iranians. Why now? Victory by Ahmadinejad's allies may be pressing his hand to act and achieve regional victory quickly. Finally, Iran may now be a committed advocate of Iraqi security; a dark ally that the US desperately needs as its era of troop presences draws to a close.
Mahdi Army: Al-Sadr's military defeats pressed his hand, in part; there was a danger of a strangle on his power base by the Iraqi Security Forces. But there are two other issues that made this choice easy for Al-Sadr.
Iraqi Government and Washington: The Iraqi government needs stability as it tries to create political unity and reconciliation. With the Al-Sadr militia on its side, it should be able to quell the anger of the restive southern Shiite regions, and Al-Sadr's Mahdi Army will become an asset, rather than a liability--this way, Iraqi Security Forces should be able to concentrate the bulk of their effort on pursuing Al-Qaeda and securing Baghdad. This advancement will become critical as US forces draw down both this summer and next winter (in particular if a Democrat wins the US election).
This new deal seems like one, long-term, that can help everyone involved. Furthermore, it ties in the security interests of the Mahdi Army, the Iraqi Government, and the Iranian leadership. I am a raging optimist, but this readjustment by the Mahdi Army and Iranian government is an excellent sign for the future security of Iraq.