Afghanistan’s President Hamid Karzai has criticized the US for failing to transfer more than 600 inmates at the Bagram Detention Center to Afghan control – even after the prison was handed over.

The US gave away control of the high-security detention center on Sept. 10, but was reluctant to hand over some of the detainees in fear that the Afghans would just let them go, the Associated Press reported. About 3,000 suspected Taliban fighters are being given back to Afghanistan, but the US is refusing to transfer 635 detainees. 

“Delay in handing over of prisoners after September 9this a serious violation of the agreement signed between Afghanistan and the United States,” Karzai said in statement Sunday.

NATO said that 600 prisoners were captured after the initial agreement to hand over control of the prison had been signed – and that these would remain in US custody for a while. The US has also kept refused to hand over 34 other prisoners that it considers too dangerous. But as the US retreats from Afghanistan and gradually gives control to Kabul, Karzai expressed disappointment with the country’s failure to fulfill the terms of the agreement.

“The people of Afghanistan consider prisons and the detention of Afghans to be an issue related to their national sovereignty and this has been repeatedly mentioned in the strategic partnership agreement between the two countries,” Karzai said.

“The continued holding of Afghans in American custody runs in contradiction with the spirit of mutual friendship and the provisions of the bilateral strategic partnership agreement,” Karzai’s office said in the statement.

Afghanistan has long sought control over the high-security prison, which has a history of abuse of its inmates. In 2002, two Bagram prisoners were chained to the ceiling and beaten to death, starting a series of cases in which prisoners were treated inhumanely. The Bagram Detention Center has accumulated more detainees than Guantanamo Bay.

As US troops gradually trickle out of Afghanistan, the Kabul government is disappointed that it will not have full control over more than 600 imprisoned Taliban fighters, raising tensions at a time when protests in the Arab world have already put the US on edge.