Fly Weight third seed Warren, competing in his third Olympic Games, lost 19-18 to Frenchman Nordine Oubaali in a nail-biting finish.
The 25-year-old said afterwards: 'I am still young. I gave my whole life to the Olympics and that was my dream. I have to work out what I am doing next.'
Spence, a Dallas-based 22-year-old, went out of the competition following a 13-11 defeat against third-seeded Indian Krishan Vikas in the Welter Weight division.
But later, boxing's world governing body AIBA upheld an appeal and reinstated the USA boxer, so Vikas has left the competition.
European champions Andrew Selby and Fred Evans extended Great Britain's fine start with last-16 victories in their respective Fly Weight and Welter Weight divisions.
Evans produced a fine last round to beat Egidijus Kavaliauskas 13-11 and Selby secured a 19-15 verdict over tough Kazakh Ilyas Suleimenov.
Belfast Fly Weight Michael Conlan clinched a straightforward 19-8 win over Ghana's Duke Micah.