Clint Dempsey scored and the United States held on after losing DeAndre Yedlin to a red card early in the second half to beat Paraguay 1-0 and book a spot in the Copa America quarterfinals.
After Dempsey scored in the first half in front of a large crowd in Philadelphia, the U.S. fought off a late Paraguay assault on Brad Guzan's goal to secure the victory in Group A.
Jurgen Klinsmann's men actually won the group after Costa Rica, who were eliminated with the U.S.' win, still managed to stun Colombia later on Saturday night.
The U.S. will next face the Group B runner-up -- likely either Ecuador or Peru -- on Thursday in Seattle. The late result means the U.S. will likely avoid Brazil, which would likely have to lose to Peru on Sunday to not win Group B.
On Saturday, the U.S. found the breakthrough in the 27th minute as Zardes raced down the wing and pulled back a low cross that Dempsey shot home.
But early in the second half, the game turned as Yedlin picked up two yellow cards in as many minutes, reducing the U.S. to 10 men.
The U.S. still held its own down a man, and center-back John Brooks and Guzan made some spectacular defensive efforts to preserve the victory.
Colombia was made to pay for making 10 changes and settled for second place in Group A after their 3-2 defeat to Costa Rica.
Having already qualified for the knockout stage, Colombia only needed a draw to win Group A, but Jose Pekerman decided to rest his first-choice team, including stars James Rodriguez and Juan Cuadrado, and his team now faces the prospect of facing Brazil in the quarterfinal.
Costa Rica needed less than two minutes to take the lead as Johan Venegas turned and unleashed a marvelous curling strike from outside the box that found the top corner.
But Colombia hit back almost immediately through Frank Fabra, who ran into the box and poked the ball past Costa Rica keeper Patrick Pemberton in the seventh minute.
Colombia kept pressing the attack but it was Costa Rica who retook the lead in the 34th minute as Fabra inadvertently turned in a cross into his own net.
Pekerman had seen enough and sent on James at half-time, along with Edwin Cardona. But again Costa Rica stunned the crowd in Houston as Borges fired in a third goal from a Bryan Oviedo cross on 58 minutes.
Soon after, Cuadrado came on and delivered a cross for Marlos Moreno to score in the 73rd minute, but Costa Rica withstood a late flurry to defeat Colombia for the first time since 1980.