Rodriguez at the double as Bolivia pay penalty
The goal gave Colombia a precious three points and moved them up into fourth spot in the South American standings.
Rodriguez rammed home his finish at the second attempt after Bolivia goalkeeper Carlos Lampe had saved the playmaker's spotkick on 83 minutes.
The goal gave Colombia a precious three points and moved them up into fourth spot in the South American standings ahead of other matches being played Thursday, with 21 points from 13 games.
Until Rodriguez's late winner, it looked as if Bolivia had done enough to earn a point, defending in depth and frustrating the hosts at every turn.
The Colombians almost opened the scoring after eight minutes when Pablo Armero's cross into the area was met by Sampdoria striker Luis Muriel, who clipped his left-foot shot against the bar.
Mateus Uribe rattled Bolivian nerves next after 17 minutes, shooting low after a deft layoff from MacNelly Torres only to see his effort parried and gathered by Lampe.
Colombia were forced into a reshuffle just after the half-hour mark when Muriel limped off with what looked like an injury to his right hamstring.
Rodriguez threatened to open the scoring on 34 minutes but Cristian Coimbra did just enough with his covering tackle to throw the Real Madrid star off balance and his shot went wide.
Rodriguez almost created an opening for Juan Cuadrado on the stroke of half-time but his chip into the box was directed just wide by the Juventus winger.
Colombia cranked up the pressure in the second half and carved out a series of good chances in quick succession which began when Carlos Bacca's header was turned around the post by Lampe in the 63rd minute.
Centre-back Yerry Mina crashed a thumping header off the underside of the bar from the ensuing corner and Torres then sent a low shot from just outside the area skidding wide of the post.
Bolivia continued to ride their luck and Coimbra was fortunate not to concede a penalty for handball as the minutes wound down.
But Bolivia's luck ran out when Coimbra brought down Cuadrado, allowing Rodriguez to score from the spot at the second attempt.