It was the third match of the day in Abidjan and followed Nigeria's draw with Equatorial Guinea in Group A while Egypt needed a last-gasp penalty to draw with Mozambique in the opening Group B match at the Felix Houphouet-Boigny Stadium.
Cape Verde went ahead in the 17th minute when Dutch-born Rodrigues's shot was parried by Ghana goalkeeper Richard Ofori, falling for Jamiro Monteiro to tuck away from close range.
Ghana thought they had equalised 10 minutes before halftime when Majeed Ashimeru struck a powerful shot into the net from outside Cape Verde's penalty area.
But after a VAR check it was disallowed because Ransford Konigsdorffer, in an offside position, was deemed to be interfering by being in the goalkeeper's line of sight.
Konigsdorffer was making his way back after inadvertently striking the upright when a punched clearance from Cape Verde goalkeeper Vozinha hit him flush on and cannoned back towards goal.
Ghana did find parity 11 minutes into the second half with Alexander Djiku heading home from Jordan Ayew's corner, bending down to get his head to the ball as the islanders' defence failed to clear an innocuous delivery.
The Black Stars might well have gone ahead with Joseph Paintsil narrowly missing out at the back post after being teed up by Antoine Semenyo.
But after that chances repeatedly fell Cape Verde's way.
Djiku made a goal-line clearance to deny Bebe 20 minutes from time while Rodrigues was twice denied in the last 10 minutes.
The breakthrough came in stoppage time after Mohammed Salisu failed to clear and a clever touch from debutant substitute Gilson Benchimol took it past Ofori and into Rodrigues's path.
Ghana veteran Dede Ayew came on with 28 minutes left to mark his participation in his eighth Cup of Nations tournament, equalling the record held by Cameroon's Rigobert Song and Egyptian Ahmed Hassan.
The earlier Group B game saw Mohamed Salah convert a stoppage-time penalty to spare Egypt's blushes in a 2-2 draw with Mozambique, playing at their first finals since 2010.
Ghana meet Egypt next on Thursday while Cape Verde take on Mozambique on Friday, both at the Felix Houphouet-Boigny Stadium