Samuel Grandsir poked in an opener for the hosts after defensive mistakes from both Saul and Angel Correa.
Diego Costa converted Antoine Griezmann’s excellent pass to level and Jose Gimenez headed in Koke’s corner to give the visitors the lead.
But Atletico, with boss Diego Simeone in the stands, were almost pegged back late on when Kamil Glik headed wide.
The Argentine was serving the final game of a four-match European touchline ban dating back to last season’s Europa League semi-final tie with Arsenal.
Winger Thomas Lemar, a £62m summer signing from Monaco, got a good reception when he came on in the second half for Atletico.
Atletico – whose Wanda Metropolitano ground hosts this season’s final – are looking to avoid a repeat of last year when they went out in the group stages.
In the other game in Group A, Borussia Dortmund beat Club Brugge 1-0 thanks to Christian Pulisic’s fortunate late winner on his 20th birthday. Dortmund’s English teenager Jadon Sancho made his Champions League debut in the match.
Jose Gimenez got what proved to be the winning goal in France in a closely-fought match
The defender rose highest under the challenge of Radamel Falaco from a corner to score
Atletico boss Diego Simeone watched the comeback Group A win from the stands in France