Chasing 179 they finished 182-4 with three balls to spare against the co-hosts at the R. Premadasa stadium in Colombo.
Ranked 11th in the world, Zimbabwe upset former champions Australia last week.
They also beat Oman and shared the points when their game against Ireland was washed out on Tuesday.
Zimbabwe needed a tricky 65 off 36 balls when captain Sikandar Raza blasted 20 off an over from leg-spinner Dushan Hemantha, including two sixes and a four, to swing the match.
Raza and Brian Bennett added 69 off 40 deliveries for the third wicket and though Raza fell in the 19th over for 45, Zimbabwe eased to their target.
Opener Bennett batted through the innings to be 63 not out.
Needing eight off the final over, Tony Munyonga clubbed Maheesh Theekshana's first ball for six to calm any Zimbabwe nerves before Bennett hit the winning boundary.
"We have trained for every situation. Then we have the right personnel to send in at the right time," said Raza, who was named player of the match.
"We have got all those roles clear and that is why you see the confidence in the changing room."
It was Zimbabwe's second highest successful run chase in T20 internationals.
After opting to bat first, Pathum Nissanka held Sri Lanka’s innings together again.
Nissanka had become the first centurion of the tournament against Australia on Monday and he followed that up with a solid 62 off 47 balls.
It was his seventh fifty in T20 World Cups, equalling Mahela Jayawardene for the most by a Sri Lankan.
Sri Lanka open their Super Eights campaign against England in Kandy on Sunday, while Zimbabwe will face the West Indies in Mumbai on Monday.
