It comes after British lawmakers across the political spectrum voiced strong opposition to the plans this week.
Speaking on the BBC's "The Andrew Marr Show" on Sunday, the health secretary said he "never liked the idea" of forcing people to show their papers in everyday activities, but the government was right to look at the evidence.
"What I can say is that we've looked at it properly and whilst we should keep it in reserve as a potential option, I'm pleased to say that we will not be going ahead with plans for vaccine passports," Javid said.
The health secretary added that the government shouldn't be doing things for the sake of it or because others are doing them.
"So many countries, at the time they implemented it, was to try and boost their vaccination rates and you can understand why they might have done that," he said.
Javid said that England has so far been "very successful" with its vaccination rates, with 55 percent of 16- to 17- year-olds having had their first doses only a month after the jabs were offered to this age group.
Shortly before his announcement on the BBC, the health secretary said the government hadn't made a final decision on domestic vaccine passport in a separate interview with Sky News. He also said that he wants to get rid of PCR tests for international travel "as soon as [he] possibly can."