Washington (AFP) - Islamic State fighters have streamed into Libya in recent months, a US official said Thursday, heightening fears the extremists are gaining ground and influence in the north African country.

About 5,000 IS jihadists are now in Libya, the defense official said, approximately double earlier estimates, while the number of IS extremists in Iraq and Syria has dropped.

The updated tallies come as the administration of President Barack Obama faces growing calls for the US military to step up action against the IS group in Libya, where the jihadists have already seized the city of Sirte and an adjoining length of Mediterranean coastline.

NATO defense ministers are meeting in Brussels next week to evaluate the ongoing US-led coalition campaign against the IS group and to discuss ways of redoubling efforts.

Patrick Skinner, a former CIA case officer now with The Soufan Group consultancy, said the United States has few good Libya options, but there is growing consensus something must be done.

"They are just terrified of it getting much worse, fast," Skinner told AFP.

"Once the Islamic State takes something, it's really hard and really bloody and really expensive to take it back."

The United States now believes there to be between 19,000 and 25,000 IS fighters in Iraq and Syria -- the group's so-called "caliphate" -- down from a previous count of 20,000-33,000.

But there are growing fears about the fate of Libya, which has been in chaos since the NATO-backed ouster of longtime dictator Moamer Kadhafi in 2011.

Skinner pointed to the irony of Western leaders now scrambling for solutions in Libya.