The four would be expelled and barred from reentering Poland and the rest of Europe's Schengen area, Stanislaw Zaryn, the spokesman for the minister coordinating special forces, said.
"They were trying to recruit people who could be used to commit some kind of terrorist crimes," Zaryn told Reuters.
"They were inspired by ISIS (Islamic State) ... but they were not a part of the organization," he said.
A Lebanese citizen was detained in April accused of trying to organize an Islamist network in Poland and other EU countries.
In 2019 Polish security services arrested two far-right sympathizers accused of planning attacks against Muslims.
Internal Border Guard officers detained the Tajik men on May 7 on the basis of evidence collected by the Internal Security Agency (ABW), Zaryn said in a statement.Special forces are under the control of interior minister Mariusz Kaminski.