Hundreds of students from madrasas, or Islamic religious schools, clashed with police and border troops in the eastern Brahmanbaria district. Police said they had to open fire to control the violence.
Many Islamist groups in Bangladesh accuse Modi of alienating minority Muslims in Hindu-majority India. The two countries issued a joint statement celebrating their cooperation and partnership but the Bangladesh government made no comment about the protests.
"We received three bullet-hit dead bodies and two others succumbed to their injuries later," Abdullah Al Mamun, a doctor at the government-owned Brahmanbaria General Hospital, told Reuters.
A local police officer confirmed five had died but declined to be named as he was not authorized to speak to media. Bangladeshi police did not officially confirm the death toll.
Protests raged across Bangladesh against Modi but also over the police killing of Islamists who had demonstrated against his two-day visit.
Four supporters of the Islamist group Hefazat-e-Islam were killed on Friday after police opened fire when protesters allegedly attacked a police station in the southeastern town of Chittagong.
Dozens were also hurt in the capital Dhaka on Friday when police used rubber bullets and teargas in clashes with protesters.
On Saturday, hundreds of members of Hefazat-e-Islam and other Islamist groups marched through Chittagong and Dhaka protesting the deaths of their supporters.
"Police opened fire on our peaceful supporters," the group's organizing secretary Azizul Haque told a rally in Chittagong. "We will not let the blood of our brothers go in vain."
POLICE ACTION
Hefazat-e-Islam, which translates as Protection of Islam, has called for a nationwide strike on Sunday to protest the killings.
Non-profit group Amnesty International also criticized the police action in Chittagong.
"The right to peaceful protest has come under concerted attack, particularly during the coronavirus pandemic, culminating in this type of bloody repression," Sultan Mohammed Zakaria, Amnesty's south Asia researcher, said in a statement.
Modi landed in Dhaka on Friday, his first international trip since the outbreak of the coronavirus pandemic last year, to celebrate the 50th anniversary of Bangladesh's independence.
He left the country on Saturday after holding talks with Bangladesh Prime Minister Sheikh Hasina and gifting the country 1.2 million COVID-19 vaccine shots.
Facebook services were unavailable in Bangladesh on Saturday, the social network said, adding it had serious concerns about the manner in which it was being restricted at a time when effective communication was necessary to tackle the coronavirus pandemic.
"We're aware that our services have been restricted in Bangladesh," Facebook said in a statement. "We're working to understand more and hope to have full access restored as soon as possible."The Bangladesh government did not comment on whether it had blocked Facebook and its Messenger app, but it has previously used internet shutdowns as a tool to curb the spread of protests.