Johnson, who is pushing for a snap election, was speaking to a BBC television crew in the northern city of Leeds on Thursday when a man pushing a child in a stroller interrupted the interview.
"You're playing games with parliament. You're playing games with the public," the man shouted at Johnson. "You should be in Brussels negotiating. Where's the negotiation going on? Where is it? You're in Morley in Leeds. You should be in Brussels."
Johnson was initially unable to respond before challenging the man - who spoke with an Irish accent - when he said the government's negotiations with the European Union about a new Brexit deal were going nowhere.
"Actually that's not true at all. We are on the verge of getting a deal," Johnson said before repeating his criticism of the opposition Labour Party for not agreeing immediately to his demand for an early election.
A crowd gathered and some people began to argue with Johnson's challenger.
Johnson turned to the onlookers and said: "Can I just ask people here: do you think we should get out on October 31st?"
Some people shouted: "Yes!" and cheered. Others shouted: "No!" Johnson got another mixed response when he asked the crowd if he thought Labour's leader, Jeremy Corbyn, was standing in the way of Brexit.
Johnson was given a clearer message in a separate encounter with a member of the public during his trip to Leeds.
Video footage showed the prime minister shaking hands with a man on a street who smiled before telling him with a smile: "Please leave my town.""I will very soon," Johnson replied.