Arpaio, 88, who billed himself as "America's toughest sheriff," trailed his former chief deputy, Jerry Sheridan, by 6,280 votes out of 443,056 ballots cast in Tuesday's four-way Republican primary, according to the county elections department.
The latest results showed Sheridan with 156,396 votes, compared with 150,116 for Arpaio, leaving the former long-time sheriff of Maricopa County no chance of closing the gap with just 2,385 ballots still to be counted.
In the November general election, Sheridan will face incumbent Democrat Paul Penzone, who ousted Arpaio from office in a 2016 landslide victory.
Arpaio, damaged by a series of court judgments that cost local taxpayers more than $178 million to date, went on to lose a race two years later to fill the seat of the late Republican U.S. Senator John McCain.
A federal judge in 2017 found Arpaio guilty of criminal contempt of court, ruling that the sheriff had willfully violated a 2011 injunction barring his officers from stopping and detaining Latino motorists solely on suspicion that they were in the country illegally.
Trump granted Arpaio clemency before he was to be sentenced, marking the first pardon of his presidency. Sheridan also was cited for contempt by the same judge, but was not prosecuted.
In his comeback bid this year, Arpaio vowed to renew controversial policing tactics he which includes Phoenix. Those policies included housing county jail inmates in tents and regular immigration enforcement sweeps that landed Arpaio in hot water with the federal courts.
Political strategists were watching the Arpaio race for signs of how receptive his get-tough messaging on immigration and law and order - echoing Trump’s - would be to voters in a crucial swing state for the Nov. 3 national elections.
Trump carried Arizona by 5 percentage points in 2016.