The busy waterway is one of a growing number of flashpoints in the U.S.-China relationship, which include a trade war, U.S. sanctions and Taiwan.
China struck a more aggressive tone in its trade war with the United States on Friday.
The tough talk capped a week that saw China unveil new retaliatory tariffs in response to a U.S. decision to raise its levies on $200 billion of Chinese imports to 25% from 10%.
The U.S. destroyer Preble carried out the operation, a U.S. military spokesman told Reuters.
"Preble sailed within 12 nautical miles of Scarborough Reef in order to challenge excessive maritime claims and preserve access to the waterways as governed by international law," said Commander Clay Doss, a spokesman for the Seventh Fleet.
Speaking in Beijing, Chinese foreign ministry spokesman Lu Kang said the ship had entered waters near the shoal without China's permission, and the Chinese navy had warned it to leave.
"I must stress once again that the U.S. warship's relevant actions have violated China's sovereignty and undermined the peace, security and good order in the relevant sea areas. China is firmly opposed to this," Lu told a daily news briefing.
The United States was trying to disturb regional peace and stability by using the issue of freedom of navigation and flight, he added.
"We strongly urge the United States to immediately stop such provocative actions so as not to undermine Sino-U.S. relations and regional peace and stability."
CONSTRUCTION
It was the second such U.S. military operation in the South China Sea in the last month. On Wednesday, the chief of the U.S. Navy said its freedom of navigation movements in the disputed South China Sea drew more attention than they deserved.
The U.S. military has a long-standing position that its operations are carried out throughout the world, including areas claimed by allies, and they are separate from political considerations.
The operation was the latest attempt to counter what Washington sees as Beijing's efforts to limit freedom of navigation in the strategic waters, where Chinese, Japanese and some Southeast Asian navies operate.
China claims almost all of the strategic South China Sea and frequently lambastes the United States and its allies over naval operations near Chinese-occupied islands.
Brunei, Indonesia, Malaysia, the Philippines, Taiwan and Vietnam have competing claims in the region.
China and the United States have repeatedly traded barbs in the past over what Washington says is Beijing's militarization of the South China Sea by building military installations on artificial islands and reefs.
China defends its construction as necessary for self-defense and says the United States is responsible for ratcheting up tension by sending warships and military planes close to islands Beijing claims.Last month, China's navy chief said freedom of navigation should not be used to infringe upon the rights of other nations.