The men did not respond when asked who they were by reporters. Earlier, the same group of men was seen padlocking a door on another side of the building.
After the men went inside, two uniformed members of the U.S. State Department's Bureau of Diplomatic Security arrived to guard the door. They also did not respond to questions from reporters.
The Chinese embassy in Washington and the U.S. State Department did not immediately respond to requests for comment on this activity.
According to a Reuters witness, consulate staff had exited the building shortly after 4 p.m. and left in vehicles before the back door was forced open.