A short time later, members of the public detained the man before police attended and arrested him.
He has since been taken to Day Street Police Station.
Officers from Sydney City Police Area Command, and various specialist police, are conducting proactive patrols of the area, but it is believed there is no further threat to public safety.
A number of crime scenes have been established and a number of road closures are in place, which is believed to have significant impact on traffic and public transport this afternoon.
Members of the public are urged to avoid the area.