The man, believed to be in his mid-40s was stopped by armed officers, the Metropolitan Police said.
He was detained on suspicion of carrying an offensive weapon.
Photos from witnesses on Twitter appeared to show several officers surrounding the man near the Institute of Contemporary Arts building.
The man has been taken to Charing Cross Police Station for further questioning. The Met Police said that the incident was not being treated as terror related.
The are no reports of any injuries sustained during the incident.
Anyone with any information is asked to contact police on 101 or Tweet @MetCC quoting CAD 6041/13APR.
To remain 100% anonymous call the independent charity Crimestoppers on 0800 555 111 or visit Crimestoppers_uk.org.