The key to this type of photography is to stop looking at buildings as whole structures and focus on the small pockets of interesting patterns and shapes they're made up of. Don't overlook the ugly looking buildings either as most of the time, once you're zoomed in and focused on one part, you'll find they're perfect subjects when you're on the hunt for architectural patterns. If you can't zoom in or use your feet to help you get a frame-filling shot, capture the image any way as you may be able to crop out the distractions in your editing software once back home.
If you find people or clutter are getting in your way move your feet to find a clearer angle or try taking your eyes away from street level all together – you'll be surprised at what you see. Just watch your shutter speeds if you're working hand-held and make sure you're focusing accurately as you don't want a blurry shot.
Don't be afraid of cloudy, damp days as these can give your shots a moody feel while a bright blue sky will contrast well against a strong metal frame.
Some ideas to try:
Old vs new – Find points where old architecture meets and merges with the new
Broken/cracked windows – The shapes formed in the glass make interesting patterns
Metal structures – Fantastic strong, bold and often symmetrical shapes