The cycle, and how many times it runs
Freeze-thaw damage does not need a deep freeze; it needs the temperature to cross freezing while the masonry is wet. In this climate that happens repeatedly through late autumn and again through the spring thaw, not just in midwinter. Each crossing that catches water inside the brick does a little more work. The number of cycles, not the coldest night that determines how much damage a winter does, so a mild wet winter can be harder on a chimney than a cold dry one.
Why the chimney and not the wall
Three reasons compound. Exposure: a chimney has weather on four sides and a horizontal top surface, while a wall has a roof over it and is sheltered on one side. Wetting: wind-driven rain reaches faces that vertical rain never touches, and snow sits on the crown instead of running off. And warmth: a wall is heated from inside the building, while a chimney above the roofline is heated only when the appliance runs: an unused chimney never dries out correctly from November to April.
Where it shows first, in order
The crown goes first, since it is horizontal and it takes standing water and snow. Then the mortar joints, which are softer than the brick and lose material from the surface inward. Then the brick faces spall, and once a face has popped off that brick absorbs water far faster than its neighbours, so the damage accelerates locally. By the time you can see missing brick from the ground, the crown above it has commonly been failing for several seasons.
What stops it
Keeping water out, in the right order. A sound crown with an overhang and a drip edge, a cap over the flue, and intact flashing where the stack passes the roof. Then sound mortar joints. Only after all of that is a breathable water repellent useful, and a film-forming sealer applied over masonry that is already taking water traps it inside, where it goes on freezing out of sight and does more damage than doing nothing.