Bricks are porous. This makes it easy for water to seeps into the cracks and tiny holes. Once temperature drop water expands as it freezes thus causing further cracks and larger holes. 
The mortar that holds the bricks together soaks in a larger amount of water. Overtime, this will erode the mortar thus loosening the bricks.