There has been a church on this site since Saxon times. Roger de Newburgh, the 2nd Norman Earl of Warwick, rebuilt the original building when he completed the foundation of a college at St. Mary's in 1123. During the medieval period, St. Mary's had many relics including a fragment of Christ's crucifixion cross.

The church stood until September 5, 1694 when a fire raged through the town and destroyed over 450 houses within five hours. Townsfolk, seeking refuge from the fire ran to the church, some of them bringing their smoldering belongings. Disastrously, the church caught fire. The remains of the church had to be demolished. Rebuilding began in 1697 and was mostly completed by 1704. The architect chosen for the job was Sir William Wilson although Sir Christopher Wren is reputed to have advised on the eventual placing of the tower. The current building represents architectural styles from the 12th to the 20th century.