Orford is the earliest English castle for which detailed documentary evidence for the construction has survived. These records show that between 1165 and 1173, Henry spent more money on Orford than any other castle. Quickly built upon his accession, Orford was a projection of royal power in East Anglia, where there were no royal castles. Orford castle immediately proved its worth by holding out during the rebellion of Henry's oldest son Henry 'The Young King' in 1173-74. The advanced design of the castle must have acted as a deterrent as it was not directly attacked although fighting did occur nearby.

Though briefly surrendered to Price Louis of France in the war of 1216-17, Orford remained a royal castle. By 1336, it had declined in importance to such an extent that Edward III granted it in perpetuity to Robert de Ufford, Earl of Suffolk. East Anglia's adherence to the Parliamentarian cause saved Orford from damage during the Civil War.