One of the most beautiful and most famous sites in the Val d'Orcia is St. Anthimus's Abbey, Abbazia di Sant'Antimo in Italian, a Benedictine monastery that dates back to the Middle Ages.
(Here's the link to the abbey's official site: http://www.antimo.it/.)
To visit the abbey is to return back in time. If you consult the official schedule of the abbey, you can visit during the services in which the friars still use ancient Gregorian chants in Latin. It is one of the most inspiring experiences of any sojourn in the Val d'Orcia.
This detail above from the abbey reveals its Romanic origins.