In brief
Roman monuments and Romanesque architecture form historical layers that must be distinguished.
Useful to know
Arles is an example of the adaptation of an ancient city to medieval European life. UNESCO includes in the story the Roman theatre, the arena, the underground galleries, the baths of Constantine and the necropolis of Alicecamp. Among the important Romanesque monuments is the Saint Trophimus with the adjacent cloister. [1]
When considering, distinguish "Roman" from "Roman". The first word here refers to antiquity, and the second to a medieval architectural tradition. The similarity of names does not mean simultaneity.
The most interesting question is not which object is the oldest, but how different periods are present in a city. Compare the types of spaces: a venue for performances, an underground infrastructure and a church ensemble. This shows a change in functions and scale without requiring any visible wall to present itself as an intact remnant of the same historical moment.
Sources
For further detail and to check the information: