Fully automated drive still lies far ahead in the future. Therefore, vehicles with multiple modes of operation will not disappear fully as many road types, traffic and weather conditions will not allow fully automated drive. Instead, fragmented trips with regard to automation will prevail, where drivers will have different levels of automation available at different times. Given this scenario and the complexity of vehicles offering multiple levels of automation with different driving modes depending on prevailing conditions, the need for drivers to understand their responsibility during the different modes becomes critical. The aim of this paper is to contribute to further understanding of how perceived control influences the driver's mode awareness of and responsibility for the driving task by reporting on an on-road Wizard-of-Oz study under real driving conditions. The results show that when confronted with a vehicle offering both a level 2 and a level 4 driving automation system, drivers have difficulty in determining whether control is allocated to them or to the system. Further results show that perceived control and responsibility for the driving task are closely linked, and that the driver's perception of the driving system influence how they interact with it. Finally, conclusions are drawn regarding the way perceived control influences mode awareness when interacting with a vehicle that features multiple levels of automation.