Supervisory control usually involves timely switching among different courses of action over multiple time scales. In this work, intelligent process supervision is addressed in the context of semi-Markov decision processes and reinforcement learning. Temporally extended actions that represent a way of behaving together with a termination condition are used to achieve a set of operational goals and sub-goals comprising a supervision task. The control strategy resorts to a hierarchy of macro-actions or options which are made up of closed-loop sequences of low-level, primitive actions. Supervisory control of a buffer tank is discussed as a representative example.