This paper proposes a behavior based scheme for high level control of autonomous robots. Two main characteristics can be highlighted in the control scheme. Behavior coordination is done through a hybrid methodology, which takes in advantages of the robustness and modularity in competitive approaches, as well as optimized trajectories in cooperative ones. As a second feature, behavior state action mapping is learnt by means of Reinforcement Learning, RL. A new continuous approach of the Qlearning algorithm, implemented with a multilayer neural network, is used. The behavior based scheme attempts to fulfill simple missions in which several behaviors or tasks compete for the vehicle control. This paper is centered in the RL based behaviors. In order to test the feasibility of the proposed Neural Qlearning scheme, real experiments with the underwater robot ODIN in a target following behavior were done. Results showed the convergence of the behavior into an optimal state action mapping. Discussion about the proposed approach is given, as well as an overall description of the high level control scheme.