In this paper the problem of least-squares (LS) identification of ARMAX models is investigated from a new point of view. An efficient scheme for estimating the noise-induced bias in the LS parameter is introduced by exploiting the unique structure of the ARMAX model and utilising extra delayed outputs. Then a new type of LS based method is developed in combination with the bias correction technique. The proposed method makes no use of a prefilter and deals directly with the underlying ARMAX model. The important characteristics of the proposed method includes desired computational efficiency and superior estimation accuracy. The behaviour of the proposed LS based method is also substantiated using simulation data while in comparison with other identification methods.