Robust identification of multivariable regression models
Vojislav Filipovic
Faculty of Mechanical Engineering, University of Kragujevac
Kraljevo, Serbia
In this paper robust (in the statistical sense) identification of multivariable systems with finite impulse response (FIR) is considered. The disturbance has non-Gaussian distribution. Using Huber's concept of min-max estimation we determined nonlinear transformation of prediction error. That introduces nonlinearity in identification algorithm. Also, the identification algorithm carries a priori information about the class of distributions to which belongs the real noise. The analysis of convergence uses the martingale theory and concept of stochastic Lyapunov function. It is shown that strong consistency holds under assumption, representing a special case of the general form of the strictly positive-real condition.
1. Introduction
Estimation algorithms based on Gaussian model have been found to be inefficient when the real distribution belongs to the heavy-tailed probabilities [1].
Considerable efforts have been oriented towards the design of robust estimation algorithm possessing a low sensitivity to distribution changes. The fundamental contribution has been given by Huber [2, 3]. The application of Huber's methodology in different fields is given in [4, 5].
Analysis of robust recursive algorithms is considered in the next author's papers. The paper [6] considers the strong consistency for robust AML algorithm. In that paper the new general form of strictly positive-real condition using passive operator theory has been introduced. The papers [7, 8] consider the adaptive minimum variance controllers for SISO and MIMO systems respectively. In those papers the global stability of adaptive controllers is shown. In the reference [9] the global convergence for a robust adaptive one-step ahead predictor is proved.
In this paper the robust identification of multivariable FIR models is considered. First we propose robust identification algorithms and then prove the strong consistency of estimated parameters.
2. The robust recursive algorithm
Let the system under consideration be described by a linear multi-input, multi-output FIR model with p-dimensional output and r-dimensional input
(1)
, 
, 
where
is matrix
polynomial in the shift-back operator 
. The order of polynomial is m.
(2)
The noise 
is assumed
to be a martingale-difference sequence with respect to a nondecreasing family
of s-algebras
.
The unknown matrix coefficients are
(3)
Model (1) can be rewritten in the form
(4)
Where
, 
(5)
Let us introduce
(6)
where 
stands for
the Kronecker product. Also, a new vector 
is constructed by stacking the
column of the 
matrix. The relation (4) now has the
form
(7)
where
, 
(8)
The algorithm for estimating the unknown parameters can be reduced to the minimization of the next functional [5, 6]
, 
(9)
where
(10)
is the prediction error.
The functional 
depends on
the probability of observations, which is in general, non- Gaussian. From
identification theory [10] it is known that
, 
(11)
where 
is a non-
Gaussian probability density.
From relation (9) we can define the empirical functional
(12)
The recursive minimization of a criterion can be done by using the approximate Newton-Raphson type method [6]
(13)
Moreover, with large k and by virtue of the approximate truth of the optimality conditions, yielding
(14)
One obtains
(15)
(16)
Where
, 
(17)
, , 
(18)
Let us introduce the matrix
(19)
By using the matrix inversion lemma we obtain recursive algorithm
(20)
(21)
, 
, 
(22)
Remark 1. Determination of
matrix M is important for applications of algorithm (20)-(22). Let us
suppose that disturbance 
in model (7) has non-Gaussian
distribution and those components of multidimensional process are
independent, i.e.
(23)
where 
, 
are functions of probability densities of i-th component of
vector .
The Fisher information has the form
(24)
The matrix M is
(25)
Remark 2. If we can use a priori assumption that distribution of real noise lies in specified class of distributions F which are convex and vaguely compact [2, 3], it is possible to construct robust real-time procedure in min-max sense. Members of F are symmetric and contain standard normal distribution N. The two important classes are
a) The gross error model
(26)
b) The Kolmogorov model
(27)
In practice we
usually use the class of distributions described by the relation (26). That is 
-
contaminated model with the mix of two normal distributions
(28)
where
(30)
In that case Fisher information is
(31)
Now the Matrix M has a form
(32)
This form of matrix M is used for implementation of algorithm (20)-(22).
3. Convergence analysis
The convergence property of the proposed robust recursive algorithm can be investigated using the martingale theory [11]. We will first consider the next two lemmas.
Lemma 1. [12, p. 242]. Suppose that the matrix A is partitioned as
Then
Lemma 2. Consider the model (7) and algorithm (20)-(22) subject to the next assumptions
A1: The quantity
satisfies
where 
denotes
the trace of matrix
A2: 
A3: The
vector nonlinear function 
satisfies
Then
, 
, .
Proof: Let us define the matrix A in the partitioned form as
(33)
By using the Lemma 1 we have
(34)
Let us introduce
for
det. From last relation it follows that
(35)
Using (35) one can get
(36)
For inverse
matrix we have 
(37)
From last two relations, one concludes
(38)
From relation (6) we have
(39)
And, also, for matrix M 
(40)
where 
is an
eigenvalue.
By using relation (37)-(40) we have
(41)
For matrix 
holds 
, 
(42)
According to assumption A1 we have
(43)
From last two relations it follows that
(44)
Using assumption A3 of lemma, we also have
(45)
Using relation (41), (44) and (45), one obtains
(46)
Here next facts, based on assumption A1, are used
, 
(47)
, (48)
The proof is completed.
Now the main result of the paper will be formulated.
Theorem. Consider the model (7) and the algorithms (20)-(22) subject to the assumptions of the lemma, and assume further that the following hypotheses are satisfied
H1: 
is a martingale difference with
symmetric distribution 
and
,
H2: The
function 
is
odd and continuous almost everywhere
H3: There exists a passive operator H such that
H4: There
exists a constant 
such that
,
where
denotes
the minimal eigenvalue.
Then
.
Proof. Introducing Lyapunov`s stochastic function
, (49)
We obtain from (20)
(50)
where 
is
prediction error 
.
By using the matrix inversion lemma, (21) can be rewritten as
(51)
From relations (50) it follows that
(52)
Under the hypotheses H2 and H3, one concludes
(53)
By using the definition of the passive operator [13], from H4 it follows that
, 
(54)
Let us define a quantity
, 
(55)
Using (53)-(55)
and since 
,
one concludes
(56)
According to Lemma 2
, (57)
The martingale convergence theorem [11] implies
, 
, (58)
From (58) it follows that
, 
, (59)
Since
(60)
From H4 it follows that
, (61)
This completes the proof.
Remark 3. Assumption H3 is a special case of general passivity conditions which, for single-input, single-output ARMAX model, first appeared in [7]. The condition follows from the theory of passive operators [13].
4. Conclusion
In the paper the robust identification of multivariable FIR models is considered. The proposed algorithm differs from the standard linear algorithms by the insertion of suitable chosen nonlinear transformation of the prediction errors, which has to cut-off the outliers. As a nonlinear function the Huber's function is used. Strong consistency is proved using martingale theory and stochastic Lyapunov function. The condition H3 of Theorem is a special form of a generalized SPR (strict positive real) condition [6].
References
1. V.D.Barnet, T.Levis. Outliers in Statistical Data. John Wiley, New York, 1978.
2. P.J.Huber. Robust estimation of location parameter. Annals of Mathematical Statistics, 35, 1964, 73-101.
3. P.J.Huber, E.M.Ronchetti. Robust Statistics. John Wiley, New York, 2009.
4. C.I.Masreliez, R.D.Martin. Robust Bayesian estimation for the linear models and robustifying the Kalman filter. IEEE Transaction on Automatic Control, 22, 1977, 361-371.
5. Ya.Z.Tsipkin. Foundation of Informational Identification Theory. Nauka, Moscow (in Russian), 1984.
6. V.Z.Filipovic, B.Kovacevic. On robust AML identification algorithms. Automatica, 30, 1994, 1775-1778.
7. V.Z.Filipovic, B.Kovacevic. On robustified adaptive minimum-variance controller. International Journal of Control, 65, 1996, 117-129.
8. V.Z.Filipovic. Stochastic multivariable self-tuning tracker for non-Gaussian Systems. International Journal of Applied Mathematical and Computer Sciences, 15, 2005, 351-357.
9. V.Z.Filipovic. Robust adaptive one-step ahead predictor. IMA Journal of Mathematical Control and Information, 18, 2001, 491-500.
10. L.Ljung, T.Soderstrom. Theory and Practice of Recursive Identification. MIT Press, USA, 1983.
11. W.T.Stout. Almost Sure Convergence. Academic Press, New York, 1974.
12. G.C.Goodwin, R.L.Payne. Dynamic System Identification: Experiment Design and Data Analysis. Academic Press, New York, 1977.
13. C.A.Desoer, M.Vidyasagar. Feedback Systems: Input Output Properties. Academic Press, New York, 1975.
[Contents]
