Updating least squares

However, beyond a particular point a further decrease in one type of specification error will typically come at the cost of an increase in another.

For example, suppose a researcher interested in modeling a real-world system S has specified a family F of possible parameterized models M(b) for S, where the parameter vector b ranges over a given parameter set B.

The estimated model corresponding to ordinary least squares (OLS) linear regression with time-invariant coefficient estimates is obtained at the limit point of the REF where RD=0 and RM attains its largest possible REF value.

updating least squares-40

The modeler could then combine this prior probability distribution with a likelihood function Prob(Y|z) to obtain a posterior probability distribution Prob(z|Y) = Prob(Y|z)Prob(z)/Prob(Y) from which to derive some form of Bayes estimate for z, e.g., a MAP (maximum a posteriori) estimate.

An issue here, of course, is that different modelers will typically have different priors that cannot be brought to conformity on the basis of available empirical evidence.

Suppose, also, that a modeler has specified a K-dimensional vector of incompatibility cost functions for measuring the degree of incompatibility between theory and data in accordance with K different goodness-of-fit criteria, where K is greater or equal to 1.

In a series of studies listed in the publications section below, and summarized in Kalaba and Tesfatsion (1996), Bob Kalaba and I develop an extension of the basic FLS/GFLS algorithms to handle this K-dimensional goodness-of-fit problem.

We also derive a recurrence relation for updating the CEF at time t 1 as a function of the CEF at time t together with a K-dimensional vector of incremental incompatibility costs associated with new data obtained between time t and time t 1.

We show that this general K-dimensional CEF construction encompasses a number of other estimation methods.

More precisely, we develop a constructive procedure for the determination of a Cost-Efficient Frontier (CEF) for this problem.

The points along the CEF correspond to the family of all estimated models that are equally efficient with respect to achieving vector-minimalization of these K incompatibility cost functions.

One way to proceed here would be to induce commensurability among specification error terms by taking a Bayesian approach.

A modeler could first associate a prior probability distribution Prob(z) with a vector z =(b,d) consisting of the parameter vector b augmented with a vector d of specification error terms.

Any such fitting will result in conceptually distinct types of specification errors measuring the extent to which conceptually distinct types of theoretical relationships are incompatible with the data set.

Tags: , ,