Some procedures for function approximation based on the use of sample data and their application in heuristic methods for solving practical problems

Let f(x) be a member of a set of functions over a probability space. Samples of f(x) are 2-tuples (xⁱ,f(xⁱ) where xⁱ is a sample of the random variable X and f(xⁱ) is a sample of f(x) at x = xⁱ. Some procedures and analysis are presented for the approximation of such functions by systems of orthonormal functions. The approximations are based on the data samples. The analysis includes the case of error in the measurement of f(xⁱ). The properties of the expected square error in the approximation are examined for a number of different estimators for the coefficients in the expansion and these well-behaved and easily analyzed estimators are compared to those obtained using the method of least squares. The effectiveness of different sets of basis functions, those involved in the Karhunen-Loeve expansion and others, can be compared and an approach is suggested to adaptive basis selection in order to select that basis which is most efficient in approximating the particular function under examination. The connection between results and applications are discussed in the introduction and conclusion.