Role of Dependence in Chance-constrained and Robust Programming

The paper deals with two methods of solving optimization programs where uncertainties occur: stochastic (in particular chance-constrained) programming and robust programming. We review briefly how these two methods deal with uncertainty and what approximations are commonly used. Furthermore, we are concentrated on approximations based on sample sets where some type of weak dependence occurs. We demonstrate that such kind of dependence does not imply any important malfunction of optimization methods used there. Numerical illustration on simple optimization program is given.


Introduction
Mathematical programming, a discipline covering methods that try to select the "best" solution is widely used by economic and engineering researchers and practitioners.The breakthrough was certainly the development of the simplex method that served a very useful purpose on solving linear optimization problems, by G. B. Dantzig (1947) -he provided an instrument for automated searching of solutions of such problems and also its applications to solve real-world problems.Methods of linear and (later developed) nonlinear programming deal with deterministic problems.But uncertainty is a natural property of all the real-world models.In the early paper, Dantzig (1955) himself realized very large importance of uncertainty in optimization.Sometimes, unknown parameters can be replaced without worries (approximated with some "good" estimates), but otherwise such simplification is not possible and could lead to false conclusions.

Origin of the uncertainty in optimization models
Consider an optimization problem of the form; Such a frame can be followed for a great variety of decision-making problems but different optimization methods have to be used according to the nature of structural and data elements of the problem.We extend the description of the problem by the following characterization: 1. the knowledge of the data is insufficient, all that is known about  is that it belongs to a given uncertainty set s R   ; 2. the objective is required to be the best possible given an actual realization of  ; 3. the constraints of the problem (1) are required to be satisfied as much as possible given the actual realization of  .


The article is processed as an output of the research projects Nonlinear Dynamical Economic Systems: Theory and Applications and Economic Systems Under Uncertainty: Optimization and Approximation, registered by the Czech Science Foundation under the registration numbers 402/03/H057 and 402/07/1113.
If a realization of  is known and fixed in advance (before the decision has to be taken), and the problem elements have suitable properties, standard deterministic optimization algorithms can be used to solve the problem (1).This is rarely the case; in practice, uncertainty of the data is to be taken into account.For example,  the data  is not known at the time when a decision has to be taken and it is observed later in the future;  the data  cannot be measured or estimated exactly even if it is realized before a decision is taken;  the solution of the problem cannot be implemented exactly due to its physical characterization -this can be transformed to the model as uncertainty of the data (not uncertainty of the solution);  the model itself is an approximation of a real problem.

Dealing with uncertainty of the data
During the history of mathematical programming, several approaches were developed to deal with uncertainty in optimization problems.First, the uncertainty is simply ignored, a fixed (nominal) value of the data is chosen and the problem is solved.The accuracy of the optimal solution is examined ex-post by the traditional sensitivity analysis.The usability of this kind of stability analysis is limited to a single, already generated solution; and, unfortunately, these solutions often happen to be very sensitive to perturbations of  .
If we declare  to be a random vector with known distribution, we can incorporate such information into the model -this is what stochastic programming does.The idea is to replace uncertainty of the data by known probability distribution and solve the newly created, deterministic optimization problem.There are many ways how to do that; among them, the most popular are  penalization/recourse problems: the potential loss due to the uncertainty is compensated ex-post by solving another optimization program that depends on the first-stage decision x and the (realized) data ; in the first-stage program, the expected value of the (uncertain) second-stage optimal value is taken into account;  chance-constrained programming problems: the constraints of the original problem are not required to be satisfied in all cases but "only" with enough high probability.
Completely different approach to solve the problem (1) is the "worst-case" analysis, also known as robust programming.The key is to find a solution that satisfies the constraints of (1) for all possible realization of  belonging to some given set  .
(In real applications, this set has not to be identical with the uncertainty set in (1), but the distinction is not important here and we shall use the same symbol  for both meanings.)The framework of the worst-case analysis was introduced by Ben-Tal and Nemirovski (1998) and developed by other authors in various directions, see e. g. (El Ghaoui -Oustry -Lebret, 1998).

Uncertain, stochastic and robust programming problems
Let us start with the following family of optimization programs (1) parameterized by where n X R  is convex and closed set.Without loss of generality, we assume that the objective is linear (non-linear objective can be transferred to the constraint part of the problem by minimizing t subject to Conditions on P and f under which the set  X is convex are treated in the literature (Prékopa, 1995).Computational issues arise when we want to compute the probability measure in the definition of  X because it involves multidimensional integrals.For that reason, we are interested in approximations of the program (3), in particular by a sampled version (see next parts of the paper).
In the robust convex program we look for a solution of (2) that is feasible for all instances of The program (4) has infinite number of constraints and some relaxation techniques are needed in order to solve the problem.In the next section, we concentrate on the "randomized" approach to approximate the problem (4).

Empirical approximations of optimization problems
where   denotes the Dirac measure placing the unit mass at  .For a given realization of samples, the empirical distribution function possesses the properties of distribution functions and, under rather general conditions, converges almost surely to the distribution function of the original distribution P. We are now ready to use the empirical distribution function as an approximation of P in the problem (3); we obtain the following "sampled" chance-constrained problem: The essential idea of such approximation is that the relative frequency of constraint violations in ( 2) is approximately the desired level of infeasibility in (3).Under suitable assumptions, convergence properties of the program (6) are assured; see (Römisch, 2003).
The robust programming problem does not involve any information about stochastic nature of the uncertainty parameter  in its original form (4). Calafiore and Campi (2005) and de Farias and Van Roy (2004) proposed so-called randomized approach considering a set of independent samples from a chosen distribution P, and so put back the stochastic nature of the uncertainty.The robust sampled convex problem reads: The idea of the problem ( 7) is relaxing the original problem (4) and requiring the constraints to be satisfied not for all the possible realizations of  , but for a sufficiently high number of samples which are in addition most probably to happen.
Properties of the problems ( 6) and ( 7) are deeply examined in (Houda, 2006a), (Houda, 2006b); we demonstrated that both the programs ( 6) and (7) (and/or their original non-sampled versions (3) and ( 4)) can lead to very different solutions although they are solving mathematically the same optimization problem (1).In this paper we continue in this direction and take into account dependence properties that sample sets may acquire.

Financial application of stochastic and robust programming
Stochastic programming has a long history and therefore a great number of applications in many scientific areas.Typical applications of chance-constrained programming include financial problems; constraint inequalities involve uncertainties in the model parameters like product demand, currency exchange rates, etc.The other typical property of chance-constrained models is that infeasibility in constraints may not be expressed in terms of "compensation costs", as is typical for models from another area of stochastic programming (problems with recourse).To illustrate one of possible applications, we give an example of the portfolio selection (cash matching) model (Dentcheva -Lai -Ruszczyński, 2004): the objective is to find an optimal portfolio of financial instruments that maximizes the final amount of money and ensures that all requested payments during the considered period is made.The uncertainty of the model (random yields and/or payments) is contained in liquidity constraints and we require that the constraints are satisfied with a sufficiently high probability.
Robust programming is recently used in many application contexts (truss design, reliability systems, filtering, etc.).As an example of a finance application we mention the robust portfolio selection problem (Goldfarb -Iyengar, 2003).In this framework, we consider perturbations in the market parameters.The uncertain parameters are considered bounded and the optimization problem is solved assuming worst-case behavior of the perturbations.

Example: simple optimization program and dependent samples
We adopt the settings of (Houda, 2006a) and consider the following simple uncertain convex program: where  is a random variable distributed according to standard normal distribution N(0;1).The distribution function will be denoted by F. Following the direction outlined in previous sections, the chance-constrained program ( 9) and the approximated versions of chance-constrained (10) and robust program (11) of the case problem ( 8) are defined by: The robust program is not well defined here -the support R   is unbounded and there is no solution x that is feasible for all instances of  .
The solution of ( 9) is easy to calculate; with 05 .0   it is the 0.95-quantile of the standard normal distribution, which is approximately 64 . 1  x .As shown in (Houda, 2006a) the optimal solution of (10) approximates this value (more the number N of samples grows, better is the approximation) rather than the optimal solution of (11) which tends to "beat" this value (more the number N of samples grows, more the probability not to exceed the value is smaller).We will now take into account two types of dependence and compare the same results obtained with dependent and with independent sets of samples.
In the sequel we replace in (10) and (11) the set of independent samples by another one of samples that are weakly dependent.We take into consideration two particular series of samples: 1. moving-average process MA(2), defined by 2. autoregressive process AR(1), defined by 1 5 .0 : The series  , , 2 1   in (12) has independent samples with the same normal distribution with zero mean and variance 1/0.38 so the variance of k  is equal 1 by convolution.In the case (13), the members of series  , , 2 1   have the same normal distribution with zero mean and variance 0.75 so the variance of k  is equal 1 again.
Moving-average process MA(2) is a special case of so-called m-dependent processes, here for m=3.A process } { k  is m-dependent if for any k the  -algebra generated by the events where A, B are events belonging respectively to the two  -algebras defined above.m-dependent sequences as well as strong mixing sequences have suitable convergence properties and are of best interest to deal with them along the independent sequences.For theoretical background of weakly dependent sequences see for example (Yoshihara, 1992).
Numerical results are shown in the array of graphics (Figure 1 and 2) for various types of dependence and sample sizes.In each of the graphs, a short tick mark on the xaxis represents the optimal solution of the chance-constrained problem (9), dotted lines are estimated densities of the optimal solutions of the approximated chance-constrained problem (10), and solid lines represent optimal solutions of the robust sampled problem (11).The sampling procedure is repeated 200 times in order to estimate densities of optimal solutions in (10) and (11).
Convergence properties and differences between chance-constrained and robust programming were discussed in (Houda, 2006a) from a mathematical and in (Houda, 2006b) from an application point of view.We concentrate here on comparison concerning influence of weak dependence of sample sets.
Based on our expectation and conclusions of the theory of weakly dependent processes, such kind of dependence does not introduce any remarkable issue.With small sample sizes (30 samples, less with 300 samples) the only observable fact is the slightly augmented variance of approximated solutions, but importance of this weakness diminishes with growing number of samples at disposition.The important fact is that basic convergence properties of approximated stochastic as well as robust programs remain valid even if we consider some kind of weak dependence of sample sets instead of independent samples.This has a very large impact on practical applications in operation research because data series that we usually have at hand are rarely pure independent but some kind of weak dependence is frequent.The theoretical confirmation of this experimental fact remains still an open question.

Conclusion
In the paper we have concentrated on special cases of approximations of uncertain convex program based on stochastic programming on one hand and on robust programming on the other hand.We took into consideration possible dependence of samples that one has at hand and demonstrated that this fact should not become an obstacle, especially if the number of samples is high.More consideration should be devoted to the correct choice of the optimization method because stochastic and robust programming lead to very different solutions although they are both solving the same original optimization program.
distributed according to P and define the empirical distribution function -a discrete random variable of the form

Figure 1 : 5 AR( 1 ) 5 AR( 1 ) 5 MA( 2 ) 5 MA( 2 )Figure 2 : 5 AR( 1 ) 5 MA( 2 )
Figure 1: Dependent and independent samples in optimization parameterized program is known as uncertain convex program, and if the realization of  is known and fixed, we can use standard convex programming tools to solve the problem.