Evaluating Institutional Performance: An Ex Post Analysis of Water Law in Colombia

In recent years, there have been many changes in environmental law and policy around the globe targeted at improving both economic and environmental conditions. Policy makers and research analysts in many countries are interested in evaluating the success of these changes. Ex-post analysis may be useful in assessing economic and environmental performance. However, the methodology for this type of analysis is still developing. This paper presents one method for performing ex-post analysis using the case of recent changes in water law in the country of Colombia.


I. Introduction
In countries around the world, the ability to perform ex post analyses of the impact of institutions (laws and policies) can be extremely useful in understanding important sectors of the economy.In many countries, relatively recent innovations in environmental institutions are of particular interest.Policy makers and research analysts would like to determine the degree to which past changes in environmental policy have been successful.This information would, in turn, help them make wise decisions in future.
Ideally, when policy makers design an innovation in policy, the plan should include an explicit method for evaluating the final results.The ability to assess the efficacy of existing institutions may suggest changes that can be made to improve the efficiency of natural resource and environmental use in a way that increases prosperity and sustainability.The question becomes what method of analysis can be used for ex-post analysis?
In general, the methodology used to perform an ex-post analysis is fairly undeveloped.Examining institutional performance is a relatively new theme in environmental economics but there have been various attempts at ex post analysis.The approaches vary in scale and precision.They range from in depth single case studies to cross country comparisons with many observations.In the particular area of water resource institutions, the relatively recent study conducted by Saleth and Dinar (2004) may be the most sophisticated and quantitative ex post analysis to date.
Because the impact of institutions is so pervasive and because institutions tend to change over long periods of time, it is notoriously challenging to design an ex post analysis that is empirically rigorous.Analysts invariably struggle with observation and measurement techniques to test cause and effect relationships.The nature of institutions does not usually allow researchers the ability to apply a strict positive methodology, meaning experiments would be conducted and individual institutional variables would be manipulated to determine measurable differential impact.
The traditional scientific method includes the multiple steps including 1) observation of worldly phenomenon, 2) induction, where observed patterns are generalized, 3) statement of hypotheses about cause and effect relationship, 4) deduction, wherein the general Background information on Colombia's water resources and institutions were gathered in connection with the Colorado State University's "Proyecto Ley del Agua Colombia" as part of Japanese Grant No. TF051886.The methodological discussion of ex post analysis presented here was developed as part of the Czech project entitled "Developing a Methodology for Ex-post Evaluation of Impact of Institutional Change and Policy Reforms on Environmental Performance" conducted at the Department of Environmental Economics in Prague and supported by The Czech Grant Agency (Project No. GA 402/06/0806).
statement is used to predict results in a particular case, and 5) testing the hypothesis by comparing observations to predicted outcomes.
The methodology used for ex post analysis (and for most economic analyses in general) is based on the second half of the scientific method, meaning it is based on logical deduction and explanatory power.That is, the research question is usually centered around "if -then" hypotheses.Regardless of the scale of ex post analyses, they must rest on a conceptual basis that reflects hypotheses about the cause and effect relationships between institutions and economic performance.In other words, what does a specific model suggest in terms of the expected results under particular circumstances and do the observable facts in a particular case bear those supposed results out?How much of the observed phenomenon can be explained by a particular model?By extension the better model is the one with the greatest explanatory power.This paper presents an ex post analysis of environmental performance, specifically in the area of water resources.Wise use of water resources is central to natural resource policy in many countries around the world.Law and policy makers in various countries, regions, and river basin face the challenging and very important task of reforming the configuration of institutions that govern water resources in order to meet national economic and social goals.Modernizing water institutions in a way that allows for more rational, efficient and environmentally sensitive water use is critical to the long run economic development of most countries.This article will demonstrate how institutional analyses can help to explain the past performance of water institutions and serve for formulating suggestions for future institutional transformations, specifically for the country of Colombia.
Colombia is a developing country in South American that is rich in water resources but arguably very poor in resource management (Grigg et. al, 2004).There are serious problems in terms of both water quantity and quality for the people of Colombia.Over 50% of the population faces basic access problems during normal conditions; over 80% suffer access problems under drought conditions.Most municipal and industrial sewage is untreated and many rivers, including those that serve the major city of Bogota carry extreme levels of chemical and biological pollution.There are significant hospitalizations due to waterborne disease.Declining water quality, combined with rapid deforestation, is causing significant environmental degradation and the loss of species in a country with rich biodiversity.As a result of these problems, there are serious equity issues between rural and urban users.
The objectives of this paper are two fold.The first objective is to outline some basic principles that are useful in evaluating environmental institutions.The second objective is to perform an ex post analysis, using the single case of Colombia by applying the basic principles to the existing situation in Colombia.Economics, as a discipline, has much to contribute to the discussion of principles that can be used as a framework for thinking about the impact of institutions on environmental performance.They may be used as a foundation for analyzing strengths and weaknesses in the existing institutional framework as well as proposals for reform.

II. Understanding Institutions
It is important to understand the meaning of the term "institution" as it is used in economics in general and in this ex post analysis.Institutions have been defined as "a set of behavioral rules that govern a particular pattern of action and relationships" (Binswanger and Ruttan (1978, p. 329).D. W. Bromley (1982, p. 11) suggests that institutions "control and liberate humans in their dealings with each other." A. A. Schmid (1972, p. 893) says "insti-tutions are sets of ordered relationships among people which define their rights, exposure to rights of others, privileges and responsibilities."Within this general categorization one may distinguish between three levels of institutions: 1) informal institutions including the cultural values, more and religions active in society, 2) the laws and regulations that constitute formal government and 3) contractual arrangements which are used to conduct transactions (Adelman and Head, 1983).
This ex post analysis of water management in Colombia, like most economic analyses of environmental and natural resource management focuses solely on the second level of institutional arrangements.Specifically, institutions are the formal laws, rules and policies governing water allocation and use.Institutions set the "rules of the game" within which individuals and agencies make decisions about water use.
Institutions are "nested" at several levels (Saleth and Dinar, 2004).For example, water resource use in the United States is governed, at the highest level, by the U.S. Constitution, which includes the power (via the Commerce Clause) to allocate water between states.Each of the states has the power to legislate laws governing water within their particular states, and various agencies (e.g.Department of Natural Resources, Department of Agriculture, and Department of Health) develop policies and regulations to carry out and enforce the laws of the states.
By establishing who can claim income from resource use and who is responsible for various costs, institutions establish the incentives that are so important in determining economic performance.Formal institutions define choice domains for individuals, firms and groups.Given a range of choice and the preferences of individual actors, certain behaviors arise.And finally the sum total of all behavior determines the amount, type and distribution of economic activity.This is sometimes referred to as the "structure -conduct -performance" model of impact.(See Saleth and Dinar, p. 33-36 for a good discussion of these basic connections and how they tie to institutional change.)The structure -conduct-performance model was first developed to explain how a particular market structure influenced firm behavior and the resulting economic outcomes.The model has been extended to address institutions in general.Institutions determine the structure of choices people face, individuals decide on their behavior (conduct) and aggregate behavior determines overall performance.
The purpose of this ex post analysis is to determine the extent to which our conceptual understanding of institutional arrangements and their impact on environmental performance can explain actual circumstances in Colombian water resources management.In order to facilitate the ex post analysis, it is useful to outline some basic economic principles that stem from an understanding of the way institutions effect structure, conduct and performance.These five principles outlined will be used to examine circumstances in Colombia.
First, resource institutions must reflect the actual boundaries of economic impact when the resource is used, and resource rules must be integrated with related institutions in order to prevent contradictory incentives.Second, legislative and regulatory authorities must decide the range in which markets will be allowed to operate and in what ways the market will be constrained.Thirdly, institutions must be carefully crafted so that accurate signals are sent to the resource user concerning the actual benefits and costs of their actions.Fourthly, policymakers must be cognizant of the distributional implications of an institutional system, i.e., the incidence of benefits and costs on various interest groups.And finally, institutions should be designed so that the costs of actually administering and enforcing the rules is low (otherwise, management costs will at into efficiency gains. In the following sections, actual resource and institutional conditions in Colombia are compared to the institutional design principles outlined above.The method suggests that this comparison can generate insight into how well, or how poorly, water institutions are functioning.In each section, the principle is used to analyze the existing conditions in Colombia.This analysis relies on background information gathered from a report entitled "Assessment of Colombia's National Environmental System" completed by Resources for the Future (2004).A bibliography of economic literature that informs the analysis is included at the end of the paper.

III. Aligning Institutions: Boundaries, Coordination and Consistency
Because water resources are re-usable and mobile, water users are tied together in ways that users of other resources are not.These physical, hydrological connections must be recognized in designing rational water institutions.When upstream use has downstream impacts, water institutions should incorporate this consideration.Where surface and groundwater are connected, water institutions must reflect that connection.The basic principle in designing rational water resource institutions is that the full range of impacts (benefits and costs) of water allocation and use must be considered.
From an economic perspective, water management institutions should be designed with a river basin perspective in mind.Otherwise, the economic benefits or costs imposed by particular users might be ignored, with serious negative economic consequences.Water institutions designed on a smaller scale may miss important connections within the river basin.This principle does not suggest that one particular entity manage the river basin.Rather, it suggests that the fundamental laws in the country must be structured with the river basin perspective in mind in a way that enables decentralized decision making under more centralized rules.
Almost always, water institutions are "nested" at several levels.For example, water policy may be connected to agricultural policy, which is turn affected by food security issues, which are partially determined by macroeconomic policies.One of the most difficult institutional principles to achieve is coordination among related agencies.Conflicting regulations certainly frustrate wise water management.
Currently in Colombia, the Ministerio de Ambiente, Viviendas, y Desarrollo Territorial (MAVDT) is responsible for environmental policy and the Regional Autonomous Corporations (CARs) and Urban Environmental Authorities (AAUs) are charged with implementing policy.In analyzing the current situation in Colombia, it appears that the regional CARs may constitute a very rough match with hydrological boundaries.However, this is difficult to assess, especially given the lack of hydrologic data within the country.Prior to Law 99, there was very incomplete coverage with only one quarter of the country represented by CARs.In cases where the regional CAR boundary does not reflect actual hydrologic connections and impacts, economists would expect problems with policies that ignore impacts outside the institutional boundary.
Based on the RFF report, Columbian policymakers and water managers clearly recognize the need for greater integration of water institutions in the country.The lack of integration and coordination between the MAVDT and the CARs is cited as a significant problem.Potential conflict stemming from the merger of the Ministry of Environment and the Ministry of Development is also a concern.Apparently, providing a coordination mechanism was one of the objectives behind the development of Law 99 and the National Environmental System (SINA).While some progress has been made, there is still a significant problem and the country could benefit from greater coordination.

IV. Prioritizing the Multiple Uses of Water: Competition and Constraints
Maximizing the social welfare derived from water resources depends on allocating water among uses in a way that recognizes the differential value of water in alternative uses.This means that when water is very scarce, only the highest valued uses should receive allocation.In a fully developed system, this might be achieved through direct competition between users and uses, where individuals "bid" for water, thus revealing the value in a particular use.Over time, water is reallocated among uses when short run or long run transfers between uses are made.Economists like to imagine a world where all transfers are accomplished through markets, but given the complexities inherent in water resource management, government agencies are usually involved in significant transfers.
Within traditional economic uses (e.g.domestic consumption, agriculture, industry) it is generally recognized that direct consumption by individuals carries a very high social value.Of course, water for consumption is much more valuable that water used for bathing, washing clothing, etc.Within agriculture, there is a range of values for water, largely dependent on the value (price) of the crop grown.In addition, permanent crops like orchards typically carrying much greater value than crops that can be grown on a yearly basis.
A serious problem with evaluating the relative value of water in alternative uses is that some very important and valuable uses of water do not carry direct market value.Many environmental uses are in this category.Some kinds of environmental values for water (e.g.recreational boating) can be estimated if, in fact, people pay for the activity (an entrance fee, etc.).However, some of the highest environmental values, like preserving bio-diversity, do not carry market prices.These values must be estimated and integrated into decisions about water allocation.Otherwise, there will be a systematic negative bias against these uses.
Clearly, the perceived environmental crisis in Colombia points to the importance of assessing the relative value of water in alternative uses.Background documents cite progressive degradation in natural resources including loss of natural habitats and bio-diversity, pending water scarcity along the Caribbean coast and within the Magdalena and Cauca basins, serious water quality problems in large urban centers and lack of wastewater treatment throughout the country.
While there is a clear environmental emphasis in the 1991 Constitution, there is conflict in the priorities stated in Law 99 because it gives bio-diversity protection priority, but also indicates that human consumption has ultimate priority.Certainly, these would be two of the most highly valued uses of water.How to balance them remains an extremely important problem to resolve at the national level.
At the federal level, in the absence of markets for water, and because bio-diversity is a non-market value, the most likely solution to the problem of priorities would be for the MAVDT to establish targets, or minimum flows, to preserve habitats and bio-diversity.It seems clear that the state has the authority to protect bio-diversity in the public interest.Once this constraint is established, other uses may be allowed to compete for remaining water.Some reports suggest that while using economic instruments for environmental purposes may be an ultimate goal, these instruments are too sophisticated and perhaps inappropriate for Colombia's stage of development.
The problem of priorities and determining the relative value of water is just as significant at the regional level.Information on Colombia consistently questions the investment decisions made by the CARs.There is virtually no information on return on investment on various projects and therefore, it is difficult to tell whether scarce dollars are being invested in water projects that have the highest net value to the citizens of Columbia.There is evidence to suggest that decisions by the CARs may be made on political, rather than economic grounds.This problem is explained in greater detail in the section on incentives, below.

V. Efficiency: Information and Incentives
Efficient allocation and use of water resources hinges critically on making rational decisions based on accurate information about benefits and costs.Institutions establish who can reap benefits from resource use and who must pay the costs.Human beings are resourceful and will respond naturally to the incentives embodied in institutional arrangements.The challenge is to present individuals with accurate signals about the cost of their water use on the entire system.
The same principle applies to agencies when they are the primary decision makers in water allocation.Institutions should be configured so agencies recognize the benefits and costs of each alternative investment in, or allocation of, water before making a particular decision.In order to maximize the net value of water, water managers must pay attention to economic information rather than political incentives.
Real information problems arise when externalities exist.When individuals or agencies impose benefits or costs on parties outside their purview, and do not consider all costs and benefits to the system in their decision making, serious externalities exist (by definition).In these cases, individuals and agencies face incentives that are based on inaccurate information, and inefficient water use will result (too much or too little will be allocated to a particular use).
In the case of Colombia, it is useful to look at the incentives faced by people in key agencies, as well as individual user incentives.Certainly, the CARs are the key water resource agency due to the high proportion of monetary resources they control.Background information suggests that CARs are influenced by both financial and political incentives, which are often interrelated.
Much of the controversy and debate in Colombia concerns the appropriate degree of autonomy between federal (MAVDT) and regional agencies (CARs).Even though this is essentially a political question, economists are interested in it because it affects the incentives faced by officials and water resource managers.On one hand, some argue that the existing level of autonomy helps the CARs to make decisions that are closely matched with true regional water concerns, without federal obstruction.
Others argue that "regulatory capture" is a serious problem, meaning that regional agencies are controlled by powerful special interests.As an extreme example, some believe NGOs are just a front for specific municipal interests.To the degree regulatory capture exists, this would give agencies the incentive to consider only the benefits and costs of water investments to particular (municipal) interest groups rather than impacts (including national environmental impacts) throughout the system.
The large fraction of CAR funding comes from municipal property taxes (usually between 15-26% of total property taxes).A smaller and diminishing proportion of funding comes from the federal MAVDT.Therefore, it is not surprising that the CARs are suspected of being heavily influenced by municipal officials, especially mayors.If it is true that the agencies focus on municipalities, rather than the impact of their investments on the entire system, significant misallocations in water will occur.There may well be a systematic bias against environmental uses like bio-diversity.
CARs also derive funding from licensing and user fees.It appears that CARs have encountered problems in taking advantage of this source of funding, as evidence by the fact that effluent fees, for example, make up only 2% of total revenues.Studies indicate that the agencies face serious information problems in this area.Basic cartography is often deficient, hydrological data is lacking and incompatible with CAR systems and environmental data is almost non-existent.Under these conditions, it would be extremely difficult for CARs to make rational decisions, even if they were inclined to do so.
The institutional arrangements that create incentives for individual water users also have serious problems.Law 99 requires the national government to impose charges on all uses of water according to social and environmental cost, which is a principle that economists would embrace.However it is not clear if, or how, water charges for normal use are used at the federal or regional level.It is clear that other CAR water regulations faced by individuals are unclear and inconsistent between regions.
Existing water regulations are voluntary and take the form of "clean production agreements" and "environmental guides".Under clean production agreements, polluters agree to reduce pollution and the agencies grant a grace period within which they are not liable for non-compliance.The evidence suggests that while the agreements are attracted politically, they produce poor incentives and have been very ineffective in achieving environmental goals.
Environmental guides are intended to provide information to companies about pollution prevention.As a practical matter they have been used by companies as a "how to" manual for obtaining licenses (since no clear standards exist).Unfortunately, licensing standards vary between CARs and the guides are incomplete in this context.These guides could be economically valuable if they evolve in a direction that provides relevant, complete information about licensing requirements that have been agreed on by the CARs and are consistently applied.

VI. Fundamental Economic Equity Issues
As stated in the introduction, economically efficient water allocation and use means maximizing the aggregate net value of the resource to the entire society.Of course, for each allocation, there are winners and losers.Each particular allocation involves a particular distribution of benefits and costs.Therefore, equity issues are often critical in designing water institutions.
Equity issues surrounding water use in Colombia can be placed into (at least) three overlapping categories.The first issue is the overall allocation between traditional economic uses of water and non-market (but economically valuable) environmental uses.In general, there is concern that there may be a systematic and inequitable bias against environmental uses, even though they carry very high value to the country.
The second issue concerns the distribution of water benefits and costs between urban and rural areas.Urban issues are characterized as "brown" issues tied to waste water treatment and poor water quality.Rural issues (sometimes labeled as "green") concern both bio-diversity and basic access to drinking water for poor Colombians.Basic access to water is often considered a basic right and the cost of water to low income people is an important concern.In this case, there is often a conflict between the economic goals of "full cost recovery" and poverty alleviation.Economically the problems of drinking water and water quality/wastewater are inextricably linked, given the reusable nature of water.Addressing water quality issues will have big payoffs in terms of access to drinking water downstream.
A third equity issue in Colombia concerns the great variation in resources and institutional capacity between CARs.Because funding is largely derived from municipalities, this problem overlaps with rural/urban divisions.It may well be the case that relatively high value potential projects in rural areas go unnoticed and undone simply due to the lack of institutional capacity and low funding.

VII. Economic Issues in Strategic Administration
Any economic analysis of alternative water institutions should take into account the cost of actually implementing the arrangement.Administration costs can be significant and can undermine even the most sophisticated regulations, especially in developing countries.Administration includes monitoring water users and enforcing the "rules of the game" and is typically the responsibility of government agencies (except in self-regulated systems).This means personnel in government agencies must observe activity and measure impacts and must have the resources to impose sanctions where warranted.
As a practical matter, it is often wise to choose an institutional arrangement that requires relatively less information to administer effectively.Once an institutional approach is selected, the agency with limited resources must also be strategic in monitoring and enforcement.Typically, monitoring efforts are focused on large water users that have relatively large impacts on the system.Ideally, agencies can play a role in lowering the total cost associated with a water institution by providing information to water users that makes it easier for them to comply with regulations (the environmental guides are a step in this direction).
In the case of Colombia, it appears that monitoring and enforcement have been largely absent.Basic information about the environmental impact of water activities is also lacking.In the 1990s, the National Planning Department proposed using a set of 256 indicators to monitor environmental conditions.Not surprisingly, the indicators were abandoned because they were too complex.In 2002, the MAVDT suggested a new set of 32 environmental indicators.There is inconsistency in the use of indicators between CARs.Some CARs use the indicators developed by the MAVDT; others do not.

VIII. Conclusion: The Utility of Ex Post Analysis
Gaining the ability to conduct solid ex post evaluations of environmental institutions constitutes an important goal for countries around the world.Progress in this area is bound to make significant improvements in resource use, economic prosperity and environmental quality.A comparison of actual circumstances in a particular country to the conceptual elements in models of institutional impact can contribute to understanding the key elements of good institutional design.If addressed in a positive way, these ex post analysis can also help policymakers to design environmental institutions that yield real improvement for society.
The particular ex post analysis presented here useful in understanding problems inherent in Colombia's water institutions.From an economic perspective, policymakers in Colombia must strive to coordinate water institutions at the national, regional and local levels and align them with the true impacts of water use and investment.Balancing traditional and environmental uses of water in a way that maximizes net value to the country continues to be a challenge.Serious equity issues concerning the distribution of benefits and costs between urban and rural Colombia remain.And institutional arrangements currently do not transmit information that is accurate.Strategic monitoring and enforcement of water institutions could also carry substantial economic benefits.
As is often the case, Colombia faces tremendous institutional inertia in trying to move forward in building rational water institutions.However, even small positive changes in water institutions can lay the foundation for more substantial changes in the future.With determination and foresight, Colombia can make changes in water institutions that facilitate achievement of national environmental and economic goals.
Similar ex post analyses can generate insights into the relationship between institutional structure, user behavior and environmental performance in the context of various resource issues.Certainly, the ex post method is not perfect: it rarely meets the empirical standard of falsify-ability.A quantitative analysis requires in-depth analysis of very specific institutional phenomenon.Nonetheless, ex post analysis allows researchers to suggest plausible explanations for environmental performance based on current theory.These explanations can be debated, modified or rejected as our understanding of institutional performance improves.Ex post analyses are often very instrumental in determining the potential for improvements in environmental performance through modifications in institutional structure and in that regard can be an important asset to policymakers.