METHODOLOGICAL PROBLEMS OF THE SOCIAL PROJECTS’ EFFICIENCY ASSESSMENT
Abstract
The analysis of methodological problems of an assessment of the social projects’ efficiency is presented in the article. The authors come to the conclusions that one of the most essential problems is that of identification, ranging and analysis of the social effects. The solution of this problem is connected with the introduction of the long-term administrative thinking, search of indicators for the assessment of progress, its driving forces and obstacles, research of a wide range of the social initiatives and results. The problem of a quantitative assessment and its justification demands the obligatory accounting of a context of the realized social project, attraction of a wide range of the interested participants of process (researchers, program specialists, experts), formation of the mechanisms of feedback with the beneficiaries and donors of the social project.
Introduction. Change, expansion and development of social practices, emergence of new technologies in social sphere, and also the growing requirements to efficiency of state programs for infrastructure, ecological, educational and public health orientation cause need of the methodological apparatus development for adoption of reasonable administrative decisions in the social sphere. Currently a considerable number of the works devoted to researches of social programs and project sassessment methods appear in domestic and foreign literature [ 5, 14, 16, 20, 21, 22, 28, 29 ]. In spite of the fact that instrumental approaches to the solution of problems in the field of efficiency assessment for social projects are a constant research object, in scientific community there are no standard universal mechanisms of efficiency assessment for decisions in this sphere [31].
First of all, vital methodological issues are, in our opinion, identification, analysis and assessment of various, mech anisms of influence and terms of manifestation of the social effects connected with implementation of specific social projects by properties.
The methodological problem of effects identification from implementation of the social project, in turn, is conne cted with definition of the indicators system or effects connecting the purposes and results of the project. It is one of the most difficult and significant stages of design planning process.
Essential element of social effects identification is their cla ssification. When determining effect from the social project it is necessary to distinguish its direct and indirect, universal and specific, positive and negative types. The direct type is understood as the effect which is directly connected with implement ation of the social project, with achievement of its specific goals. Indirect effect arises owing to development of the external processes initiated by the social project. Often it is very difficult to estimate such effect quantitatively. The universal eff ect is the effect arising at implementation practically of any social project (improvement of life quality, change of social climate, etc.) and specific is caused by features of the specific social project, project scales, features of the countries and reg ions in the territory on which the project is realized. Besides, it is impossible to estimate such project, as causing only positive effects. Modern researches show that negative effects, such as decrease in motivation, increase in the taxation and protes t moods quite often accompany imple mentation of social projects [10].
Main Part. Methodological approaches to the social projects efficiency assessment The analysis of effects assumes profound classification depending on scales, purposes, and direction of social projects. So, the analysis of the social projects at the moment supported by Agency of strategic initiatives and Our Future fund [18] shows that social effect of these projects can be devided into five classes conditionally:
1. Creation of workplaces for the social and unprotected citizens (employment of disabled people, mothers having many children, etc.).
2. Adaptation into society the actually or potentially asocial citizens (adaptation of addicts, children from orphanages, etc.).
3. Improvement of social conditions in the region (creation of children's interest groups, pools, hospitals with preferential prices, programs of motherhood and childhood support, etc.).
4. Rehabilitation of disabled people (creation of conditions for normal activity of disabled peo ple, programs of treatment of disabled people, inclusive programs).
5. Improvement of the human capital quality in the region (educational projects, cultural and educational projects).
And this list, certainly, can be supplemented and expanded, building clas sification in a foreshortening of the maximum coverage of all social effects range. It is important that it is necessary to pick up or develop the technique, on one hand, providing assessment for each class of effects' adequacy and completeness, and, on t he other hand, comparability to other effects.
On analysis stage of social effects it is necessary to carefully study starting conditions for target social group, to reveal the alternative mechanisms allowing to achieve goals, spending smaller quantity of resources.
Classification of social effects can be constructed on definition of beneficiaries groups as well. Thus, C. Gonzales suggests to use concept public benefit: the benefit due to which the project can be realized for a ssessment of social projects [9].
F. Vanklya offers three essential principles necessary to observe during identification and analysis of social effects:
1) need of essence understanding and social effect source;
2) definition as key indicator of social effect the quality improvement, but not the level of living improvement;
3) research and assessment of public opinion [30].
The following "problem zone" of social projects management is definition of social effects quantitative characteristics, (including the influence vector – positive and ne gative), shown at various levels, in various temporary periods; integrated assessment of efficiency.
At the high level of generalization it is possible to say that the key purpose of the social project consists in combination of economic efficiency (econo mic effect), social justice (effect of justice) and improvement of life quality for target social groups (social effect). Thus in a general view, the problem of quantitative assessment of the social project efficiency consists in assessment of each of the se effects and its integrated size.
Modern ways of social effects quantitative assessment can be divided into three big groups: comparative, indicative and indistinct - and - interval methods. The first group is comparative methods in which social, economic ef fects and effect of justice are not shared, and are compared with expenses or selling costs of the social project:
1) method of cost - benefit analysis (cost - benefit analysis – CBA);
2) method of cost effectiveness analysis (cost effectiveness analysis – CEA);
3) method of cost - utility analysis (cost - utility analysis – CUA);
4) method of weighted cost effectiveness analysis (weighted cost effectiveness analysis – wCEA);
5) method of hedonistic usefulness assessment.
From above - mentioned methods of efficiency assessment fo r social projects the greatest distribution was gained by a method of the expense and benefit analysis (CBA) [4]. It consists in comparison of benefits (the aggregated effects) estimated in terms of money, and the made expenses. And depending on the assess ment purposes the method can be used in two modifications. The first is an effect assessment (true current benefits) in the short - term period:
NBst = B – C, (1),
where NBst stands for short - term net present benefit;
B (benefits) – benefits (social effect) in the current period,
C (costs) – expenses in the current period.
At the second modification the indicator of long - term effect of the target program implementation is estimated:
where NBlt s tands for long - term net present benefit;
Bt (benefits) – benefits (social effect) in moment of time t;
Ct (costs) – budget outlays in the moment of time t;
d (di scount rate) – discount rate [26 ].
This method can be applied in the case when benefits refl ecting direct effect from the social project can be estimated in terms of money. Advantages of the method are: the universality of expenses and benefits indicators allowing to compare various projects; opportunity to estimate long - term effect on the basis of benefit indicators discounting.
Restrictions of this method are that in social sector of benefit it is difficult to estimate in value terms, and, costs for collection of information can be unfairly high. Besides, social effects have to be estimated from a position of all society, but not separate social group, i.e. it is necessary to take both positive, and negative components of outer effects into account.
Use of the expense and productivity analysis method (CEA), assumes an assessment of ratio of expen ses and result, the aggregated benefit from the project. Benefit from the project is estimated not in terms of money, but in physical units. In fact, the productivity gain is calculated. It does not allow to compare benefit directly to costs of the project implementation. Therefore the conditional cost of effect unit as the relation of expense volume to the extent of the created social effect is calculated. Works are devoted to comparison of the CBA and CEA methods written by P. Dolan, J. Lezurin, M. Levin , B. Hansen, L. Jacobson [13, 15 ]. In modern literature even more often the CEA method is considered as the most universal and attractive approach from the practical point of view. The main advantage of this tool is that rather simple idea is its cornerston e and thus results of the analysis are easily interpreted.
Generally when using the CEA method the result is expressed in incremental sizes. Therefore it is accepted to call a classical method of expense and productivity incremental (ICEA – incremental cos t - effectiveness analysis) [17 ]:
(3)
where ICER stands for incremental cost - effectiveness ratio);
∆C – gain of expenses as a result of interventions;
∆E – productivity gain (social effect) as a result of interventions.
The value of ICER indicator is lower, the less expenses are connected with achievement of a certain level of productivity and therefore, the considered option of soci al changes is more effective [27 ]. As researches of var ious authors showed, as top limit the level of values of this assessment it is possible to take the indicator of WTP (willingness - to - pay) reflecting tendency of the subject making financial decisions to pay for the considered social project.
Essential res trictions of the method are:
- assumption of linear nature of expenses on productivity dependence;
- comparison of social parameters, various by nature;
- complexity of the expenses and effects accounting during various periods of time;
- complexity of outer effec ts influence assessment on result;
- high degree of result sensitivity to a choice of indicator characterizing social effect.
The term "analysis of expenses and usefulness" is used in V. Goel and A. Detski's works, by C. Gerard, J. Torrens [ 6, 7, 8 ]. The me thod of the expenses and usefulness analysis (CUA) is based on comparison of expenses in terms of money and the benefits for concrete target group exp ressed in terms of usefulness [24 ]. The method is most often applied by consideration of budget outlays on health care. The method is effectively used in the pharmacological economy studying a ratio between expenses and efficiency, safety, life quality at alternative schemes of disease treatment (prevention). The way of usefulness measurement is specific and i s defined specifically for each project, for example, can be expressed in terms of QALY (quality adjusted life years) – number of the prolonged years of life. The assessment of social projects implementation expediency is made proceeding from the analysis of C/U criterion:
Let us emphasize that unlike the method of expenses and productivity, this method gives the chance of aggregating effects in a usefulness indicator. Despite this advantage, the restrictions co nnected with linearity of approach and complexity of uniform criterion choice for usefulness are also applicable to this method.
When using a method of expenses and weighed productivity analysis (weighted cost effectiveness analysis – wCEA) it is offered t o unite different effects in uniform integrated effect and to correlate them to the size of the spent resources. In this case its conditional expression via the aggregated indicator including various characteristics of assessment object acts as social effect:
Such approach demands determination of weight or importance of a contribution of each characteristic to the general result. In recommendations of the World bank it is advised to define weight, based on opinions of experts, the persons making decisions, and views of socie ty on the considered problem [12 ]. It should be noted that assignment of scales is one of the most difficult and subjective moments when carrying out the similar analysis. The method lifts some limits in a choice of specific criteria for assessment since means use a set of criteria, but generates the organizational and information restrictions connected with def inition of their weight characteristics.
The hedonistic method is based on use of property value for assessment of public benefit. It is meant that implementation of the social project changes various characteristics and properties of environment, thereby influences the property value as well. The change in property price connected with change of properties as a result of the project implementation is considered a criterion or public benefit assessment, and, therefor e, efficiency of the project [9 ].
Advantage of a method is that for obtaining the expected values of different types of benefit there is no need of carrying out the separate analysis of each type – the property value increment acts as the aggregated indicator.
The method assumes use of the develo ped econometric model for which it is important to make selection of property in and out of project zones, and also to consider all properties of infrastructure (type of land plot, the characteristic of property, existence of services, etc.). The property price before implementation of the project is offered to be determined the next three ways: to question owners, to consult experts, to use a property assessment for taxation. It is obvious that this method of assessment is more applicable for the large - sca le state projects or projects of public and private partnership aimed on development of territories and the social sphere.
If comparison of indicators is the cornerstone of comparative methods (the income and expenses, usefulness and expenses, property val ue before implementation of the project), collecting and generalization of various indicators for removal of the uniform aggregated effect assessment from implementation of the social project is the cornerstone of indicative approach. From our point of view, most brightly interpret indicative approach: efficiency assessment method on the basis of indicators of population life quality and a method of public welfare function assessment (social welfare function).
Inclusion of indicators of life quality in as sessment of social programs efficiency is logical and proved by several reasons. First, an ultimate goal of the majority of the realized social programs is improvement of the population life quality and this criterion is put in a basis of making decisions on social modernization. Secondly, indicators of life quality not so significantly depend on value judgment, personal relations and public conduct, allowing to measure progress, being not always based on personal assessment of consumers [ 31 ]. Thirdly, th e social effects having various nature, the purposes and tasks of projects are often crossed or even completely coincide with various indicators of the population life quality.
Let us note that concept of life quality in itself is a difficult, ambiguous an d multi - layered phenomenon which is defined by a wide range of indicators. Today researchers allocate three types of life quality indicators: objective, subjective and integrated [ 2, 27 ]. All three types of indicators, in our opinion, are applicable in ass essment of social projects efficiency for the different directions, scales and spheres of responsibility.
Objective indicators of life quality characterize social structures of different level of community, they are estimated through parameters of objectiv e conditions and processes of activity. Indisputable advantage of objective indicators use for assessment is possibility of their selection for statistical collections and reports.
Subjective indicators of life quality are based only on value judgment and mean inclusion in research of questioning, polls, focus groups, and, therefore, demand serious costs of the analysis and information processing. A number of authors consider it necessary to define life quality, measuring degree of satisfaction of the pop ulation in the following directions: health, level of income, family happiness, housing conditions, level and quality of education, peace of mind, independence and freedom, respect of people around, employment securities, quality of medical attendance, con fidence in the future, security from criminal encroachments, ecological situation, peace and harmony in society, leisure and rest, comfort of settleme nt, power, religious beliefs [19 ]. Unlike objective, above - mentioned indicators characterize more function al requirements of individuals and degree of their satisfaction that also is essential during assessment of social projects.
The integrated way unites subjective and objective indicators of life quality, expanding with that, possibilities of indicators cho ice for social projects. But, as well as any integrated method, it not only expands a range of opportunities, but also aggregates shortcomings and restrictions, the united methods. Considering a wide range of purposes and tasks of social projects, it is po ssible to say that association of indicators of life quality in assessment of efficiency is productive at careful selection of indicators by certain criteria. The structure of such criteria is given in work of M. Hagerty [11 ], the most significant for ass essment of social projects criteria are generalized by A. Yemelyanov [31 ], which in their structure: practical importance; possibility of aggregation at various levels; reliability and validity of components of an indicator; possibility of decomposition of an indicator; objectivity of reflection of the main categories of life quality; potential measurability in objective and subjective terms.
The main problem of indicators of life quality method application in assessment of social projects efficiency, from our point of view, is selection of adequate selection of the subjective and objective indicators characterizing concrete effects, both flowing, and postponed in time.
Let us note also that in the majority of indicative methods, after definition of a set of the indicators characterizing effects of the social project and corresponding to the above - named criteria there is a problem of their integration into a uniform indicator – an assessment of the social project. It is a serious problem since indicators are qualitatively diverse and characterize various components of the population level of living which are difficult for uniting in a uniform quantitative index.
In modern scientific literature two groups of the methods allowing to integrate indicators are off ered: methods of rationin g and methods of aggregation [2 ]. Methods of rationing are: method of linear scaling and method of mark assessment. Rationing methods, in our opinion, are suitable for the comparative analysis, identification of preferences, creati on of ratings and acceptance on their basis of administrative decisions on investment, prolongation, implementation of the social project. We believe that methods of rationing can also be used before aggregation methods for receiving more uniform indicators.
The method of linear scaling is based on definition of reference points (the maximum and minimum values of indicators). These values can be presented by statistical data or expert opinion.
Further for each indicator the norm on formulas if the quantit ative assessment of an indicator positively influences social effect (life quality) is calculated,
If the quantitative assessment is negatively connected with social effect (for example, number of the unemp loyed),
The method of linear scaling gives good base for further aggregation (summation) of indicators since they become more comparable, defining situation between reference points. In a method of mark assessment actual data are estimate d in points concerning any standards or standards of indicators on the region, branch, similar projects, etc. The formula of rationing for indicator is calculated:
It is necessary to refer cost intensity, complexity of justification and subjectivity of choice both reference points, and reference indicators to shortcomings of rationing methods.
Methods of aggregation are: simple summation of indicators, calculation of arithmetic - mean value of all indicators and calcula tion of the average value of indicators, taking into account the scales specified by experts. Methods of aggregation can be used independently, as well as together with rationing methods, for providing uniform assessment of the social project efficiency.
At the heart of the following indicative method – a method of an assessment of public welfare use of universal approaches to identification of social effects and generalization of several indicators lies. Application of the method becomes possible in the case when the list of actions, the purposes and tasks of the social project are well structured and can be defined in advance, besides, there is a number of projects or programs from which it is necessary to choose. The purpose of application of this metho d is drawing up a rating of the project on an integrated indicator which is estimated in the following parameters:
- degree of the social project target group priority;
- the importance degree of the direction on condition improvement for a target group (the p urpose and tasks);
- degree of potential efficiency and adequacy of the method used wit hin the respective direction [26 ]. All these indicators have to be estimated quantitatively within each social project.
In the work by I. Shakina the SWF function (public welfare) for each i p roject is offered to be counted on a formula:
where SWF i – the size of function of public welfare of i project;
TGj– priority degree from a set j target group of a set of K revea led in i project;
NKjk – importance degree k directions on improvement of a state j target group of the set K revealed in i project;
MLKl – potential efficiency and adequacy of a method l within which the direction from the great number of L revealed in i project.
Let us note that values of indicators can be both positive, and negative.
It is necessary to carry a ready formula of indicators integration and the accounting of the project context to advantages of the method, but essential restr iction of its application is the complex and uncommon development challenge of system assessment of the project parameters: priority of target groups, the importance of the directions on improvement of their state, potential efficiency and adequacy of the applied methods.
Application of indistinct and interval estimates to efficiency of social projects assumes that there are
V = { v1, v2, ..., vn } – a set of social projects which are subject to the multi - criteria analysis;
C = { c1, s2, ..., cm } – a set of quantitative and qualitative criteria by which options are estimated;
B = { b1, b2, ..., bk } – competence of estimates of k - the experts who are carrying out expertize.
The problem of assessment consists in ordering elements of a set of V by criteria from a set C taking into account competence of experts. Such problem definition is typical in the sphere of assessment for projects and demands application of algebra for indistinct sets. By analogy with application of algebra for indistinct sets for the asses sment of innovative projects presented in works of the Kazan university scientists [1 ], procedure of assessment of the social project can be presented the following sequence:
- Use of preliminary expert examination for elimination of obviously unpromising, i nadequate social projects.
- Application of the analysis of hierarchies method by T. Saati for decomposition of a multi - criteria, complex challenge of efficiency assessment for social projects on simpler components and carrying out paired comparison of crite ria [31].
- The analysis of criteria as indistinct sets which are set on universal sets of options by means of accessory function in the form of triangular or trapezoid indistinct numbers.
- Ranging of options on the basis of indistinct sets crossing - criteri a which answer the scheme Bellmana - Zade [32], known in the theory of decision - making. At estimation of indicators experts set the lower – pessimistic estimates and top – optimistic estimates. Further processing of indistinctly formulated opinions of expert s is offered to be carried out by dephazification, i.e. transfer to correct quantitative estimates, and their further processing in a dephazified look.
Let us note that use in assessment procedure of algebra for indistinct sets allows to process a wide ran ge of expert estimates, to carry out the multi - criteria analysis of social projects, using, including, data and indicators of comparative and indicative methods. It is necessary to carry the difficult mathematical apparatus necessary for the description of compatibility functions for linguistic variables, attraction of a big number of experts and creation of the system criteria adequate to social projects to restrictions of the method application.
Conclusions . Summing up the result of the carried - out analys is, it is possible to draw a conclusion that as the most actual problems of an assessment of social projects the following is distinctly allocated.
Problem of identification, ranging and analysis of social effects. Solution of this problem is connected wit h introduction of the long - term administrative thinking, search of indicators for an assessment of progress, its driving forces and obstacles, research of a wide range of social initiatives and results.
The problem of a quantitative assessment and its jus tification demands the obligatory accounting of a context of the realized social project, attraction of a wide range of the interested participants of process (researchers, program experts, experts), formations of feedback mechanisms with beneficiaries and donors of the social project.
Reference lists