Не всегда получается интересно, но тут, как мне кажется, вышло именно так. Задача была описать сложности, которые возникают при оценке успешности той или иной государственной политики.
Я описал сначала теорию, потом нашёл интересный материал, подробно описывающий, как оценивать действия государства в той или иной сфере, а потом оценил этими способами Горбачевскую антиалкогольную кампанию и Олимпиаду в Сочи.
По антиалкогольной кампании интересные и неожиданные выводы. Вообще она часто в литературе попадается как один из самых успешных примеров краткосрочных государственных интервенций для увеличения продолжительности жизни, а кто бы мог подумать.
Материал, конечно, на английском (если вдруг у кого-то есть настроение перевести — буду рад).
В конце добавил оценку и мнение преподавателя, чтобы те из вас, кому интересно как тут в университете всё это работает почитали.
В начале под спойлер убрал довольно скучную теорию, если интересно — почитайте тоже.
The challenges in judging whether a policy has been a success illustrated with an examples of anti-alcohol campaign in USSR and Sochi winter Olympic games in Russia
Part 1: what is Policy
[Скучная теория...]The simple definition of Policy is «the actions of government and the intentions that determine those actions» (Birkland, 2005). So, Policy is a decision made by government and what we can use to explain and understand what is the intentions of this decision.
However, a number of sources define Policy almost with a completely different definition. For example «Diverse activities by different bodies are drawn together into stable and predictable patterns of action which (as often as not) come to be labelled ‘policy’» (Colebatch, 2002).
There are a number of definitions of Policy, some of them contradict each other. This happens because the issue of Public Policy and government is very complicated and involves many actors, branches of government, centers of power and decision makers (John, 1998).
One of the problems of evaluating Policy success is already in the definition of what is Policy: does it contain, for example, non-action, as some of the researchers suggested (Birkland, 2005), does it contain actions of public groups that resist the government in its attempt to change the status quo (Hill, 2009)?
A large part of Public Policy is unseen by the public or the elected politicians who are responsible for this Policy. This happens because the public can only pay attention to a small part of everything that is going on in the government, as the majority of the Public Policy activities occur in the name of the Public, but without real participation or even knowledge of the members of the Public and elected politicians. This concept is called 'semi-sovereign people', defined by Schattschneider in his book (1961).
The policy cycle is a classic description of the Public Policy process that explains how this process should work. This concept is criticized by some researchers (Benoit, 2013) (Cairney, 2011) for being too idealistic and oversimplified, not working in the real world.
- Identification of the aims that needed to be solved
- Generation of relevant solutions, picking the best one,
- Legitimization: authority from an elected body, groups of society, a referendum or other ways of legitimization,
- Implementation: ensure resources and organizational structure to implement the Policy,
- Evaluation: assessment of the Policy and the implementation, understanding to what extent the desired effect is achieved,
- Policy maintenance, termination or succession: decision if the Policy should be continued, terminated or be changed
Pic. 1: The policy cycle (source: Connors, 2016)
Rationality and Incrementalism
The classic starting point in policy analysts is a concept of Comprehensive rationality. The idea is that elected politicians translate their values into policies in a straightforward «perfect» way. As defined by professor Cairney (2016), an elected politician will need «a clear, coherent and rank-ordered set of policy preferences which neutral organizations carry out on their behalf». It means that the politician needs to separate his values completely from the way the Policy will be executed. They need to do a comprehensive analysis of full information, research and expertise on the topic, completely separate it from their own point of view, take into account the context and organizational options to maximize the benefits of the policy to the society.
As the situation described above is an ideal situation and does not happen in the real world, there is a concept of «bounded rationality».
The theory of bounded rationality defined by an economist Herbert Simon (Simon, 1957) suggests that policy decisions are affected by the absence of time, resources, information and also the inability to separate the values and beliefs of politicians from imperial facts and absence of clear objectives. So, the bounded rationality is «the limits of the policy maker omniscient» (Cairney, 2011).
A concept by Charles Lindblom: «Analysis that is limited to consideration of alternative policies all of which are only incrementally different from the status quo» (Lindblom, 1979). The theory suggests that organizations use simplifying strategies, and because of a lack of time and resources, they limit the number of choices to a small number that are only slightly different from the current situation and diverge incrementally. The decision makers tend to avoid high costs (both in term of resources and politics) of significant change of policy and tend to take non-radical steps.
Policy Implementation: Top-down and bottom-up approach
Two main strategies of organization of the Policy implementation can be articulated: the top-down and the bottom-up approach. The bottom-up approach suggests that decisions are made in the low level of government, by experts and public servants responsible for the particular policy field. If there is a disagreement among the low level, the decisions are taken to the next level of government, up until the top level elected officials.
Systems functioning like this are described as multi-level governance. The power is spread wide across the political system among national, regional, city and municipal levels and among governmental and non-governmental institutions (Cairney, 2011).
The top-down approach suggests that decisions are taken in the top level of the government, by high level elected officials, and the public servants just execute the decisions that were taken in the top levels before.
Systems functioning this way are called the Westminster systems. These systems rely on representative democracy: a party that wins an election controls the government. Systems like this have political leadership - the cabinet of ministers - that controls the vast majority of public policy, and politically neutral civil service that acts according to the political leadership's wishes. Power is centralized, and the government accounted to the public via parliament (Cairney, 2011).
There are advantages and disadvantages in both approaches (Sabatier, 1986). The advantages of the bottom-up approach is the greater level of expertise that is available for low-level decision makers who deal with a narrower policy field and can use more resources (time and money) to gather the information to make the right decisions. Also, this approach motivates members of the team as they have real impact on the final outcome and carry full responsibility for it.
The disadvantage of it is that decisions are mostly taken by unelected public servants who are invisible from the public, so there is a lack of public control and transparency in this process. It is difficult to understand who is responsible for what happens and who to blame in case something goes wrong.
The advantage of top-down approach is that the responsibility, liability and accountability can clearly be identified and controlled by the public. The disadvantage is that systems like this are less susceptible for innovations as the medium and low levels of it are demotivated and not determined to improve it.
The role of power in Policy making
Power is the ability to use or exploit what people think and what people believe, to achieve goals. The classic definition of power is «A has power over B to the extent that he can get B to do something that B would not otherwise do» (Dahl, 1957).
There are a number of power theories: the pluralist theory, that suggests that power is spread among many actors, and though it is mostly located in the framework of government, there are many other actors in this process (Dahl, 1967).
The elite theory, in opposition, suggests that privileged groups such as financial, political and technical/academic elites have power and the policy decisions are extracted from the elite consensus (Dye, 2001).
Power affects policy making not only directly, with promoting the ideas and interests of powerful groups by using resources and influence, but also by avoiding certain questions being discussed and taking them off the agenda. (Gul, 2009).
Part 2: Policy failure and success
A study (McConnel, 2010) shows a number of main approaches in current Public Policy academic discussion that describe success and failure of Public Policy.
- Detailed evaluation criteria is given in a book in 1971 (Lasswell). Seven sets of criteria are given: intelligent function, promotional function, prescribing function, invoking function, criteria for application and termination and appraisal function. This approach can be summarized as «policy science that contributes to societal betterment» (McConnel, 2010).
- Concept of public value. It describes the triangle framework to show what a successful public policy sector looks like. It has three main criteria: legitimacy and support, operational capabilities and production of value (Moore and Khagram, 2004).
- Deliberation and public engagement. This approach focuses on the process of the policy implementation and suggests that if it engages stakeholders in dialogue in order to understand and prevent implementation problems and cultivate policy legitimacy, then the programs will be visible and successful,
- Political success. This approach suggests that Policy is successful if it is able to produce benefits to political actors or groups.
- Non-failure concept. It suggest that policy is successful when it does not fail.
For the purpose of the case study analysts, I will use the definition of success given by McConnel (2010): «A policy is successful if it achieves the goals that proponents set out to achieve and attracts no criticism of any significance and/or support is virtually universal».
Process. While defining if a policy is successful as a process, we should check:
- The policy goals were preservation,
- General acceptance of legitimacy of the policy exists and that it has been produced through legal means and procedures,
- If sustainable coalition was built,
- What the policy symbolizes: innovation and movement forward, or obviously bizarre ideas,
- Existence of opposition: Does it exist and for what extent.
- Was it implemented in line with the objectives, or failed to do so?
- Have the desired outcomes been achieved?
- Were benefits created for a large target group, or did it produce damage to the groups?
- Has the policy domain criteria been meet?
- Is there an opposition to the aims and means of the program.
- Is it improving or damaging the electoral prospects and reputation of the politicians and leaders who are responsible for it?
- Is the process of controlling and endorsing of the policy agenda easing, or damaging the business of governing?
- Is it sustaining or damaging the broad values of government?
- Is there an opposition for political benefits of government?
- Resilient success,
- Conflicted success,
- Precarious success,
Challenges in judging policy success
The main challenge in policy judging is the extreme complicity of the process and multiplicity of parameters and dimensions of measurement. As shown above, there are three different dimension with five parameters in each and a five-point scale of measurement. The process of evaluation is highly subjective and in case of broad public attention, can be politicized, so even within one dimension it is difficult to reach an agreement about the evaluation.
But the main problem, as shown in a study, is that different analytics analyze policies in different dimensions and, therefore, disagree on its success or failure (Howlett, 2012).
Moreover, most policies are successful in one dimension, but fail in another. Most of the policies will not show complete success or failure in any of the dimensions, but will be partly successful in each of them. This makes politicians and administrators highly sensitive to policy risks (Howlett, 2012).
Example 1: Gorbachev anti-alcoholic campaign in USSR
In 1985 the Soviet Union government introduced an anti-alcohol campaign. The goal was to fight widespread alcoholism in the Soviet Union. The annual consumption of alcohol per capita in the USSR by 1984 reached 14 liters, 53% of it was vodka (Treml, 1997).
In 1985 an anti-drinking campaign began with cuts in the production and sale of alcoholic beverages (all the sales controlled by the government) combined with price increases and introduction of administrative penalties for alcohol abuse.
The statistical reports showed a rapid decrease of consumption of alcohol: by 66% in the first three years of the campaign (Treml, 1997) according to official statistics. Yearly statistical reports showed a decrease of alcohol-related deaths. «The average number of deaths in 1986 and 1987 compared to 1984 had declined by about 200,000; the struggle against alcoholism of the last few years contributed to the decline of mortality» (Narkhoz 1987 in Treml, 1997).
However, Treml's research (1997) shows a drastic increase of consumption of so-called «Samogon» – illegal, self-produced low quality vodka. This was heavily consumed, especially in rural areas where the conditions for brewing Samogon were better than in the cities.
If we are to include Samogon consumption in total alcohol consumption, the anti-alcohol campaign still showed a major 33% decrease of alcohol consumption (Kueng and Yakovlev, 2016).
The negative consequences of the campaign were major reductions of budget incomes from alcohol sales (Terekh, 2005), and major damage to the grape growing farms, many of which were destroyed, others broke, and an increase of illegal alcohol production and consumption.
Politically, the campaign was a disaster for Gorbachev and the Soviet government: the citizens perceived the campaign as an absurd initiative of the authorities, directed against the «small soviet citizen» (Maetnaya, 2015).
However, a recent study shows unexpected outcomes of the short-term anti-alcohol policy, that are observable only now: the campaign and liberalization of the light-alcohol market after the Soviet Union collapsed lead to a shift from heavy alcohol to light alcohol among young users (Kueng and Yakovlev, 2016). The low availability of vodka and other hard alcohol lead to a pattern for drinking light alcohol among young people. Decades after the campaign finished researchers are finding that 60% of the recent decrease in male mortality can be explained in this shift of preferences. The researchers expect the male mortality rate will continue to decrease by another 23% over the next twenty years.
So if we are to use McConnel's (2010) system of measurement of public policy we will see the next results:
Anti-alcohol policy in the USSR as process:
- The government's goals have been preserved: alcohol consumption decreased. But a major increase of illegal alcohol production and consumption and budget losses, lead to unwilling consequences. Precarious success
- Policy legitimacy was doubtful: even though the high government officials supported the program, the wine producers, economists and law-enforcement agencies opposed it (Grushko, 1997). Conflicted success
- There was no need for building a coalition in the Soviet Systems to implement policies, however, as described above, this policy had some opposition. This dimension is not useful for a Soviet system in my opinion.
- The program symbolized a move forward and was supported with major propaganda efforts to insure this impression. However, these efforts did not work and the public did not accept the campaign (Maetnaya, 2015). Failure,
- Opposition to the process was high in Soviet terms, as described above. Precarious success
- Mixed results of the implementation: a big decrease of alcohol consumption accompanied by an increase of illegal alcohol production, yet the program resulted in a decrease of 33% of alcohol consumption including illegal alcohol. Resilient success,
- Desired outcomes have been achieved, especially if we count the long-term outcomes: the consumption of alcohol and the mortality among men decreased (Kueng and Yakovlev, 2016). Success.
- The target group of the campaign - male population - enjoyed the benefits of it: mortality decreased during and after the campaign (Kueng and Yakovlev, 2016) (Treml, 1997). Success,
- The program generally met the policy domain criteria, but part of the campaign drew unwanted public attention and scandals: the demolition of vineyards and samogon production for example. Precarious success.
- Despite support from parts of society, the program had major opposition, as the values and means were not understood. Precarious success
- Hard to define the electoral prospects in the Soviet system, there are no fair and open elections.
- The program brought heavy destruction into the government's functioning: decrease in budget incomes, necessity of enforcement of alcohol bans and prosecution of illegal alcohol producers. Failure,
- It is hard to say what the broad values and directions of the Soviet government were, but if we look into Lenin’s works on the Soviet state we can see some of them (Lenin, 1918). It is hard to say if these values had something to do with reality in 1985.
- The program had high opposition to political benefits from the general public and major stakeholders. Failure.
So it is possible to say that the anti-alcohol campaign was a success as a program, since it achieved most of the goals and benefited the target group, and was precariously successful as a process and a failure politically. This shows that the extreme complexity and multiple dimensions of assessment of policy success are a major challenge to evaluate whether it was a success or a failure.
In fact even the author of the program, Mikhail Gorbachev, when asked about the program said «Because of mistakes in the process good and big program ended as a failure» (Gorbachev, 2005).
However, a report (The European Bank for Reconstruction and Development, 1999) shows a completely opposite evaluation of the program: «The significance of alcohol seems to be confirmed by the fact that the anti-alcohol campaign of Gorbachev during the mid-1980s resulted in a very notable improvement in male life expectancy by almost three years between 1980 and 1985». The same assessment is given by another report (United Nations, 1999).
Recent research (Kueng and Yakovlev, 2016) shows a significant decrease in male mortality decades after the three-year-long campaign and confirms the positive assessment of the program.
Interesting fact: while writing this paper I asked my twitter followers to answer a quick survey: do you assess the Gorbachev anti-alcohol program as a success, 87% of the 4271 participants answered «no» and «probably no». This is just an interesting fact and not an attempt to make a survey.
The different opinions on success and failure of the same program show that the assessment of public policy was very challenging because of different points of view, different assessors and the complexity of all the factors that needed to be weighed up during the evaluation.
Example 2. Sochi 2014 winter olympic games
Sometimes there are situations when the assessment of a policy's success or failure is challenging because of a different understanding of the policy goals, and wrong sets of goals for a policy in the first place.
It can be called «icon politics»: when a project is being done for symbolic value, to contribute to the reputation of a political leader or national prestige. Projects like this can be unsuccessful economically, environmentally, and with other regular-world assessment measures, but can still be perceived and declared as a success by politicians and even perceived like this by the public opinion (Carter, 2011).
In 2007 Sochi was selected to host the Winter Olympic Games in 2014. This project was quickly declared as «national priority» and personally controlled the preparations to the games (Putin, 2012). A report prepared by Russian opposition leaders suggests that the Olympic Games for Putin was a personal priority:
«The Winter Olympics in Sochi are a personal project of Putin’s. He thought (and probably still does) that the Olympic Games would be his triumph, and that the participation of athletes from all over the world would be an indisputable recognition of his leadership both in Russia and in the world» (Nemtsov and Martynyuk, 2013).
In Guatemala during the elections of the city that will host the game, Putin declared that 10 billion dollars would be spent for construction of brand new arenas and venues (RBC, 2007). However, the final official spending ended up to be 55 billion dollars (Müller, 2014). It made the Sochi Olympics the most expensive olympic games in history.
The games was finished with a triumph of the Russian Olympic team – Russia won 33 medals, first place in total medal counts and in gold medal counts. In Russia, the games were perceived as a national triumph, and a survey showed 77% of Russians evaluated the games as a success (Levada, 2014).
As this survey applies more for the sport results of the games, there were other questions, and 81% of the respondents said that the games «caused a surge of patriotic feeling».
Recent doping allegations question the athletic success of the Russian Olympic team (McVeigh, 2016).
A study (Müller, 2014) shows the main issues were particularly important for the Russian leadership in this project:
- Grandeur: Building a state-of-the-art sporting object in the open country with major governmental spending was supposed to show the greatness of Russia.
- Domestic spectacle: The project's aim was to promote Russia's self-pride and self-confidence by showing the possibility of organizing such an event and sport success.
- Soft power: Putin wanted to demonstrate to the world as much as to himself that Russia was still, and again, a force to be reckoned with.
- Neopatrimonialism: Redistribution of wealth resources to insure loyalty of Russian elites. Opposition claims that major corruption and fraud happened during the construction of the objects (The Anti-Corruption Foundation and Navalny, 2014),
- Post-occupancy of the Olympic park: currently, the sport venues are not used; Sochi was not transformed into an attractive winter destination, and the coastal claster (place with 14 venues including 48,000 seat football arena and 12,000 ice hockey arena) has not become an integrated piece of the city.
- Physical point of view: the Olympic park is located 30km from Sochi's city center. It is difficult to reach as most of the rail connections were canceled after the games. The space is not attractive, is surrounded by a major road, and does not offer attractive activities for tourists, except for the October Formula 1 competition, the rest of the year the park stands empty.
- Economic: The event cost 55 billion dollars, all the sports venues are now closed or underutilized; the park, with the exception of weeks before the F1 race, is abandoned.
However, Vladimir Putin declared the games as a major success. Recently (Putin, 2016), after the doping allegations and the evaluation of the legacy of the games shown above, he reminded all about the «major success».
«Exactly two years ago the Olympic flame was lit in Sochi. Across the country, we were responsibly preparing for this event and are proud that we held the best in the history of the Olympic Games - modern transport, resorts, recreational infrastructure and unique sports facilities were built, created. Thanks to this creative work - XXII Olympic and XI Paralympic Winter Games were held at the highest level, which left a unique legacy that will long serve the people» (Putin, 2016).
This case shows the challenges on assessment «icon policies». Evidence-based assessment of the facts, process, and program shows clear failure. But the political side can be evaluated as a success. Therefore, people that oppose the initial goals of the policy will not take into account the later «success» because they opposed the goals in the first place.[Источники...]
- The Anti-Corruption Foundation and Navalny, A. (2014) Sochi 2014: The comprehensive report. Available at: http://sochi.fbk.info/en/report/ (Accessed: 22 January 2017).
- Azzali, S. (2016) ‘The legacies of Sochi 2014 Winter Olympics: An evaluation of the Adler Olympic park’, Urban Research & Practice, , pp. 1–21. doi: 10.1080/17535069.2016.1216586.
- Benoit, F. (2013) Public Policy Models and Their Usefulness in Public Health: The Stages Model. Available at: http://www.ncchpp.ca/docs/ModeleEtapesPolPubliques_EN.pdf (Accessed: 20 January 2017).
- Birkland, T.A. (2005) An introduction to the policy process: Theories, concepts, and models of public policy making. 2nd edn. United States: M.E. Sharpe.
- Cairney, P. (2011) Understanding public policy: Theories and issues. Houndmills, Basingstoke, Hampshire: Palgrave Macmillan.
- Cairney, P. (2015) Policy policymaking in the UK. Available at: https://paulcairney.files.wordpress.com/2013/08/chapter-2-20-8-13-cairney-policy-policymaking-uk.pdf (Accessed: 19 January 2017).
- Cairney, P. (2016) Paul Cairney: Politics & public policy. Available at: https://paulcairney.wordpress.com (Accessed: 20 January 2017).
- Carter, P. (2011) ‘Governing welfare reform symbolically: Evidence based or iconic policy?’, Critical Policy Studies, 5(3), pp. 247–263. doi: 10.1080/19460171.2011.606298.
- Cherney, A. and Head, B. (2010) ‘Evidence-based policy and practice: Key challenges for improvement’, Australian Journal of Social Issues, 45(4), pp. 509–526. doi: 10.1002/j.1839-4655.2010.tb00195.x.
- Colebatch, H.K. (2002) Policy. 2nd edn. Maidenhead, U.K.: Open University Press.
- Connors, S. (2016) ‘GeoPolicy: Science and the policy cycle’, European Geosciences Union, .
- Dahl, R.A. (1957) ‘The concept of power’, Behavioral Science, 2(3), pp. 201–215. doi: 10.1002/bs.3830020303.
- Dahl, R.A.A. (1961) Who governs? Democracy and power in the American city. 26th edn. New Haven: Yale University Press.
- Dye, T.R. (2001) Top down policymaking. New York: Chatham House Publishers Inc.,U.S.
- Dye, T.R. (2005) Understanding public policy. 11th edn. New-Jersey: Pearson.
- The European Bank for Reconstruction and Development (1999) Transition report 1999. Available at: http://www.ebrd.com/downloads/research/transition/TR99.pdf (Accessed: 21 January 2017).
- Gorbachev, M. (2005) ‘20 years after the start of anti-alcohol campaign’. Interview with Interfax, .
- Grushko, V. (1997) Spy’s destiny: book of memories. .
- Gul, S.K. (2009) ‘Power and power relationships in the public policy making process’ by Serdar Kenan Gul. Available at: https://works.bepress.com/skenangul/8/ (Accessed: 21 January 2017).
- Hill, M.J. (2009) The public policy process. 5th edn. Harlow, England: Pearson Education.
- Howlett, M. (2012) ‘The lessons of failure: Learning and blame avoidance in public policy-making’, International Political Science Review, 33(5), pp. 539–555. doi: 10.1177/0192512112453603.
- Kueng, L. and Yakovlev, E. (2016) ‘Long-run effects of public policies: Endogenous alcohol preferences and life expectancy in Russia’, SSRN Electronic Journal, . doi: 10.2139/ssrn.2776422.
- Lasswell, H.D. (1971) A pre-view of policy sciences. New York: American Elsevier Pub. Co.
- Lenin, V. (1918) Ocherednie zadachi Sovetskoi vlasti. Moscow: .
- Levada (2014) The results of the Olympic Games in Sochi, a public opinion poll. Available at: http://www.levada.ru/old/03-03-2014/itogi-olimpiiskikh-igr-v-sochi (Accessed: 22 January 2017).
- Lindblom, C.E. (1959) ‘The science of “muddling through”’, Public Administration Review, 19(2), p. 79. doi: 10.2307/973677.
- Lindblom, C.E. (1979) ‘Still muddling, not yet through’, Public Administration Review, 39(6), pp. 517–526. doi: 10.2307/976178.
- Maetnaya, E. (2015) ‘The people poisoned and cursed those who introduced Prohibition’. Available at: https://www.gazeta.ru/social/2015/04/25/suhoi_zakon.shtml (Accessed: 21 January 2017).
- McConnell, A. (2010) ‘Policy success, policy failure and grey areas in-between’, Journal of Public Policy, 30(03), pp. 345–362. doi: 10.1017/s0143814x10000152.
- McVeigh, N. (2016) Wada: Russian government oversaw widespread doping – as it happened. Available at: https://www.theguardian.com/sport/live/2016/jul/18/wada-mclaren-report-russia-sochi-2014-doping-live (Accessed: 22 January 2017).
- Moore, M. and Khagram, S. (2004) Mark Moore on creating public value what business might learn from government about strategic management. Available at: https://www.hks.harvard.edu/m-rcbg/CSRI/research/publications/workingpaper_3_moore_khagram.pdf (Accessed: 21 January 2017).
- Müller, M. (2014) ‘Introduction: Winter Olympics Sochi 2014: What is at stake?’, East European Politics, 30(2), pp. 153–157. doi: 10.1080/21599165.2014.880694.
- Nemtsov, B. and Martynyuk, L. (2013) Winter Olympics in the Subtropics. Available at: http://www.putin-itogi.ru/winter-olympics-in-the-subtropics/ (Accessed: 22 January 2017).
- Parsons, W. (1995) Public policy: An introduction to the theory and practice of policy analysis. Aldershot, UK: Edward Elgar Publishing.
- Petersson, B. (2014) ‘Still embodying the myth?’, Problems of Post-Communism, 61(1), pp. 30–40. doi: 10.2753/ppc1075-8216610102.
- Putin, V. (2012) ‘On the readiness of facilities for the Olympics in Sochi the President spoke with Vladimir Potanin. News. First channel’. Interview with 23 November, .
- Putin, V. (2016) Vladimir Putin speech on Sport Day 7 February.http://www.pravda.ru/news/society/07-02-2016/1291349-densporta-0/
- RBC (2007) Vladimir Putin presented in Guatemala Sochi’s bid to host the Olympics. Available at: http://www.rbc.ru/society/04/07/2007/5703c8199a79470eaf7660b9 (Accessed: 22 January 2017).
- Sabatier, P.A. (1986) ‘Top-down and bottom-up approaches to implementation research: A critical analysis and suggested synthesis’, Journal of Public Policy, 6(01), p. 21. doi: 10.1017/s0143814x00003846.
- Schattschneider, E.E. (1961) The semisovereign people: A realist’s view of democracy in America. 14th edn. New York: Holt,Rinehart & Winston of Canada.
- Simon, H.A. (1957) Models of man, social and rational: Mathematical essays on rational human behavior in a social setting. New York: John Wiley and Sons.
- Simon, H.A.A. (1997) Administrative behavior: A study of decision-making processes in administrative organizations. 4th edn. New York: Simon & Schuster Adult Publishing Group.
- Terekh, K. (2005) ‘Last minister of trade in the USSR’. Interview with Viktor Pushkarev for 5 December, p. http://www.businesspress.ru/newspaper/article_mId_21962_aId_362860.html.
- Treml, V.G. (1997) ‘Soviet and Russian statistics on alcohol consumption and abuse’, SSRN Electronic Journal, . doi: 10.2139/ssrn.2293.
- United Nations (1999) Human Development Report 1999: Russian Federation. .