Why Policies Fail: An Institutional Perspective
Casual policy implementation and unsatisfactory public services are ubiquitous in India. But there is little study and analysis of the underlying causes that block effective reform. This article argues that these shortfalls are embedded in the very structure and functioning of the implementing organisations, collectively known as the “field administration”. This is illustrated through a case study of the field administration in a district in Madhya Pradesh. As the dysfunctions of the present-day field administration have their roots in the past, the key developments that shaped it are described briefly below.
History of Neglect
The field administration’s colonial legacy comprised a centralised and authoritarian system of regulatory administration. As the colonists attempted to maximise land revenue, the district, headed by the district collector, emerged as the key unit. Varied sub-district administrative systems existed across British India, but in the model influential after independence the district was divided into sub-divisions and the tehsils were headed by revenue officers.
To minimise administrative costs, frontline workers, or ‘street bureaucrats’, such as police constables and forest guards, were paid paltry salaries and discouraged from taking any initiative. The field administration served the state, not citizens.
A police system was established to maintain order and a forest administration system to enhance income from forests. Field organisations for school education and health, with limited outreach, were set up towards the latter part of colonial rule. Weak attempts were made to institute partially representative local bodies, which remained disempowered (Misra 1983; Potter 1996).
The district collector was usually a British official, and subordinate personnel, Indian. The latter were viewed with suspicion and an elaborate system of checks, inspections, and record-keeping were developed. To minimise administrative costs, frontline workers, or “street bureaucrats”, such as police constables and forest guards, were paid paltry salaries and discouraged from taking any initiative. The field administration served the state, not citizens. For example, the police focused on maintaining order and neglected crime detection (Arnold 1976; Misra 1983; Potter 1996).
Post-independence, this regulatory system was continued and expanded. But socio-economic development became an important goal, requiring changes in the field administration. Initially, as the government promoted a holistic socio-economic development of villages, an appropriate administrative system was deliberated. A new administrative unit, the block, as a site for local planning, and local elected governments or panchayats at the district, block, and village levels, expected to lead socio-economic development, were established. Extension officers for agriculture,education and other sectors were provided at the block and village levels to serve five or so villages (Dantwala and Barmeda 1995; GoI 1985).
However, in later years, scant attention was paid to organisational effectiveness. In the mid 1960s, holistic village development was abandoned when a new technology-based agriculture policy prioritising areas with high productivity was adopted. Panchayats were disempowered in most states and each state department began to operate its own programmes and organisations in the field. Over time, the number of departments, departmental organisations, and employees grew. In most states, the district collector, not panchayats, led and coordinated socio-economic development (GoI 1985).
New shifts came about after 1991 when containing government expenditure became a priority. Several state governments stopped filling up vacant posts, privatised some services, and recruited frontline workers on contracts at very low salaries.
After the 1970s, patronage-based postings of officials increased steadily, just as did rent-seeking (Potter 1996). Subsequently, in the mid-1980s, dissatisfaction with public services within and outside government led to a renewed interest in panchayats. In 1993, a constitutional amendment made panchayat elections mandatory, but most state governments devolved few powers to them. The departments and district collectors remained dominant.
New shifts came about after 1991 when containing government expenditure became a priority. Several state governments stopped filling up vacant posts, privatised some services, and recruited frontline workers on contracts at very low salaries. These policies continued even after the late 1990s when government revenues rose after the economy improved and new programmes for socio-economic development were initiated.
These developments have resulted in a very inadequate field administration as a case study below illustrates.
In the case study from Madhya Pradesh, the structure and functioning of the field administration and its impact on public services and policy implementation were analysed. The field work, conducted in 2017-18, included the study of 56 field organisations from eight departments (that is, government offices, panchayats, schools, anganwadis, sub-health centres, and cooperative societies); interviews with elected panchayat representatives, officials, frontline workers, personnel from non-government organisations, and journalists; and focus group discussions (FGDs) with different social groups. (For a detailed case study, see “Government at the Grassroots: A case study of field Administration” and for a detailed discussion, see “Organization through Neglect: Understanding Field Administration in India”, a forthcoming working paper for the Centre for Policy Research, New Delhi).
One finding was that a wide range of services was provided to citizens. The district, with a population of around 1.2 million in the 2011 census, had 5,425 government organisations that had direct dealings with citizens. Almost three quarters, such as schools, health centres, and anganwadis provided social services, while others included law enforcement agencies such as police stations and revenue courts, and commercial organisations such as cooperative banks and ration shops. The scattered residential pattern, with an average village population of 886 and a population below 500 in more than a third of the villages (2011 census), posed a challenge. Only low-cost services, such as primary schools and anganwadis, were provided in each village. Health services were sparse and people consulted traditional healers and dubious “doctors” with no medical qualifications.
The capacity to respond to situations and take coordinated action was constrained by the fragmented administrative structure and narrow definition of the roles of the implementing organisations. The field administration was organised along three axes—the “departmental axis” comprising the organisations of 37 state departments; the “local government axis” comprising zilla parishads at the district level, janpad panchayats at the block level, and gram panchayats for one to four villages in rural areas, and municipalities in urban areas; and the “district collector axis” with the district collector as the administrative head of the district.
Coordinated action was difficult because of the large number of departmental organisations. Moreover, departments functioned in silos. There was no common geographical administrative unit below the district. Some departments adopted sub-divisions, others blocks, and some individualised units.
Contextual needs could not be addressed because state departments linked nearly all funds to specific programmes. Moreover, nearly a third of the departments had no organisation or personnel below the district level and therefore very limited contact with citizens. Less than a third had organisations or personnel below the block level. Though many of the panchayat representatives interviewed were aware of local issues and motivated to resolve them, this potential remained unexploited because the panchayats lacked funds.
Coordinated action was difficult because of the large number of departmental organisations. Moreover, departments functioned in silos. There was no common geographical administrative unit below the district. Some departments adopted sub-divisions, others blocks, and some individualised units. Administrative units below the block were even more varied. In addition, there were two coordinating agencies. While the law mandated panchayats to lead socio-economic development, state departments delegated authority to the district collectors. Consequently, panchayat representatives complained that officials did not provide them with information and ignored panchayat resolutions. Parallel to this, the district collector had powers under 71 laws and was chair or member secretary of 82 committees, an impossible mandate to fulfil.
The district had 11,769 government personnel comprising managers and experts at the district and sub-district levels and frontline workers at the sub-block and village levels. Structural fragmentation ensured that most sample district and sub-district organisations had just one manager or expert. Grassroots organisations had inadequate staff as well. For example, at the anganwadi, a worker and a helper were supposed to address malnourishment and the health of children under the age of six and their mothers, provide pre-school education, and conduct activities related to adolescent girls and domestic violence. At the same time, 41% of the posts among the sample district and sub-district organisations and 20% among the frontline workers of the sample departments were vacant.
Government employees were well qualified. All the managers and experts and nearly two thirds of the frontline workers were college educated. But, across the district, there was no expertise, or very little of it, in three critical areas—human resource management, law, and social communication and mobilisation. Moreover, departmental organisations did not have the varied skills they needed because their employees were drawn from a few departmental “services”, each service comprising personnel with similar expertise. For example, the school education department lacked specialists for community mobilisation and education of marginalised children. The women and child development department, charged with addressing child malnutrition, had no nutritionists.
Contractual employees ... mainly frontline workers, lacked job security, had lower salaries than regular employees, and no avenues for promotion. They formed associations that agitated and litigated for better service conditions. Sometimes they were successful.
There was a perverse incentive structure for employees. Regular employees (78% of all employees) had security of tenure, reasonable salaries, and other benefits but their seniority-based promotions were infrequent, creating little positive incentive. Worse, with patronage-based postings, they gained by pleasing patrons, not by working hard. Several officials were involved in rent-seeking rackets along with powerful politicians, while others reported being transferred for resisting illegitimate orders. The average tenure of organisational heads was very short, less than a year. Further, interviewees reported that politically connected employees could not be disciplined.
Contractual employees (23% of all employees), mainly frontline workers, lacked job security, had lower salaries than regular employees, and no avenues for promotion. They formed associations that agitated and litigated for better service conditions. Sometimes they were successful. School teachers had improved their salary significantly through agitations. Thus, contractual employees made gains with skilful agitation and litigation, not hard work.
An unproductive working style added to the above deficiencies. In addition to specifying activities through programmes, state departments issued numerous instructions on day-to-day matters. For example, the school education department prescribed not only the curriculum and textbooks, but also the school timetable and pace of lessons. With such centralisation, field officials described their most important activities as implementing programmes, following directions, and supervising junior officials.
Officials said that state reviews and reporting took up a substantial amount of time. Schools and anganwadis had to maintain 20 and 15 registers, respectively. In addition, field officials often received directions to participate in activities that were state priorities (such as sanitation, tree planting, or cultural events) at the expense of their core work. Further, the state government initiated frequent “campaigns” on various issues. Officials said that before one campaign could take off, a new one began, and long-term tasks were neglected.
Research shows that an analysis of the relevant information and a culture of inquiry and openness leads to organisations devising better strategies (Argyris and Schon 1978). But field organisations relied on extreme hierarchy. Several interviewees reported that they could not question senior officials, and supervisors saw disciplinary action as the main tool to motivate employees. Information was used for reporting and day-to-day management, not for planning and strategising. For example, extensive data on students’ test scores was collected, but not analysed.
The ability to achieve outcomes was negatively affected by widespread rent-seeking as officials colluded with law-breakers and pilfered funds. ...[P]eople reported paying bribes for every government service.
Officials did not try to learn from other agencies. For instance, officials charged with reducing domestic violence were oblivious of the successful strategies followed by the self-help group (SHG) federation in the district. Field organisations were not “learning” organisations, or organisations where new skills and insights acquired by individuals became a part of standard operating procedures (Popper and Lipshitz 1998).
The ability to achieve outcomes was negatively affected by widespread rent-seeking as officials colluded with law-breakers and pilfered funds. In FGDs, people reported paying bribes for every government service. Panchayat representatives and journalists pointed to extensive illegal sand-mining from river beds, which was only possible with official collusion. Though a majority of the officials interviewed denied any rent-seeking, a few said that it was widespread and even described the sharing arrangements for money siphoned off from projects.
The fault lines in the administrative system were sharply reflected in the deficient quality of grassroots institutions. In three of the five sample anganwadis, though there was no indication of gross neglect, there was no pre-school education because of the unrealistic workload of those in charge. The most frequent activity observed in anganwadis was the filling up of registers. In the sample schools, teachers engaged with students for around only half the school time, being busy preparing information or attending meetings. In three of the five sample schools, many children in grade 4 and 5 could not read.
A positive feature of the field administration was its close contact with the community. While frontline workers frequently interacted with the community, other officials toured the area, participated in fairs, and met citizens in their offices. However, officials often could not address people’s concerns, such as when labourers’ wages in employment generation programmes, which were to be paid directly to their bank accounts from the state level, were stalled. Teachers saw students as problematic because they lacked the skills to teach children from deprived backgrounds. Rent-seeking added a vicious aspect to all this, with forest guards extorting money from poor tribals.
The real goals of government programmes became casualties when the larger context was ignored. Children below the age of six were provided supplementary nutrition at anganwadis but many missed it as they only attended irregularly because their parents took them along to work.
The deficient administration was an obstacle to problem solving. In FGDs, people identified inadequate livelihoods as their most important issue, but little headway was made in creating more livelihood opportunities in spite of the availability of funds. No district-level analysis was undertaken and an initiative to get SHGs to supply goods purchased by the government was in jeopardy because of rent-seeking. Nor could the drinking water shortage, the second most important problem identified in FGDs, be addressed. The core problem was that the falling ground water table made water sources dry up. But field officials could do little as the departmental scheme focused on provision and repair of hand pumps. The problem was exacerbated through illegal mining on river banks, a consequence of rent-seeking, and the water evaporated faster because of increased exposure to the sun.
The real goals of government programmes became casualties when the larger context was ignored. Children below the age of six were provided supplementary nutrition at anganwadis but many missed it as they only attended irregularly because their parents took them along to work. In the sanitation programme, in the rush to meet targets, the sample villages had been declared open defecation free but an estimated 12% to 40% of the people continued to defecate in the open.
The Way Ahead
Public services and policy implementation cannot improve without understanding and addressing the core problems of the field administration. For example, at present, a key strategy for improving the administration is to increase the use of technology. However, in the case study, while technology had led to better accounting and record keeping, centralisation, a core problem, had increased because of frequent reviews from the state level via video conferences. Moreover, as digital technology was introduced at great speed, software glitches led to new problems, such as students not getting their scholarship money.
The reform of the field administration should be based on more studies and extensive deliberation. But some ideas emerging from this study can be used as starting points. Fundamentally, meaningful reform requires a shift in the conceptualisation of field organisations, from executors of orders to sites for analysis of the local situation and formulation of contextual strategies. For this, local governments need to be empowered to take context-appropriate decisions.
The fragmented structure can be addressed by fusing the numerous field organisations mirroring state departments into five or so organisations. Three organisations—concerned with agriculture, livelihoods and allied activities; infrastructure development and maintenance; and social welfare—can be overseen by local governments. The remaining two—concerned with law enforcement and general administration and revenue collection—can be supervised by the district collector. These can have offices at the district and two sub-district levels to promote local decision-making and action.
Changing the working style in the field is difficult without parallel changes at other levels of government, but some broad protocols for working methods, including situational analysis, consultation, weighing pros and cons, and eliciting feedback, can be developed and codified.
With fewer organisations, each of these can be provided with a skilled administrator and the varied expertise needed. The current ad-hoc policy of not filling up vacancies needs to be discontinued. To increase the motivation of government personnel, the recruitment of poorly paid contractual employees and the patronage-based postings of regular employees have to be eliminated, and promotion avenues increased substantially.
Changing the working style in the field is difficult without parallel changes at other levels of government, but some broad protocols for working methods, including situational analysis, consultation, weighing pros and cons, and eliciting feedback, can be developed and codified. While the use of technology should be promoted, better outcomes would be achieved with a slower pace that allows glitches to be sorted out.
Eliminating rent-seeking is a challenge because it is widespread across all levels of government. Stopping patronage-based postings would help to some extent, as would strengthening accountability mechanisms such as the Right to Information and social audit. Establishing autonomous investigating agencies to which citizens can complain could be considered, though these could also be captured by rent-seeking nexuses.
Acknowledgements: The work on this paper was supported by the Professional Assistance for Development Action (PRADAN) and the Centre for Policy Research (CPR), New Delhi.
Rashmi Sharma is a Senior Visiting Fellow at the Indian Council for Research on International Economic Relations (ICRIER), and a former IAS officer.
Argyris, C. and Schön, D. A. (1978): Organisational Learning: A Theory of Action Perspective. Boston: Addison-Wesley.
Arnold, D. (1976): “The Police and Colonial Control in South India”, Social Scientist, 4 (10), pp. 3–16.
Dantwala, M.L. and Barmeda, J. N. (1995): “Rural Development”, in R.S. Ganpati et al. (Eds.), Public Policy and Policy Analysis in India. New Delhi: Sage.
Government of India (1985): “Report of the Committee to Review Existing Administrative Arrangements for Rural Development and Poverty Alleviation Programmes (CAARD)”, Department of Rural Development, Ministry of Agriculture and Irrigation.
Misra, B.B. (1983): District Administration and Rural Development in India: Policy Objectives and Administrative Change in Historical Perspective. Oxford University Press.
Popper, M. and Lipshitz, R. (1998): “Organizational Learning Mechanisms: A Structural and Cultural Approach to Organizational Learning”, Journal of Applied Behavioural Science, 34 (2), pp. 161–179.
Potter, D.C. (1996): India’s Political Administrators. Oxford University Press.
Sign up for The India Forum Updates
Get new articles delivered to your inbox every Friday as soon as fresh articles are published.
The India Forum seeks your support...
to sustain its effort to deliver thoughtful analysis and commentary that is without noise, abuse and fake news.