CITATION AND COPYRIGHT INFORMATION:
Cite as: Shields, P. (1999, May). The role of Sense-Making in critical policy research: theoretical and methodological issues. Paper presented at a non-divisional workshop held at the meeting of the International Communication Association, San Francisco.
© Peter Shields (1999).

ESSENCE OF PROJECT:
In our project, Brenda Dervin and I have been using Sense-Making to interrogate dominant policy discourses on residential or household telecommunications users in the United States. In these discourses, actors from government, industry, consumers groups and academia ritualistically construct the residential user as citizens and/or consumers with particular needs, behaviors, interests and desires. Most of the time, these constructs are presented as self-evident and beyond dispute, with little or no empirical support. Aside from legitimating often self-serving rationales, these constructs have material consequences (in terms of who gets what access under what conditions, for example). We have been concerned with addressing this dearth in empirical research. Our motivation for doing so is to add the missing voice of the residential user to the policy discourse. We believe this goal cannot be realized by conducting empirical research that is guided solely by the traditional categories of users that pervades policy and academic discourses (users defined demographically or geographically, for example). These categories are typically imposed externally on the worlds of users and lead to questions (and answers) that are based on observer rather than user perspectives. In our research, we have used Sense-Making to enter the lives and worlds of users so they could talk on their own terms and in the contexts of their own lives about telecommunications issues. This allowed us to generate alternative categories of users based on residential users’ perspectives. A key objective of our project is to compare and contrast traditional and alternative categories and to draw out the policy implications of this analysis.

THE REASONS I TOOK THIS ROAD:
Two “ideal-types” of policy research can be identified. Most telecommunications policy research can be characterized as problem-solving or “administrative” in nature. That is to say, in this research the prevailing institutional arrangements, power relations and structures of knowledge are taken as the given framework for action. The general aim of this research is to make these relations and institutions work more smoothly by dealing with particular sources of trouble. Often this research, wittingly or unwittingly, justifies or condones the interests and actions of dominant actors. By contrast, critical policy research directs itself toward a critical appraisal of the framework that problem-solving research takes as a given. Drawing on political economy theory, social theory and/or textual analysis, critical policy researchers examine who benefits or is likely to benefit from policies that are made within the existing institutional arrangements, power relations and dominant structures of knowledge. They seek to show how biases in the dynamics of the policy process function to privilege some policy options while muting others. Some critical policy researchers are also interested in policy advocacy. That is, they seek to broaden the menu of policy options to include feasible alternatives that are often aimed at changing existing institutional arrangements in some way.

As a self-identified critical policy researcher, I believe Sense-Making can offer much to the critical policy research project. For example, Sense-Making immediately challenges the dominant policy discourse referred to above by providing a critique of the alien and imposed institutional frameworks that are used to portray users. As already mentioned, Sense-Making provides the analyst with the means to enter the lives and worlds of the users so the latter can talk in their own terms about their telecommunications needs, behaviors and assessments. This can provide critical policy researchers with analyses that illuminate the extent to which policy discourses (mis)represent residential users. Furthermore, Sense-Making’s normative commitment to the notion that users should not only be listened to but be able to define the cognitive frameworks within which they speak, guides the policy researcher to develop the often unorthodox policy implications of user views and assessments. These implications often include calling for the reconfiguration of particular institutional practices.

THE BEST OF WHAT I HAVE ACHIEVED:
Part of our project focuses on telecommunications privacy. Along with Katz (1991), we found that there is a substantial gulf between privacy concerns as expressed by residential users and as expressed by telecommunications policy experts. The residential users we talked to were concerned about forms of privacy invasion that touched on them in their daily lives in visible and direct ways (marketing calls, phone harassment, for example). By contrast, the experts focus on processes about which most people are unaware (the large-scale collection and sharing of huge quantities of calling information about telephone users, for example). These “behind the scenes” processes while central to the professional lives of the experts, are not central to everyday life as lived by lay people. The implications of our findings is not that experts should abandon their concerns. This work, we believe, is crucial. Rather, our results suggest that experts must find some way of making their insights relevant to the everyday lives of users, illuminating for users how these hidden processes have concrete privacy implications for them. One way may be the design of properly constructed information campaigns (see Shields & Dervin, 1998-6).

As discussed above, a goal of our overall project is to compare, contrast, and examine the interplay between traditional versus alternative categorizations of users, and, in particular, to address the value of going beyond the traditional demographic user classifications. To this end, our research on telephone privacy compared the power of traditional and alternative user categories in predicting user views and behaviors. As exemplars we used four privacy criterions - felt privacy violated by phone, done anything to protect privacy, report protection helped, and reported protection hindered. These four criterions were selected because they offer a range of experiential moments ranging from externally imposed events (i.e. imposition of privacy violations from outside), to self-controlled actions (i.e. whether action was taken to protect privacy), to cognitive/emotional events (i.e. evaluations of helpfulness). First, we found that as a group the demography categorizations were not the most significant predictors in any of the four predictive analyses. Second, we found that as a group the demography measures played less of a role the more experiential involvement moves toward user control and assessment. Thus, demography did better in predicting whether respondents felt their privacy and been violated by phone than whether they took action; and demography was not even near-significant in predicting either evaluation measure. The alternative categorizations exhibited the most strength in predicting evaluations of how actions hindered. These findings are suggestive of possible gains to be garnered by moving policy analysis beyond traditional demographic predictors (see Dervin & Shields, forthcoming).

WHAT HAS BEEN HELPFUL:
Giddens’ stratification model of the agent has been quite helpful in conceptualizing the problems discussed above (see Giddens, 1984; especially his discussion of discursive and practical consciousness). However, his model does not seem to offer much in the way of helping to address the problems.

WHAT I HAVE STRUGGLED WITH:
As pointed out earlier, I believe Sense-Making can offer much to the critical policy research project. Yet if this claim is to be persuasive, at least two important methodological and theoretical issues need to be addressed. One issue has to do with the role of quantification in critical policy research. Sense-Making provides policy researchers with the means to generate a rich mix of qualitative and quantitative data (on user needs and assessments, for example). Quantitative data can be a particularly important resource in policy making circles. But as Murdock (1997) points out, many critical scholars “persist in seeing any move to incorporate what can be usefully counted as a concession to empiricism. This thins out the materials the analyst has to hand in constructing explanations and restricts the scope of inquiry.” I think Murdock is right to castigate these scholars for their a priori rejection of all forms of quantification. However, I am struggling with the proposition that there may be forms of quantification (one-way analyses of variance and multiple regression, for example) that rest on an ontology (assumptions about causality and relation of the individual to society, for example) that may be at odds with the ontological assumptions that undergirds much critical research (see Morrow, 1994).

Another issue that needs to be addressed, and the one I would like to focus on at the workshop, has to do with the perennial structure-agency problem. Drawing on political economy theory, macro-sociological theory, and/or textual analysis, critical policy researchers tend to focus on how actors (users, policy makers, for example) are differentially embedded in, constrained or even constituted by various sets of social relations. On the other hand, Sense-Making in interviewing practices focuses much more on cognitive-emotional processes and frameworks of actors in attempting to understand actors’ beliefs and behaviors. At the meta-theoretic level Sense-Making does not focus on the actor as its unit of analysis but rather the actor-situation-practice moment. While this removes Sense-Making from the traditional polarity between approaches that theorize structures versus approaches that theorize individuals, I am still struggling with the tendency in much political economy theory to reduce actors to social relations. For example, to what extent are actors’ cognitive frameworks already shaped by social discourses and forces?

The following gives some idea of how this tension plays as I examine how and why residential users use technology in the ways that they do. On the one hand, I tend to draw on structural analysis to situate users within a context of conditions of which they may or may not be aware. These conditions do not specify the course of action to be pursued in every foreseeable situation, but as simply constraining or enabling the production of particular acts. On the other hand, I draw on Sense-Making in order to understand user perspectives on why they act and think as they do in particular situations. One of the problems I am struggling with is how to interpret and weight the utterances of users in light of structural analysis. How can I identify the extent to which users are knowledgeable about the conditions of existence of the various situations they discuss? In other words, I am not sure how to identify the bounds of agents’ knowledgeability in particular situations. This also raises the question of precisely what “knowledgeability” means. How does one set up a dialogue between the knowledge resulting from structural analysis and the perspectives and understandings of users?

WHAT WOULD HELP NOW:
I would benefit from discussion of the problems raised above. I am sure that many of the workshop participants have in various ways confonted these problems.

PROJECT ABSTRACT:
Our book project which is tenatively titled “Telecommunication in the Eye of the Beholder: User Perspectives, Uses, Evaluations and their Policy Implications” uses Sense-Making to interrogate dominant policy discourses on residential or household telecommunications users in the United States. Specifically, we focus on dominant policy discourses pertaining to telecommunications privacy and universal service. In these discourses, residential users are constructed as citizens and/or consumers with particular needs, behaviors, interests and desires. Most of the time, these constructs are presented as self-evident and beyond dispute, with little or no empirical support. Our project addresses this dearth in empirical research. Our motivation for doing so is to add the missing voice of the residential user to the policy discourse. We believe this goal cannot be realized by conducting empirical research that is guided solely by the traditional categories of users that pervades policy and academic discourses. These categories are typically imposed externally on the worlds of users and lead to questions that are based on observer rather than user perspectives. In our research, we have used Sense-Making to enter the lives and worlds of users so they could talk on their own terms and in the contexts of their own lives about telecommunications issues. This allowed us to generate alternative categories of users based on residential users’ perspectives. Key objectives of our project is to compare and contrast traditional and alternative categories and to draw out the policy implications of this analysis.

REFERENCES:
(For references to works by Dervin and colleagues, see Dervin’s writings: Chronological listing.)

Giddens, A. (1984). The Constitution of Society. Berkeley, CA: University of California Press.

Katz, J. E. (1991). Public Concerns Over Privacy: The Phone is the Focus. Telecommunications Policy, 15 (2), 166-68.

Morrow, R. A. (1994). Critical Theory and Methodology. Thousand Oaks, CA: Sage.

Murdock, G. (1997). Thin Descriptions: Questions of Method in Cultural Analysis. In J. McGuigan (Ed.), Cultural Methodologies, (pp 178-192). London: Sage.

Shields, P., & Dervin, B. (1998-6). Telephone Privacy: Residential User Perspectives and Strategies. Media International Australia, 87, 95-113.

OTHER MATERIALS BY THIS AUTHOR ON THIS WEB SITE:
See: http://communication.sbs.ohio-state.edu/sense-making/AAauthors/authorlistshieldsp.html