Validity – Let me count the ways…

I was having a little problem understanding what exactly was being referred to as “validity” during the project presentations. I finally figured out it was a difference in discourses (surprise!). My discipline has taught me these discourses around validity constructs:

·         Face validity – the degree to which a measure seems to be measuring what it is intended to measure

·         Content validity – how well a measure covers the range of meaning associated with a concept

·         Predictive validity – how well a measure is associated with future behaviors, etc.

·         Construct validity – how well a measure of a concept is associated with a measure of another concept of a theory that it should be associated with

·         Internal validity – agreement between a study’s conclusions about causal connections and what is actually true (I figured out I was getting closer to the class objective with this one)

Needless to say, I was hearing validity with my “measurement” ears and I wasn’t sure what I was supposed to be addressing.  My bookshelf, however, contains a nice selection of research methodology resources and in Action Research (McNiff and Whitehead, 2006) I found this: “…when we claim our theory has “validity” we say it has truth value and is trustworthy… (has been) demonstrated as having truth value through a rigorous process of public critique” (Pp20-21).

McNiff and Whitehead (2006), discuss the researcher’s validity by raising questions about whether a “practitioner is competent to judge their own work and how they will assure the “watching public” that their findings are credible and trustworthy.” To do this the practitioner/evaluator must make their “evaluation processes visible and show that (processes) are rigorous and robust, and produce strong evidence to show that they as practitioner-researchers are competent and capable” (p. 70).  My methodology entailed having the class “discover” the text alongside me in order to make the processes visible. My qualifications as a research-practitioner, while frequently referenced in class, were not explicitly stated in the presentation.  Further reading discussed “claims-making” and how the researcher’s awareness of validity threats and attempts to control for those are documented.  It is important for the researcher-practitioner to articulate the values that inspired the work, as well.

I began to understand that I needed to ask myself – why did I choose this topic? After giving that some thought, I came up with the following, which I hope will address my bias and improve the validity of my analysis.

Scholarly activity and research are essential activities of the university, as is teaching, creative expression and service.  A review of evaluation practices show a mix of ¼ service and ¾ teaching and research are generally acceptable, teaching success demonstrated by class loads, evaluations, etc.,  and research by publications and/or funding.   “Scholarly activity and research” for teaching faculty have, largely, been measured by the number of quality publications.  My claim: There is a culture shift wherein “funded” research is being weighted more heavily in importance than ever before. Dollars count. Why is this important to me? I am not a tenure/tenure-track faculty, nor am I evaluated by the same standards as tenured faculty, so what is in it for me? I realized this had to do with my values and beliefs.

I am a student of organizational culture and change management. I gravitate towards looking at things through a cultural lens. Throughout my career, I have worked to create environments that can bridge workplace obstacles so that people can “do what they are there to do” – yes, I think I can say I coined that in as much as any of us “coin” commonsense.  I am also a “research practitioner” meaning I do research within the organizational structure of the university – and routinely navigate all of the requisite policies and procedures. I know what it means from a practitioner point of view to conduct research within the university structure. I know how time-consuming and confusing the system can be.  I am aware of what supports are available and what ones are lacking or non-existent. 

My bias? – concern about how this culture shift is going to be managed – about how the organization is going to address the problem of a lack of infrastructure to support the expectation for faculty to pursue ever greater amounts of “funded” research – and all layers that go with that expectation.  How do my concerns affect my study? I am motivated to do something, like maybe raise my voice in the context of my project.  But I have tried to do so in such a way that other “experts” can draw their own conclusion about the discourse in Theme II.  

Thanks for reading....susangale