What do researchers use to reduce bias and error in measurement of data?
RESEARCH BIASbias = unknown or unacknowledged error created during the design, measurement, sampling, procedure, or choice of problem studiedbias is so pervasive because we want to confirm our beliefsscience is organized around proving itself wrong not rightkey difference between qualitative and quantitative research isattempts to eliminate bias by quantitative researcherexplicit acknowledgement of bias by qualitative researchers(1) design biasresearch design bias is introduced NOT when the study fails to control for threats to internal and external validity BUT RATHERwhen the study fails to identify the validity problems ORwhen publicity about the research fails to incorporate the researchers cautionse.g. Judith Wallerstein's longitudinal study of divorced children is based on a very small sample of white, upper middle class, California families and no control group BUT it has confirmed the basic belief that divorce is bad for kids, so it is difficult to argue that divorce can be either harmless or useful.confirms the bias that children of divorced parents are damaged goodsevidence to the contrary does not fit the public bias against divorce and gets little exposure2nd eg: selecting the most or least of anything creates a regression effectselecting the poorest peple to study the effects of an anti-poverty programselecting the lowest functioning mentally ill people to study the effects of a therapy programselecting chronic homeless to study the effects of a housing programMolidar selected incarcerated women in his qualitative study of female gang membersSUMMARY: unless your report addresses the problem of regression, it will be a biased report (Molidar does not address this)3rd eg: study dropouts (attrition effect)people who drop out may be the ones who needed it mostalternately, people also drop out when the program worksSUMMARY: unless your report addresses the problem of attrition or experimental mortality, it will be a biased report(2) measurement biasmeasurement bias exists when reseacher fails to contol for the effects of data collection and measuremente.g. tendency of people to give socially desirable answersbig problems when asking about violence, sex, money, criminal behavior, or when the person perceives there is something to loose by their answerusing self report is often biased by social desirabilityin self report, we often use a "lie scale" or a social desirability index to control for "impression management"e.g. plant a series of questions like:I have never told a lie on purposeI always know the difference between right and wrongthe score on the social desiability or lie scale is then used to statistically control for self-reporting biasmost clinical research is highly vulnerable to measurement biase.g. using an invalid measurethe "self esteem" problem: tendency to think self esteem covers everything(3) sampling biassampling bias exists (beyond regression) when the sampling procedure introduces biasKey Sampling Problem #1: omission of women, Hispanics or other minorities from samples OR studying only minoritiesmost medical studies have been done on white or black maleswe know little about women and heart diseaseALL child abuse studies have been done on women abuserswe know virtually nothing about child abuse by menalmost ALL partner abuse studies have been done on heterosexual coupleswe know next to nothing about gay and lesbian partner violenceKey Sampling Problem #2: targeting the most desirable or most accessible samplee.g. in research on the effectiveness of batterers treatment programs, some researchers use conflictual couples seeking marriage counseling, and exclude court referred batterers, batterers with co-existing mental disorders, batterers who are severely violent, and batterers who are substance abusers . . . and then conduct the research in suburban university settingsthe problem is not the sample, it's the failure to acknowledge the bias the sample brings(4) procedural biasprocedural bias exists most often when we administer the research interview or questionnaire under adverse conditionsusing studentse.g. using psych students for course creditpaying subjectse.g. my own study of addiction and domestic violence--Ss were paid $25 each, or $50 a couplefor a couple of crack addicts, $50 is a powerful incentive to date for a few hourse.g. administering questionnaires in a brief intervale.g. our own study of domestic violence in a Tennessee garment factoryIn addition to social desirability and anti-feminist measurement bias, we also have the questionnaires group administered in a small room during a meeting, and the subjects are paid by the piece so the longer it takes to fill out the questionnaire, the more money it costs them.(5) "type III " error or problem biastype 1 error or false positiveindependent variable had no effect, but you erroneously think it didbias usually results from not acknowledging other factors which could account for the same resulttype 2 error or detection failureindependent variable had an effect, but you didn't noticebias usually results from not acknowledging eitheryour sample was too smallyour measurement was too gross, oryou didn't do a good statistical analysistype 3 error: solving the wrong problembias results fromfailing to acknowledge we asked the wrong questions to the wrong people while trying to solve the wrong problem, orwe fail to adequately identify more problematic background assumptionsEXAMPLE #1: RQ: WHY DO BLACK STUDENTS SCORE LOWER ON IQ TESTS THAN WHITE STUDENTS?asks wrong question; some black students do score lower on IQ tests & some do notunstated and untested assumption: black students are dumber than white studentsbetter question #1: WHICH BLACK STUDENTS SCORE LOWER ON IQ TESTS THAN WHICH WHITE STUDENTS?better question #2: WHAT FACTORS PREDICT BLACK STUDENTS SCORING HIGHER ON IQ TESTS THAN WHITE STUDENTS?EXAMPLE #2: RQ: "WHY DO SOME RAPE VICTIMS HAVE A HIGHER INCIDENCE OF CHILDHOOD INCEST THAN NON RAPE VICTIMS?"this question is better than the previous question because it admits the possibility that some rape victims do not have a history of childhood incest, and implicitly wonders what factors predict which victims do and which victims dont.3 problem w/ this question are:ignores the rapist and focuses on the victimimplicitly assumes that one of the causes of rape is prior victimizationcasts rape as a mental health problem rather than a criminal justice problemBETTER QUESTION #1: WHAT ARE THE CHARACTERISTICS OF MEN WHO RAPE?BETTER QUESTION #2: WHAT FACTORS ARE ASSOCIATED WITH SUCCESSFUL RAPE PROSECUTIONThis question part of a whole class of research which is victim-blamingwill talk more about doing victim-blaming research in a minuteEXAMPLE #3: WHAT EFFECT DO COMPANY PROVISION OF DAYCARE SERVICES HAVE ON THE JOB SATISFACTION OF WOMEN?Implicitly assumes:women are (and should be) the caretakers of childrenwomen should have children and be parentswomen need to have children to be satisfied with their jobBETTER QUESTION: WHAT EFFECT DO COMPANY PROVISION OF DAYCARE SERVICES HAVE ON THE JOB SATISFACTION OF EMPLOYEES WHO HAVE CHILDREN and EMPLOYEES WHO DO NOT HAVE CHILDREN?all above questionsselect vulnerable or minority populations to studyask questions which hinge on an unstated assumptionasking the wrong question and solving the wrong problem (Type 3 error) tends to support the phenomena of "blaming the victim" or what sociologists call thejust world hypothesisJust World Hypothesis = in a fair and just world, good things happen to good people and bad things happen to bad peopleever since adam & eve (in the levantine religions such as Judaism, Christianity, and Islam) anything connected to or close to nature is seen as implicitly BAD (bad=ill, mentally ill, the cause of problems, or the reason problems get worse)women are ascribed with mysterious natural powers (she casts a spell on men)non-white people, native or indigenous people are ascribed with dark powers (especially physical and sexual powers)animals or raw nature or "drive" are ascribed with uncontrollable impulses, which are seen as bad, "Libido"How to blame victims in research:1. Assume homogeneity. Assume a minority or at-risk population is homogenous"all MOYCHes are alike" (MOYCH=Minority of Your Choice)e.g. all women on welfare are similar2. Wonder about variation. Ask the basic question of research: "why dont they score the average?"why arent they like the majority, not-at-risk group?why are black women on welfare?why are latinos so uneducated?3. Speculate on a predictor of variation. Find some characteristic of the subgroup which may explain the deviancemost black women on welfare are unmarriedmany latinos speak spanish in their home4. Assume causality. If the characteristic is found in the minority subgroup, assume it explains the deviance from the normthe reason women are on welfare is that black men have abdicated their role as provider and black women have assumed leadership in the familywe now have a African American matriarchy of dominant, welfare dependent single mothers in the inner city (Plausible, except it ignores facts that: most women on welfare are white and child support non-payment, not race, is the major cause of welfare dependency)assume the reason latinos fail in school is speaking spanish; latino families do not care to be integrated into the mainstream by speaking englishwe now see latino families as non-compliant, resistent, and mono-cultural (Plausible, but ignores the facts that most latino dropouts speak english; many latinos are bicultural, while most mainstream white people are mono-cultural;(Also ignores an alternative conclusion: the reason some latinos fail in school is not because they dont speak english, but because the schools dont speak spanish)5. Attribution. Begin to view the characteristic as an attribute of the groupAfrican americans are on welfare--thats how they areconclude: black women are welfare queensLatino families are not interested in the "melting pot"--they want to do things the way they did in mexico or cuba or puerto ricoconclude; latinos are lazyAnother example: why do rape victims have a higher incidence of incest?rape victims are over sexualized (victims of early abuse, so they are tuned in to sex)study the sexual attitudes,belief & experiences of rape victimsif the results are different than non-rape victims (they will be), assume this supports the theory that rape victims are more sexualized than non-rape victimstherefore, sexualization is a predictor (cause) of rapei.e. the victim's sexuality causes the raperesearch with minorities require extra efforts to keep from biasminority trait, not statekey factor is power, not numberBlack africans in south africa used to be a minority, but in 1994 became a true majority, even though their numbers remained the same"colored" and whites are now minorities in south africacolored=anyone who is not black or whiteminority is anyone who is not (in USA):whitemaleChristianheterosexualabled (physically & mentally)age 18-65H2 do research with minorities:be accountable to your Sscarry out research with full involvement of Ss, from conceptualization to interpretation of resultsuse combination of qualitative & quantitative designsparticipant observationsemi-structured interviews"back validation"panel/pretest/pilotin depth interviewslanguage validationnarrative data (eg self anchored test)avoid standardized instrumentsavoid post-hoc studies; front load minority input and longitateavoid assumption of homogeneityselect problems that are the felt difficulty of the community rather than the researcherLiane Davis' critique of "masculine research"feminine research: connection, collaboration between Ss and rschr, studied in natural context, behaviors not studied in isolationmasculine rsch: hierarchal, researcher defines parameters in advance, Ss provides the information, phenomena & Ss are studied apart from their world, out of contextmain result was the complexity revealed in the feminine study. Argues that social work problems are, in fact, complex, and therefore require a finine approach to researchMartha Heineman Pieper's label of social work research as a "psuedoscience", and contasts to "huerism"PSUEDOSCIENCElogicalpositivistic: all information must come through the senses; if it isnt "sensible" it is not appropriate data; ultimately, if you cant measure it, it isnt realempirical: observations representing realityHUERISM (eg ethnography, phenomenology, ecology, structuralism)Hueristic challenges:controlled experiments create a researcher-practitioner conflictresearcher is always biased, non-neutral observerclinical problems are complicated, cant be reduced to testable statementssituational knowledge is more important than universal lawsstatistical significance NE importanceclinical significancerecognition of bias is more important tahn attempts to eliminate it"solving the wrong problem" is Pieper's overall critiqie: she say that "it is better to find an approximate answer to the right question than an exact answer to the wrong question"
