Sociology 357 Piliavin

FIELD OBSERVATION EXERCISE

DUE DATES: See deadlines

For this exercise, you plan and carry out a structured field observation. You first will do an unstructured observation for 30-60 minutes, in order to develop a hypothesis. In this first part, your task is to train yourself to see as many concrete behaviors as you can without filtering them through any interpretive process. You will discover that this is very difficult to do. In the second part, you select a hypothesis, operationalize the independent and dependent variables, do another 30-60 minutes of structured observation to test that hypothesis, and perform reliability checks on the accuracy of your observations. In this second part, your attention is very narrowly focussed on only the very few variables you have decided to observe. We will have a class workshop after you have done the unstructured observation to help you develop the hypothesis for the structured study.

I suggest that you observe someplace where people are entering a room, building, or other well-defined space. Your dependent variable will be some dimension along which their behavior varies as they enter: "grooming," getting out money, pausing and scanning the room, etc. You are not required to select an "entry" location. You may observe any sort of behavior that interests you. I suggest focusing on entering because it eliminates a lot of the sampling problems in deciding whom to focus on when. Any other locale would work if people flow through fast enough to give you enough subjects but slow enough to permit accurate observation. (An average of one per minute is a good rate. The acceptable range is as fast as three a minute or as slowly as one every three minutes, on average.) People buying things at a sales counter or vending machine might also give a good flow, as might people going through some sort of exhibit or waiting in line. I suggest entering behaviors because they have enough variability that you should get interesting data, and they require some effort to operationalize properly. Any dependent variable that your preliminary observation tells you does indeed vary, and that is of some difficulty to operationalize is also acceptable. The central point of this assignment is to show that you can operationalize an observable variable, so do not attempt to avoid the problem by picking a variable that is so obvious that you cannot imagine there being more than one way to measure it. (e.g. what product a person buys) You will have to do it over if you do this.

About Teams.

You are strongly encouraged but not required to do this exercise with one other class member. Teams have two options:

write a joint report, or
each person write the whole report individually.

Option 1 is appropriate when team members are truly working and learning together. It is unfair and unethical for one student to do most of the studying and writing while another "free rides" under the guise of option 1. Those doing the work have the right to refuse to "give" partners papers they did not help write. If you choose option 2, you work together until the data are collected and, if you wish, put into a statistical table, but you must not collaborate in writing your separate reports. If you find yourself in an ambiguous position about these options because of unforeseen problems, speak to me and I will help you to determine the fairest thing to do.

Steps in Execution

Preliminary Unstructured Observation

With your partner, select a place and time to observe for about 30 minutes. Select the same place (or type of place) and same general time of day as you plan to use for your structured observation. If as you observe it becomes clear that people move through this place too quickly or too slowly, look around for another place.
Your task is to force yourself to see the concrete details of people's actual behavior, not your interpretations of them. Watch what people actually do; suppress the normal instinct to evaluate people or to presume motives. Look at behavioral details like patterns of movement through space, hand gestures, posture, positions of legs or arms, ways of eating or drinking, eye or head movements, amount or volume of talking. "Friendly smile", "in a hurry," "flirting," and "nervous" are interpretations, not actions. As you observe, take brief "jotted notes" of specific behaviors worth observing.

It is OK to be thinking about the kind of behavior you might want to do your structured observation on, but do NOT focus narrowly on ONLY that kind of behavior. Whatever you are thinking about, train yourself to look for the variations and differences among people in these behaviors. Also keep your eyes and mind open to other kinds of behaviors that might be more interesting to you to study. More often than not, one's initial idea turns out to be wrong or uninteresting.
When the time is up, stop and write down what you can recall of the behaviors you saw. Your jotted notes may remind you of things, and there may be others that did not make it into your notes. Make a special note of details you saw that seem worth remembering for your structured research. Then write a short paragraph explaining where and when you observed and attach it to these observation notes. List 1-3 possible hypotheses that arise from this observation. YOU WILL TURN a xerox of THIS IN AS HOMEWORK #2. This doesn't have to be neat. Keep the original for yourself to help you with part 2, the structured observation, and to attach as an appendix to your final report. It is perfectly OK if this is in your normal illegible handwriting with spelling and grammar errors. I just want to know that you did this step, before the specified date. It is NOT worth recopying or typing it.
Compare notes with your partner(s) on what you saw, and if you have time talk over which behaviors seem a promising basis for your structured observation.

Structured Observation

Plan for Structured Observation

Pick your dependent variable, the behavior you will observe. You need to operationalize this variable. This means choosing a level of measurement, deciding on the attributes, and then carefully spelling out what observable cues you will use for observing people's behavior. Basically you will either count how many times, measure for how long, or rate how intensely a person does something (leading to a quantitative measure), or you will categorize their behavior (leading to a qualitative measure). If you categorize, your operationalization is focused on defining the differences among the categories. If you count, time, or rate, your operationalization is focused on defining exactly what the behavior you are counting or timing or rating is, when it begins and ends, and how to distinguish degrees of it. We will discuss this in class.

You need to understand what operationalization of a variable means so that you can carefully define the observational rules for distinguishing when the behavior begins and ends or the exhaustive and mutually exclusive categories of this dependent variable. This operationalization should be so complete that another person could use only your written instructions and record the variable the same way you would. You will operationalize only one dependent variable. Remember to plan to have an "other" category in case something unexpected comes up in the field.
Pick your independent variable and explain its operationalization. It will usually be gender (male, female) or some other obvious physical characteristic. (You may use a more "difficult" independent variable if you wish; just be sure to explain how you operationalized it.) If you use gender, YOU MUST GIVE THE EXPLICIT RULES YOU USE TO DECIDE THE PERSON'S SEX, just as with any other operational definition.

Create a recording sheet for your structured observations. PLEASE NOTE: This format is almost always the best one; students often invent their own formats, which usually are more error-prone and unreliable than this one. Please use this format unless I tell you that another idea is better for your particular project. Let each line be a different subject (unit of analysis). If you are counting, you will make tally marks and then turn them into a number. If you are measuring time or using a continuous rating scale, you will write down the appropriate number. If you are categorizing, select consistent shorthand symbols for each category of the independent and dependent variables. Suppose you agreed to use M and F for male and female, and to use L for licked ice cream cone, B for bit it, and O for anything else. Then your recording sheet might look like this:

Sex	Eating	Identifier	Comment
M	B	Badger sports jacket
F	L	blonde, pink dress
M	O	red hair, freckles, green shirt	some of each
M	B	Asian, yellow shirt	used lips, not teeth
F	L	Af-Am, maroon shirt

Regardless of how you are measuring your dependent variable, you will need "identifiers" for the individuals observed. The "identifiers" are so that you and your partner can go back over your individual data later and check your reliability. Generally, use hair, skin, and shirt/blouse colors as identifiers.

Before the observation, agree upon the symbols and identifiers and set up the columns, being sure to have a few extra data sheets made up in case you get more subjects than you expect. Comment whenever it is difficult to decide how to categorize someone on the independent or dependent variable, or to explain "other" codes.

Plan your sampling procedure. That is, explicitly decide who will count as a subject. In this study, you are "sampling" one period of time, and you should try to study everyone who comes into the setting during that time who is an appropriate subject. However, you should define who "counts" as a subject, both in terms of being in a position for your dependent variable to be relevant, and in terms of their personal characteristics. For example, will you consider children, or only adults? Will people have to enter a certain space or spend a certain amount of time in the setting to count as subjects? Will you exclude certain people (e.g. those wearing employee uniforms)? Will you include only those people who enter a certain physical area, or spend a minimum amount of time in the area? What will you do if too many people come in at once for you to observe? (Don't use a setting where this is the normal problem, but even when the flow is usually OK, you might have occasional problems.) NOTE: You do not have to worry about random or representative sampling at this point.

Carry Out Your Research

Using the form you have developed, you and your partner observe independently (each uses a form, both watch the same people, no discussion) for a minimum of 30 minutes until you obtain a minimum of 40 observations.

NOTE: If you are not working with another member of the class, you must take someone with you as a reliability checker.If it turns out that the pace is so slow that you cannot observe at least 40 people in the time you have available, move to a different setting, or come back at a different time. If it turns out that people are coming in so fast that you cannot possibly observe them, stop and move to a better location or come back at a slower time.

If you realize part way through that there is a problem with your operationalization or sampling, use the comments column to note the details for difficult cases and keep going. If you add a decision rule part way through, record it on your sheet and keep observing. Quit observing early only if the problem

s are so bad that you just cannot use your scheme at all, in which case you need to fix the problem and start over.

Roster your data and enter it into the computer

Set up a final data sheet in which the rows are individuals and the columns are variables. Set up five columns. The first one is for a case identification number. You should then have two columns for the independent variable, and two columns for the dependent variable. These refer to the two partners' codings. You label the independent variable columns with a name indicating what the variable is, such as sex1 (coding of the sex of the case as recorded by partner 1) and sex2 (ditto for partner 2), using a name with no more than 8 letters and numbers, beginning with a letter. Do the same for the dependent variable columns. Then compare your raw data sheets to match up people you observed, using the identifiers and the person's sex (which is almost always recorded accurately, unless ambiguous). Record all of the data from people you both observed onto the data sheet first. At the bottom of the data sheet, record first the ones that partner 1 saw but partner 2 did not, and then those partner 2 saw but partner 1 did not.

You will learn to use SPSS for data analysis. Data entry will be discussed and demonstrated in class, and there will then be a lab in which you will enter and analyze your own data. Data from both partners are entered into the same data file, so that the extent to which both partners have seen the same thing can be calculated. We will be able to calculate both a sample selection reliability (the proportion of cases on which both partners saw the same individuals) and coding reliability (the proportion of cases in which the same people were seen in which the partners agreed on the coding of behavior).

WRITTEN REPORT

PLEASE FOLLOW THIS FORMAT EXACTLY.

About Truthfulness.

Science depends on researchers telling the truth about what really happened in their research, not what they wish had happened. At the same time, students worry that they will be graded down if they tell the truth. So, for each question, I insist that you tell the truth about what really happened in the research, but then follow it with an opportunity to explain what you now think you should have done. If there was a mistake and your self-criticism gives a correct statement about what you should have done, you will receive full credit as if you had done things right in the first place.

OUTLINE

Title page. Title of report, author(s), date. Put partner's name in parentheses at the bottom of the page if you worked with someone but wrote reports separately.
Abstract. Write one paragraph which summarizes your research methods, hypotheses, and findings. You may include this on the title page if you wish.
Body of paper.

Introduction. Write a paragraph stating your topic and why it is worth researching. Discuss in some detail what you saw in your unstructured observation that made you develop your hypothesis. State your bivariate hypothesis. (Note: we will NOT normally do literature reviews in our course assignments, but this is where it would go, and if something you read went into your thinking on this project, this is the appropriate place to mention it.)
Methods of research. (Note: We will write this section in a more closely structured format than the usual research article. This is so I can more easily grade your paper. Number each section of this discussion as it is numbered here, e.g. 2b for operationalization of dependent variable.)

Sampling.

Describe the setting of your research, the date and time of day you conducted it, and any details relevant to understanding your data. A diagram of the physical setting is often helpful here. Discuss any differences between settings or times of day between the structured and unstructured observations. Talk about how you fit into the setting, and how you presented yourself so as not to influence the data collection.
Describe your sampling procedures, including any restrictions placed on eligible subjects, or other procedures for deciding whom to study within the setting.
Evaluation: why you think these procedures were good, or what you now believe should have been done differently.

Dependent variable.

Why you chose your particular conceptual variable and its operationalization.
Complete details on your operationalization as you planned it.
Evaluation: How the operationalization actually worked out, why you think these procedures were good, or what you now believe should have been done differently.

Independent variable.

Why you chose your particular conceptual variable and its operationalization.
Complete details on your operationalization as you planned it. (Note: This answer is typically short, but it must be worded to show that you know what an operationalization is. Even sex must be operationalized.)
Evaluation: How the operationalization actually worked out, why you think these procedures were good, or what you now believe should have been done differently.

Results. (Attach the original messy structured data collection sheet to the back of your paper as an appendix. You will be graded down if it is missing.)

Reliability analysis.

Present the results of the reliability analysis you performed; include the computer printout in an appendix.
Discuss these results. Did you have more than trivial disagreements (more than one case) over either sampling or coding? Can you figure out what caused the problem(s)? (Examples: You had different views of the room. There was one coding category that you were defining differently, and you didn't realize it at the time.)
Evaluation: why you conclude your procedures were good, or what you now believe you should have done differently.

Test of hypothesis.

Prepare a bivariate statistical table to show the relationship between your independent variable and dependent variable, BASED ON the computer printout. If you categorized, use a contingency table, with frequencies and percentages; if you timed or counted or rated, present frequency distributions and a difference of means table. (We will discuss this in class.) The table must have a title, and the variables must be identified.
Write a paragraph discussing your statistical results, saying what they show and whether your hypothesis is confirmed or disconfirmed.
Discuss anything else worth mentioning that you learned in your research, including unexpected events or surprising findings.

Ethical implications. Discuss the ethical implications of your research. Do you feel that your work invaded anyone's privacy? How did you feel about doing covert observations?
Conclusions and interpretations. This is where you talk about the larger issues your research raises, whether you feel that your findings are likely to be more generally true, and what research, if any, you would like to see pursued by yourself or others as a consequence of your research. For this class, it is also a chance to talk informally about what you liked or didn't like about the assignment or the way you did your research.

Appendices

Your unstructured observation notes and hypotheses. (the original messy version.)
Your original structured observation data collection sheet, the one you actually used in the field to get the data. DO NOT recopy or retype this sheet. I want to see the real data.
The computer printout, including frequencies on your variables, reliability analyses, and hypothesis tests. I check them against your data sheet, to be sure you did not make an error. I actually do this.

Group process report. Pick the category that applies to you and answer the relevant questions. I WILL NORMALLY NOT RETURN THESE STATEMENTS, BUT WILL KEEP THEM FOR MY RECORDS. PARTNERS MUST HAND THESE IN SEPARATELY SO THEY CANNOT POSSIBLY HAVE ACCESS TO EACH OTHER'S ANSWERS.

No partner.

How did you feel about working alone? Would you do it again, or would you prefer a group?
How much effort did you have to put into this project?
How well prepared did you feel in terms of course materials and understanding what to do?
Tell me if there is anything I should know about you or your life that you want me to know, especially if it might affect your grade or my ability to be fair in grading your work.

Had partner, wrote separate papers.

Compare you and your partner in the effort you put into the project.
Compare you and your partner in the extent to which you studied course materials and knew what to do for the assignment.
Who did your statistical analysis?
Did you start trying to work together before deciding to write separate papers? How far did you get?
Were there some things you found necessary to discuss in preparation for writing your papers? What?
How did the group process work out? Was it a positive or negative experience? Would you do things differently in the future?
Tell me anything else I should know that might affect your grade or your partner's, or that I should know to be fair in grading your work, or that you would like me to know even if it is not relevant to your grade.

Wrote joint paper.

Do you stand by the paper as written, or is there something you feel should have been said differently? Any corrections you offer at this point will be factored into your grade. This answer may be as long or short as you feel is appropriate.
Compare you and your partner in the effort you put into the project.
Compare you and your partner in the extent to which you studied course materials and knew what to do for the assignment.
Who did your statistical analysis?
How did you go about getting the writing done?
How did the group process work out? Was it a positive or negative experience? Would you do things differently in the future?
Tell me anything else I should know that might affect your grade or your partner's, or that I should know to be fair in grading your work., or that you would like me to know even if it is not relevant to your grade.

Top

Questions? Comments? Please contact jane.piliavi n@wisc.edu