Types of experiment. Impeccable and good experiment The concept of an impeccable experiment and its types

22.11.2021

Bodhi: Common Mistakes in Performing Social Experiments.

Bodhi: "The Purpose of Social Experimentation (SE)".

Chapter 2. BASICS OF EXPERIMENT PLANNING

If you want to experimentally test whether music radio programs help memorize French words, you can easily do this by repeating one of the experiments described in the previous chapter. Most likely, you will design your experiment along the lines of Jack Mozart. You will pre-determine both conditions of the independent variable, practice at the same time of day, and document each step of the experiment. Instead of four piano pieces, you could memorize four lists of words like this: listening to the radio, no radio, no radio, with radio. In other words, you can apply the same experimental design, which is Jack.

It is possible that you will understand some of the reasons for your own actions. But something will surely remain unclear, and above all - the sequence of conditions of the independent variable, i.e., the experimental scheme itself. This is not your fault, because you have not yet passed the experimental schemes. In this chapter, this shortcoming will be eliminated. Of course, you can conduct an experiment by simply imitating a model, but it is much better to understand what you are doing. There are no two identical experiments, and blind copying of an experimental scheme often leads to difficulties. For example, Yoko could use regular alternation of two conditions (juice varieties) in her experiment, as she did in the experiment with weavers (use or not use of headphones). But then she would probably know the name of the juice being tested, and "this is exactly what she tried to avoid by using a random sequence. Also, if you do not know the reasons for the various plans and schemes, it will be difficult for you to assess the quality of the experiments that you will read about And, as you remember, to teach you this is one of the main goals of our book.

In this chapter, we will compare those plans that

The experiments in Chapter 1 were built, with less successful plans for doing the same experiments. The model for their comparison will be a "flawless" experiment (which is practically impossible). An analysis of this here will allow us to consider the main ideas that guide us in creating and evaluating experiments. In the course of this analysis, we will introduce several new terms into our vocabulary. Finally, we will determine what is perfect and what is not in the three experimental schemes that were used in Chapter 1. And these schemes represent three ways of ordering, or three kinds of sequences of presentation of various conditions of the independent variable used in the experiment with one subject.

After studying the material in this chapter, you will be able to competently and not imitate someone else's experiment to design your own. At the end of the chapter, we will be asked questions on the following topics:

1. The degree of approximation of a real experiment to an impeccable one.

2. Factors violating the internal validity of the experiment.

3. Systematic and non-systematic sources of violation of internal validity.

4. Methods for increasing internal validity, methods of primary control and experimental schemes.

5. Some new terms from the dictionary of the experimenter.

JUST PLANS AND MORE SUCCESSFUL PLANS

Undoubtedly, the first condition for conducting an experiment is its organization, the existence of a plan. But not every plan can be considered successful. Suppose that the experiments described in chapter 1, carried out differently, according to the following plans.

1. In the first experiment, let the weaver wear headphones for 13 weeks, and then work without them for 13 weeks.

2. Suppose Yoko decided to use only two cans of each type of juice in her experiment, and the whole experiment took four days instead of 36.

3. Jack decided to apply the partial method of memorization to the first two pieces, and the holistic method to the next two.

4. Or, keeping the same sequence of methods, Jack chose short waltzes for the experiment, rather than the longer pieces he usually learned.

We feel quite clearly that, compared with the experiments previously described, all these plans are unsuccessful. And if we had comparison sample, then we could definitely say why exactly the original plans were better. The flawless experiment serves as such a model. In the next section, we will discuss it in detail and then see how it applies to evaluate our experiments.

PERFECT EXPERIMENT

We now have examples of successful and unsuccessfully designed experiments. Can a well-designed experiment be further improved? And is it possible to make the experiment absolutely flawless? The answer is: any experiment can be improved indefinitely, or - which is the same thing - a flawless experiment cannot be carried out. Real experiments improve yourself as you get closer to perfection.

Flawless Experiment

Flawless Experiment- an experimental model that is not feasible in practice, used by experimental psychologists as a standard. This term was introduced into experimental psychology by Robert Gottsdanker, the author of the well-known book Fundamentals of Psychological Experiment, who believed that the use of such a model for comparison would lead to a more effective improvement of experimental methods and the identification of possible errors in planning and conducting a psychological experiment.

Criteria for a flawless experiment

A flawless experiment, according to Gottsdanker, must satisfy three criteria:

Ideal experiment (only independent and dependent variables change, there is no influence of external, or additional, variables on it)
Infinite experiment (the experiment must continue indefinitely, since there is always the possibility of a manifestation of a previously unknown factor)
An experiment of full correspondence (the experimental situation must be completely identical to how it would happen "in reality")

Literature

Gottsdanker R. Fundamentals of psychological experiment. M.: MGPPIA, 1982. S. 51-54.

Wikimedia Foundation. 2010 .

See what "Flawless Experiment" is in other dictionaries:

This page is proposed to be merged with Laboratory experiment (psychology) ... Wikipedia

This term has other meanings, see Experiment (meanings). Check information. It is necessary to check the accuracy of the facts and the reliability of the information presented in this article. There should be explanations on the talk page ... Wikipedia

Held in special conditions experience for obtaining new scientific knowledge through the targeted intervention of the researcher in the life of the subject. The concept of "psychological experiment" is interpreted ambiguously by various authors ... Wikipedia

Validity is a measure of compliance with the extent to which the methodology and results of the study correspond to the tasks. In particular, validity is considered a fundamental concept of experimental psychology and psychodiagnostics. ... ... Wikipedia

- (English validity) a measure of compliance with the extent to which the methodology and results of the study correspond to the tasks set. In particular, validity is considered a fundamental concept of experimental psychology and psychodiagnostics. As in ... ... Wikipedia - Below is a list and summary of the episodes of the television series Happy Together, the first episode of which aired on March 8, 2006. The series tells about a dissatisfied shoe salesman Gennady Bukin and his family, as well as neighbors and others ... Wikipedia

old age- the final period of life, the conditional beginning of which is associated with a departure from direct participation in the productive life of society. The chronological definition of the boundary separating old age from maturity is not always justified because of the huge ... ... Great Psychological Encyclopedia

The Perfect Experiment

Perfection is best defined in terms of the ideal experiment (Keppel, 1973, p. 23). In an ideal experiment, only the independent variable is allowed to change (and, of course, the dependent variable, which takes on different values under different conditions). Everything else stays the same, and so only the independent variable affects the dependent variable. In our three well-designed experiments, this is of course not the case. Weavers wore headphones and worked without them at different times - on even or odd weeks. The plays that Jack memorized using the whole and partial methods were also different. Youko never drank both varieties of tomato juice on the same day. In each case, something else changed besides the independent variable. In later chapters, we will discuss another type of experiment that uses different subjects for each condition of the independent variable to eliminate temporal variations (such as odd and even weeks) and task differences (memorized pieces). But even they do not meet all the requirements of an ideal experiment, because the subjects will also be different. As you will soon see, a perfect experiment is impossible. However, the idea itself is useful, it is by it that we are guided by the improvement of real experiments.

In an ideal (impossible) experiment, the weaver would have to work with and without headphones at the same time! Jack Mozart would memorize the same piece at the same time in whole and in part methods. In both of these cases, the difference in the values of the dependent variable would be due only to the independent variable, the difference in its conditions. In other words, all incidental circumstances, all other potential variables would remain at the same unchanged level.

The concept of flawless experiment was introduced into psychology by Robert Gottsdanker. Psychologists view a flawless experiment as a model in which all requirements are embodied and nothing threatens a reliable conclusion. Such an impeccable research model is unattainable in reality. However, this concept contributes to the development and improvement of experimental methods, avoiding possible errors in the experiment.

R. Gottsdanker defined the criteria for an ideal experiment: it must be ideal, infinite and an experiment of full compliance.

In an ideal experiment, only the independent and dependent variables change, there is no influence of external or additional variables on them. A variation of his pure experiment, in which the researcher operates with only one independent variable and its fully refined conditions.

In an infinite experiment, the number of trials and subjects make it possible to cover all sources of variable variability. Such an experiment can continue indefinitely, since there is the possibility of an unknown factor acting. In order to find out all the side effects that distort the effect of the independent variable, the researcher must continue research indefinitely both in time and in number of attempts, since there is always the possibility that in the next trial something can violate the ideality of the experiment.

In a full fit experiment, the additional variables must fully match the reality counterparts of those variables. The experimental situation is completely identical to the actual real situation.

The model of an ideal experiment is an unattainable ideal to which one must strive. The closer the real experiment of this model, the better it is.

D. Campbell offers the following criteria for a good experiment:

1. A good experiment determines a clear temporal sequence: the cause has in time to precede the effect.

2. The influence or influences must be statistically related to each other. If the possible cause and effect are not related (no covariance), then one phenomenon cannot be the cause of the other.

3. There should be no alternative plausible explanations for the causes of the effect, that is, it is necessary to exclude the influence of side variables or at least control them.

4. It is important to correctly identify the cause and effect in terms and concepts.

The use of experiment in various areas of psychology has its own specifics. Psychological experiments are aimed at investigating certain problems, and in various branches of psychology, their specific problems occupy a central place. The main problems are determined by the researcher's curiosity, his versatility, imagination, and also by the fact that there are opportunities for the implementation of experimental designs. For example, the eminent psychologist John Watson, in his book "Behaviorism" (1924), expressing conviction in his ideas, suggested the following: "Entrust me with a dozen healthy normal children and let me bring them up as I see fit; I guarantee that by choosing each of them randomly, I will make him what I think: a doctor, a lawyer, an artist, a merchant, even a beggar or a thief, regardless of the data, abilities, vocation or race of his ancestors. The design of such an experiment may be astounding, but of course such a proposal goes too far beyond what most of us consider acceptable.

Professional requirements for psychologists do not clearly define the nature and content of their research activities, teaching and counseling. In any environment, not just at a university or scientific institution, there is an opportunity for research. The experiment is possible in a school, office, state or commercial organization, in everyday life, on vacation.

If you want to experimentally test whether music radio programs help memorize French words, you can easily do this by repeating one of the experiments described in the previous chapter. Most likely, you will design your experiment along the lines of Jack Mozart. You will pre-determine both conditions of the independent variable, practice at the same time of day, and document each step of the experiment. Instead of four piano pieces, you could memorize four lists of words. in the following way: listening to the radio, no radio, no radio, with radio. In other words, you will be able to apply the same experimental design as Jack.

It is possible that you will understand some of the reasons for your own actions. But something will surely remain unclear, and above all - the sequence of conditions of the independent variable, i.e., the experimental scheme itself. This is not your fault, because you have not yet passed the experimental schemes. In this chapter, this shortcoming will be eliminated. Of course, you can conduct an experiment by simply imitating a model, but it is much better to understand what you are doing. There are no two identical experiments, and blind copying of the experimental design often leads to difficulties. For example, Yoko could use regular alternation of two conditions (juice varieties) in her experiment, as she did in the experiment with weavers (use or not use of headphones). But then she would probably know the name of the juice being tested, and that's exactly what she was trying to avoid" - using a random sequence. Also, if you don't know the rationale behind the various plans and schemes, it will be difficult for you to judge the quality of the experiments you read about. And, as you remember, to teach you this is one of the main goals of our book.

In this chapter, we will compare the design of the experiments in Chapter 1 with less successful designs for the same experiments. The model for their comparison will be a "flawless" experiment (which is practically impossible). An analysis of this kind will allow us to consider the main ideas that guide us in creating and evaluating experiments. In the course of this analysis, we will introduce several new terms into our vocabulary. In the end, we will determine what is perfect and what is not in the three experimental designs that were used in Chapter 1. And these designs represent three ways of ordering, or three kinds of sequences of presenting different conditions of the independent variable, used in the experiment with one subject.

1. The degree of approximation of a real experiment to an impeccable one.

2. Factors violating the internal validity of the experiment.

3. Systematic and non-systematic sources of violation of internal validity.

4. Methods for increasing internal validity, methods of primary control and experimental schemes. 5. Some new terms from the dictionary of the experimenter.

JUST PLANS AND MORE SUCCESSFUL PLANS

Undoubtedly, the first condition for conducting an experiment is its organization, the existence of a plan. But not every plan can be considered successful. Suppose that the experiments described in Chapter 1 were carried out differently, according to the following plans.

1. In the first experiment, let the weaver wear headphones for 13 weeks, and then work without them for 13 weeks.

2. Suppose Yoko decided to use only two cans of each type of juice in her experiment, and the whole experiment took four days instead of 36.

3. Jack decided to apply the partial memorization method to the first two pieces, and the holistic method to the next two.

4. Or, keeping the same sequence of methods, Jack chose short waltzes for the experiment, and not the longer pieces that he usually learned.

We feel quite clearly that, compared with the experiments previously described, all these plans are unsuccessful. And if we had a sample for comparison, then we could definitely say why the original plans were better. A flawless experiment serves as such a model. In the next section, we will discuss it in detail and then see how it is applied to evaluate our experiments.

PERFECT EXPERIMENT

The Perfect Experiment

Perfection is best defined in terms of the ideal experiment (Keppel, 1973, p. 23). In an ideal experiment, only the independent variable is allowed to change (and, of course, the dependent variable, which takes on different values under different conditions). Everything else stays the same, and so only the independent variable affects the dependent variable. In our three well-designed experiments, this is of course not the case. Weavers wore headphones and worked without them at different times - on even or odd weeks. The plays that Jack memorized using the whole and partial methods were also different. Youko never drank both varieties of tomato juice on the same day. In each case, something else changed besides the independent variable. In the following chapters, we will talk about the Other Type of Experiments, in which different subjects are used for each of the conditions of the independent variable, which allows us to eliminate time variations (like even and odd weeks) and differences from tasks (learned pieces). But even they do not meet all the requirements of an ideal experiment, because the subjects will also be different. As you will soon see, a perfect experiment is impossible. However, the idea itself is useful, it is by it that we are guided by the improvement of real experiments.

Endless Experiment

Poor Yoko! In her case, even a perfect experiment will not be flawless. It is not for nothing that she fears that in different jars tomato juice of the same variety differs in quality. Even if she were to conduct a perfect experiment, contriving to drink two different varieties of juice from the same glass at the same time, her assessments would still apply only to particular examples of each variety. Yet Yoko could have eliminated the effects of juice quality variability across cans by performing another impossible feat. "All" she needs is not to stop her experiment after 36 days and continue it indefinitely. Then she would be able to average not only the variability of each type of juice, but also possible fluctuations in her own assessments of its taste qualities. This is an endless experiment. It is easy to see that it is not only impossible, but also meaningless. After all, the general meaning of the experiment is to draw conclusions based on a limited amount of data that have a wider application. However, the infinite experiment, like the ideal one, also serves as our guiding idea.

As a matter of fact, Jack Mozart and the authors of the study in the weaving shop could also be asked to conduct an endless experiment instead of an ideal one. For even if, in an ideal experiment, Jack finds that the partial method is more effective for this particular piece, the question remains whether the advantages of this method will continue when learning other pieces. The first experiment raises the same doubts: what if the weaver worked better with headphones only during the experiment? However, they (and you) need to be warned that the endless experiment also has its drawbacks. The very fact of presenting one of the experimental conditions to the subjects may affect (during the study period) their work under another condition. Perhaps the partial method was more effective during the experiment only due to the contrast with the holistic method. And after the experiment, a single method will be applied, and the contrast factor will disappear. All this proves that neither ideal nor endless experiments are completely flawless. Fortunately, they have not only different disadvantages, but also different advantages and can serve to evaluate real experiments, which are far from flawless.

Full match experiment

Neither ideal nor endless experimentation can eliminate the shortcomings of Jack Mozart's unsuccessful study of learning waltzes instead of sonatas. At best, Jack could do a brilliant experiment on waltzes - which, however, will not make them sonatas!

In order to completely exclude defects of this kind, an experiment of full correspondence is needed. This experiment is also meaningless, although it is practically feasible. In his research, Jack would have to memorize the same pieces that he will learn after him. There is no benefit from such an experiment, as well as from the infinite. But then no one will be able to point out to Jack the inconsistency of the pieces that he learned in his experiment.

All three kinds of flawless (almost) experiments are unrealistic. An ideal experiment is impossible, an experiment of full compliance is meaningless, and an infinite experiment is both. They are useful as "thought" experiments. They tell us what to do to create an effective experiment. Ideal and infinite experiments show how to avoid extraneous influences and thereby gain greater confidence that the experimental results do indeed reflect the connection. independent and dependent variables. The full fit experiment reminds us of the need to control other important experimental variables, which we keep unchanged.

GENERALIZATION, REPRESENTATIVENESS AND VALIDITY

As we established in Chapter 1, the goal of any experimental study is to ensure that conclusions based on a limited amount of data remain valid outside of the experiment. This is called generalization. Our analysis of an impeccable experiment shows that the reliability of experimental conclusions is determined by at least two requirements. The legitimacy of possible generalizations also depends on them. The first requirement is that the relationship between the independent and dependent variables found in the experiment should be free from the influence of other variables. The second requirement is that the constant level of the additional variable involved in the experiment be consistent with its level in the wider field of practice.

Representativeness

We already know that a flawless experiment is impossible, but it gives us guidelines for properly designing real experiments. We can now ask the question of the application of these principles. The answer is simple - you need to determine how successfully a really conducted experiment represents (represents) an impeccable experiment. First of all, let's see to what extent the possibility of extraneous influences on the dependent variable is excluded in our experiments.

In the original study, conducted in a weaving shop, the subject worked for 13 weeks with headphones and 13 alternating weeks without headphones. In the "unsuccessful" revision of the experiment, she wore headphones for the first 13 weeks, and the next 13 worked without them. In an ideal experiment, the subject would have to work with and without headphones at the same time. It is clear that the scheme of alternating weeks approaches this ideal to a greater extent. An alternation of two conditions, or ABABABABAB, etc., is more representative of their simultaneous presentation than a sequence of only A and B.

In his original experiment, Jack Mozart learned the pieces in the following order: holistic method - partial - partial - holistic. In the "unsuccessful" experiment, the sequence was different: integral - integral - partial - partial. In the first case, the average positions of the holistic and partial methods were the same. The holistic method occupied positions 1 and 4 in the sequence with an average of 2.5. Partial method positions were 2 and 3, mean 2.5. On the contrary, in the "unsuccessful" experiment, the holistic method occupied positions 1 and 2. the average - 1.5, and the partial - 3 and 4, the average - 3.5. The original experiment again turned out to be more representative for the simultaneous presentation of two conditions.

In the original version of her experiment, Yoko drank both Rittenhouse and BuddinBeadle juices at random over a period of 36 days. In the "unsuccessfully" modified version, it ended in 4 days. It is clear that 36 is closer to infinity, not 4. The original plan better represents the infinite experiment than the modified plan.

The full fit experiment is better represented in Jack's original study than in his modified waltz version. Although Jack did not learn all the pieces that he intended to learn in the future, he took exactly the same type of pieces, that is, he chose the appropriate level of the additional variable. And the waltz version turns out to be “inadequate,” since these pieces differ in their level from those that Jack would have learned in the experiment of full compliance.

Summing up, we can say that more reliable information about the relationship between the independent and dependent variables is given by those experiments that better represent the ideal and infinite experiments. And the closer the level of a significant additional variable in the conducted experiment to its level in the full correspondence experiment, the better the real situation under study is represented in it.

Validity

Depending on how real experiments represent flawless, they are called more or less valid. An impeccable experiment would allow one to unmistakably separate the right hypothesis from the wrong one. If Jack Mozart could run a flawless experiment, he would know exactly which of his hypotheses is correct: the partial method is better or the holistic method is better. Thus, when you talk about the validity of an experiment, you are assessing the quality of the work that you intend to do to determine the validity of one of the competing hypotheses.

Internal validity

All three of the “failed” experiments we described lacked internal validity. This means that they do not allow us to consider the resulting picture of the relationship between the independent and dependent variables as reliable. And as we have seen, all sorts of extraneous influences are to blame for this. An experiment that lacks internal validity cannot be used to determine which hypothesis about the relationship between the independent and dependent variables is true and which is false. For example, if it is not clear to us why the weaver worked better: because she wore headphones, or because the weather was fine, we cannot consider the results of the experiment sufficient to determine the true and false hypotheses about the effect of headphones on labor productivity.

The term "intrinsic" emphasizes the essence of this kind of validity. It can be said that an experiment devoid of internal validity is a failure, so to speak, from the inside, by its very nature. Indeed, if it does not allow you to verify the reliability of the found ratio of the independent and dependent variables, it is simply useless.

External validity

An "inadequate" experiment that Jack could have carried out, learning waltzes instead of sonatas, would not have been a failure in principle. It would be a perfectly normal experiment in memorizing waltzes. It cannot be considered useless. Jack could have used his results if he had thought in hindsight that he was actually looking for the most effective method learning waltzes. However, this experiment lacks external validity. It does not provide sufficient grounds for determining the correct and incorrect hypotheses about the best method for memorizing sonatas.

The term "external" refers to the definition of the subject of the experiment - what exactly is it dedicated to. In this case, the experiment was not outwardly valid because "sonatas" are the same necessary component of the tested hypothesis as the independent and dependent variables.

General definitions

The concepts of external and internal validity are central to our entire book. Their application in later chapters is largely determined by what we have just said. We now give more formal definitions of these concepts. True, you will understand their full significance only when you get acquainted with experimental problems of a higher order. But you will already have a basis for a general understanding and further refinement of what validity is and its two types.

Let's start with a schematic representation of the experimental hypothesis:

Independent variable... Ratio... Dependent variable... Levels of other variables. Thus, the hypothesis includes the relation itself and the designations of both its sides. The definition of the validity of an experiment, both internal and external, is as follows. This is the degree of legitimacy of the conclusion about the experimental hypothesis, which is provided by the results of this experiment in comparison with the results of an experiment that is flawless in all three aspects.

The concept of internal validity of an experiment concerns only the relation itself and does not affect what exactly is related. Hence, internal validity is the degree of validity of the conclusion about the experimental hypothesis based on the results of this experiment, in comparison with the conclusion based on the results of ideal and infinite experiments, where changes in the independent and dependent variables occur under the same conditions, and all other side factors remain unchanged.

Any experiment also faces the problem of matching the situation under study with the real one. The question of whether the level corresponds to an additional variable, such as music, has already arisen. Somewhat later, we will discuss similar issues for independent and dependent variables. It is clear that questions about correspondence concern the content of what stands on both sides of the relation under study. These are questions of external validity. It can be defined as the degree of legitimacy of a given conclusion about an experimental hypothesis in comparison with the conclusion based on the results of an experiment with full agreement between the independent, dependent and levels of all additional variables.

In this chapter, we will mainly discuss the problem of internal validity. In any experiment, you will run into this problem from the very beginning; if internal validity is not achieved, it makes no sense to consider external validity. Recall that Chapter 1 presented experiments of a type for which there is little discussion of external validity. And in the next chapter, we will consider experiments in which these questions come to the fore.

No guarantees

We can say that an experiment is valid without actually knowing whether the conclusions are correct. We can prove that it is invalid without knowing that the conclusions are wrong. The reason is that we cannot know in advance which of the two competing hypotheses is correct. After all, if we knew about it, we would not have to carry out the experiment. If Jack had known in advance which of his two hypotheses was correct: (1) the partial method is better or (2) the holistic method is better, he might not have done his research.

In determining the validity of real experiments, we must compare the very procedures for conducting them with the procedures for "conducting" a flawless experiment. A valid experiment represents a flawless experiment better than an invalid one. therefore, in a valid experiment, we are more likely to get the same results that we could achieve in a flawless experiment. However, it is important to remember that limited - and always imperfect - experimental evidence is associated with risk. Even the most highly valid experiment may give inaccurate information about the correctness of the experimental hypothesis, and the information obtained in an invalid experiment may turn out to be accurate. The reasons for this risk and its impact on the interpretation of experimental results will be discussed in the following chapters, especially in Chapter 6 (“Significant results”).

FACTORS THREATING INTERNAL VALIDITY

We can now apply the notion of a flawless experiment (perfect and infinite) to describe what hinders the achievement of internal validity in real experiments. As we shall see, some of these interferences cannot be eliminated; they are necessarily related to the procedures for conducting our not-quite-perfect experiments. Let's say that if Jack needs to learn two pieces, he will inevitably learn one of them first. There are, however, some difficulties that can be overcome if this is taken care of in advance. So, Jack already knew not to use partial and holistic methods at different times of the day.

Changes in time

known side effects. In an ideal experiment, different states of the independent variable are presented to the subject at the same time. Jack couldn't do that, but he could at least study at the same time of day. The time of day is a known side variable (i.e., different from the independent) that can affect the effectiveness of the session, and it must be guarded unchanged. If Jack had been inattentive, then on different days of the experiment, he could have been working either with closed doors, or with open windows. And street noise can greatly affect the effectiveness of classes. Therefore, it is better to keep it unchanged by keeping the windows closed. In the headphone experiment, which lasted more than six months, the researchers were aware of possible changes in temperature and humidity in the weaving shop. Unfortunately, the conditions of the experiment did not allow them to exclude these changes. But the experimenters recorded and tried to take into account the influence of these factors. And most importantly, the alternation of two conditions of the independent variable reduced the influence of these factors. The experimenter should try to determine in advance all possible factors that may change over time. And most importantly, try to keep them at a constant level with each new sample.

Instability over time. But even with the best efforts, the experimenter will not be able to make one sample exactly (except for the difference in the levels of the independent variable) similar to the others. There will always be some instability over time. In the experiment, it manifests itself in the variability of side factors, as well as in some variations of the independent variable itself. Finally, there are always completely unclear sources of strong fluctuations in the answers of the subjects, leading to an increase in the scatter of experimental data. let's consider concrete examples each of these three forms of instability over time.

Variability of side factors. It often happens that way. that the experimenter is aware of the existence of extraneous factors affecting the dependent variable, but cannot directly control them. Some day in the work of a weaver could turn out to be “not the most successful” due to the fact that the day before she went to bed late. Of course, the experimenter could try to convince her not to do this until the experiment was over. But the experiment lasted six months! Having dined the night before in a restaurant, Jack did not feel well while learning one of the pieces - another time he should be more careful.

From sample to sample, the environmental conditions never remain the same. Describing the experiment in the weaving shop, the researchers state:

“It is well known that weaving productivity can be affected by atmospheric conditions. So, with an increase in temperature and relative humidity, the number of thread breaks decreases. On the other hand, a further increase in both, while continuing to favorably affect the physical properties of the yarn, adversely affects the physiological state of people, whose performance may decrease so that it negates any positive effects ”(Weston and Adams, 1932, p. 56 ).

Therefore, even by measuring temperature and humidity, it is impossible to establish precisely their influence on labor productivity. The list of side variables could be continued indefinitely, including subjective factors, such as, for example, the good or bad health of the subject during the experiment. A conscientious experimenter can detect some of these changes, but cannot avoid them. Now you understand why the experimenter seeks to escape from the real world into beautiful soundproof laboratories and deal with such subjects (white rats) whose behavior he can control 24 hours a day. But even there, the heaters sometimes get cold, the water bottles get clogged, and the rats catch a "cold".

Being in the experimental situation itself can cause lasting changes in the behavior of the subject. This was the main conclusion of the famous Hawthorne experiments, a conclusion important to all experimental psychologists. At the Western Electric Plant in Hawthorne, Illinois, a study was conducted on the effect of shop floor lighting on productivity assembly work. Preliminary attempts to establish any pattern ended in failure. A systematic study of the working conditions of workers was then undertaken (Roethlisberger and Dixon, 1946). A major part of this research was experimenting with the switch assembly task. It was “an assembly of telephone relays; this is an operation that is usually performed by women: it is necessary to connect about 35 small parts into a “prefabricated armature” and fix it with four screws” (p. 20).

A special room was equipped for the experiment so that the researchers could control the working conditions and adequately evaluate the activities of the operators. As subjects in the experiment, five young women took part, who had fully mastered this species work. Two independent variables were studied: the distribution of rest periods, as well as the length of the working day and working week. Labor was paid according to the total number of switches assembled by a team of five people.

It was found that regardless of the distribution of rest periods and the length of the working day and week, labor productivity continued to grow for two years! The researchers report, firstly, "a gradual change in social relations within a group of operators towards group cohesion and solidarity and, secondly, a change in the relationship between operators and their controllers. The organizers of the experiment sought to create an atmosphere of mutual support and cooperation among the girls, to save them from unnecessary worries and anxieties. These efforts to create the necessary conditions for the experiment indirectly led to a change in relations between people” (pp. 58-59).

Using our terminology, this situation can be described as follows. Before the experiment, the social working conditions of the subjects were at the same level. In the experimental situation, this “side variable” moved to another level. This led to a long-term change in the dependent variable - labor productivity, despite the fact that objectively the social conditions in the experiment remained unchanged.

The independent variable is i. We cannot count on the complete identity of each of the conditions of the independent variable throughout the experiment. On some days or even weeks, the headphones may not be as comfortable to put on as on the rest. Despite Jack's best efforts, he may have different attitudes, such as the partial method, while learning different pieces. And Youko was aware of the variations in each of the conditions of her independent variable. Juice of the same variety in any two jars is never the same, and the difference between eggs is sometimes very large. Some changes will occur even in those experiments in which, it would seem, complete uniformity of conditions has been achieved. The brightness of the electric light (as a stimulus) will change with voltage fluctuations in the network, and they happen quite often. During the experiment, regular changes may also occur, for example, as the life of a light bulb increases, its light may become less and less bright.

dependent variable. Under the action of the same independent variable, the subject will not always give the same answer. This will be the case even if the experimenter is unusually skillful and punctual in eliminating the instability of side factors and the independent variable.

The instability of the dependent variable is very effectively represented in the graphs showing the results of the two experiments. On fig. 2.1 shows the weekly output of the subject D. in the experiment with headphones. As you can see, she missed the least blows from the tenth to the twelfth week and from the eighteenth to the twenty-second. And her most unfortunate indicators - the largest number of missed strokes - fall on the fourteenth week and the end of the experiment. And what is especially interesting - for both operating conditions, the curves rise and fall together. Changes in productivity over time are undoubtedly more significant than the differences between using and not using headphones.

On fig. 2.2 shows the changes in the answers of the subject in the experiment for the time of the choice reaction. Samples were given every six seconds; the subject had to move the handle towards or away from himself and thereby combine the two points of light. Of course, the points were presented in random order. Over the 70 scheduled consecutive trials, both short fluctuations and more regular deviations were observed in the reaction time of the subject. The shortest reaction time was shown between about the thirtieth and fortieth samples, and the longest between the sixtieth and seventieth. And this increase cannot be considered the result of fatigue, because just before the fortieth test the subject was resting. As a result, the largest indicators slightly exceeded 400 ms, and the smallest - 200 ms, i.e., the reaction time changed in a ratio of two to one.


Rice. 2.1. Weekly labor productivity of subject D. The abscissa axis is the sequence of weeks of the experiment. The y-axis is the number of missed strokes (on average per hour). Dotted line - work without headphones, solid line - with headphones	Rice. 2.2. Selection reaction time for 70 consecutive samples. The abscissa axis is the sample numbers (the dotted line marks the rest period). The y-axis is the reaction time (in ms). The dotted line is the shift of the handle towards you, the solid line is away from you; triangles mark answers with errors

Thus, in the study of reaction time, minute-by-minute and even second-by-second changes were found. They are not associated with fatigue, rather they can be explained by fluctuations in attention. Weaver D.'s performance chart shows significant fluctuations in her productivity. In this case, the rises and falls of the curves, apparently, do not depend on temperature and humidity. True, the increase in the number of missed strokes by the end of the experiment can be explained by the use of artificial (gas) lighting; it was necessary, since the experiment ended in the autumn.

Even when the responses of the subjects themselves are constant, changes can be made by the procedure for measuring them. The counter records each movement of the shuttle, making a new strike. However, the instruments are not always correct. And if measurements are associated with subjective judgments, then they will certainly turn out to be less stable. Jack considered the piece to be completely learned after two unmistakable renditions by heart. However, there were quite a few small errors, almost mistakes, in the performance of the pieces. Sometimes Jack might consider them mistakes, and sometimes not. And this was explained by quite natural fluctuations in his subjective state. Changes in the evaluation of the performance of plays could be regular. For example, during the course of the experiment, Jack could become more and more strict about his mistakes.

Differences in Experimental Problems

One and the same piece cannot be memorized (as ideally) by two various methods simultaneously. But even if the methods follow one another, they still cannot be applied to the same piece. If a piece is memorized, it is memorized. There are experiments in which it is necessary not only to present different experimental conditions at different times, but also to change the difficulty of the tasks. This is a very significant difference from the ideal experiment. How can Jack be sure that the pieces he has chosen are of equal difficulty? But in any learning experiment involving the same subjects, the tasks for different conditions of the independent variable will necessarily be different.

Sequence Effects

In an unsuccessful version of his experiment, Jack first memorized two pieces in the partial method, and then the other two in the whole. We already know that any (including those just described) factors that change over time can affect the quality of his game. However, there are other influences associated with the position of each of the conditions of the independent variable in the sequence of their presentation. The effects of one of the conditions on the following ones are called sequence effects, order effects, or transfer effects. They can be positive or negative, general or specific. The use of the partial method could have had a positive effect on Jack's further studies in the holistic method by increasing practice or getting used to the experimental regimen. It could also have a negative effect: the habit of memorizing pieces in short passages could prevent memorizing large parts, or Jack could simply be tired of studying.

Experimenter's Bias

At the time of the appearance of the car, there was such an anecdote in the form of a riddle. Q: What is the most important screw in a car? Answer: The one that holds the steering wheel. We may ask in the same vein. Question: Which of the factors that threaten the validity of the experiment is the most dangerous? Answer: Experimenter

If the researcher has any expectations about the results of the experiment, especially those related to the preference for one of the conditions of the independent variable, then these expectations will somehow manifest themselves in the experiments, Yoko knew well that the main thing is to randomize the sequence of both varieties of juice. She wanted to eliminate any hint of which variety she was judging each morning. But Jack did not show proper accuracy. First, he picked up pairs of pieces that seemed to him the same in difficulty (in order to learn each of the pieces in different ways), and then he himself arranged them in a certain sequence. But if at the same time he counted on the greater efficiency of the partial method, he could unwittingly select the more difficult pieces from each pair for the integral method.

In addition, subjective assessments of the quality of the performance of plays could fluctuate in a non-random manner (as was shown above). Jack might unwittingly favor one of the methods. Therefore, when evaluating the performance of both pieces of each pair, Jack should not put too much faith in the partial method, but also try to achieve the highest results when applying the holistic one.

In the experiment with headphones, the researchers naturally expected to increase productivity with their help and could well convey their confidence to the participants in the experiment. Therefore, it is possible that weavers (on average) tried to work better with headphones.

One of the most insidious consequences of the experimenter's prejudice is the unwillingness to take into account some experimental data, as if they were obtained under atypical conditions, for example, under strong street noise. Unfortunately, the experimenter's opinion about atypical conditions is often very subjective. Hence, the same noise level will be considered atypical in one state of the independent variable, but quite normal in another.

Even the accuracy of data recording may depend on the bias of the experimenter. It is shown, for example, that in the protocols of experiments on the study of extrasensory perception there are errors in favor of the presence of the corresponding phenomena, if the recorder believes in their existence. Those who do not believe in extrasensory perception do not allow such distortions (Kennedy, 1939). A thorough analysis of this problem as a whole is presented in The Influence of the Experimenter in Psychological Research (Rosenthal, 1976).