Copyright, 1994, Yogesh
Malhotra, Ph.D., @BRINT Institute,
All Rights Reserved
Malhotra, Yogesh. (1994). On Science, Scientific Method And Evolution Of Scientific Thought: A Philosophy Of Science Perspective Of Quasi-Experimentation [WWW document]. URL http://www.brint.com/papers/science.htm
Quasi-experiments (Campbell and Stanley, 1963) constitute a class of empirical studies that lack two of the usual features of experimentation -- the lack of full control and absence of randomization. They may be defined as "experiments that have treatments, outcome measures, and experimental units, but do not use random assignment to create the comparisons from which treatment-caused change is inferred" (Cook and Campbell, 1979: p. 6). Their function is to probe causal relations between manipulated independent variables (treatments) and measured outcomes (effects), and their structure involves one or more treatments, measures taken after a treatment, and -- usually -- more than one unit receiving each treatment (Cook, 1983).
Suggesting a normative standard for quasi-experimentation, Cook (1983) argued that:
"Quasi-experimentation should be ontologically realist, but also epistemologically fallibilist, critical, and hence postpositivist, and it should pursue publicly specified methods that try both to verify causal relationships and to falsify them through the judicious use of experimental designs, statistical analyses, and a critically appraised common sense that is heavily dependent on past knowledge in a particular substantive area."Furthermore, as noted by Campbell and Stanley (1963: p. 34), quasi- experimentation differs from pure experimentation in the sense that the researcher "lacks the full control of scheduling of experimental stimuli." Moreover, they have emphasized "because full experimental control is lacking..." the researcher should be "fully aware of the points on which the results are equivocal." Therefore, the judgement of the researcher plays a much greater role in the interpretation of the results than in pure experimentation. This is explicit in their reminder: "While this awareness is important for experiments in which "full" control has been exercised, it is crucial for quasi-experimental design." Therefore, it becomes imperative for a researcher involved in quasi-experimentation to be aware of the various issues that might determine the "equivocality" of results in order to make a better informed judgement (interpretation) of the results. The critical nature of this judgement needs to be given due importance because of the enormity of its impact on society at large.
Cook and Campbell (1979: p. 28), in their argument on the importance of causal phenomena, have emphasized its importance in the social policy process: "Knowledge of causal manipulada, even the tentative, partial, and probabilistic knowledge of which we are capable, can help improve social life." Without denying their argument, we would like to emphasize that in quasi-experiments [that are heavily dependent upon researcher judgement] particularly, incorrect interpretation of results [resulting in incorrect choices or decisions] may have significant detrimental effect on social policy.
The significance of these issues can be appreciated with reference to the [scientific] process of creating "true" and "objective" knowledge. This is one of the main objectives of this paper. Besides serving this purpose, the following discussion is aimed at two other objectives: (a) to familiarize potential researchers with various existing philosophies of the scientific process of knowledge creation, and (b) to facilitate the development of an informed perspective of research activity in general and quasi-experimentation in particular. Essentially, this paper puts together various authors' interpretations of the philosophy of science in order to bring about a balanced opinion on the various aspects of quasi-experimentation approach.
Campbell and Cook (1979) have argued that "quasi-experiments require making explicit the irrelevant causal forces hidden within the ceteris paribus of random assignment." The primary focus of their interpretation is on causation and the related issues of falsification [and truth] through statistical analyses and critical judgement. A brief review of the scientific method and its goals would help elucidate the ongoing debate on the key elements of quasi-experimentation. We provide a review of various interpretations of the role of science in discovering truth in the process of theory development and knowledge creation. This process is then analyzed with respect to the evolution of scientific thought from logical positivism to scientific realism.
The Role Of Science In Knowledge Creation
Hunt (1991, p. 17-18) argued that the major purpose of science is to develop laws and theories to explain, predict, understand, and control phenomena. He suggested that a science must have a distinct subject matter, a set of phenomena which serves as a focal point for investigation. The discovery of the underlying uniformities among these phenomena yields empirical regularities, lawlike generalizations, laws, principles, and theories. Through this process, science aims to produce knowledge of the world by establishment of generalizations governing the behavior of the world (Chalmers, 1990). How does this process relate to the "scientific method"? We explain this in the following discussion.
The Scientific Method
The word science has its origins in the Latin verb scire, meaning "to know." Although, one can "know" through tenacity, authority, faith, intuition, or science, the method of science [or the "scientific method"] is distinct in its notion of intersubjective certification. In other words, it should be possible for other investigators to ascertain the truth content of scientific explanation(s). "Scientific knowledge thus rests on the bedrock of empirical testability" (Hunt, 1991: p. 197). Empirical replication depends on a comparison of "objective" observations of different researchers studying the phenomenon.
Science And Objectivity
All observation is potentially contaminated, whether by our theories or our worldview or our past experiences, but we should deny the conclusion that science cannot, therefore, objectively choose from among rival theories on the basis of empirical testing. Obviously, if objectivity requires that the choice between rival theories be made with certainty (no possibility of error), then science is not objective. In science, all knowledge claims are tentative, subject to revision on the basis of new evidence. Although science cannot provide one with hundred percent certainty, yet it is the most, if not the only, objective mode of pursuing knowledge (Hunt, 1991: p, 200-201). This pursuit is dependent upon the imagination as well as critical and analytical skills of the scientist. It is generally believed that the goal of the pursuit is the discovery of truth.
Science And Truth
Two conceptions of science embody two different valuations of scientific life and of the purpose of scientific enquiry. According to the first conception, science is above all else an imaginative and exploratory activity, and the scientist is a person taking part in a great intellectual adventure. The alternative conception suggests that science is above all else a critical and analytical activity and the scientist is pre-eminently a person who requires evidence before he or she delivers an opinion, and when it comes to evidence is hard to please (Medawar, 1991: p. 30-31).
In the first conception, truth takes shape in the mind of the observer: it is his imaginative grasp of what might be true that provides the incentive for finding out, so far as he can, what is true. This viewpoint is supported by other scholars of science. For instance, Greenwald, et al. (1986) argue that: "One's preliminary hypotheses have a decided advantage in the judgement process."
According to the second conception, truth resides in nature and is to be got at only through the evidence of the senses: apprehension leads by a direct pathway to comprehension, and the scientist's task is essentially one of discernment (Medawar, 1991: p. 30-31).
Inasmuch as these two sets of opinions contradict each other flatly in every particular, it seems hardly possible that they should both be true; but anyone who has actually done or reflected deeply upon scientific research knows that there is in fact a great deal of truth in both of them. For a scientist must indeed be freely imaginative and yet skeptical, creative and yet a critic. What are usually thought of as two alternative and indeed competing accounts of the two successive and complementary episodes of thought that occur in every advance of scientific understanding. This general conception of science which reconciles the two sets of contradictory opinions is sometimes called the 'hypothetico-deductive' conception (Medawar, 1991: p. 32-33, p. 231; Popper, 1959).
Besides these two accounts of the purpose of scientific inquiry, there are two other [mutually competing] conceptions that provide direction to the process of scientific inquiry: consensual view of science and the dissension view of science.
Science As Consensus
According to this approach, scientific knowledge is the product of a collective human enterprise to which scientists make individual contributions which are purified and extended by mutual criticism and intellectual cooperation. According to this theory the goal of science is a consensus of rational opinion over the widest possible field (Ziman, 1967). The two concepts of consensibility and consensuality need to be differentiated for understanding of this goal.
Scientific knowledge is distinguished from other intellectual artefacts of human society by the fact that its contents are consensible. This implies that each message should not be so obscure or ambiguous that the recipient is unable either to give it whole-hearted assent or to offer well-founded objections. The goal of science, moreover, is to achieve the maximum degree of consensuality. Ideally the general body of scientific knowledge should consist of facts and principles that are firmly established and accepted without serious doubt, by an overwhelming majority of competent, well-informed scientists. A consensible message is one which has the potentiality for eventually contributing to a consensus, and a consensual statement is one which has been fully tested and is universally agreed. We may say, indeed, that consensibility is a necessary condition for any scientific communication, whereas only a small proportion of the whole body of science is undeniably consensual at a given moment (Ziman, 1978)
Whereas philosophers located the source of the consensual character of science in the scientist's adherence to the canons of a logic of scientific inference, sociologists argued that science exhibited so high a degree of agreement because scientists shared a set of norms or standards which governed the professional life of the scientific community. Based upon the consensual view of science, science was thought to be strictly cumulative (Laudan, 1984). The opposing view of science is that of dissension.
Science As Dissension
There are four lines of argument which undermine the classical preoccupation with scientific consensus: the discovery that scientific research is much more controversy-laden than the older view would lead one to expect; the thesis of theory incommensurability; the thesis of the underdetermination of theories; and the phenomenon of successful counternormal behavior (Laudan, 1984).
The ubiquity of controversy is succinctly captured by Kuhn (1977) in his objection to the consensual approach: the emergence of new scientific ideas "requires a decision process which permits rational men to disagree, and such disagreement would generally be barred by the shared algorithm which philosophers have generally sought. If it were at hand, all conforming scientists would make the same decision at the same time." Kuhn maintains that it is only the existence of differential preferences and values among scientists which allows new theories to flower. What makes the broad degree of agreement in science even more perplexing is the fact that the theories around which consensus forms do themselves rapidly come and go (Laudan, 1984).
The thesis of incommensurability implies that rival theories are radically incommensurable. The impossibility of full translation between rival paradigms is further exacerbated by the fact that the advocates of different paradigms often subscribe to different methodological standards and have nonidentical sets of cognitive values (Kuhn, 1977).
The underdetermination of data amounts to the claim that the rules or evaluative criteria of science do not pick out one theory uniquely or unambiguously to the exclusion of all its competitors. Feyerabend (1978) and Mittroff (1974) have both argued that many highly successful scientists have repeatedly violated the norms or canons usually called scientific. Specifically, Feyerabend believed that it is undesirable for scientists to ever reach consensus about anything. His ideal of science is the sort of endless questioning of fundamentals which one associates with pre-Socratic natural philosophy: nothing is taken as given, everything can reasonably be denied or affirmed. Indeed, many of the most noteworthy instances of scientific progress seem to have involved scientists who have repeatedly violated the norms or canons usually called scientific. For the supporters of this doctrine, scientific debate and disagreement is far more likely the "natural" state of science than consensus is (Laudan, 1984).
This philosophy of science as dissonance would benefit from a brief elaboration of objectivity [of human beings] and its contribution to knowledge.
Knowledge And Objectivity - A Different Point Of View
Living in a particular world, an individual needs knowledge. An enormous amount of knowledge resides in the ability to notice and to interpret phenomena such as clouds, the sound patterns in a wood, the behavior of a person believed to be sick, and so on. Knowledge resides in the ways we speak, the flexibility inherent in linguistic behavior included (Feyerabend, 1987: p. 106). Knowledge can be stable and it can be in a state of flux. It may be available in the form of public beliefs shared by all, and it may reside in special individuals. It may reside in them in the form of general rules that are learned by rote, or as an ability to treat new situations in an imaginative way (Feyerabend, 1987: p. 109).
As Hanson (1958), Kuhn (1962), Popper (1972), and others have noted, observations are always interpreted in the context of a priori knowledge. The history of science provides numerous examples of the fact that "what a man sees depends both upon what he looks at and also upon what his previous visual-conceptual experience has taught him to see" (Kuhn 1970, p. 113). Even the most 'objective' written presentation is comprehended only by virtue of a process of instruction that conditions the reader to interpret standard phrases in standard ways that would collapse without a community of thinkers arguing in this manner (Feyerabend, 1987, p. 111). Language and perception interact. Every description of observable events has what one might call an 'objective' side -- we recognize that it 'fits' in a particular situation -- and 'subjective' ingredients: the process of fitting description to situation modifies the situation. Features lacking in the description tend to recede into the background, outlines emphasized by the description become more distinct. The changes are noticed when the description is first introduced; they disappear when using it has become routine. The apparent objectivity of familiar 'facts' is a result of training combined with forgetfulness and supported by genetic dispositions; it is not the result of deepened insight (Feyerabend, 1987: p. 106).
Events are structured and arranged in special ways, the structures and the arrangements gain in popularity, they become routine, intellectuals interested in perpetuating the routine provide it with a 'foundation' by showing that and how it leads to important results. Far-reaching practices and views have been supported by a 'reality' that was shaped by them in the first place (Feyerabend, 1987: p. 107).
Evolution Of Scientific Thought - Positivism To Scientific Realism
The following discussion relates the concepts discussed so far to the evolution of scientific thought from logical positivism to postpositivism and scientific realism. The structure of the following discussion is based upon Anderson's (1983) narrative of "scientific progress."
During much of this century "positivism" has dominated discussions of scientific method. The term was popularized by Comte, and generally refers to a strict empiricism which recognizes as valid only knowledge claims based on experience (Abbagnano, 1967; Brown 1977).
During the 1920s positivism emerged as a full-fledged philosophy of science in the form of logical positivism. Developed by the Vienna Circle, a group of scientists and philosophers, logical positivism accepted as its central doctrine Wittgenstein's verification theory of meaning (Brown, 1977; Passmore, 1967). The verification theory holds that statements or propositions are meaningful only if they can be empirically verified. This criterion was adopted in an attempt to differentiate scientific (meaningful) statements from purely metaphysical (meaningless) statements (Anderson, 1983).
According to logical positivists, universal scientific propositions are true according to whether they have been verified by empirical tests -- yet no finite number of empirical tests can ever guarantee the truth of universal statements (Black, 1967; Brown, 1977; Chalmers, 1976). In short, inductive inference can never be justified on purely logical grounds (Hempel 1965).
As a result of these difficulties, Carnap (1936, 1937) developed a more moderate version of positivism, which has come to be known as logical empiricism which became the "received view" in the philosophy of science for approximately next 20 years (Suppe 1974).
Essentially, Carnap replaced the concept of verification with the idea of "gradually increasing confirmation" (1953, p. 48). He argued that if verification is taken to mean the "complete and definitive establishment of truth," then universal statements can never be verified. However, they may be "confirmed" by the accumulation of successful empirical tests. Thus, science progresses through the accumulation of multiple confirming instances obtained under a wide variety of circumstances and conditions.
Logical empiricists believe that all knowledge begins with observation. This leads to empirical generalizations among observable entities. As our ideas progress, theories are formulated deductively to explain the generalizations, and new evidence is required to confirm or disconfirm the theories. Throughout the process, data are given precedence. Indeed, the entire process is viewed as essentially an inductive one. Science in general and knowledge in particular are believed to occur in an upward fashion: from data to theory to understanding (Bagozzi, 1984). Feigl (1970: p. 7) terms this as "an 'upward seepage' of meaning from the observational terms to the theoretical concepts," and it is construed in a similar way by Hempel (1952: p. 36), Carnap (1939: p. 65) and others logical empiricists.
Logical empiricism is characterized by the inductive statistical method. In this view, science begins with observation, and its theories are ultimately justified by the accumulation of further observations, which provide probabilistic support for its conclusion. Of course, the logical empiricist's use of a probabilistic linkage between the explanans and the explanandum does not avoid the problem of induction. It remains to be shown how a finite number of observations can lead to the logical conclusion that a universal statement is "probably true" (Black, 1967). Moreover, attempts to justify induction on the basis of experience are necessary circular. The argument that induction has worked successfully in the past is itself an inductive argument and cannot be used to support the principle of induction (Chalmers, 1976).
In addition to the problem of induction, logical empiricism encounters further difficulties because of its insistence that science rests on a secure observational base. There are at least two problems here (Anderson, 1983). The first is that observations are always subject to measurement error. The second, and perhaps more significant, problem concerns the theory dependence of observation. We have discussed some aspects of this issue under the section on Knowledge and Objectivity. The fact that observation is theory laden does not, by itself, refute the logical empiricist position. It does, however, call into question the claim that science is securely anchored by the objective observation of "reality."
In his development of falsificationism, Popper has offered an alternative method of theory justification which is designed to overcome some of the difficulties inherent in logical empiricism.
Popper And Falsificationism
Unlike positivists, Popper accepted the fact that "observation always presupposes the existence of some system of expectations" (1972: p. 344). For Popper, the scientific process begins when observations clash with existing theories or preconceptions. To solve this scientific problem, a theory is proposed and the logical consequences of the theory (hypotheses) are subjected to rigorous empirical tests. The objective of testing is the refutation of the hypothesis. When a theory's predictions are falsified, it is to be ruthlessly rejected. Those theories that survive falsification are said to be corroborated and tentatively accepted (Anderson, 1983).
In contrast to the gradually increasing confirmation of induction, falsificationism substitutes the logical necessity of deduction. Popper exploits the fact that a universal hypothesis can be falsified by a single negative instance (Chalmers, 1976). In Popper's approach, if the deductively derived hypotheses are shown to be false, the theory itself is taken to be false. Thus the problem of induction is seemingly avoided by denying that science rests on inductive inference. Anderson (1983) notes that Popper's notion of corroboration itself depends on an inductive inference. According to falsificationism, then, science progresses by a process of "conjectures and refutations" (Popper 1962, p. 46). In this perspective, the objective of science is to solve problems.
Despite the apparent conformity of much scientific practice with the falsificationist account, serious problems remain with Popper's version of the scientific method. For example, Duhem (1953) has noted that it is impossible to conclusively refute a theory because realistic test situations depend on much more than just the theory that is under investigation. Quine-Duhem thesis (Quine, 1953; Duhem, 1962) points out that because of all of the background assumptions that might be wrong -- flaws in the equipment, the effects of unknown or wrongly disregarded physical processes, and the like -- any outcome can be rationally distrusted and explained away by ad hoc hypotheses that alter the background assumptions. Falsification can thus be regarded as particularly equivocal (Cook and Campbell, 1979).
The recognition that established theories often resist refutation by anomalies while new theories frequently progress despite their empirical failures, led a number of writers in the 1950s to challenge the positivistic views of Popper and the logical empiricists (Suppe 1974). Various philosophers and historians noted that scientific practice is often governed by a conceptual framework or world view that is highly resistant to change. In particular, Kuhn pointed out that the established framework is rarely, if ever, overturned by a single anomaly (1962). Kuhn's model helped to initiate a new approach in the philosophy of science in which emphasis is placed on the conceptual frameworks that guide research activities.
Kuhn's Scientific Revolutions
Central to the Kuhnian argument is the concept of a "paradigm." A paradigm constitutes the world view of a scientific community (Laudan, 1977; Suppe, 1974). The paradigm will include a number of specific theories which depend, in part, on the shared metaphysical beliefs of the community (Kuhn, 1970). In Kuhn's view, the individual scientist's decision to pursue a new paradigm must be made on faith in its "future promise" (Kuhn 1970: p. 158). Furthermore, in his view, science progresses through "paradigm shifts," but there is no guarantee that it progresses toward anything -- least of all toward "the truth" (Kuhn 1970, p. 170).
Given its (seeming) advocacy of relativism, Kuhn's Structure of Scientific Revolutions became one of the most carefully analyzed and evaluated works in the philosophy of science. [Relativism is discussed further in the paper.] In criticism of Kuhn, some writers (such as Lakatos, 1974; Laudan, 1977) have suggested alternative world view models. Here we briefly explain Laudan's "research tradition" concept, which attempts to restore rationality to theory selection by expanding the concept of rationality itself.
Like Kuhn and Lakatos, Laudan sees science operating within a conceptual framework that he calls a research tradition (Anderson, 1983). The research tradition consists of a number of specific theories, along with a set of metaphysical and conceptual assumptions that are shared by those scientists who adhere to the tradition. A major function of the research tradition is to provide a set of methodological and philosophical guidelines for the further development of the tradition (Anderson, 1982).
Following both Kuhn and Popper, Laudan argues that the objective of science is to solve problems -- that is to provide "acceptable answers to interesting questions" (Laudan, 1977, p. 13). On this view, the "truth" or "falsity" of a theory is irrelevant as an appraisal criterion. The key question is whether the theory offers an explanation for problems that arise when we encounter something in the natural or social environment which clashes with our preconceived notions or which is otherwise in need of explanation (Anderson, 1983).
Critical relativism is a multifaceted philosophy of science: one of its major assertions is that there exists no single "scientific method." Instead, disciplinary knowledge claims are viewed as contingent upon the particular beliefs, values, standards, methods, and cognitive aims of its practitioners. Moreover, critical relativism recognizes that knowledge production in the social sciences is impacted by the broader cultural milieu in which it is embedded (Anderson, 1986).
Critical relativism is skeptical of all claims to scientific knowledge because it recognizes that there are multiple scientific objectives and alternative methods for attaining these objectives (Laudan, 1984). Moreover, critical relativism recognizes that the value of such claims must be assessed in light of their unique modes of production and their methods of justification. To suggest that the hallmark of scientific knowledge is its empirical testability is to settle for far less than we should demand of such an important enterprise as science. Anderson (1986) further argued that the requirement of empirical testability is notoriously ambiguous within the recognized sciences, and it is a criterion that is allegedly met by patently "nonscientific" disciplines (Laudan, 1983).
Critical relativism rejects the basic premise of the positivistic approaches that there is a single knowable reality waiting "out there" to be discovered via the scientific method (Olson 1981). The critical relativist has no quarrel with the metaphysical notion that there may well be a single social and natural reality, but he or she will resist the assertion that science is capable of revealing or even converging upon this "reality" (Laudan, 1981). Instead, the relativist accepts competing research programs for what they are -- different ways of exploring and analyzing natural phenomena, each with its own advantages and liabilities (Anderson, 1986).
After its brief excursion into the relativism, constructivism, and irrationalism of Kuhn and Feyerabend in the 1960s, philosophy of science turned toward realism in the 1970s (Suppe 1977). In other words, the reasoned pursuit of truth returned to the philosophy of science.
Classical realism believes that the world exists independently of its being perceived (Hunt, 1990). A fundamental tenet of modern-day, scientific realism is the classical realist view that the world exists independently of its being perceived. This is contra Olson's (1981) relativism: there really is something "out there" for science to theorize about (Hunt, 1990). However, scientific realism does not embrace "direct" realism which holds that our perceptual processes result in a direct awareness of or straightforward confrontation with objects in the external world. Advocates of scientific realism, though agreeing that our perceptual processes can yield genuine knowledge about an external world, emphatically reject direct realism. They argue for a fallibilistic and critical realism. Hence scientific realism is a middle-ground position between direct realism and relativism. Scientific realism is also a critical realism, contending that the job of science is to use its method to improve our perceptual (measurement) processes, separate illusion from reality, and thereby generate the most accurate possible description and understanding of the world (Hunt, 1990). The practice of developing multiple measures of constructs and testing them in multiple contexts in social science stems from this critical orientation (Cook and Campbell, 1986). In short, scientific realism proposes that (1) the world exists independently of its being perceived (classical realism), (2) the job of science is to develop genuine knowledge about the world, even though such knowledge will never be known with certainty (fallibilistic realism), and (3) all knowledge claims must be critically evaluated and tested to determine the extent to which they do, or do not, truly represent or correspond to that world (critical realism). In conclusion, with respect to truth and scientific realism, the perspective of Siegel (1983, p. 82) seems a fair summary statement: "To claim that a scientific proposition is true is not to claim that it is certain; rather, it is to claim that the world is as the proposition says it is."
In this paper we focused on the philosophy of science issues and attempted to relate them to the interpretation of quasi-experimentation as provided by Cook (1983):
"Quasi-experimentation should be ontologically realist, but also epistemologically fallibilist, critical, and hence postpositivist, and it should pursue publicly specified methods that try both to verify causal relationships and to falsify them through the judicious use of experimental designs, statistical analyses, and a critically appraised common sense that is heavily dependent on past knowledge in a particular substantive area."Having reviewed the various interpretations of "realist," "fallibilist," "critical," and "postpositivist" philosophies, the potential researcher should be better equipped to determine causal relationships and to "falsify" them by making judicious use of various tools and critical "common sense."
With reference to our earlier discussion on the interpretation of quasi-experimentation [with its loss of control and hence requiring greater judgement on part of the researcher] (Campbell and Stanley 1963: p. 34), being aware of the various issues discussed in this paper, the potential researcher would be better equipped to determine the "equivocality" of the results in order to make a more informed judgement (interpretation) of the results.
Abbagnano, N. (1967), "Positivism," Encyclopedia of Philosophy, 6, Paul Edwards, ed., New York, NY: Macmillan.
Anderson, P.F. (1982), "Marketing, Strategic Planning and the Theory of the Firm," Journal of Marketing, 46 (Spring), 15-26.
Anderson, P.F. (1983), "Marketing, Scientific Progress, and Scientific Method, Journal of Marketing, 47, (Fall), 18-31.
Anderson, P.F. (1986), "On Method in Consumer Research: A Critical Relativist Perspective," Journal of Consumer Research, 13, (September), 155-173.
Bagozzi, R.P. (1984), "A Prospectus for Theory Construction in Marketing," Journal of Marketing, 48 (Winter), pp. 11- 29.
Black, M. (1967), "Induction," Encyclopedia of Philosophy, 6, Paul Edwards, ed., New York, NY: Macmillan.
Brown, H.I. (1977), Perception, Theory and Commitment, Chicago, IL: University of Chicago Press.
Campbell, D.T. and J.C. Stanley (1963), Experimental and Quasi- Experimental Designs for Research, Boston, MA:Houghton Mifflin Co.
Carnap, R. (1936), "Testability and Meaning," Philosophy of Science, 3, 419-71.
Carnap, R. (1937), "Testability and Meaning," Philosophy of Science, 4, 1-40.
Carnap, R. (1953), "Testability and Meaning," in Readings in the Philosophy of Science, Herbert Feigel and May Broddeck, eds., New York, NY: Appleton-Century-Crofts, 47-92.
Carnap, R. (1966), The Philosophy of Science, New York, NY: Basic Books.
Carver, R.P. (1978), "The Case Against Statistical Significance Testing," Harvard Educational Review, 48(3), 378-399.
Chalmers, A. (1976), What is This Thing Called Science?, St. Lucia, Australia: University of Queensland Press.
Chalmers, A. (1990), Science and Its Fabrication, Minnesota Press.
Chandrashekhar, S. (1987), Truth and Beauty: Aesthetics and Motivations in Science, Chicago, IL: The University of Chicago Press.
Cook, T.D. (1983), "Quasi-Experimentation: Its Ontology, Epistemology, and Methodology," In Gareth Morgan (Ed.), Beyond Method, Newbury Park, CA: Sage.
Cook, T.D. and Campbell, D.T. (1979), Quasi-Experimentation: Design & Analysis Issues for Field Settings, Boston, MA:Houghton Mifflin Co.
Cook, T.D. and Campbell, D.T. (1986), "The Causal Assumptions of Quasi- Experimental Practice," Synthese, 68, 141-180.
Duhem, P. (1953), "Physical Theory and Experiment," in Readings in the Philosophy of Science, Herbert Feigl and May Brodbeck, eds., New York, NY: Appleton-Century-Crofts, 235-52.
Duhem, P. (1962), The Aim and Structure of Physical Theory, New York, NY: Atheneum. (Originally published, 1908).
Feyerabend, P. (1978), Against Method, London, U.K.: Verso.
Feyerabend, P. (1987), Farewell to Reason, London, UK: Verso.
Greenwald, A.G., Leippe, M.R., Pratkanis, A.R. and M.H. Baumgardner (1986), "Under What Conditions Does Theory Obstruct Research Progress," Psychological Review, 93(2), 216-229.
Hanson, N.R. (1958), Patterns of Discovery, Cambridge, U.K.: Cambridge University Press.
Hunt, S.D. (1990), "Truth in Marketing Theory and Research, " Journal of Marketing, 54 (July), 1-15.
Hunt, S. D. (1991), Modern Marketing Theory: Conceptual Foundations of Research in Marketing, Southwestern Publishing.
Kuhn, T. (1962), The Structure of Scientific Revolutions, Chicago: University of Chicago Press.
Kuhn, T. (1977), The Essential Tension, Chicago, IL: The University of Chicago Press.
Lakatos, I. (1974), "Falsification and the Methodology of Scientific Research Programs," in Criticism and The Growth of Knowledge, Imre Lakatos and Alan Musgrave, eds., Cambridge, U.K.: Cambridge University Press, 91-195.
Laudan, L. (1977), Progress and Its Problems, Berkeley, CA: University of California Press.
Laudan, L. (1981), "A Confutation of Convergent Realism," Philosophy of Science, 48 (March, 19-49.
Laudan, L. (1983), "The Demise of the Demarcation Problem," in The Demarcation between Science and Pseudo-Science, Vol. 2, ed. Rachael Laudan, Blacksburg, VA: Center for the Study of Science in Society, Virginia Polytechnic Institute.
Laudan, L. (1984), Science and Values, Berkeley, CA: University of California Press.
Medawar, P. (1990), The Threat and the Glory: Reflections on Science and Scientists, Oxford, U.K.: Oxford University Press.
Olson, J.C. (1981), "Towards a Science of Consumer Behavior," in Advances in Consumer Research, Vol. 9, ed. Andrew A. Mitchell, Ann Arbor, MI: Association for Consumer Research, v- x.
Passmore, John (1967), "Logical Positivism," Encyclopedia of Philosophy, 5, Paul Edwards, ed., New York, NY: Macmillan.
Popper, K.R. 1959), The Logic of Scientific Discovery, New York, NY: Basic Books.
Popper, K.R. (1972), Objective Knowledge, Oxford, U.K.: Clarendon.
Quine, W.V. (1953), From a Logical Point of View, Cambridge, MA: Harvard University Press, 1953.
Siegel (1983), "Brown on Epistemology and the New Philosophy of Science," Synthese, 14, 61-89.
Suppe, F. (1974), The Structure of Scientific Theories, Urbana, IL: University of Illinois Press.
Suppe, F. (1977), The Structure of Scientific Theories, 2nd ed., Chicago, IL: University of Illinois Press.
Ziman, J. (1967), Public Knowledge, Cambridge, U.K.: Cambridge University Press.
Ziman, J. (1978), Reliable Knowledge, Cambridge, U.K.: Cambridge University Press.