For the special issue on the philosophical foundations of AI, Journal of
Experimental and Theoretical AI (JETAI)

Design as Inquiry:  The Methodological Legacy of AI

R. P. Loui

Washington University in St. Louis

*with some input from Prof. Miriam Thalos, Prof. Michael Loui, Prof.
Fernando Tohme, and James Meqguier.  This paper is dedicated to Prof.
Henry Kyburg on his seventy-first birthday.

			Method and Discipline

Method counts.  Method, in theory, is what defines disciplines, and it
will not be surprising to anyone that AI, which struggles to be
recognized as a valid and independent discipline, has novel method.
Fundamental research in AI has always been hard to appreciate because
of its methodological novelty.  The main claim of this paper, that AI
has a its own method, is thus not contentious.  But it is rarely
acknowledged (one exception perhaps is Alan Bundy, though John
Haugeland, among others, has pointed from outside).  More frequently,
researchers in AI attempt to assimilate their work to the culture of
another discipline (notably mathematics, psychology, and physics; see
Paul Cohen, for example, for a polemic on empirical method; see Matt
Ginsberg for a canonical mathematical and physics sensibility).  This
paper seeks to reaffirm the claim that AI's method is novel, to depict
more clearly than before what is the method and where is the novelty.

"Method" is the centerpiece of the philosophy of science.  It is a
topic that consumed the attention of the post-Viennese, logic-minded
philosophers for decades (and is assigned to names such as Rudolph
Carnap, Karl Popper, and Imre Lakatos).  There had been a deductive
method, and there needed to be an inductive method.  Amazingly, there
had been little dispute over the method of deductive calculation; yet
there was plenty of discussion when the method was alleged to be
inductive.  Experimental method was a good thing, certainly compared to
the worlds of humors, mesomorphs, and mesmers, but the prescription for
inductive confirmation was elusive (which is not to say that the
rejected pseudo-sciences failed to practice method; whether they did or
not, it is method that led to their rejection).  Method was so central
that there was even a famous reaction, "Against Method," (Paul
Feyerabend) which argued that the vestments of method-prescribed or
method-based inquiry stifled creativity.  What is scientific method?
What methods are non-scientific?  Why is methodic inquiry supposed to
be better than inquiry that is uncalculated, unprogrammed,
unregularized?  I do not believe that anyone thinks these questions are
fully answered, but they have certainly gotten their due philosophical
attention.

Beyond inductive and deductive logics, there are patterns of
"non-scientific inquiry," many of which ring plausible as methods:
dialectical method, historical method, literary method (for example,
Hilary Putnam has been discussing "non-scientific knowledge" lately).
But within the mathematical and scientific disciplines, only two
methods are recognized:  analytic method (analysis, the
logico-mathematical propagation of constraints), and empirical method
(induction, the data-directed construction of theory and confirmation
of hypothesis).  It is my observation that most AI practitioners assume
one of these to be their method.  Roughly, those who prove theorems
think they are analytic, and those who build systems think they are
empirical.  It is fairly obvious upon reflection however that AI should
be neither mainly analytic nor mainly empirical.


			AI is Design for Inquiry

AI, like computer "science", is design.  The method of AI is design.
AI shares its emphasis on design with all of the engineering fields
(mechanical, electrical, chemical, and civil engineering, historically,
but now including genetic, economic, control, symbolic systems, and
biomedical engineering).  There, design has long been acknowledged as
fundamental, characteristic, and constitutive.  Design permeates
engineering.  Not all design is engineering and engineering is not just
design.  There are fields that do not fit entirely within engineering
but which continue to emphasize design:  consider architecture,
industrial design, business organization, pharmaceutical design,
political and legislative design, graphic design, and interior design.
Conversely, there are problem-solving aspects to engineering which need
not figure in design.  Thus, some kind of flow control might be
designed in order to solve a problem.   Meanwhile, architecural
vernacular could be used for expression or cultural affirmation rather
than problem-solving.  Nevertheless, it is not too misleading to think
of engineering when it is claimed that AI has a design method.
Computer "science" in general has an emphasis on design, and to the
extent that AI might really be a part of informatiques, AI inherits
computing's design emphasis.  But I do not want to claim that AI and
computer "science" have the same method of inquiry (though I clearly do
not agree that computer "science" is science) or that either is merely
engineering.

In order to separate design from analysis and experiment, it is useful
to be clear about the latter pair.  Then it will be important to
explain how design could be a method:  how it is that there is a method
of inquiry that utilizes design, not just fields of activity that
employ designers.  That is, how it could be that mind design (and now
social design and even drug design) could differ fundamentally from,
say, costume design.  It will be our task to ponder how it is that so
much of AI design has been thought to be analysis or experiment.
Finally, it will be worth noting why design method has emerged,
particularly with AI, and why it is not going to disappear even as AI
problems and paradigms mutate from decade to decade.

			Two Famous Methods

What is analysis?  Analysis is the use of logic in deductive
calculation, that is, the propagation of constraints which produces
incontrovertible, inarguable, undeniable, infallible, "demonstrated"
conclusions.  Perhaps the fallacy of analysis is the idea in the first
place that there is a way of propagating constraints that provides such
a guarantee.  Analysis includes axiomatic method, complexity analysis,
real analysis, decision analysis, numerical analysis, statistical
analysis qua analysis (as opposed to statistical analysis in
induction), probabilistic analysis (including Bayesian reasoning),
queueing analysis, and analysis of variation, among others.  It
probably does not include performance analysis, chemical analysis,
psychoanalysis, DNA analysis, or hair analysis, where "analysis" is
functioning as a synonym for "inquiry," or "study," in all of its
generality (except to the extent that the study or inquiry is
theory-laden).

A celebrated example of analysis is Kenneth Arrow's "Impossibility
Theorem," according to which a "social choice function" for
"individuals" with "preference relations" does not exist which is
simultaneously "non-dictatorial," "anonymous," et cetera.  Out of the
definition of the terms, and the rules of the language in which the
definitions appear, comes tumbling the contradiction.  From the will to
avoid contradiction are forged the shackles of analytic constraint: if
f is a "social choice function" that is "non-dictatorial," "anonymous,"
et cetera, then the individuals must not have "preferences."  Perhaps
the clearest example of analysis is in the analysis of preferences
themselves:  X "prefers" A to B; and "prefers" B to C; thus, by the
meaning of "prefers", X "prefers" A to C.

Analysis at root is really about language and the rules of language
use.  It is the examination of the consequences of adopting a
descriptive linguistic manner (and it does not really matter what that
language is, so long as its rules take a certain consequence-oriented
form; thus one can do deontic analysis, alethic analysis, or fuzzy
analysis if one writes deontic, alethic, or fuzzy postulates).
Analysis is a kind of semantic activity.  It is a priori:  it does not
require observation of the world; and it is non-ampliative:  it works
from assumptions that are logically strong toward unavoidable
conclusions that are logically weaker, not venturing beyond the
"meaning" of the words used in the assumptions.  Thus, the use of logic
to derive conclusions from assumptions is an analysis of meaning, a
working from meaning postulates to semantic entailments.

Induction is about describing the world, finding regularities in the
world, and resolving the tension between the two.  It is not to be
confused with finite and transfinite "mathematical induction" (a
confusion so unfortunately frequent in AI).  Induction includes
inference to the best explanation, scientific theory construction,
hypothesis testing, estimation, inductive confirmation, classification,
and diagnosis.  Induction is how one is supposed to get a handle on
those assumptions, claims, axioms, postulates, and principles with
which one starts the deductive engine.

A celebrated example of induction is that "the next crow will be black"
on the basis that one hundred prior crows have been black.  Another,
more interesting induction is that "rational preference is not
representable with a real-valued extensive measure" because so many
rational preferences violate the "sure-thing" and "independence of
irrelevant alternatives" properties that are implied by utility
functions.  This kind of induction, which results in the rejection of
axioms, is manifest in lofty "paradigm shifts" as well as lowly
"auxiliary" assumption appraisal.

Induction is a posteriori:  it requires data that represent contingent
facts about the world.  It purports to reveal the nomological and
"natural"; it attempts to characterize an immutable reality that exists
whether one theorizes about it or not, and the existence of which is
not altered by the language used to describe it.  Induction is thought
to be ampliative, to be informative, though it is really just a
nonampliative use of metalanguage to produce conclusions that appear
ampliative with respect to the object language.  To hold that "the next
crow will probably be black, given the past sample of one hundred black
crows" is a deductive consequence which must occur in the metalanguage
of the language in which "the next crow will be a crow" occurs.  It is
not yet an induction.  To detach the probability qualification and
assert "the next crow will be black" indeed appears ampliative with
respect to the claims "the past one hundred crows have been black" and
"the next crow will be a crow."  But it really depends on the meaning
of certain metalinguistic assertions such as "and the next crow will be
random" or "I know of no exceptional conditions," with which AI is
familiar from formalizing nonmonotonic reasoning.  (This same
demystification of induction can be found in AI from Henry Kyburg to
Henry Kautz, from David Poole to Judea Pearl.)

The tension in induction is to find a language that permits description
without too much inconvenience, yet is a language that conveniently
regularizes descriptions:  PV=nRT, for ideal gases, but if too many
gases we encounter are therefore non-ideal, an inconvenience, then
perhaps PV does not equal nRT.  Powerful predictions are convenient,
but correctness is also convenient (there is nothing wrong with
thinking of type I and type II error here).

So far, everything should be familiar to the student of method:  these
are the methods of deduction and induction that dominate mathematical
and scientific work.  The domination extends to computing, and into
AI.  One alleges that one has discovered an algorithm or a complexity
result as though one had discovered a sequence of DNA.  "The
algorithmic world has revealed itself!"  One judges the suitability of
a representational system based on its mathematical properties:  the
more mathematically characterizable the properties, the better the
system.  "Good results come from good analysis."  What data support
your affection for this heurisic?  What axiom system vindicates your
learning algorithm?  One should perform an empirical study of a
system's performance, especially under real-world conditions.  One
should prove a theorem, and the more analysis required, the better.

			Method:  Design

Design is different.  Design is the production of an artifact to meet a
purpose.  The artifact is often physical, such as a device or a
product, but it is increasingly important to design abstract artifacts,
especially information artifacts:  methods, algorithms, and symbol
systems.  Even in the examples where the result is alleged to be a
physical object, the main product of the design activity may be the
concept and method of construction, the economy of manufacture broadly
speaking, not just the instantiation of the object itself.  Celebrated
designs outside of computing include the Sheaffer classic fountain pen,
the Volkswagen Beetle, the Saturn V rocket, the Golden Gate Bridge, the
steel backbone skyscraper, the Vickrey auction, the commercial tax
code, the Amsterdam canals, the black cocktail dress, and Esperanto.
Within computing, every system, every language, every algorithm, and
every mathematical model is an exercise in design:  from Apple
Macintosh to FORTRAN to Rivest-Shamir-Adelman encryption to pi
calculus.  Within AI, there are not only famous designs, such as Deep
Blue, but more importantly, design paradigms, such as neural networks,
backtrack search algorithms, nonmonotonic logics, chart parsers, and
image segmenters.

Design is a priori in the sense that the organization and configuration
of components does not depend on the contingencies of the world, though
it is a posteriori in the sense that its effectiveness at meeting its
purposes depends crucially on the physics, chemistry, and biology, and
on the stage of human history in which the design is embedded.  Design
is intentional and teleological (purposive).  To be a design, there
must be design goals and design uses, designers and users.  The
discipline that is surprisingly well suited to appraise design is
history.

Naturally, design proceeds hand in hand with invention, and here
philosophy must nod to antecedent legal thought:  good design should be
"useful and nonobvious"; it should "advance the state of the art" and
be "reducible to practice."  Good design is memorable.  It is a
landmark in the design space; it declares its tradeoffs clearly:  the
methods of its construction, and the integrity of its concept.  It
opens a manner of theme and variation or closes the discussion about
possibility.  It is an existence proof and it contributes to
aesthetics.  The first design of its kind may be memorable; copies and
repetitions add nothing, nor do interpolations or obfuscations.
Probably every computer science doctoral dissertation committee has
puzzled over the standard for acceptable work and reached this same
conclusion.  Good computer "science" is good design.  It is not merely
empirical confirmation or disconfirmation, tour de force depth of
calculation, plausible falsifiable proposition, erudition of
speculation, literary value of exposition, or possession and exhibition
of expert ability.

There is no question that analysis and empirical work can serve
design.  There is equally no question that experimental design can be
memorable, and so too can be memorable the design of a proof.  But in
those cases, the ultimate aims are empiricial and analytic; in the
other cases, the ultimate aim is to provide information to designers.

If good design produces landmarks, design for inquiry produces maps.
Design for inquiry has long been AI's stated creed (one need only
consult Herbert Simon's last invited AAAI talk and the themes that
originate in _Sciences of the Artificial_), though it has perhaps never
been called quite that.  Design for inquiry seeks to populate the space
of possibilities with examples that represent the variety and extent of
capability.  The line between what computers can and cannot do is drawn
by designing systems, observing their successes and understanding their
failures.  It is usually not hard to verify that a design makes use of
its alleged principles, since the principles are simple enough for all
to see.  It is also not hard to ascertain a design's successes and
failures since the purposes of design tend to be transparent.  Usually
the hard part is designing the bridge for four lanes of traffic, not
verifying that the bridge has four lanes.

Design for inquiry is a special kind of design.  To design for inquiry
is to suppose that there is a phenomenon that can be understood by
constructing artifacts.  Somehow, intelligence, the mental and the
social, can be understood this way.  How?

It cannot simply be that designing computer systems that are
quasi-intelligent produces metaphors for the construction of
psychological (scientific) theories.  This has long been the official
relation between AI and cognitive psychology:  a servitude of the
former to the latter (though psychology, it was allowed, could in turn
provide ideas for design).  Design is not just a "logic of discovery"
(N. R. Hanson) that provides hypotheses, which are then ultimately
subordinated to empirical validation as nomological claims.  And it
cannot be as simple as the claim that the design of bridges serves the
inquiry into bridge-phenomena, or even transportation-phenomena, both
of which seem hopelessly tautological.  It might be, however, that the
design of intelligent systems shares more with the design of fonts,
design of cities, and design of games, and some even more surprising
design activities, than it does with the design of disk drives,
pipeline caches, and motherboards.

			Designable Phenomena

One thing that AI clearly shares with design fields is that its
phenomenon is open; the object of inquiry is largely defined as the
output of the designers.  That is, the scope of intelligence is not
fully defined, nor at any time is it analytically defineable.  It is
not considered to be a circumscribable target, like the world's
regularities or a logico-mathematical system.  (This is not to say that
scientific method is inapplicable to designable moving targets, such as
social phenomena; but that the method treats the enlargeable phenomenon
as unenlargeable.)

Intelligence is subject to further definition as it is encountered.
Encountering intelligence redefines intelligence.  Engineerable
phenomena are all like that.  They behave like the technical terms of
law that are open:  "reasonable" search, "due" process, "effective"
alternatives.  (I have chosen not to use Herbert Hart's full phrase
"open-texured" or Friedrich Waisman's antecedent non-legal term
"porous", much less the derivative "accordian" metaphor of Joel
Fineberg.)  The case law, as cases are decided, makes the definition
clear over time, the phenomenon developed, the legislation evolved.
Revision is forseen.  The same happens with flight, bridge, and
operating system.

Some phenomena, like social phenomena and unlike scientific phenomena,
are constructible.  They are not there until they are made.  Constructs
do not serve roles and purposes until they are constructed, so it would
be impossible to conceive of the entire space of potential (designable)
phenomena.  The scope of potential intelligence makes no more sense
than the scope of potential insurance legislation.  These belong to the
third baseball umpire, "Ain't nothin' 'till I call 'em".  (Recall that
the first two umpires sit on the horns of an empiricist's dilemma:  "I
call 'em as I see 'em" and "I always get 'em right".)  So Grand
Unification, scientific progress, and reduction are all ideas that are
better left to physicists than to engineering fields.

AI is prepared to revise more open concepts now than it has in the past
few decades.  As AI pushes into the social sciences, its contribution
is the design of rule systems, normative systems, games of social
interaction, and political economies.  AI can deliver the game itself,
not just uncover the game or study the efficiency of strategies for
playing it.  AI has changed what is meant by intelligence and with
computing more generally, will soon change what is meant by society.

But in the concepts that AI revises, AI differentiates itself from
engineering.  There are constructible categories which are conceived
more as phenomena, and less as solutions to problems.  So an economic
system is more phenomenon than solution, while an airplane is more a
piece of problem-solving than it is a phenomenon worthy of independent
study.  Bridge phenomena can be understood by designing bridges, to be
sure, but mostly bridges solve problems -- people care about chasms
that need spanning, not bridges that appear spuriously and demand
explanation.  

It is hard to undertake empirical study of constructible
phenomena.  Empirical study is supposed to reveal the inherited,
contingent world, and constructivity seems to be about the exact
opposite:  the making of fairly arbitrary worlds.  Somehow, there are
things that are both constructible, but also naturally occurring.
Somehow, there have been things like economies, and intelligences,
which have been the subjects of empirical study and are now the
subjects of design for inquiry.

Still, there could have been designers of magnets and agricultural
feeding schedules, tall temples and sneakers who were interested in the
phenomena of magnets, feeding, temples, and sneakers rather than the
solution of particular puzzles.  The accident of an antecedent design
history would then have led to a non-engineering category of human
design activity:  magnet lust, feed fancy, temple chic and sneaker
fashion, phenomena worthy of statisticians and scientists.  Is the
difference between AI and chemical engineering only that intelligence
was there before AI applications were recognized and before problems
that required a designed intelligence for their solution were framed
(whereas chemical engineering pops into existence because people need
chemicals engineered)?

			Design of the Evolvable

The most marked difference, I claim, between AI and both the
traditional sciences and the traditional engineering fields is that AI
studies a class of phenomena that are both evolvable and
constructible.  So it is not simply that there had been
phenomenon-defining design before there was design for
problem-solving.  The really queer thing about AI is that intelligence
can be both the result of evolution and of design.  The preexistence of
the designable phenomenon is fact and observation, the sort of thing
that is scientifically addressable, if it does fall a bit short of
nomology.  More importantly, it is evolution that created the
pheonomenon:  intentionless but teleological selection.  No one, we may
suppose, engineered intelligence with the same kinds of intentions with
which one engineers a sneaker until there was quite a bit of
intelligent phenomena walking about.  But intelligence had a purpose
and even something akin to a design history.

So the apt and well-used comparison of AI to flight remains:  one can
inquire about flight through design, since flight occurs before
airplanes, it occurs through natural selection, and its concept can be
extended through design.  Intentional design sheds light on
intentionless "design."  And the similie of flight is better than that
of bridges:  studying naturally occurring bridges through bridge design
sheds light only on the cleverness of the first people who decided to
walk over happy rock formations.  Selection is different from
serendipity.  The fact that there is already a strong teleological
approach to a pre-engineered phenomenon is what makes AI seem
scientific even though it does not use scientific method.

What artificial intelligence says about naturally selected intelligence
is what did not occur naturally.  Sometimes it is not obvious what
could have been but wasn't.  It is worth noting that Gary Kasparov does
not marshal his associative mental energies in a linearizable regimen
of search.  Knowing some alternatives illuminates the process and
constraint that favored one of the alternatives.  With the space of
rule sets for chess-playing well mapped, we can locate and characterize
the individual artifact.  Sometimes the characterization is precise:
"no grandmaster has an endgame table complete to seventeen ply"; and
sometimes it is not so precise:  "Deep blue is the brute Ferrari of
chess programs, not the beautiful Bugatti."  Once the claim is made,
there is no question of falsification, but instead, an effort to
understand the metaphor.  The way to appreciate the fact that a Bugatti
has sixteen cylinders is not by multiplication of bore and stroke, but
by comparison to six-cylinder and four-cylinder designs.  Design
comparisons and metaphor are no less important when an intelligence is
said to use A*.

Even this is not surprising, though it should make us in AI all feel
guilty for reading too much philosophy of physics and philosophy of
mathematics and not enough philosophy of biology.  Teleological
explanation, like design for inquiry, has had trouble finding fame, and
sometimes even acceptance, in the philosophy of science.

Now here is the surprise.  AI is soon to have company; it is not alone
among the disciplines that study naturally occurring and evolvable
phenomena through design.  Biomedical engineering and economic
engineering, genetic engineering and cultural engineering are on the
rise, and will use exactly the same design method as AI.

To understand hearing, engineer some hearing devices.  To understand
auctions and markets, build some artificial ones.  Which genetic
combinations are robust; which matriarchies are stable?  The greater
the extent of speculative design (to see what is possible with what
techniques) and the less design that is tightly targeted
problem-solving, the more these emerging fields will look like AI.

Perhaps it is the utter uselessness of design that AI permitted itself
for decades that makes it now the clearest instance of a field that
understands design for inquiry.  Engineered flight certainly came
first, but it was coeval with the problem-solving and the commerce of
engineered flight.  To inquire, one must have the luxury of time to
inquire.  AI once had this luxury.  It would indeed be nice if AI could
find a methodological excuse to permit itself the will to pure design
once more.