From: D. F. Siemens, Jr. (firstname.lastname@example.org)
Date: Fri May 09 2003 - 15:56:08 EDT
Iain and Jim,
It strikes me that Iain is emphasizing the need for multiple changes
because engineers demand answers /now/, not eventually. Jim approaches
matters from the viewpoint of eventual improvement, with no reason to
label anything "rush!" Both outlooks are functional in their place,
though it must be granted that comprehensive alterations would radically
speed up evolution.
On Fri, 9 May 2003 19:11:08 +0100 "Iain Strachan"
Jim Armstrong wrote:
What I was struck by was that the complex objective was not reached until
the "environment" was (apparently slightly?) enriched by the presence and
interaction of other essentially similar processes. As an ensemble, they
interacted constructively in sufficient measure to increase the system
creativity past the tipping point, enabling the achievement of the
challenging simulation objective. I was also struck by the small number
and simplicity of the "rules" that governed the elements of this
simulation and provided this powerful insight. However, this is
consistent with the lessons of complexity and chaos.
Having worked for around 15 years in the field of numerical optimization
& in providing and maintaining large scale numerical optimization solvers
for engineering problems, I look at this somewhat differently.
All that is happening in the paper is that a very simple function is
being optimised, with incremental values of fitness, such that a simple
and smooth path exists from the start position to the final objective,
like Dawkins's metaphor of "climbing mount improbable". Think of the
population of a Genetic Algorithm as just a cloud of points in
configuration space that wander around under mutation. Then natural
selection biases the "cloud" towards the ones that happen to be better at
any one point. In order for this to work, it must be possible to find an
improvement with just a very few mutations at any one point. This is
exactly what was the case in the paper. In fact they write:
"We also identified every mutation that fell along the line of descent in
this population, and characterized the phenotypic changes associated with
every step (Supplementary Information). The EQU function first appeared
at step 111 (update 27,450). There were 103 single mutations, six double
mutations and two triple mutations among these steps."
(They do not discuss whether the double and triple mutations were
improving of the fitness; some steps produced a drop in fitness).
Now the reason why I uphold strongly to the "deck stacking" argument, and
object to it being called "weak" (as you do below) is that in just about
any non-trivial engineering problem that you want to solve, the above
situation (where improvements can be made by one or two small localised
changes) almost NEVER occurs in practice. I support
optimization/Non-Linear Programming/Equation solving software that has to
deal with systems of anything from a few tens to hundreds of thousands of
equations. They are ALL based around computing the gradient with respect
to all the variables simultaneously & update all the variables at once in
each step. (This is like applying a mutation to every part of the
genome). No one would seriously suggest a solver that only adjusted one
or two variables at a time, leaving the rest unchanged, because they just
don't work. In nearly all engineering problems of interest (certainly in
numerical optimization), the variables are tightly coupled, which means
that you have to make multiple coordinated changes to variables, not just
one or two of them.
But if you think I'm just a crank in the minority here, let me give you
an independent assessment. One of the Fortran routines in the library
that we use happens to be a Genetic Algorithm. It's very well written
and incorporates all the latest heuristics. One developer in my company
asked about using it for a problem that they had. As it turns out, I met
the person who wrote the GA routine a few days later (as he works on an
adjoining site). He is a visiting Prof. at Edinburgh University, a world
expert in optimization, and the author of several large scale codes.
When I told him that someone was interested in his GA code he just
laughed. "Good grief!! He doesn't want to use that _seriously_ ? I only
wrote it for fun. Tried it on a couple of problems. It was _awful_!"
Since Nature seems to have done a pretty good job at solving difficult
engineering problems, I'm therefore distinctly underwhelmed at the claim
that this toy one-variable-at-a-time problem is a demonstration of the
evolutionary origin of complex features.
In a way, this is not too surprising an outcome, resembling parallel
processing in computer terms, as opposed to serial processing. A little
bit of parallel processing can be a very powerful thing - the most
powerful computers on Earth embody a great deal of parallel processing to
achieve their performance!
I don't see how this resembles parallel processing, except that GA's can
exploit parallel processing by having all the members of the population
(the cloud of points in configuration space) evaluated on different
processors. But having also worked quite a bit on parallel processing
myself, the plain fact is that it can at _best_ achieve a linear speed-up
in processing (linear in the number of processors), and that is only
true if there is _no_ interaction between the elements running on
different processors. As soon as interaction is required, then further
slow-downs are incurred because of the communication overheads in sending
data between different processors, and the improvement is sub-linear. If
you don't get it right, the parallel processing results in a dramatic
slow-down. And furthermore, with the rate at which processor speed is
increasing (Moore's Law), parallelism only buys you a few years before
your array will be beaten by a single processor. In the early '90's I
worked on a small array of Transputers, which were the big thing in
parallel processing, using a special language called Occam to incorporate
parallel constructs in the programming language. But the brutal fact is
that the computer I'm typing this email at, which is just a common or
garden entry level domestic machine, is the equivalent of around 1000
transputers in floating point performance! (The Transputer gave around
0.7 million floating point operations per second, or FLOPS. I've tested
this machine at around 700 MegaFlops. My laptop, which is about a year
later vintage than this machine is twice as fast as that again).
The argument concerning the evolution of things of "irrreducible
complexity" seems to embody the notion of a rather linear evolution
process. But the remarkable "processors" of nature are HIGHLY parallel
and, taken with the environments in which they occur, are extremely
(unimaginably) rich and redundant with respect to materials, processes,
and instantiations of things of a given kind and slight variations
thereof (e.g., a given protein). Equally important is the fact that they
are also rich with respect to the huge numbers of potential interactions
among them and their products. The result is essentially EXTREME parallel
processing with arguably exponential outcome possibilities.
Sorry, but I don't follow this at all. What do you mean by "linear
evolution process?". Unless that's exactly what was demonstrated in the
paper. Gradual increments with one change at a time. At the risk of
repeating myself, the paper explicitly stated (in the Discussion) that
if multiple changes were required (i.e. you didn't reward the
intermediate stages), then _it did not work_. I don't agree that
parallel processing has exponential outcome possibilities. This is only
the case if there is interaction between the parallel elements. But the
paper didn't employ any form of "crossover" (which is normally used in
GA's to simulate sexual reproduction). Hence there is no interaction
between the elements, except in the natural selection process. Even if
there were interactions, the fact that exponential outcome possibilities
are available is irrelevant. Sure, the number of _favourable_
possibilities increases exponentially, but so does the number of
_unfavourable_ possibilities, at a much faster rate! (As they say in the
paper, most mutations are damaging; only a very few actually improve the
fitness). What this means is that the probability of finding a
favourable one _decreases_ exponentially, because you are dividing an
exponentially increasing number by another one that is also exponentially
increasing at a faster rate; e^x / e^2x = e^(-x).
It seems to me that this result - including the evidence for different
evolutionary rates - does below-the-waterline damage to some of the ID
arguments. I would not be surprised to find that "stacked deck" response
taken up as a rallying (but weak) riposte. But if so, it will only
demonstrate that the point of this work was missed.
Actually, I think it's the critics of the "irreducible complexity"
argument who are missing the point. Meanwhile, I'll carry on making
money for my company by solving problems that are "irreducibly complex"
every step of the way. (Irreducibly complex, in my definition, means:
"requiring several coordinated changes to make any improvement").
This archive was generated by hypermail 2.1.4 : Fri May 09 2003 - 16:01:43 EDT