Re: Evolutionary rate

From: D. F. Siemens, Jr. (
Date: Fri May 09 2003 - 15:56:08 EDT

  • Next message: Jim Armstrong: "Re: Evolutionary rate"

    Iain and Jim,
    It strikes me that Iain is emphasizing the need for multiple changes
    because engineers demand answers /now/, not eventually. Jim approaches
    matters from the viewpoint of eventual improvement, with no reason to
    label anything "rush!" Both outlooks are functional in their place,
    though it must be granted that comprehensive alterations would radically
    speed up evolution.

    On Fri, 9 May 2003 19:11:08 +0100 "Iain Strachan"
    <> writes:
    Jim Armstrong wrote:

    What I was struck by was that the complex objective was not reached until
    the "environment" was (apparently slightly?) enriched by the presence and
    interaction of other essentially similar processes. As an ensemble, they
    interacted constructively in sufficient measure to increase the system
    creativity past the tipping point, enabling the achievement of the
    challenging simulation objective. I was also struck by the small number
    and simplicity of the "rules" that governed the elements of this
    simulation and provided this powerful insight. However, this is
    consistent with the lessons of complexity and chaos.
    My reply:

    Having worked for around 15 years in the field of numerical optimization
    & in providing and maintaining large scale numerical optimization solvers
    for engineering problems, I look at this somewhat differently.

    All that is happening in the paper is that a very simple function is
    being optimised, with incremental values of fitness, such that a simple
    and smooth path exists from the start position to the final objective,
    like Dawkins's metaphor of "climbing mount improbable". Think of the
    population of a Genetic Algorithm as just a cloud of points in
    configuration space that wander around under mutation. Then natural
    selection biases the "cloud" towards the ones that happen to be better at
    any one point. In order for this to work, it must be possible to find an
    improvement with just a very few mutations at any one point. This is
    exactly what was the case in the paper. In fact they write:

    "We also identified every mutation that fell along the line of descent in
    this population, and characterized the phenotypic changes associated with
    every step (Supplementary Information). The EQU function first appeared
    at step 111 (update 27,450). There were 103 single mutations, six double
    mutations and two triple mutations among these steps."

    (They do not discuss whether the double and triple mutations were
    improving of the fitness; some steps produced a drop in fitness).

    Now the reason why I uphold strongly to the "deck stacking" argument, and
    object to it being called "weak" (as you do below) is that in just about
    any non-trivial engineering problem that you want to solve, the above
    situation (where improvements can be made by one or two small localised
    changes) almost NEVER occurs in practice. I support
    optimization/Non-Linear Programming/Equation solving software that has to
    deal with systems of anything from a few tens to hundreds of thousands of
    equations. They are ALL based around computing the gradient with respect
    to all the variables simultaneously & update all the variables at once in
    each step. (This is like applying a mutation to every part of the
    genome). No one would seriously suggest a solver that only adjusted one
    or two variables at a time, leaving the rest unchanged, because they just
    don't work. In nearly all engineering problems of interest (certainly in
    numerical optimization), the variables are tightly coupled, which means
    that you have to make multiple coordinated changes to variables, not just
    one or two of them.

    But if you think I'm just a crank in the minority here, let me give you
    an independent assessment. One of the Fortran routines in the library
    that we use happens to be a Genetic Algorithm. It's very well written
    and incorporates all the latest heuristics. One developer in my company
    asked about using it for a problem that they had. As it turns out, I met
    the person who wrote the GA routine a few days later (as he works on an
    adjoining site). He is a visiting Prof. at Edinburgh University, a world
    expert in optimization, and the author of several large scale codes.
    When I told him that someone was interested in his GA code he just
    laughed. "Good grief!! He doesn't want to use that _seriously_ ? I only
    wrote it for fun. Tried it on a couple of problems. It was _awful_!"

    Since Nature seems to have done a pretty good job at solving difficult
    engineering problems, I'm therefore distinctly underwhelmed at the claim
    that this toy one-variable-at-a-time problem is a demonstration of the
    evolutionary origin of complex features.


    In a way, this is not too surprising an outcome, resembling parallel
    processing in computer terms, as opposed to serial processing. A little
    bit of parallel processing can be a very powerful thing - the most
    powerful computers on Earth embody a great deal of parallel processing to
    achieve their performance!

    I don't see how this resembles parallel processing, except that GA's can
    exploit parallel processing by having all the members of the population
    (the cloud of points in configuration space) evaluated on different
    processors. But having also worked quite a bit on parallel processing
    myself, the plain fact is that it can at _best_ achieve a linear speed-up
    in processing (linear in the number of processors), and that is only
    true if there is _no_ interaction between the elements running on
    different processors. As soon as interaction is required, then further
    slow-downs are incurred because of the communication overheads in sending
    data between different processors, and the improvement is sub-linear. If
    you don't get it right, the parallel processing results in a dramatic
    slow-down. And furthermore, with the rate at which processor speed is
    increasing (Moore's Law), parallelism only buys you a few years before
    your array will be beaten by a single processor. In the early '90's I
    worked on a small array of Transputers, which were the big thing in
    parallel processing, using a special language called Occam to incorporate
    parallel constructs in the programming language. But the brutal fact is
    that the computer I'm typing this email at, which is just a common or
    garden entry level domestic machine, is the equivalent of around 1000
    transputers in floating point performance! (The Transputer gave around
    0.7 million floating point operations per second, or FLOPS. I've tested
    this machine at around 700 MegaFlops. My laptop, which is about a year
    later vintage than this machine is twice as fast as that again).

    The argument concerning the evolution of things of "irrreducible
    complexity" seems to embody the notion of a rather linear evolution
    process. But the remarkable "processors" of nature are HIGHLY parallel
    and, taken with the environments in which they occur, are extremely
    (unimaginably) rich and redundant with respect to materials, processes,
    and instantiations of things of a given kind and slight variations
    thereof (e.g., a given protein). Equally important is the fact that they
    are also rich with respect to the huge numbers of potential interactions
    among them and their products. The result is essentially EXTREME parallel
    processing with arguably exponential outcome possibilities.

    Sorry, but I don't follow this at all. What do you mean by "linear
    evolution process?". Unless that's exactly what was demonstrated in the
    paper. Gradual increments with one change at a time. At the risk of
    repeating myself, the paper explicitly stated (in the Discussion) that
    if multiple changes were required (i.e. you didn't reward the
    intermediate stages), then _it did not work_. I don't agree that
    parallel processing has exponential outcome possibilities. This is only
    the case if there is interaction between the parallel elements. But the
    paper didn't employ any form of "crossover" (which is normally used in
    GA's to simulate sexual reproduction). Hence there is no interaction
    between the elements, except in the natural selection process. Even if
    there were interactions, the fact that exponential outcome possibilities
    are available is irrelevant. Sure, the number of _favourable_
    possibilities increases exponentially, but so does the number of
    _unfavourable_ possibilities, at a much faster rate! (As they say in the
    paper, most mutations are damaging; only a very few actually improve the
    fitness). What this means is that the probability of finding a
    favourable one _decreases_ exponentially, because you are dividing an
    exponentially increasing number by another one that is also exponentially
    increasing at a faster rate; e^x / e^2x = e^(-x).

    It seems to me that this result - including the evidence for different
    evolutionary rates - does below-the-waterline damage to some of the ID
    arguments. I would not be surprised to find that "stacked deck" response
    taken up as a rallying (but weak) riposte. But if so, it will only
    demonstrate that the point of this work was missed.

    Actually, I think it's the critics of the "irreducible complexity"
    argument who are missing the point. Meanwhile, I'll carry on making
    money for my company by solving problems that are "irreducibly complex"
    every step of the way. (Irreducibly complex, in my definition, means:
    "requiring several coordinated changes to make any improvement").


    This archive was generated by hypermail 2.1.4 : Fri May 09 2003 - 16:01:43 EDT