Index


1.0 Introduction

Top

This document is a development specification that will evolve into a user manual. The product it describes has been evolving in several prototypes and this document currently spans two versions, GAEMY 1 and GAEMY 2. Most of this html file explains GAEMY 1, which has a simpler architecture and is therefore easier to comprehend on the first read. GAEMY 1 is in an operations state at this time, whereas GAEMY 2 is still in the design stage. The new design additions pertaining to GAEMY 2 are in a separate html file linked from this one. When GAEMY 2 has completed development, this whole document will be revised to reflect its design, eliminating reference to GAEMY 1.

1.1 RIP - Rapid Iterative Prototyping

Compiler-Compiler or "Metacompiler" technology has a rich heritage that was largely derailed with the advent of Yacc (Yet another compiler compiler), which only partly addressed the task of compiling a compiler--syntax parsing. Yacc never came close to the intent of compiling compilers, largely because it utilized the C-language as its own backend subsystem, essentially defaulting to procedural programming for most of the compiler-building task. As a surrogate for assembly languages, C was certainly a major force in achieving software portability, as it was used to implement the first portable operating system, Unix. But Yacc merely became a way of conveniently handling language grammar specifications for pseudo-conventional compiler implementation via C programming.

Earlier compiler compilers such as Tree Meta were far more ambitious in that they were designed for evolution, of themselves and of the languages they were used to produce translators for. At TRW in 1970 we first used Tree Meta to developed a compiler for a problem-oriented language, the ATLAS test language. It was a cross compiler that generated code for what surely was one of the very first portable computers--automatic test equipment (ATE) used for diagnistic maintenance testing of army vehicles.

Although Yacc, as a Unix utility, became the only "standard compiler compiler", and has been used extensively in implementing compilers, it is remarkable how little it has evolved itself, when you consider that metacompilers were originally designed to evolve by recompiling themselves in particular. Even its open-source version, Bison, has adhered very closely to the original limited design of Yacc itself. This is hardly surprising, of course, as it is merely a parser generating specification preprocessor for C-programmers, there is never any intent to evolve the source language or its grammar, which is usually regarded as immutable.

1.1.1 Vertical Evolution of Application Languages

Metacybernetics has a different agenda entirely, which in concert with the original intent of metacompiler technology, is to promote rapid vertical evolution of languages into becoming metaphoric for end-user expression of scientific models, and adaptive to as many problem domains as there are science and engineering specialties. This requires rapid iterative prototyping (RIP) not only of new compilers, but of a new variation called compiler-publishers (CPs), which generate code-flow wiki websites from code, along with translating the code for execution.

As a development methodology, RIP is a radical departure from most sanctioned software-development practice, and is downright uncomfortable to many programmers, because programmers are not used to developing a new design completely from their own inspiration. They usually work from a specification produced by somebody else, or from a "perfect specification", a fully operational program that is to be ported or rendered into a new environment.

Working strictly from inspiration means "evolving from scratch", before what is being designed is completely understood, or even what the final end product will look like. It is largely an experimental process in which the final end-item only emerges after several attempts to "close" an open-ended design, only to detect a flaw that requires a new attack. A single program may end-up becoming several programs, or even a program that regenerates itself.

1.1.1.1 Top to Bottom Design Iteration (Mutation and Surgery)

The major flaw in parser generators and GUI builders is that they are incomplete tools that are not designed for top-to-bottom design iteration. Once a top-level design is generated in code from the metaphoric specification (BNF metalanguage or GUI-pallet artwork), the major work that follows is surgery on that top-level design, which takes a long time to get something running which can be tested, before the design can be iterated. Any regeneration of the top-level design has to be reconciled with the low-level surgery thereafter. So design iteration is laborious and prolonged. When it does occur, it is a lot like starting over.

RIP, on the other hand, is a process of rapid top-to-bottom design iteration, where each iteration is a small step, and is merely regarded as a trial for rapid review and improvement, based on "feel". Design increments in this case are so "shallow", as to approach continuity "in the limit". That way, each transition is similar enough to its preceeding step that a comparative editing process, facilitated by difference highlighting editors like Meld, can be extremely effective.

1.1.1.2 Learning by Doing

Another extremely important aspect of this approach is that it requires far less programming skill with a particular language or API, because the metaphoric code-generating medium, organizes the design, and shows the novice how the code is properly used. Understanding an API like GTK is a daunting task for the novice to overcome, if there are no realistic examples to learn from. The proverbial "Hello World" and calculator design are lame shadows of what is usually needed to immerse a novice in what is often an alien design paradigm.

1.2 GAEMY is a RIP Tool

The major focus of GAEMY is on the part that Yacc leaves undone, the action semantics which accomplishes the major work of translation. This work is all about surgery (manual editing) of a mutation of the system architecture (a change in the grammar of the language to be compiled), and the automated reconciliation of the surgery with the grammar change.

Reconcilation employs synchronized comparative editing via difference-detecting and highlighting multi-pane editors such as Meld, or un-synchronized multi-pane gang editors such as jEdit or Kate.

1.2.1 The MIDDLE Principle

Multi-level equivalence of the associative design patterns is made simple by their comparative juxtaposition, and by a principle of configuring the data ordering and juxtaposition so as to minimize incremental design difference for lockstep equivalence (MIDDLE). This synchronization principle maximizes suggestive adaptations to the user, even when the data being compared and changed are functionally in separate phases of design transformation. Difference-minimizing synchronization is managed through an assembly-line architecture of incremental archives. Each archive in the assembly line is a tree of directories in which different states of code are maintained, and from which separate states may be compared and edited to reconcile them with changes made upstream by generation or downstream via editing.

Archives are named by a project name appended to a serial name composed of a date stamp and a serial letter. They are generally associated with a mutation change (from editing the pure grammar of the language), and a new archive in the assembly-line may be created for each mutation.

The most recent archive (head of the assembly line) is paired with a "baseline tree" which is associated with the latest surgical changes. This tree is maintained under version control as a subversion working directory.

1.2.2 GAEMY Subsystems

GAEMY has two primary subsystems:

  1. A Yacc grammar preprocessor that translates a pure Yacc Grammar into:
    • an augmented grammar with action predicates in the form of C-function calls to semantic functions, and
    • an archive tree containing:
      • A semantics subdirectory containing the semantic function templates, and
      • A website of the augmented grammar with links to the semantic functions;
    • a baseline tree containing a corresponding semantics subdirectory containing the surgically modified versions of the semantic functions;
  2. GUI control panels for accessing the website and controlling gang and meld editing of the pure grammar and the semantic functions in various multi-pane editing sessions comparing incremental code variations of the same functionality in two or more files, as required to evolve a compiler-publisher design.

The sequentual date-stamped archive tree and the baseline tree constitute an assembly-line architecture for evolutionary development and maintenance.

1.3 Grammar Forms

Currently, the input (pure) Yacc grammar must adhere to a "canonical" form in which each syntactic alternative appears on a separate line in the following form:

<rule_name>
     :  <syntax_alternative_1 pattern terms>
     |  <syntax_alternative_2 pattern terms>
     |  <syntax_alternative_3 pattern terms>
          :
          :
     |  <syntax_alternative_N pattern terms>
     ;

The action predicates inserted into the pure Yacc input grammar are C-function calls mirroring the syntactic format of the BNF alternative, to create the corresponding augmented grammar, i.e,

<rule_name>
     :  <syntax_alternative_1 pattern terms>
        {
            <rule_name_1>(<syntax_alternative_1 pattern arguments>);
        }
     |  <syntax_alternative_2 pattern terms>
        {
            <rule_name_2>(<syntax_alternative_2 pattern arguments>);
        }
     |  <syntax_alternative_3 pattern terms>
        {
            <rule_name_3>(<syntax_alternative_3 pattern arguments>);
        }
            :
            :
     |  <syntax_alternative_N pattern terms>
        {
            <rule_name_N>(<syntax_alternative_N pattern arguments>);
        }
     ;

where the action predicates constitute the C-code { ... } blocks, which solely contain generated function calls to functions named as numbered sub-rule action functions, with arguments mirroring the syntax terms of the syntax alternative patterns.

1.4 Example Rule in Grammars and Action Semantics

The following shows an example rule in pure grammar form, augmented grammar with grammar predicates, and action semantics.

1.4.1 Pure Grammar Rule

First we show the pure grammar rule as it is input to GAEMY. We have chosen the select_case_statement rule because it shows all types of token symbols so that we can show how they are translated by GAEMY, including terminal symbols (literal characters in single quotes), upper-case token symbols, converted to integers by the lexer, and lower-case non-terminal symbols (rule names).

/*
=================================================
             select_case_statement
=================================================
*/
select_case_statement
  : name  ':'  SELECT  CASE  '('  case_expr  ')'          '\n'
  |            SELECT  CASE  '('  case_expr  ')'          '\n'
  | name  ':'  SELECT  CASE       error                   '\n'
  |            SELECT  CASE       error                   '\n'
  ;

1.4.2 Augmented Grammar Rule

Next we show the equivalent rule in the augmented grammar file generated by GAEMY (with action predicates shown in blue).

/*========== select_case_statement =============================================*/
select_case_statement
  : name  ':'  SELECT  CASE  '('  case_expr  ')'          '\n'
    {
       $$ = select_case_statement_1($1,":","SELECT","CASE","(",$6,")","\n");
    }
  |            SELECT  CASE  '('  case_expr  ')'          '\n'
    {
       $$ = select_case_statement_2("SELECT","CASE","(",$4,")","\n");
    }
  | name  ':'  SELECT  CASE       error                   '\n'
    {
       $$ = select_case_statement_3($1,":","SELECT","CASE",$5,"\n");
    }
  |            SELECT  CASE       error                   '\n'
    {
       $$ = select_case_statement_4("SELECT","CASE",$3,"\n");
    }
  ;
/*-----------------------------------------------------------------------------*/

1.4.3 Yacc Union Pseudo-Variable Translation

Pseudo-Variable Types - The types of the pseudo-variables (assigned to the symbol by the lexer) appearing in the embedded action predicates depend upon the kind of symbol that was recognized. If it is a symbol recognized as a linguistic token (delimiter) of the language (e.g SELECT and CASE above), then the type of the pseudo-variable will be a preset integer associated with that symbol (standard Yacc mechanics). If the symbol is a floating-point number, then the type of the pseudo-variable will be a form of real. If the symbol is an identifier or a quoted string, then the type of the pseudo-variable will be a (char *) pointer to a string. What is actually on the stack is a union of any of these types, as defined in the Yacc declaration:

     %union {
           int i;
           double d;
           char *s;
     }

However, if the symbol recognized by the rule is a non-terminal symbol (ie. the name of another rule), then its type will be affected by whatever reductions that non-terminal symbol represents, and may not be knowable from the context of the action predicate. This is normally a context-dependency which Yacc is not designed for in its normal use. Non-terminal symbols may have a declared type using the %type declaration, specifying one of the members of the union. For example, the declaration

     %type  some_number

specifies that the non-terminal symbol "some_number" is type double, according to the union definition.

In the case of our compiler-publishers, the union declaration in the Yacc BNF file heading is:

     %union {
          int i;    // integer type associated with tokens (terminal symbols)
          nont *n;  // pointer to a struct typedef for non terminal symbols
          symt *id; // pointer to a symbol table struct typedef for identifiers
          char *s;  // pointer to a string for quoted strings and numbers
     }

In the augmented grammar file generated by GAEMY, a %type declaration with <n> is generated for every non-terminal symbol.

Note that in the select_case_statement rule embedded function call, the Yacc feature, <n>, is also used to select nont type from the union being used in the predicate. The arguments represented by non-terminals are represented by the standard Yacc pseudo-variables of the form $<n>1, 2, etc.

Note also, that GAEMY translates upper-case tokens and character literals into string literals in the calls to the action functions, and generates type declarations in the action semantics function stubs to match the types in which the pseudo-variables are passed in the calls

1.4.4 Action Semantics Rule Module Files

Finally, we show the action semantics rule C-module file, containing all of the sub-rule action function templates (stubs). This file is generated by the GAEMY preprocessor.

/*========== select_case_statement ============================================*/
#include 
#include "common.h"
#include "fc90actmacros.h"
// select_case_statement
//   : name  ':'  SELECT  CASE  '('  case_expr  ')'          '\n'
nont select_case_statement_1(nont *n1,char *s2,char *s3,char *s4,char *s5,nont *n6,char *s7,char *s8)
{
  DBC("[%s,%s,%s,%s,%s,%s,%s,%s]\n",n1->s,s2,s3,s4,s5,n6->s,s7,s8)
  {
     char *ss;
     nont *n;
     *ss=TS("%s%s%s%s%s%s%s%s",n1->s,s2,s3,s4,s5,n6->s,s7,s8);
     n->s = ss;
  }
  DBE("\n") 
  return(n);
}
//   |            SELECT  CASE  '('  case_expr  ')'          '\n'
nont select_case_statement_2(char *s1,char *s2,char *s3,nont *n4,char *s5,char *s6)
{
  DBC("[%s,%s,%s,%s,%s,%s]\n",s1,s2,s3,n4->s,s5,s6) 
  {
     char *ss;
     nont *n;
     *ss=TS("%s%s%s%s%s%s",s1,s2,s3,n4->s,s5,s6);
     n->s = ss;
  }
  DBE("\n") 
  return(n);
}
//   | name  ':'  SELECT  CASE       error                   '\n'
nont select_case_statement_3(nont *n1,char *s2,char *s3,char *s4,nont *n5,char *s6)
{
  DBC("[%s,%s,%s,%s,%s,%s]\n",n1->s,s2,s3,s4,n5->s,s6) 
  {
     char *ss;
     nont *n;
     *ss=TS("%s%s%s%s%s%s",n1->s,s2,s3,s4,n5->s,s6);
     n->s = ss;
  }
  DBE("\n") 
  return(n);
}
//   |            SELECT  CASE       error                   '\n'
nont select_case_statement_4(char *s1,char *s2,nont *n3,char *s4)
{
  DBC("[%s,%s,%s,%s]\n",s1,s2,n3->s,s4) 
  {
     char *ss;
     nont *n;
     *ss=TS("%s%s%s%s",s1,s2,n3->s,s4); 
     n->s = ss;
  }
  DBE("\n") 
  return(n);
}
/*----------------------------------------------------------------------------*/

Note the form of each function template. The syntax of the sub-rule is displayed as a comment (magenta), followed by the function heading (blue). Then the function body first contains a call (red) to a debug logging system using the DBC macro, which passes a formatted string and output parameters (in this case the function arguments). At the function exit, another call is made to the debug logging system, using the DBE macro.

Inside the function body is a statement that concatenates all of the input symbols together. This is the default stub (green), which simply mirrors the input syntax of the sub-rule into its output. In a source-to-source translation, like Fortran Calculus, this will be correct for a great part of the syntax which does not require translation. Note also that this form allows for simple testing of grammars for syntactic shift-reduce conflicts. If the complete output mirrors the input, then no conflicts are present.

2.0 Metacybernetic Evolution Concepts

Top

In this section, we define some terms and acronyms that assist in understanding the different GAEMY evolution phases, processes and data architecture.

2.1 Mutation, Surgery, and Vertical Evolution

Files referred to in the aggregate as a "mutation" refer to generated files that contain function template stubs, derived from the pure grammar by the GAEMY preprocessor. Such files always reside in an "archive" tree, namely the "current" archive tree produced by the last run of the preprocessor. This current archive is the head of the generated part of the "assembly line", which consists of the sequence of such archives.

Files referred to as "surgery" are files that have been manually edited. Such files reside in the "baseline" tree, which always structurally corresponds to the current archive. Although the baseline tree is considered part of the assembly line, there is only one baseline tree. It is the only part of the assembly line that is maintained under active version control. That is, it is a local working directory of a Subversion repository.

In another sense, all surgery files are part of a mutation, the current system architecture design.

2.1.1 Mutation Evolution - Syntax Grammar Changes

Changes in the syntax of a language being implemented through the design of a compiler or compiler-publisher is the most monumental change that such a translator must undergo. It requires a change in the translator's system architecture, so it is the structural equivalent of an earthquake. Its impact cannot usually be predicted in advance, but must be inspected after the fact.

This is especially the case when the translator is implemented with a compiler compiler like Yacc. Although in principle it is easy to modify the Yacc grammar file, it is almost never done once a grammar has been completely specified, because of this "earthquake effect".

Thus new language mutations, especially those that evolve vertically into specialized application domains, i.e. what used to be called "problem-oriented" languages, are so rare as to be virtually non-existent in today's IT world.

The purpose of GAEMY is to remedy this situation--to simplify the management of mutation evolution, such that proliferation of vertically evolved problem-oriented languages can rapidly emerge in R&D application environments.

2.1.2 Surgical Evolution - Semantic Changes - Bulk Design

The bulk of design work of a translator is done manually in the semantic action functions called from the action predicates in the augmented grammar. The evolution of this design is not metaphoric, like grammar design, but requires extensive coding in the algorithmic C-language, and possibly other languages and API libraries. Each time the grammar changes, this code must be revisited and adapted.

But such code may extensively evolve through many generations when there are no grammar changes at all. All of this is surgical evolution of a given mutation (the grammar). With the possibility of rapid and experimental grammar evolution, the burden of surgical evolution becomes prohibitive without automation, such as that provided by GAEMY.

2.1.2.1 Code Compilation

The traditional purpose of compilers is the generation of binary executable code from the source language, for various execution platforms, usually with multiple stages of code optimization. This purpose is extended dramatically with parallelization of code for multi-threading and multi-core parallel-processors partitioning.

2.1.2.2 WebCode Publishing

A new purpose of Yacc and other metacompiler tools was dramatically extended with advent of the web, through the translation of source languages into wiki webcode sites that were navigable just like source debuggers. Our prototype publisher, EvoScope, was developed with Yacc and used to compile the Apache 2 source code into a wiki website, containing permanent footnote triggers that popped up tooltip sub-hypertext (containing links) on mouseover, and temporary wiki "collab notes" could be posted on the site after the fact, for email sharing by collaborators. This website would expand and explain C-macros, which in Apache 2 were often nested to nine or ten layers.

2.1.3 Vertical Evolution of "Wrapper" Compiler-Publishers

The primary intent of GAEMY is to facilitate the rapid evolution of multi-level compiler-publishers for MetaScience languages which generate and publish code in multiple translation stages. At the base level will be the next generation MetaCalculus languages now in development. Progressing upward will be several translation layers, where there can be multiple, stylistically diversified languages, many being derivatives of conventional languages like Python, Java, Perl, etc. But ultimately the intent is to produce problem-oriented optimization languages (POOLs), formed out of DNA-like modeling alphabet APIs of MetaEngineering modeling elements re-cast from optimization re-engineering of legacy software. Ultimately the POOLs will be wrapped in GUIs to populate diversified MetaCAD appliance products.

The Metacybernetic tools GAEMY and EverGlade are essential to bringing the kind of RIP leverage needed to empower vertical evolution into application modeling domains in R&D. It can only progress via peer-development involving application-science modeling architects and IT systems programmers working together.

2.1.3.1 Base Level - MetaCalculus Target Languages

The final target languages, MetaFor and MetaCalc, at the bottom of the wrapper-CP stack, belong to the next generation MetaCalculus API, MC8, a complete re-engineering of the MC7 MetaCalculus language Fortran Calculus and its API of optimization and simulation solvers, This re-engineering was accomplished in 1994 largely by my PROSE design-partner, Mike Robinson, but later shelved for lack of continuation funding.

MetaFor adds MetaCalculus modeling templates to Fortran 95 syntax, making it highly suited for optimization re-engineering of legacy Fortran code. MetaCalc is a more dynamic MetaCalculus array language in the tradition of PROSE, suitable for end-users who have more casual programming skills. Both languages will contain the Metaphorio GUI wrapper interface, implementing format-independent verb templates to map data to and from Glade-designed GUIs and databases. These languages are interoperable, so programs combining both languages can be generated from higher-level translators.

2.1.3.2 Optimization Broadcasting Level

Optimization broadcasting (OB) is a method of parallel processing that searches the same model formulation simultaneously in many instances, resulting in many mathematical search paths. It is a kind of "swarm searching". The swarm of paths may derive from replicating a given search method from many different starting points, or different search methods from the same set of starting points, or any combination of both. Historically, MetaCalculus users, restricted to mono-processing computers, would apply one solver (search method) at a time, trying a set of starting points one at time, until a solution point matching a convergence criterion was found. If a solution could not be found after a series of starting points were tried, then the process was repeated using a different solver. Since Fortran Calculus had twenty optimization solvers in its library, many sequential runs might be necessary to find the right combination of search method and starting point to find an optimum solution for very complex models.

OB is the next evolution of MetaCalculus. This approach to enhance computer solution power is both straightforward, simple, and elegant. MetaCalculus swarm searching of pathological hypersurfaces can become a mathematical science all its own, probably in combination with fractal geometry. With the availability of multi-core processors or cluster computers, OB swarm searching makes more sense than sequential searching. Particularly since the model programming is the same. As a practical matter, what this means is that MetaCalculus and OB parallelism will become mainstream together. Together they make casual supercomputing a practical proposition for every computer user in R&D, education, and ultimately business. Once the first OB MetaCalculus language product is defined, it will obviate all mono-processing languages, merely because it easily subsumes their capabilities in a context (many-core chips seeking a market) that multiplies these capabilities beyond measure.

It is clearly going to take a lot of study, over the myriad kinds of optimization models that can emerge rapildly from re-engineering legacy high-fidelity simulation models. It is appropriate that it will first emerge through the simple distribution of whole mono-processing MC programs to networked cluster machines to provide the interprocess communication transparency needed to track and study swarm-searching behavior and the patterns that will emerge from it.

Extending the MetaFor and MetaCalc base languages for OB multi-processing will employ a second layer of wrapper compilers which translates a mono-source MC gang-solver-nest program invoking a group of solver nests (e.g. optimization solvers with nested correlation and simulation solvers) and a group of starting condition data-sets into a set of distributed MC mono-source single-solver-nest programs each invoking a separate Unix process routed to a networked server or one of the cores of a many-core processor.

The objective will be to vertically evolve the ninth-generation MetaCalculus API (MC9), with solvers in shared-object libraries on multi-core servers designed to coordinate in OB swarm searching for automated topography mapping of hyperdimensional mathematical surfaces. These swarm-solvers will output data to and recieve swarming instructions from backend databases, having triggers to spawn secondary and tertiary swarm-searching MC programs to flesh-out and interpolate hypersurfaces and drive hyper-VR visualization programs.

Development of these OB wrapper compiler-publishers can be easily derived in peer application/system programming environments from the Fortran-DVM and C-DVM source-to-source compilers developed under the GNU public license at the Keldysh Institute of Applied Mathematics of the Russian Academy of Sciences.

These MC9 "OB under the hood" compiler-publishers can be for any style of language translatable into MetaFor or MetaCalc. The easiest, of course, will be MC9 gang-solver-nest versions of MetaFor and MetaCalc themselves, which will be developed first. Then MC9 versions of other languages like C, Python, Java, and other languages could be easily translated into MC9 MetaFor and MetaCalc, by simply adding another layer of wrapper compiler-publishers.

2.1.3.3 Problem-Oriented Optimization Languages (POOLs)

The next level of vertical evolution represents a fanning out of diversified application languages built-upon MetaEngineering component model bases. As MetaCalculus refers to canned mathematical templates and solvers, MetaEngineering refers to canned engineering modeling components, derived from re-engineering legacy simulation code into MC optimization code, and then modularizing the engineering models into constituent components. Then over time refining these components, arriving from many legacy sources, into universal component libraries organized in problem-oriented API alphabets.

Such POOLs will simply be wrapper languages for MC9 MetaCalculus languages and associated MetaEngineering API libraries. But, being problem-oriented, they can aggregate more tailored parallelism than the general purpose MetaCalculus languages. Learned characteristics of the canned model response surfaces can be used to prescribe canned solvers, advantageous scaling, and other canned parameter settings to enable more efficient and reliable searching. Thus problem-oriented solution advantages can be just as important as language specialization in supporting higher productivity in modeling language diversification.

2.1.3.4 MetaCAD Appliances

Pallet-based GUI modelers will be the logical extensions of the POOLs, as they mature, trading graphics expression of application models design for language expression. GUI Evolvers like EverGlade will become the primary tools for extending the POOLs in this arena. But these POOL-extended appliances will become just as noted for their graphical output devices, designed for problem-oriented hyperdimensional visualization. HyperVR visualization techniques that transduce partial differential information, fractal signatures, and other derivative output data into sensory analogs that couple to human perception for interpretation, will need to be studied and applied. This will open up new challenges in hypergeometric data management, data modeling, and interpretation, tasking all our database skills.

2.1.3.5 Summary of Wrapper Layers

To summarize the vertical evolution from a top-down perspective, these are the compiler-publisher classes strata:

  1. Wrapper CPs for Problem-Oriented Optimization Languages – Modeling API verb alphabets which operate meta-engineering library modeling components. They translate into:
  2. Wrapper CPs for General Purpose MetaScience Modeling Languages – General programming languages of various genres, having Find statements for invoking optimization broadcast swarm searching in parallel on multi-core platforms, each exercising the same model with different solvers and from different search starting points. They translate into:
  3. Wrapper CPs for Fortran-DVM or C-DVM augmented intermediate languages – Parallel distribution compilers (with programming via distibution directives in comment fields to create monoprograms in:
  4. MetaFor and MetaCalc – MC8 MetaCalculus languages.

Note that levels 1, 2, and 3 are wrapper CPs generating code for high-level languages in each case. Each of these target languages will in essence be design "substrates". Macro processors will be used in each substrate for programming algorithmic design patterns for translation into from the CP above it.

These are examples of what we mean by vertical evolution.

2.1.4 Automatic/Manual Design Reconciliation

The term "reconcilation" comes up again and again in this discussion because most of the use of GAEMY is about reconciling automatically generated stub code to code that has been fleshed-out by manual editing. Reconcilation implies comparison. So except for new code design when it is first inserted into the generated function stubs, all design evolves by comparison with what went before.

This is why comparative editors like Meld are so important. Even when Meld cannot be applied, and we have to resort to gang editors like jEdit and Kate, it is only because, as yet, they have more flexibility for comparison than Meld, because they are not constrained by synchonous scrolling. But like the advent of jet engines in 1943, with expected improvement in Meld, it is only a matter of time until it becomes the editing norm.

Comparative editing is so fundamental to design evolution in Metacybernetic tools like GAEMY and EverGlade, that the other apparatus important to this evolution process is but a context to facilitate the Meld process by maintaining data in a structure that minimizes incremental difference in each step of the process--the assembly line.

2.2 Assembly-Line Evolution Management

2.2.1 Code Repository Trees

For every GAEMY project, there are two repositories:

  1. GAEMY Home (~/GAEMY) - The repository of GAEMY scripts, documentation, and project archive assembly lines, which contain surgery websites.
  2. PROJECT Home (~/PROJECT ) - The repository where Yacc takes its input for compilation of the executable translator. It is where the version-controlled project baseline action rule tree resides and where the augmented grammar for Yacc processing resides.

In this prototype version of GAEMY, these are $HOME-based trees.

2.2.2 Assembly-Line Architecture

The action functions themselves are generated as template stubs in C-code module files named rule_name.c and located in subdirectory semantics in a paired set of directory trees, denoted as Base and Archive, where the Base tree is the version-controlled head of the evolution, and the Archive tree is the latest in an "archive assembly-line". The directory architecture is illustrated in Figure 2.2.2:1.


Figure 2.2.2:1


GAEMY separates the processing of the pure grammar (the system architecture metaprogram of the evolving translator), from the action semantics of its design. Two different GUI windows are used to control this separate processing.

3.0 Grammar Mutation Window

Top

Figure 3.0:1 shows the Grammar Mutation Window.




Figure 3.0:1

This is the main control window for GAEMY. Once the project name is entered, the current assembly-line archive tree is displayed below it. This field is read-only.

3.1 Mutation Buttons

All of its buttons in the lower frame pertain only to pure grammar processing. The Edit button is for editing the pure grammar file. The Apply disposes the edited pure grammar file. The Cancel button cancels the prior editing operation without disposition. The Quit button exits all GAEMY processing.

3.1.1 GAEMY Preprocessor Execution

The Execute button runs the GAEMY preprocessor to create a new mutation (i.e. a new assembly line archive), converting the pure grammar into an augmented grammar, and generating semantic action files that are partitioned into the new archive tree semantics and the baseline tree semantics, according to the surgical status of these files (further discussed below). This run also generates the webcode site and stores it in the SurgeryWeb subdirectory of the new archive.

3.1.1.1 GAEMY Options

The three radio buttons in the top middle of the window define the preprocessor options, defined in the following links:

3.2 Editors Radio Buttons

The Editors radio buttons on the upper right of the window are employed to choose which of the five editors are to be employed in the pre-loaded editing sessions to follow. Multiple files may be selected for comparative meld editing using Meld, or gang editing using the Geany, Jay, Jedit, or Kate.

This selection pertains not only to the pure grammar editing controlled by the Mutation Window, but also to surgical editing controlled by the Action Reconciliation Window discribed below.

3.2.1 Meld Editing

Meld editing is synchronized side-by-side comparative content editing of two or three windows which scroll in lock-step. The Meld editor uses a diff algorithm in the background to show colored association streams relating like text in the compared windows, and to differentially slide scrolling when like lines are not lined up. In action reconciliation editing, three different meld editing cases are specified according to the number of files selected (up to three), as in the three sections below:

3.2.2 Gang Editing

Gang editing assumes no synchronized association between the files being concurrently edited, and most of the editors have only limited side-by-side or above-below window splitting for concurrent viewing of the editing buffers. Geany and Kate separate buffers into tabbed pages. Jedit has a roladex type of file selection in each window, and allows arbitrary window splitting for side-by-side and above-below sub-window combinations.

3.3 Invoking Action Reconciliation Surgery

Once a new system mutation has been produced by execution of the GAEMY preprocessor, subsequent evolution of that mutation is performed surgically, unless subsequent changes are made to the grammar. This process is controlled by another GUI window, invoked by activating the Action Reconciliation Surgery check button in the middle of the Grammar Mutation window (see Figure 3.3:1).




Figure 3.3:1

If a new archive has just been produced via the Execute button, registering a change in the Assembly Line Archive display, then activating the Action Reconciliation Surgery check button will immediately bring up the Action Reconciliation Surgery window to the side of the Grammar Mutation window.

If this is a repeat session with the current archive, which was created in a prior session (mutation of new archives is very rare, as most design alterations only involve action semantics surgery), then the Action Reconciliation Surgery button should be activated before the project is entered. Then the Action Reconciliation Surgery window will be immediately invoked upon project-name entry.

4.0 Pure Grammar Articulation (GAEMY 2 Spec)

Top

The major focus of the original version of GAEMY was evolving the action semantics to reconcile changes in a whole grammar, as that has been the major disconnect of tools like Yacc. The rest of this html file deals with this semantics issue, and should be understood first, before visiting this section.

These more advanced concepts logically fit here in the top-down usage of GAEMY, but dramatically alter the project architecture in GAEMY 2 by subdividing whole grammars into subgrammar assemblies, which become the mutation units instead of projects, so in GAEMY 2, assembly-line archives are associated with subgrammar assemblies rather than projects, as shown in Figure 4.0:1.


Figure 4.0:1


However, at this point in the description of GAEMY action semantic processing concepts it is best not to complicate things until section 5.0 is understood. The only difference is that the pure grammar is associated with both a project (whole grammar) and a subgrammar assembly in GAEMY 2. Figure 5.0:1, as it applies only to GAEMY 1, is missing an Assembly entry widget. Since the rest of section 4.0 pertains to GAEMY 2 additions, I am putting a link to the GAEMY 2 additions in another html file. This link should be ignored until the narrative below is comprehended, but before actual use of GAEMY 2 is attempted.

5.0 Action Reconciliation Surgery Window

Top

Figure 5.0:1 shows the Action Reconciliation Surgery window that will appear. In its top frame, the Project and Assembly Line Archive displays are read-only. The buttons below the displays pertain only to the mutation represented to this archive. Below the top frame is a notebook with two tabbed pages.




Figure 5.0:1

The left tab page that is showing is a menu for selecting action-semantics files to edit. As shown in Figure 5.0:2, with selection of the "K menu", this is a check menu in which multiple files may be selected, each file-name corresponding to a rule in the Yacc grammar (pure and augmented).




Figure 5.0:2

Now clicking the right tab page (Figure 5.0:3) we see a similar but more limited check menu at the bottom, having only one letter sub-menu (T), and only one file entry in that menu, pertaining to the tap_index_property rule.




Figure 5.0:3

This page is for editing action-semantic files that were impacted by a recent grammar change, as detected by the GAEMY_preprocssor in its last run. The preprocessor compiles a list of such files and uses it to generate this page of the GUI window, if any files were impacted. Thus the purpose of this page is to present a suspense list of files that are pending reconciliation (the original contents of the check menu at the bottom). The "Pending" page presents two ways in which this suspense list can be satisfied. If it is a short list that can be satisfied in a single session (say 1-5 files), then one can select the upper radio button to invoke a script, generated by the preprocessor, for a chain of meld sessions to do all the updates in one setting.

For a larger list that may require several settings to complete the design reconciliation editing, one can choose the check menu at the bottom. Since this menu is temporary, and is reduced each time a file is edited and the Apply button is clicked, what is shown here could have represented a much larger suspense menu, which has already been reduced to one item. When this item has been edited and applied, the whole page will disappear from the window. This process is illustrated in Section 5.2.

5.1 All Action Semantics File Editing

When the Meld editor is employed, three different meld editing cases are specified according to the number of files selected (up to three), as described below:

5.1.1 Single Rule File Melding

When a single file is selected for editing by Meld, GAEMY presumes this to be two versions of the same file name taken from the baseline tree and the latest archive tree respectively. Thus it will bring up Meld with the baseline file on the left and the archive file on the right. In general this will be for reconciliation of a grammar change which affects only a single action rule, causing that rule to be regenerated by GAEMY, replacing the previous baseline file which GAEMY would have moved to the current archive. The purpose of the meld edit session would then be to update the new baseline file with the action content from the prior archived version to compensate for the grammar change.

Returning now to the left tab page we illustrate the process of meld editing the semantics of a single such grammar rule, in this case the "numeric_expression" rule (Figure 5.1.1:1) First we find it in the "N" sub-menu of the All Action Semantics File Editing menu, which contains a menu item for all rules of the grammar, and activate its check box. Then we click the WebCode button to bring up to locate the function module for this rule.




Figure 5.1.1:1

In Figure 5.1.1:2, we locate the "numeric_expression rule in the augmented grammar webpage, and selec the action fucntion call in its "numeric_constant" sub-rule".




Figure 5.1.1:2

Clicking on this takes to the webpage for the "numeric_expression" action semantic module file (Figure 5.1.1:3).




Figure 5.1.1:3

Now we click on the Edit button, which opens the Meld editor session, as shown in Figure 5.1.1:4. This compares the baseline version of this file (left side) to the latest archive version (right side) archive version. But since no prior surgery has been done on this baseline version, the two files are identical.




Figure 5.1.1:4

To illustrate the surgery process, we make a harmless change on line 12, and then save the left-side file (Figure 5.1.1:5).




Figure 5.1.1:5

Upon exiting Meld after saving the file has been saved, a popup acknowledging that the editing is complete, and the Apply button must be clicked to update the baseline tree and the website (Figure 5.2.1:6).




Figure 5.1.1:6

Clicing on the Apply button causes a popup acknowledging the files and website was updated (Figure 5.1.1:7).




Figure 5.1.1:7

Finally, refreshing the numeric_expression module-file website shows that harmless change has been make (Figure 5.1.1:8).




Figure 5.1.1:8

5.1.2 Dual (Split) Rule File Melding - Two Baseline Files and One Archive File

When two files are selected for editing by Meld, the order in which the check boxes in the menu are clicked is important. GAEMY presumes that the first file checked is a regenerated action stub file of a previously edited rule-action file in which the grammar has split into two separate rules, creating generated action files where the first has the old rule name, but its grammar pattern (calling sequence) is different from its former version in the baseline semantics directory (left side) and so this pattern must be updated to match its newly generated archive stub version (right side).

The middle-pane baseline file is a different file whose content must be partitioned between the left pane and the middle pane. It is either a new file generated for the first time containing only stubs, and its baseline version must be fleshed out from some of the left-side content, or it may be an old file containing content from the baseline.

In either case, two meld sessions for updating both baseline files will occur. The first session is for content updating (although content can be updated in both sessions, as the two left panes will be the same) from the former content of the left pane to meet whatever partitioning of the design between the two rules is required. In the second session, the two files on the left and middle panes will be the same, reflecting the content just edited, if both have been saved before exiting first Meld. The third file on the right in the second meld will be the new archive stub version of the middle file just generated. The intent of this session will be to alter the calling sequence of the middle pane file to match the right hand stub file. After this update, only the middle-pane file must be saved again, unless the left pane has been futher changed.




Figure 5.1.2:1

Note that in this meld session, the boolean.c file, chosen first in the check menu (Figure 5.1.2:1) appears twice in the left pane and right pane. But nominally these files would have different content, as the one on the left (Figure 5.1.2:2) is from the baseline tree, which nominally would have undergone prior surgery, whereas the one on the right, having just been generated by the GAEMY preprocessor in the new archive, would contain just stub functions (as they both do now, causing them to be identical).

In this case the middle pane contains the baseline version of the second file, boolean_dollar.c, which nominally in this case, would receive some of its content from the left-most copy of boolean.c in the melding process, caused by the splitting of grammar rules.

[This example needs to the fleshed out to show this process in a later version of this manual]




Figure 5.1.2:2

Figure 5.1.2.3 merely shows some editing of these files for purposes of illustrating how the results are propagated to the website below.




Figure 5.1.2:3

Figure 5.1.2.4 is the second meld session in which the left and middle panes have the same content from the first meld, but the right pane, contains the archive version of the middle pane instead of the archive version of the left pane.

If the middle pane had been previously edited (not currently generated), then its calling sequence (arguments) would have different structure than that shown here because that version of the file would have come from the baseline (i.e. old copy) rather than just having been generated in the new archive as the stub version to be stored in the baseline after melding. In this second meld session, the right pane calling sequence heading would reflect the new splitting of the rule grammar.




Figure 5.1.2:4

Figure 5.1.2:5 shows the popup following the Meld session acknowledging that the two baseline files have been edited, and must be disposed to update the baseline and the website by clicking the Apply button.




Figure 5.1.2:5

Figure 5.1.2:6 shows the acknowlegement after clicking the Apply button that the two baseline files and the website have been updated.




Figure 5.1.2:6

Figure 5.1.2:7 show validation of the website having been updated to reflect the editing of boolean.c during the above meld session.




Figure 5.1.2:7

Figure 5.1.2:8 show validation of the website having been updated to reflect the editing of boolean_dollar.c during the above meld session.

Note: The menu on the left still shows the boolean rule selected, instead of the boolean_dollar rule because both rules appear juxtaposed in the augmented grammar web page where both links to the two separate files were click in sequence.




Figure 5.1.2:8

5.1.3 Triple Rule File Melding - Two Baseline Files and One Archive File

When three files are selected for editing by Meld, GAEMY presumes that the first file checked is a regenerated action stub that is to merge the content of the semantics of two previous rules. If surgery was previously done on this regenerated action file, its prior version will be in the baseline, and that version will be loaded as the target of the merge, otherwise the preprocessor will have updated the baseline with the new stub version (as well as the new archive). GAEMY will bring up this merge target file on the left (prefixed TARG), and the two baseline files in the center and on the right in the order they were checked in the menu.




Figure 5.1.3:1

Figure 5.1.3:2 merely captures the click on the Edit button to invoke the meld sessions. The previously checked file selection is not deactivated until the Apply button is clicked. So multiple editing sessions may be invoked to accumulate edits, if there is any reason to.




Figure 5.1.3:2

Figure 5.1.3.3 shows the top scroll of the first Meld session. Since all three files are from the baseline tree, but none have had prior surgery, since they contain nothing but stubs. They represent the case where generated files and generated files are identical, causing the preprocesser to duplicate them in both the new archive and the baseline.

In future we will replace this example with one that represents a more nominal case of prior surgery.




Figure 5.1.3:3

Figure 5.1.3.4 shows the bottom scroll of the left side of th first Meld session (the middle file is considerably longer). We added a comment to the bottom of the left pane and then saved the file.




Figure 5.1.3:4

Figure 5.1.3.5 shows the top scroll of the second meld session. The nominal reason for this meld is the case where the left side merge target has had prior surgery and has just been further merge-updated from the last session in exchange with the files on the middle and right side from that sesson, but the calling sequences (headings) to its functions do not yet reflect the grammar changes reflected in the new archive. That information is provided by the newly-generated archive version of the target file (now appearing on the right side).

Of course, the content of this example does not support this yet, since the two sides are identical (the degenerated case of no prior surgery). This will be remedied with more appropriate content in the future.




Figure 5.1.3:5

Figure 5.1.3.6 shows the bottom scroll of the second meld session, where the comment was left in the first session. Now we have added to the comment.

Please note that we could keep doing this sequence of meld sessions indefinately, since there are so many different functions whose headings might need to be altered due to a non-trivial change in the pure grammar, we may want to iterate to review the design in this way. Moreover, since GAEMY is a client-server system, in which we may be collarorating with other designers, this state can be held indefinately and reviewed by others until everyone agrees before clicking the Apply button to dispose the files and update the webcode site.




Figure 5.1.3:6

Figure 5.1.3.7 merely shows the popup acknowledgement that we are done with this phase of surgery. We can continue by clicking the Edit button again or finish by clicking the Apply button.




Figure 5.1.3:7

Figure 3.1.3.8 simply shows the popup acknowledgement that we have applied the editing results and updated the Webcode site.




Figure 5.1.3:8

5.2 Pending Action Grammar Change Reconciliation File Editing

This section illustrates the completion of a reconciliation suspense list resulting from a grammar change that impacted a set of rules, which required that their semantic functions be reconciled to the grammar change.

In this example (Figure 5.2.1) this last function is checked.




Figure 5.2:1

In Figure 5.2.2, the Edit button is clicked to bring up a Meld session comparing the baseline version of the file with the newly created archive version generated by the grammar change.




Figure 5.2:2

Figure 5.2:3 shows the meld session. Of course this example is as yet degenerate because no surgery has yet been done on the baseline version on the right. So it is identical with the archive version on the left. This will be corrected in future content of this example. But it does serve to illustate the process at this point.

For illustration purposes we type in a harmless comment on line 13.




Figure 5.2:3

Then in Figure 5.2:4, we click the Save button and exit Meld.




Figure 5.2:4

Then in Figure 5.2.5 we get a popup dialog from GAEMY confirming completing of this stage of editing, and that we need to click the Apply button to update the file in the baseline tree, and update the webcode site.




Figure 5.2:5

In Figure 5.2.6 we click on the Apply button.




Figure 5.2:6

Then in Figure 5.2.7 we get popup confirmation that the 'tap_index_property' item has been removed from the Pending Action Menu, i.e. the suspense list. Of course if further editing of this file is necessary, that can be accomplished via the All-Action menu in the other tab page.




Figure 5.2:7

Since this menu item was the last one in the in the suspense list, we get a popup message notifying us that this tab page is now being deleted (Figure 5.2:8).




Figure 5.2:8

Then in Figure 5.2:9, we see that is the case, as we are now back to the All Action page, which is the only tab left in the window.




Figure 5.2:9

Next we click on the WebCode button to inspect the updated website (Figure 5.2.10). We search the sidebar menu for the 'tap_index property' rule and click on it to bring up that rule in the augmented Yacc grammar. Then in that rule we find the link to function call 'tap_index_property_1($<n>1,"=",$<n>3);', and click on it.




Figure 5.2:10

This brings up the web page containing the linked function (Figure 5.2:11), where we highlight the comment that we added in the baseline file.




Figure 5.2:11

The remainder of this document is under construction

NOTE: Unlike the EverGlade product, which is ready for beta users, having been shaken down somewhat in the development of the GAEMY GUI shown below, GAEMY has not yet been employed to create a new compiler-publisher, for which it was designed. Consequently, this version of the document is not really a manual. Rather it should be considered an evolving design specification.

Its examples are preliminary, since they reflect intended modes of use instead of actual use. Nevertheless, they will serve to explain usage concepts, and will later be replaced by actual examples of use.