GAEMY 2 Extended Functionality


These next two sections provide the new functional specifications for the GAEMY 2 additions to manage the evolution of pure grammars and mutation test parsers. These additions will greatly expand the scope of GAEMY as an experimental RIP language and compiler-publisher evolution tool, able to simi-automate software A&E in a hybrid peer discipline of applications-programming and systems-programming .

4.1 Assemblies as Units of Grammar Evolution

Grammar rules corresponding to language constructs are the basic syntactic units of a grammar, which are named by what are called non-terminal symbols. In their augmented form they contain action predicates which invoke semantics processing. These predicates appear in subrules and invoke semantic functions corresponding to the subrules, which are extremely numerous for any grammar. Even grammar rules are so numerous that alphabetization is the only practical way they can be organized when the focus is semantic design.

Yet when a grammar is initially designed, the alphabetization approach is cumbersome, and rules are grouped into "pseudo sub-programs", even though there is no procedural ordering in Yacc grammars. This is one reason for designing grammars in units that are functionally related like sub-programs.

4.1.1 Simplifying Vertical Evolution

Another key reason for shifting from the whole-grammar focus of Yacc and GAEMY 1 to subgrammar assemblies in GAEMY 2 is to facilitate the creation of portable sub-languages that can be added to the many different but functionally similar languages that we have in this fractionated software art today.

The two principal sub-languages I am focused on are MetaCalculus and Metaphorio, the former dealing with simplifying higher-mathematical modeling, and the latter enabling vastly simplified I/O that fosters orthogonal bifurcation of software architecture into "internal bodies" (proto-software engines) and "external clothes" (I/O interfaces), where the external clothes can be "unzipped" from binary programs and exchanged for clothes of a different style.

The orthogonal bifurcation process of Metaphorio produces re-usable whole application software. This is analogous to a concept that is impossible in nature according to the second law of thermodynamics: the ontogenetic-reversal of a functional body of biological cells (GUI-clothed application software) converting them back into functionally-grouped stem-cell architecture (proto-software design patterns), which can then be restyled in a new GUI by a process of metaprogramming, e.g. Glade GUI building (see APPENDIX ).

Unlike the component re-usability of OOP, this leads to complete application-system re-usability after initial implementation, where restyling for other application contexts of similar engine functionality is a trivial adaptation. The net effect is economy of scale in custom software development.

Metaphorio can foster enormous leverage with MetaCalculus in R&D application evolution, because of the universality of MetaCalculus templates, and their alphabet-like combination into standard modeling design patterns.

Although both of these sub-languages are like tips of icebergs, where the bulk is in the hidden semantics, GAEMY facilitates their addition to standard programming languages as "add-ins", causing these languages to vertically evolve from implementation languages into modeling languages, in which implementation is automated. The intent is that the common languages will now become peer media for end-user scientific modelers and software engineering implementers to work as peers, mentoring each other with their complementary different expertise--fostering the leverage of synergy.

This synergy can build a bridge across the vast gap that now blocks the vertical evolution of software technology, especially in R&D, where there is no economy of scale to support labor-intensive development. Pure Parsers to Handle Ambiguities and Special Challenges

The MetaCalculus sub-language presents a major challenge to Yacc parsing, because the structure of its primary operation statements is not bound at parser-generation time, but at compilation time, by the reading of solver config files. That is the case if solvers are not to be "built-ins" to the language. In PROSE and Fortran Calculus, all solvers were built-ins (hard-coded in their compilers). In the MC8 end-user language Sycal (now MetaCalc), solver config files were introduced into its non-canonical compiler (written in Fortran 77). Now the challenge is how to do this via Yacc/Bison, or even the backtracking version of Yacc, btyacc.

It is therefore natural to focus just on the sub-language grammar of MetaCalculus which only has three operation statements, and experiment with various grammars for just these three statements. Moreover, since two of these statements are "compound sentences" approaching "paragraphs" in size and involve the solver config files challenge, it is convenient to create an assembly for each of these statements and create a working "pure parser" for just that statement, before aggregating its (sub-)assembly into successively higher assemblies, including the MetaCalculus sub-language assembly, and finally the whole language grammar assembly.

A pure parser is one which translates nothing (generates no output). GAEMY, as we have seen, can generate an augmented grammar, which still retains a kind of purity, because its semantic functions are automatically generated, and in such a way that will translate source syntax into itself. Such a test parser is called an "augmented pure parser". Producing such parsers with GAEMY has the added advantage that it generates a website that can be used to peruse the grammar and cross-reference debugging output.

In developing a translator, it is prudent to experiment with pure parsers to work out ambiguities and special challenges like that of MetaCalculus syntax, before performing any semantic surgery on the action stub functions. Wrapper Translators

The primary use of GAEMY by the open-source community, which will employ it to add MetaCalculus and Metaphorio to conventional programming languages, is intended to be in the production of "wrapper translators" and "wrapper translator-publishers" as pre-processors to the MetaCalc compiler and compiler-publisher. The resulting composite compilers and compiler-publishers will compile enhanced MetaCalculus and Metaphorio syntax dialects of these "meta" versions of the standard languages, to generate programs linked to MC8 and MC9 APIs. MetaCalc will thereby become the intermediate language of these diversified MetaScience language processors.

Producing such wrapper translators avoids the more daunting effort of dealing with MetaCalculus or Metaphorio semantics. This process will be hardly more than source-to-source transliteration, at least for the legacy languages. For the OOP languages, slightly more work will be required until an OOP version of MetaCalc is developed, down the road.

4.2 Assembly Synthesis

First off, to articulate the process of grammar management, we must change the original parent window of GAEMY, introduced as Figure 3.0:1, into a tabbed notebook version as illustrated in Figures 4.2:1 and 4.2:2.

Notice the change in the look and feel of the GUI. This is the result of moving Glade and EverGlade to a new version of Linux (Kubuntu 9.04 instead of Fedora 8) and a new version of KDE (KDE 4 instead of KDE 3).

Figure 4.2:1 GAEMY 2 Parent Window

This is the parent window which is invoked on GAEMY 2 startup, where the lower portion (framed in the figure) has the same buttons as Figure 3.0:1, but now they apply to the subject assembly which is to be named in the second entry widget, and the added notebook tab pages are provided for definition and maintenance of such assemblies within a project.

Figure 4.2:2 Tooltip (Mouseover Assembly Entry Widget)

This tooltip appears from moving the mouse over the assembly entry widget. It describes an assembly as a named group of subgrammar partitioned files originating from the partitioning of the original project grammar file into a series of files.

4.2.1 Original Project Grammar Partitioning

The method of defining and synthesizing grammar assemblies is to manually separate the original grammar file into partition files roughly corresponding to the pseudo sub-programs of its original design, where each partition has a cardinal number prefix, a name, and a ".ysg" suffix (e.g. 1.preamble.ysg, 2.tokens.ysg, 3.precedence.ysg, etc.).

These files must be stored in the ~/GAEMY2/Projects/$project/YSG subdirectory (where $project denotes the project name).

4.2.2 Assembly Origination

GAEMY employs these "cardinal partitions" as the "initial conditions" for evolving an unbound number of partitions in the created assemblies by adding new partition files to them, where file names involve one or more upper-case letters to the cardinal prefixes, and a different name, along with the ".ysg" suffix.

The origination of an assembly (designated $assembly) creates another subdirectory ~/GAEMY2/Project/$project/$assembly where all of the *.ysg partition files will be stored. Thereafter, such partitioned files will constitute the only persistent form in which the content of the assembly will be maintained. Temporary aggregated versions of this assembly grammar will be created in single files for intermediate processing steps, including GAEMY semantics preprocessing and Yacc processing, but will not be edited nor permanently stored in this single-file form.

Figures 4.2.1:1 through 4.2.1:4 illustrate the steps that can be taken to originate assembly content, starting with assignment of original project partition files.

Figure 4.2.1:1 Second Tabbed Page Showing Subgrammar Assembly Menu

The middle notebook tab brings up a menubar from which assembly partition files can be selected and added from the original project grammer, whole assemblies can be selected and added as subassemblies, and then unwanted files can be deleted to prune the assembly.

Figure 4.2.1:2 Tooltip (Mouseover Project Partition Files Menu)

The left side menu contains the partition files of the original project grammar in the pull-down check menu. Selecting any or all files from this menu and then clicking the button below, causes these files to be copied from the ~/GAEMY2/Projects/$project/YSG subdirectory to the ~/GAEMY2/Project/$project/$assembly directory. This also causes the right-side menu to be populated to register the altered contents of the subject assembly.

Figure 4.2.1:3 Tooltip (Mouseover Project Assemblies Menu)

The items of the center pull-down check menu are the list of all pre-defined assemblies. One or more of these whole assemblies can be added to the subject assembly by first activating the One or More radio button, then selecting the assemblies in the pull-down check menu and clicking the button below. This action will then repopulate the right-side menu to show the file content of the assembly.

If the Only Assembly radio button is active, then the purpose of the center menu is to change the subject assembly menu. Upon selecting an assembly in the pull-down check menu, that assembly will appear in the Assembly entry widget at the top of the window, and the center menu will be repopulated to exclude it from this list. Also, the file contents of the right-side menu will be repopulated to list all of the files in the subject assembly.

Figure 4.2.1:4 Tooltip (Mouseover Assembly Partition Files Menu)

This menu is used to view the files belonging to the subject assembly, and to prune the assembly by selecting those to be deleted and clicking the button.

4.2.3 Assembly Content Editing

Other than the file addition or deletion, the contents of an assembly grammar may be altered by editing of the assembly using the Edit and Apply buttons of the first notebook page shown in Figure 4.2.3:1.

During the editing process, new files may be added to the assembly, and named by adding capital letters to the file-name prefix (e.g. 2A.tokens2.ysg, 3B.precedence3.ysg, etc.).

Figure 4.2.3:1 Tooltip (Mouseover First Notebook Page) Gang Editing

If any of the four gang editors (Jedit, Jay, Geany, or Kate) are chosen, then the assembly files will be loaded in the editor as separate files. In this case only the files from the current assembly archive are loaded. Any new files created and saved will be taken as additions to the assembly. As these files are preloaded into the ~/GAEMY2/EDITS subdirectory, which becomes the PWD for editing, saving the changed files from the editor, returns them to this directory. Then the editing results are disposed back to the ~/GAEMY2/Projects/$project/$assembly directory by clicking the Apply button. Meld Editing

When Meld is selected, both the current assembly archive and the previous one in the assembly-line are involved. A temporary aggregate file appending the partition files of the assembly from each of the two archives is produced. The contents of each appended partition file is separated by comment deliminter lines containing the partition file names, so as to identify partition boundaries in the aggregate files. These temporary aggregate files are contructed in the ~/GAEMY2/EDITS directory, which becomes the PWD for editing.

Then the two aggregate files are loaded into the Meld editor, current in the left-pane buffer and previous in the right-pane buffer. Only changes to the left buffer will be propagated when this file is saved (back into the ~/GAEMY/EDITS directory).

When the Apply button is clicked. This file is then parsed, using the partition separation delimeters to re-create the partition files in the ~/GAEMY/EDITS directory, and these files are then returned to the ~/GAEMY2/Projects/$project/$assembly directory.

Note that the ~/GAEMY2/EDITS directory is cleared at the start of Edit processing and the ~/GAEMY2/Projects/$project/$assembly directory is cleared (its contents are moved to a temporary archive) at the start of Apply processing. So the temporary aggregate file saved from Meld editing may completely restructure the assembly (adding partitions, deleting some, and altering content of the partitions). Then when the Apply button is clicked, the saved aggregate file will be reparsed and its contituent files moved to the ~/GAEMY2/Projects/$project/$assembly directory.

GAEMY 2 Design Media Notes


GAEMY originated as a Perl script,, in the development of the EvoScope C-language webcode parser using Yacc. This script translated canonical Yacc BNF pure grammar rules into augmented grammar rules having action predicates containing the semantic action function calls for each syntax alternative subrule. Then it generated the function stubs that each predicate would call.

The GAEMY Pure Grammar Preprocessor

The enhanced version of this script, which became the GAEMY Yacc preprocessor, generalized the translation process to handle the various Yacc pseudo-variable argument types and equivalent formal parameters in the function stub headings.

These formal parameters were then employed in generating calls to the DBC trace-logging macro which would serve to describe the full working of the grammar in execution. Since the stub functions were designed for null translation, namely translating a source syntax into itself, this would serve to make the working of the grammar transparent, as an aid to working out Yacc shift-reduce conficts to remove ambiguities and refine grammar rules.

An additional translation function added to the GAEMY preprocessor was the generation of a grammar-action navigation website, and providing for its evolution due to surgical editing of the semantic action functions.

GUI Requirements

Utilizing comparative editing as an incremental design evolution technique gave rise to the assembly-line concept to maintain files undergoing semantic-action function surgery that had to be reconciled to grammar mutations; so both levels of evolution could be maintained in lockstep. This lockstep evolution of two disparate levels of design was the central problem of Metacybernetics that I had first encountered in the design of microcoded "metacomputers" as high-level-language emulators in the writeable instruction-set computer (WISC) architecture of the Aerospace Research Computer (ARC) in the early 1980s. We called this architecture a "metacomputer host". We were evolving the Tree Meta compiler-compiler to become its equivalent of GAEMY--a tool for evolving languages, along with their instruction-set architectures, and their microcode emulators, all in lockstep.

As the same problem of lockstep evolution of two disparate design levels is also central to the evolution of GUIs, the same Metacybernetic approach could be employed for evolution of Glade-produced GUIs as for Yacc-produced compilers. Evolving a GUI for GAEMY itself was the primary challenge that gave rise to the EverGlade GUI evolver.

Dynamic Menus

The primary initial challenge of the GAEMY GUI was the generation of a very large dynamic menu for accessing the action-semantic function files for comparative Meld or gang editing. I saw immediately that this challenge lent itself to the meta-design abstraction and regeneration process that I had postulated as a key concept of Metacybernetics in 1983. So I set about building EverGlade to employ this meta-design process to generate GAEMY GUI menus.

Definite Dynamic Menus

As it turned out, the meta-design abstraction-regeneration approach is quite adequate and straightforward for medium to large-scale definite menus. Thus it was relatively simple to employ Glade-2 in this early form of EverGlade. But the meta-design regeneration process involves Make. The GAEMY preprocessor has to generate a child GUI using Make, for each grammar mutation. Yet this is quite practical because grammar mutation is rare compared to semantic surgery.

Indefinite "Running" Dynamic Menus and Notebook Pages

However, for incremental indefinite dynamic menu generation, such as I have specified for the MetaCalculus RunViewer GUI, with its running sidebar expanding notebook, in which each page contains an indefinite incremental menu to index the parallel pages of the stream notebook, a more highly-dynamic notebook and menu generation process is required. In long MetaCalculus runs, the RunViewewer, which would be employed in real-time during the run, and in post-run warehoused runscapes, could conceivably contain thousands of sidebar notebook pages and hundreds of thousands of menu items to index the multi-stream output from optimization broadcasting swarm searching.

This "running GUI generation" process is beyond the scope of the Glade-2 version of EverGlade. It requires capabilities just now being approached in the bleeding-edge Glade-3/GtkBuilder development. Specifically, it requires dynamic alteration of GtkBuilder XML strings, originally produced by Glade-3.

So far this approach has not been attempted with GtkBuilder yet, according to my research. In fact, the only published work, Foundations of Gtk+ Development, utilizes the hybrid GtkUIManager/GtkActionGroup approach, which has a serious design flaw preventing the generation of running dynamic menus. It does not allow callback generalization utilizing the user-data parameter. Each callback function must be unique to its caller signal. Thus this approach cannot even be used to produce the GAEMY action menus created by EverGlade now, where all menu signals use the same callback function.

The approach to be taken for running GUI generation must be an extension of the Glade-3/GtkBuilder approach discussed in Micah Carrick's tutorial on this subject. This calls for a new version of EverGlade (EverGlade3).

GAEMY 2 Development Continuation


As it turns out, the GtkBuilder approach is the most straightforward way to implement the subgrammar assembly management design of GAEMY 2, as well as the more daunting RunViewer GUI. So my intent is to put further work on GAEMY 2 development on hold, while I develop EverGlade3. Then I will use EverGlade3 to complete GAEMY 2.



In the analogy of the way Metaphorio is designed to work—like ontogentic reversal, mentioned in Section 4.1.1, as being impossible according to the second law of thermodynamics—it is interesting to note that a kind of natural bypass of the second law does appear to be provided for by Nature, and that this bypass is somewhat analogous to the Metacybernetic reverse engineering evolution as implemented in EverGlade. See Section 7.0 in the EverGlade Manual. Apparently, stem cells originally manufactured by the body and remaining present throughout life can be extensively duplicated in vitro and then re-inserted in the body to regrow healthy tissue. A recent experiment with a heart patient (recently announced on TV) is attemping just that.