Original Parser

version 1 by Ron Newcomb

  • Home page
  • Beginning
  • Previous
  • Next



  • Book - Tokens

    Chapter - Parse Token

    [The main parsing routine above tried a sequence of ``grammar lines'' in turn,
    matching each against the text typed until one fitted. A grammar line is
    itself a sequence of ``grammar tokens''. Here we have to parse the tokens.

    |ParseToken(type, data)| tries the match text beginning at the current
    word marker |wn| against a token of the given |type|, with the given |data|.
    The optional further arguments |token_n| and |token| supply the token
    number in the current grammar line (because some tokens do depend on what
    has happened before or is needed later) and the address of the dictionary
    word which makes up the |token|, in the case where it's a ``preposition''.

    The return values are:
    (a) |GPR_REPARSE| for ``I have rewritten the command, please re-parse from scratch'';
    (b) |GPR_PREPOSITION| for ``token accepted with no result'';
    (c) $-256 + x$ for ``please parse |ParseToken(ELEMENTARY_TT, x)| instead'';
    (d) 0 for ``token accepted, result is the multiple object list'';
    (e) 1 for ``token accepted, result is the number in |parsed_number|'';
    (f) an object number for ``token accepted with this object as result'';
    (g) $-1$ for ``token rejected''.

    Strictly speaking |ParseToken| is a shell routine which saves the current
    state on the stack, and calling |ParseToken__| to do the actual work.

    Once again the routine is traditionally divided into six letters, here named under
    paragraphs ``Parse Token Letter A'', and so on.

    (A) Analyse the token; handle all tokens not involving object lists and
    break down others into elementary tokens
    (B) Begin parsing an object list
    (C) Parse descriptors (articles, pronouns, etc.) in the list
    (D) Parse an object name
    (E) Parse connectives (AND, BUT, etc.) and go back to (C)
    (F) Return the conclusion of parsing an object list]

    [ Called from Number.i6t ]
    [ ParseTokenStopped x y;
        if (wn>WordCount()) return GPR_FAIL;
        return ParseToken(x,y);
    ]

    To decide what parser result is the parsing of (y - a grammar token) as (x - token type) if any (this is ParseTokenStopped):
        if the parser's current word position > the word count, decide on parse fails;
        decide on the parsing of y as x at 0 under 'something'.

    The parse token nesting level is a number that varies.
    The parse token nesting level variable translates into I6 as "parsetoken_nesting".
    Include (- Global parsetoken_nesting = 0; -).

    To decide what parser result is the parsing of (y - a grammar token) as (x - a token type) at (pcount - number) under (token - a grammar token) (this is ParseToken):
        let temp be an object;
        let tempnum be a number;
        if the parse token nesting level > 0:
            save the next word to parse's position;
            save the noun filter;
            save the number of words matched per object ;
            save the number of match groups;
            save where the previous typo's at;
            repeat through the match list of size the number of objects in the match list:
                now temp is the match list element;
                save temp;
                now tempnum is the match list's groups element;
                save tempnum;
                now tempnum is the the match score list element;
                save tempnum;
            save the number of objects in the match list;
        increment the parse token nesting level;
        let rv be the actual parsing of y as x at pcount and token;
        decrement the parse token nesting level;
        if the parse token nesting level > 0:
            restore the number of objects in the match list;
            repeat through the match list of size the number of objects in the match list:
                restore tempnum;
                change the match score list element to tempnum;
                restore tempnum;
                change the match list's groups element to tempnum;
                restore temp;
                change the match list element to temp;
            restore where the previous typo's at;
            restore the number of match groups;
            restore the number of words matched per object ;
            restore the noun filter;
            restore the next word to parse's position;
        decide on rv.

    [ ParseToken given_ttype given_tdata token_n token i t rv;
        if (parsetoken_nesting > 0) {
            ! save match globals
            @push match_from; @push token_filter; @push match_length;
            @push number_of_classes; @push oops_from;
            for (i=0: i<number_matched: i++) {
                t = match_list-->i; @push t;
                t = match_classes-->i; @push t;
                t = match_scores-->i; @push t;
            }
            @push number_matched;
         }

        parsetoken_nesting++;
        rv = ParseToken__(given_ttype, given_tdata, token_n, token);
        parsetoken_nesting--;

        if (parsetoken_nesting > 0) {
            ! restore match globals
            @pull number_matched;
            for (i=0: i<number_matched: i++) {
    @pull t; match_scores-->i = t;
                @pull t; match_classes-->i = t;
                @pull t; match_list-->i = t;
    }
            @pull oops_from; @pull number_of_classes;
            @pull match_length; @pull token_filter; @pull match_from;
        }
        return rv;
    ]