#

| Chapter 10 PLVprs, PLVtkn, and PLVprsps: Parsing Strings |

## 10.2 PLVtkn: Managing PL/SQL Tokens

The `PLVtkn` (PL/Vision ToKeN) package determines whether an identifier is a PL/SQL keyword. It does this by using a token table containing these keywords.

Every PL/SQL program is filled with identifiers. Identifiers are named PL/SQL language elements and include variable names, program names, and reserved words. Reserved words play a very different role in our programs than do the application-specific identifiers. I recommend strongly in Oracle PL/SQL Programming that you reflect these different roles in your program by using the UPPER-lower method: all reserved words are typed in UPPER case and all application-specific identifiers are typed in lower case. I even go so far, in PL/Vision, as to provide you with a package (PLVcase) which will automatically convert your programs to the UPPER-lower method.

### 10.2.1 Keeping Track of PL/SQL Keywords

Well, if `PLVcase` is going to uppercase only keywords, it has to know which identifiers in a PL/SQL program are the reserved words. This information is maintained in the `PLV_token` table, which has the following structure:

>      Name                            Null?    Type
>      ------------------------------- -------- ----
>      TOKEN                           NOT NULL VARCHAR2(100)
>      TOKEN_TYPE                               VARCHAR2(10)

where the `token` column is the identifier and `token_type` indicates the type.

The different token types in the `PLV_token` table are stored in the `PLV_token_type` table, which has this structure:

>      Name                            Null?    Type
>      ------------------------------- -------- ----
>      TOKEN_TYPE                               VARCHAR2(10)
>      NAME                            NOT NULL VARCHAR2(100)

The contents of the `PLV_token_type` are explained in the following table:

Token Type | Name | Description
--- | --- | ---
B | BUILT-IN | Builtin functions and procedures of the PL/SQL language, including packaged builtins.
D | DATATYPE | Different datatypes of the PL/SQL language, such as INTEGER and VARCHAR2.
DD | DATA-DICTIONARY | Views and tables from the Oracle Server data dictionary, such as ALL_SOURCE and DUAL.
E | EXCEPTION | Predefined system exceptions such as ZERO_DIVIDE.
OF | ORACLE-FORMS | Reserved words from the Oracle Forms product set
S | SYMBOL | Symbols like + and =.
SQL | SQL | Elements of the SQL language. In many cases, SQL tokens are used in PL/SQL and also in Oracle Developer/2000. These are still listed as SQL tokens.
X | SYNTAX | Syntax elements of the PL/SQL language, such as AND or LIKE.

There is a row in `PLV_token` for each reserved word in PL/SQL. You can change the contents of this table if you want. You might, for example, want to add keywords for the Oracle Developer/2000 builtins or the Oracle Web Agent PL/SQL packages. You can even add your own application-specific identifiers to the table. As long as the token type you assign is not any of those listed above, PL/Vision will not misinterpret your entries.

There are currently 1,235 rows in the `PLV_token` table, broken down by token type as follows:

Token Type | Count
--- | ---
BUILT-IN | 198
DATATYPE | 22
DATA-DICTIONARY | 168
EXCEPTION | 15
ORACLE-FORMS | 623
SYMBOL | 32
SQL | 94
SYNTAX | 83

From the PL/SQL side of things, the PLVtkn package provides an interface to the `PLV_token` table. This package is used by PLVcase to determine the case of an individual token according to the UPPER-lower method.

As you will soon see, PLVtkn is not a particularly large or complicated package. Its purpose in life is to consolidate all of the logic having to do with individual PL/SQL tokens, particularly regarding keywords. By hiding the implementation details (the name and structure of the `PLV_token` table, the particular values used to denote a symbol or syntax element or builtin function), PLVtkn makes it easier for developers to apply this information in their own programs.

### 10.2.2 Determining Token Type

PLVtkn provides a set of functions you can use to determine a string's token type in PL/SQL. The headers for these functions are shown below:

>     FUNCTION is_keyword
>        (token_in IN VARCHAR2, type_in IN VARCHAR2 := c_any) RETURN BOOLEAN;

>     FUNCTION is_syntax (token_in IN VARCHAR2) RETURN BOOLEAN;
>     FUNCTION is_builtin (token_in IN VARCHAR2) RETURN BOOLEAN;
>     FUNCTION is_symbol (token_in IN VARCHAR2) RETURN BOOLEAN;
>     FUNCTION is_datatype (token_in IN VARCHAR2) RETURN BOOLEAN;
>     FUNCTION is_exception (token_in IN VARCHAR2) RETURN BOOLEAN;

All of the functions except for `is_keyword` take a single string argument and return TRUE if the string is that type of token. The following examples illustrate the way the PLVtkn functions interpret various strings:

>     SQL> exec p.l(PLVtkn.is_builtin('to_char'));
>     TRUE
>     SQL> exec p.l(PLVtkn.is_builtin('loop'));
>     FALSE
>     SQL> exec p.l(PLVtkn.is_syntax('loop'));
>     TRUE
>     SQL> exec p.l(PLVtkn.is_syntax('='));
>     FALSE
>     SQL> exec p.l(PLVtkn.is_symbol('='));
>     TRUE

#### 10.2.2.1 Generic keyword checking

The `is_keyword` function is a more general-purpose function. It returns TRUE if the token is a keyword of the type specified by the second argument. The default value for this second parameter is `PLVprs.c_any`, which means that `is_keyword` will return TRUE if the specified token is any kind of keyword.

`PLVcase` uses the `is_keyword` to determine whether the token should be upper- or lowercase. When applying the UPPER-lower method, it doesn't matter if the token is a builtin function or a syntax element, such as the END statement. All such keywords must be uppercase. Here is the code from the `PLVcase.token` procedure which performs the actual conversion:

>     IF PLVtkn.is_keyword (v_token)
>     THEN
>       v_token := UPPER (v_token);
>     ELSE
>       v_token := LOWER (v_token);
>     END IF;

To keep code volume in PLVtkn to an absolute minimum and eliminate redundancy, I implement all of the "specialized" `is` functions (`is_builtin`, `is_syntax`, etc.) with a call to `is_keyword`, as shown below:

>     FUNCTION is_symbol (token_in IN VARCHAR2)
>        RETURN BOOLEAN
>     IS
>     BEGIN
>        RETURN (is_keyword (token_in, c_symbol));
>     END;

### 10.2.3 Retrieving Information About a Token

You will use the `get_keyword` procedure to retrieve from the `PLV_token` table all information stored about a particular token. The header of this procedure is:

>        PROCEDURE get_keyword (token_in IN VARCHAR2, kw OUT kw_rectype);

You provide the token or string and `get_keyword` returns a PL/SQL record, which is a translated version of the row in the table. The translation generally involves converting string constants to Boolean values. For example, one of the record's fields is named `is_keyword`. The expression assigned to this Boolean field is:

>        kw.is_keyword :=
>           kw_rec.token_type IN
>              (c_syntax, c_builtin, c_symbol,
>               c_sql, c_datatype, c_datadict, c_exception);

where `kw_rec` is the cursor-based record into which the `PLV_token` row is fetched.

The anonymous block below shows how to use `get_keyword`. It accepts a string from the user of this script (`plvtkn.tst`), retrieves the information about that string (as a token), and displays some of the data.

>     DECLARE
>        my_kw PLVtkn.kw_rectype;
>     BEGIN
>        PLVtkn.get_keyword ('&1', my_kw);
>        p.l (my_kw.token_type);
>        p.l (my_kw.is_keyword);
>     END;
>     /

The lines below show this script being executed for the THEN keyword.

>     SQL> @plvtkn.tst then
>     X
>     TRUE

10.1 PLVprs: Useful String Parsing Extensions |  | 10.3 PLVprsps: Parsing PL/SQL Strings

Copyright (c) 2000 O'Reilly & Associates. All rights reserved.