TABLE

From ICE Enterprises
Jump to navigation Jump to search

Perform various table related functions

<TAB>  - Table name
<FUNC> - Function Name (COPY,CREATE,EQUALS,LIST,LOAD,SAVE,...)
<P3>   - Optional parameter dependent on <FUNC>
<P4>   - Optional parameter dependent on <FUNC>

TABLE MODES:
  Each table has a mode. The mode of a table describes the underlying data
  structure that holds the table's keys and values. Each mode has slightly
  different behavior, particularly in terms of performance and iteration
  order.

  Prior to NeXtMidas 2.7.0 there were only three table modes (FB, KV and HT)
  and the default mode was hard-coded to KV mode. Several additional modes
  were added in NeXtMidas 2.7.0 as well as the ability to set the default
  table mode.

  Some of the more common table modes:
    FB   = FromBytes                 (Internal byte serialization of a table)
    HT   = HashTable                 (Uses a Java Hashtable)
    KV   = KeyVector                 (Uses NeXtMidas KeyVector)
    SLHM = SynchronizedLinkedHashMap (Uses a Java LinkedHashMap)
    STM  = SynchronizedTreeMap       (Uses a Java TreeMap)
    CHM  = ConcurrentHashMap         (Uses a Java ConcurrentHashMap) Since 3.1.1

  Details on all of the table modes are included in the documentation for the
  Table class (see nxm.sys.lib.Table).

FUNCTION: [DEF=LIST]
  COPY    - Copy a table.  Does not copy any references to other objects
            or tables, these are copied as strings.
              SYNTAX: TABLE <tabin> COPY <tabout>

  CREATE  - Creates a new table. This can (1) create an empty table, (2) create
            a table from a string, (3) create a table that is a shallow-copy of
            a Java Hashtable, (4) create a new reference to an existing Table,
            or (5) load a named .tbl file.
              SYNTAX:   TABLE <tabout> CREATE               ! (1) empty table
                     or TABLE <tabout> CREATE {x=1,y=2}     ! (2) from string
                     or TABLE <tabout> CREATE <hashtable>   ! (3) from Hashtable
                     or TABLE <tabout> CREATE <table>       ! (4) from Table
                     or TABLE <tabout> CREATE <tbl_file>    ! (5) from .tbl file

  EQUALS  - Compares two tables to see if they have the same keys and values.
            The result of the comparison ("TRUE" or "FALSE" is put in <res>).
              SYNTAX: TABLE <first> EQUALS <second> <res>

            The values are compared using Util.equals(Object,Object) method
            and does not test for nested tables. Also, this does not consider
            table order (e.g. {A=1,B=2} and {B=2,A=1} would be equal).

  GETDEFAULTFlags - Get the default table flags and put it in result <res>.
              SYNTAX: TABLE ,, GETDEFAULTFLAGS <res>

  GETESCAPEDSTRING - Get the Table as a String, with escape sequences inserted
                     where needed (in place of quotes, tabs, newlines, Unicode
                     sequences, etc). Since NeXtMidas 2.9.0.
              SYNTAX: TABLE <tab> GETESCAPEDSTRING <res>

  GETDEFAULTMODE - Get the default table mode and put it in result <res>.
              SYNTAX: TABLE ,, GETDEFAULTMODE <res>

  GETFLAGS - Get the table flags for <tab> and put it in results <res>.
             Since NeXtMidas 2.9.3.
              SYNTAX: TABLE <tab> GETFLAGS <res>

  GETKEY  - Get value of <key> in Table <tab> and put in it results <res>.
            Quote the <key> name to retrieve a mixed or all lower case key from
            the Table. If <res> is not specified, it is displayed to terminal.
            Since NeXtMidas 2.7.1.
              SYNTAX: TABLE <tab> GETKEY <key> <res>

  GETKEYS - Get all the keys in a table.
              SYNTAX: TABLE <tab> GETKEYS   <res>  ! Get the keys as an array
              SYNTAX: TABLE <tab> GETKEYS/S <res>  ! Get the keys as a string

            Valid Modifiers:
              /S - Returns a comma-separated string rather than an array.
              /R - Reverses the order of the elements returned.

  GETMODE - Get the table mode and put it in result <res>.
              SYNTAX: TABLE <tab> GETMODE <res>

  GETSIZE - Gets the size of the table (i.e. the number of keys in the table).
              SYNTAX: TABLE <tab> GETSIZE <res>

  LIST    - LIST the contents of a table
              SYNTAX: TABLE <tab> LIST

  LOAD    - Loads a table from a .tbl file.
              SYNTAX: TABLE <tabout> LOAD <filename>

            Note1: Without the /T or /X the file name MUST end in .tbl or .xml.
                   This works the same as "results t:<tabout> <filename>".
                   (When accessing a URL or a file without an extension the /T
                   or /X modifier must be used.)
            Note2: This forces output Table to KeyVector mode to allow
                   duplicates for legacy/backwards compatibility reasons (by
                   enabling the "Add" flag, but will only remove that flag on
                   the main table but not on sub-tables.)
                   To force using default Table mode and flags output table and
                   it sub-tables, use LOAD/T along with /FLAGS=default or
                   /FLAGS=<desired table flags> switch.

            Warning: The OPAL table import feature is still experimental and not
                     all OPAL table constructs are supported.

            Valid Modifiers:
              /O - File is an Midas2K OPAL table (note this modifier uses a
                   capital 'o' not the number zero).     [Since NeXtMidas 3.3.2]
              /C - Ignore normal NeXtMidas rules and treat OPAL table keys as
                   case-sensitive (this will make the table difficult to use
                   within NeXtMidas).                    [Since NeXtMidas 3.3.2]
              /T - File is a .tbl file even if does not end in .tbl.
              /X - File is a .xml file even if does not end in .xml.

  LOADCFG - Loads a table from a .cfg configuration file (such as those commonly
            used on Windows systems and with Python). Since NeXtMidas 3.1.3.
              SYNTAX: TABLE <tabout> LOADCFG <filename>

            This will take a configuration file in the form:
              [SECTION1]
              KEY1=VAL1
              KEY2=VAL2
              ...

              [SECTION2]
              KEY1=VAL1
              KEY2=VAL2
              ...

            And create a corresponding table of tables in the form:
             {SECTION1={KEY1=VAL1,KEY2=VAL2,...},
              SECTION2={KEY1=VAL1,KEY2=VAL2,...},
              ...}

            Valid Modifiers:
              /C - Ignore normal rules and treat the keys as case-sensitive
                   (does not apply to section names).
              /D - If a default section is found, automatically merge any
                   default values in with the other section entries.

  MERGE   - Merges <table1> and <table2> by adding or replacing keys in <table1>
            with those in <table2> and putting it into new <tabout> table. Inner
            tables in <table1> are replaced with any matching keys (with table
            values or otherwise) in <table2>.
              SYNTAX: TABLE <tabout> MERGE <table1> <table2>

            This is identical to the results syntax
              nM> res t3 ^{table1}^{table2}

            The MERGE/R function is particular useful to merge tables which
            are similarly formed but may have missing keys.

            Valid modifiers:
              /L - Do legacy table merge. Since NeXtMidas 2.7.1 and DEPRECATED.
                   SYNTAX: TABLE <tab> MERGE/L <table1_and_tabout>
              /R - Recursively merge the tables AND inner tables at the same
                   level and with the same key name in <table1> and <table2>.
                   Only valid without /L modifier. Since NeXtMidas 2.7.1.

            See examples below.

  SAVE    - Saves a table to a .tbl file. Note that the file name should include
            the .tbl extension.
              SYNTAX: TABLE <tabin> SAVE <filename>

  SAVECFG - Saves a table-of-tables to a Windows/Python type .cfg file. Note
            that the file name should include the .cfg extension. This is
            effectively the reverse of the LOADCFG function. Since 3.1.3.
              SYNTAX: TABLE <tabin> SAVECFG <filename>

  SETDEFAULTFLAGS - Set the default table flags.
              SYNTAX: TABLE ,, SETDEFAULTFLAGS <mode>

            Note that this can also be set using the ENVIRONMENT command (both
            commands provide identical functionality).

  SETDEFAULTMODE - Set the default table mode (note: can not set default table
            mode to FB).
              SYNTAX: TABLE ,, SETDEFAULTMODE <mode>

            Note that this can also be set using the ENVIRONMENT command (both
            commands provide identical functionality).

  SETFLAGS - Set the table flags specified in <flags> for <tab>.
             Since NeXtMidas 2.9.3.
              SYNTAX: TABLE <tab> SETFLAGS <flags>

  SETKEY  - Set a <value> at <key> in Table <tab>. Quote the <key> name to put
            a mixed or all lower case key into the Table. Since NeXtMidas 2.7.1.
              SYNTAX: TABLE <tab> SETKEY <key> <value>

  SETMODE - Set the table mode (note: can not set table mode to FB).
              SYNTAX: TABLE <tab> SETMODE <mode>

  SORT/K  - Sorts a table alphabetically based on the keys. This updates the
            specified <tab> and will change it's mode to KeyVector. Since
            NeXtMidas 2.7.0, it is significantly faster to create and use a
            table with TreeMap (TM) or STM mode.
              SYNTAX: TABLE <tab> SORT/K

            Valid Modifiers:
              /R - After sorting, reverse the sort.

  SORT/V  - Sorts a table based on the values in the table. This assumes that
            all of the values in the table are both homogeneous and Keyable.
            This updates the specified <tab>.                  (NOT IMPLEMENTED)
              SYNTAX: TABLE <tab> SORT/V <tags>

            The <tags> parameter is a is a list of the records to sort by in
            order of importance. Each tag may be prefixed with '+' (sort
            ascending, the default) or '-' (sort descending). For example
            "+FREQ|-MAG" sorts the table by FREQ (ascending) and sorts any
            records with matching FREQ values by MAG (descending).

Escape Sequences in Table:
--------------------------
Since NeXtMidas 2.9.0, escape sequences can be used when creating Tables from
Strings.

   nM> nM> res t:queen {a="\u265b"}
   nM> res queen
   T: QUEEN           = Table of 1 entries
   1S:  A             = ♛

The escape sequence is the escape character, which is a backslash ("\"),
followed by a sequence. generally, any escape sequence that can be used in Java
can be used here. This includes the Unicode escape sequences, which is "\u"
followed by 4 hex digits, such as the example above, which has the Unicode
escape sequence for the black queen chess piece.

If you have the backslash in your String and wish to preserve it, use another
backslash in front of it, just like in Java (and many other languages).

There is a Table flag to disable escape character translation, if desired. This
flag is able to be set as one of the default Table flags in the ENV table:

  nM> env set DEFAULTTABLEFLAGS +DisableEscapeSequences

Examples:
  1. Make a copy of an existing table
       nM> res tab {x=1,y=2}
       nM> tab tab COPY tab2

  2. Create a new empty table
       nM> table tab CREATE

  3. Sort a table based on its keys.
       nM> table tab SORT/K     ! Alphabetical order
       nM> table tab SORT/K/R   ! Reverse alphabetical order

  4. Compare two tables
       nM> table tab1 EQUALS tab2 equal
       nM> res equal

  5. Merge two tables without recursion (replace keys at first level)
       nM> set tbl1 {A=1,B=2,INNER={X=1,Y=1}}
       nM> set tbl2 {B=20,C=30,D=40,INNER={X=2,Z=2}}
       nM> table tbl3 MERGE tbl1 tbl2
       nM> res/all tbl3
         T: TBL3            = Table of 5 entries
         L:  A              = 1
         L:  B              = 20
         T:  INNER          = Table of 2 entries
         L:   X             = 2
         L:   Z             = 2
         L:  C              = 30
         L:  D              = 40

    Merge the same two tables WITH recursion (replace keys containing tables at
    all levels):
        nM> table tbl3 MERGE/R tbl1 tbl2
        nM> res/all tbl3
          T: TBL3            = Table of 5 entries
          L:  A              = 1
          L:  B              = 20
          T:  INNER          = Table of 2 entries
          L:   X             = 2
          L:   Z             = 2
          L:  C              = 30
          L:  D              = 40

  6. Set/Get a mixed-case key from a table
       nM> res t:tab1 {}
       nM> table tab1 SETKEY "mixedCaseKey" "value for mixed case key"
       nM> table tab1 GETKEY "mixedCaseKey"  value
       nM> res value
        24S: VALUE           = value for mixed case key
       nM> table tab1 GETKEY "mixedCaseKey"
        24S: mixedCaseKey    = value for mixed case key

  7. Save a table to a Windows/Python type .cfg file
       nM> res tmpTable {SECTION1={KEY1="VAL1",KEY2="VAL2"},&
                SECTION2={KEY3="Val3",KEY4="val4"}}
       nM> TABLE tmpTable SAVECFG test_table_cfg.cfg

  8. Load a Windows/Python type .cfg file (table of tables) into a table
       nM> res tmpTable2 {}
       nM> TABLE tmpTable2 LOADCFG test_table_cfg.cfg


A FEW NOTES REGARDING NEXTMIDAS TABLES:
=======================================
  Syntax
  ------------
  When defined on the command line or in a macro a table is specified in the
  following format:

    nM> res t:mytable {TAG1=VAL1,TAG2=VAL2,...,TAGn=VALn}

  Where the table is defined within a pair of braces ({...}) where each element
  of the table is specified by a TAG=VALUE syntax with a comma (,) separating
  subsequent elements.

  Note that no translation is done on the values of a table unless they are
  escaped with a caret:
    nM> res ten 10
    nM> res t:mytab {A=ten}
    nM> res mytab.a
     3S: MYTAB.A   = TEN
    nM> res t:mytab {A=^ten}
    nM> res mytab.a
      L: MYTAB.A   = 1

  Invalid Syntax
  ------------
  Since NeXtMidas 3.3.0 (via the Table flag CheckMalformed, which is off by
  default), if you fail to follow the syntax above, you will get an exception
  in the following circumstances:
    nM> res tab {X=""Y=2}  ! missing comma
    nM> res t:tab {X=1,Y=2 ! missing '}' at end of String

  Valid Tag Names
  ---------------
  All tags in a table must be one or more characters in length and may contain
  only numbers, upper-case letters and the underscore. This is basically the
  same as identifiers in C/C++ except that it may begin with a number and is
  case-insensitive. Tags may be specified on the command line or in a macro with
  lower-case letters since they will be automatically converted to upper case by
  the NeXtMidas shell.
    Regular Expression for Tag Names: [0-9A-Z_]+

  There is no upper limit to the length of a tag name but users are likely to
  find that tag names longer than 16 characters are unwieldy to use. Note: Older
  versions of NeXtMidas allowed tag names to contain dashes (-). This behavior
  has been deprecated since it caused confusion as to whether a statement like
  ^MYTBL.FOO-9 represented a single table reference (^{MYTBL.FOO-9}) or a table
  reference followed by a subtraction (^{MYTBL.FOO}-9).

  Types in a Table
  ----------------
  When a table is specified on the command line its type is automatically
  assigned by the NeXtMidas shell. For example:

    nM> res t:mytable {ONE=1.0,TWO=2.0,TEN=10.0}

  Would create a table with three doubles in it. It is possible to specify the
  type of the value by giving the appropriate type specifier before the tag name
  (see Typecasting in a Macro for the list of valid type specifiers). For
  example:

    nM> res t:mytable {B:ONE=1.0,B:TWO=2.0,F:TEN=10.0}

  Would create a table with two bytes and a float.

  Inside a table, strings should be quoted, numbers should not be quoted, and
  subtables should be within a pair of braces. For example:

    nM> res t:mytable {NUMBER=1,STRING="My String",SUBTABLE={ONE=1,TWO=2}}

  One exception is that single-word strings with no special characters will be
  accepted even if they are not quoted (though this practice is discouraged).

  Translation
  -----------
  When translating a table reference translation always goes to the end unless
  the translation is done using the caret-brace (^{...}) syntax. Here are a few
  examples:

     1: nM> res mytable {A=1,B=2,C=3,D={A=10,B=20,C=30},X=9,Y=99,Z=999}
     2: nM> res a d
     3: nM> res b {a="a",b="b",c="c",D={A="x",B="y",C="z",D="z"}}
     4: nM> res mytable.d.a
          L: MYTABLE.D.A     = 10
     5: nM> res mytable.^a.a
          ERROR: ...
     6: nM> res mytable.^{a}.a
          L: MYTABLE.D.A     = 10
     7: nM> res mytable.^{b.a}.d
          D: MYTABLE.A.D     = 1.0
     8: nM> res mytable.^{b.^a.d}
          ERROR: ...
     9: nM> res mytable.^{b.^{a}.d}
          L: MYTABLE.Z       = 999
    10: nM> res mytable {X=5,Y=2}{X=1,Z=3}
       is the same as
        nM> res mytable {X=1,Y=2,Z=3}

  Line 5 causes an error because it tried to access mytable.^{a.a} but a is a
  string ("d") not a table. Similarly line 8 attempts to access a.d. To avoid
  situations such as this the caret-brace syntax is always recommended when
  dealing with translation in the middle of a table name.

  While the above examples may seem a bit contrived, many applications use the
  values within one table as the keys used to access another table. It is not
  uncommon to see the nesting of these two or three levels deep.

  Altering a Table
  ----------------
  Once a table has been defined the the values in the table can be easily
  modified or new values added using the RESULT or SET commands. For example:

    nM> res mytable.a 7.0

  This will set the value A in my table to be the double value 7.0. To set it to
  a byte use the following:

    nM> res b:mytable.a 7.0

  Note that before adding values to a subtable the subtable must first be
  inserted:

    nM> res mytable {ONE=1,TWO=2}
    nM> res mytable.SUBTABLE.a 3
      ERROR: Table Entry SUBTABLE not found for put of A
    nM> res mytable.SUBTABLE { }
    nM> res mytable.SUBTABLE.a 3
    nM> res mytable.SUBTABLE.a
      L: MYTABLE.SUBTABLE.A = 3


  Values in a table can be removed using the REMOVE command. For example:

    nM> rem mytable.a

  This will remove A from MYTABLE.

  Table Modes and Sorting
  -----------------------
  As listed above, table can be set to use one of several internal modes. For
  comparison purposes, three are described below:

  Mode  Underlying Storage Mechanism  Nominal Order of Operation  Sortable
                                      Get  Add  Remove
  ----  ----------------------------  ---  ---  ------            --------
  KV    NeXtMidas KeyVector           O(n) O(1) O(n)              YES
  SLHM* Java Synchronized HashMap     O(1) O(1) O(1)              NO
  FB**  Byte Buffer                   n/a  n/a  n/a               n/a

  *  SLHM is the default mode.
  ** FB mode is used only for serialization of the table. A table in FB mode
     will automatically switch to KV mode on first use.

  The tradeoff between KV mode and SLHM mode is sortability versus access time.
  The TABLE command provides functions that can change the mode of a table and
  sort a table.

  View the Table javadocs for a complete comparison of modes.

  CLONE -vs- REFERENCE
  --------------------
  A table can be cloned (copied) such that the copy is stored in a separate
  memory location from the original.  A change to the clone will not affect the
  original.  When two tables need to refer to the same result a reference
  can be created such that a change using either reference variable changes
  the value seen by both.  A results table example illustrates these
  relationships:

    nM> set origtable {x=1,y=2} (create a table and assign some values)
    nM> set table1 origtable    (create a reference to the original table)
    nM> set table2 ^origtable   (clone/copy the original table)
    nM> set table1.x 9          (modify x value in table 1)
    nM> res table1.x            (x=9 because table 1 value was modified)
    nM> res table2.x            (x=1 because the copy remains unchanged)
    nM> res origtable.x         (x=9 table1 references same memory as origtable)

Switches:
  /FLAGS= - Table flags mask/settings to use for LOAD/SAVE functions.
            Use LOAD/T function along with this switch to ensure that the output
            table and all it's sub-tables use specified flags. The SAVE function
            only uses the DisableEscapeSequences and SerializeTimeAsTColon flags.
            Mask Options are:
              Add,AddRoot,AutoReadPrev,AutoWritePrev,
              CaseSensitive,DisableEscapeSequences,ExceptionOnNull,
              Force,OnlyDotIntoSubTables,ReadOnly,CheckMalformed,
              SerializeTimeAsTColon
            (Since 3.1.2 for LOAD and 3.5.3 for SAVE/SerializeTimeAsTColon)

See Also: ENVIRONMENT, nxm.sys.lib.Table