ConvertBot: Perform various table related functions

2020-04-27T22:05:03Z

Perform various table related functions

New page

Perform various table related functions
<pre>
<TAB> - Table name
<FUNC> - Function Name (COPY,CREATE,EQUALS,LIST,LOAD,SAVE,...)
<P3> - Optional parameter dependent on <FUNC>
<P4> - Optional parameter dependent on <FUNC>

TABLE MODES:
Each table has a mode. The mode of a table describes the underlying data
structure that holds the table's keys and values. Each mode has slightly
different behavior, particularly in terms of performance and iteration
order.

Prior to NeXtMidas 2.7.0 there were only three table modes (FB, KV and HT)
and the default mode was hard-coded to KV mode. Several additional modes
were added in NeXtMidas 2.7.0 as well as the ability to set the default
table mode.

Some of the more common table modes:
FB = FromBytes (Internal byte serialization of a table)
HT = HashTable (Uses a Java Hashtable)
KV = KeyVector (Uses NeXtMidas KeyVector)
SLHM = SynchronizedLinkedHashMap (Uses a Java LinkedHashMap)
STM = SynchronizedTreeMap (Uses a Java TreeMap)
CHM = ConcurrentHashMap (Uses a Java ConcurrentHashMap) Since 3.1.1

Details on all of the table modes are included in the documentation for the
Table class (see nxm.sys.lib.Table).

FUNCTION: [DEF=LIST]
COPY - Copy a table. Does not copy any references to other objects
or tables, these are copied as strings.
SYNTAX: TABLE <tabin> COPY <tabout>

CREATE - Creates a new table. This can (1) create an empty table, (2) create
a table from a string, (3) create a table that is a shallow-copy of
a Java Hashtable, (4) create a new reference to an existing Table,
or (5) load a named .tbl file.
SYNTAX: TABLE <tabout> CREATE ! (1) empty table
or TABLE <tabout> CREATE {x=1,y=2} ! (2) from string
or TABLE <tabout> CREATE <hashtable> ! (3) from Hashtable
or TABLE <tabout> CREATE <table> ! (4) from Table
or TABLE <tabout> CREATE <tbl_file> ! (5) from .tbl file

EQUALS - Compares two tables to see if they have the same keys and values.
The result of the comparison ("TRUE" or "FALSE" is put in <res>).
SYNTAX: TABLE <first> EQUALS <second> <res>

The values are compared using Util.equals(Object,Object) method
and does not test for nested tables. Also, this does not consider
table order (e.g. {A=1,B=2} and {B=2,A=1} would be equal).

GETDEFAULTFlags - Get the default table flags and put it in result <res>.
SYNTAX: TABLE ,, GETDEFAULTFLAGS <res>

GETESCAPEDSTRING - Get the Table as a String, with escape sequences inserted
where needed (in place of quotes, tabs, newlines, Unicode
sequences, etc). Since NeXtMidas 2.9.0.
SYNTAX: TABLE <tab> GETESCAPEDSTRING <res>

GETDEFAULTMODE - Get the default table mode and put it in result <res>.
SYNTAX: TABLE ,, GETDEFAULTMODE <res>

GETFLAGS - Get the table flags for <tab> and put it in results <res>.
Since NeXtMidas 2.9.3.
SYNTAX: TABLE <tab> GETFLAGS <res>

GETKEY - Get value of <key> in Table <tab> and put in it results <res>.
Quote the <key> name to retrieve a mixed or all lower case key from
the Table. If <res> is not specified, it is displayed to terminal.
Since NeXtMidas 2.7.1.
SYNTAX: TABLE <tab> GETKEY <key> <res>

GETKEYS - Get all the keys in a table.
SYNTAX: TABLE <tab> GETKEYS <res> ! Get the keys as an array
SYNTAX: TABLE <tab> GETKEYS/S <res> ! Get the keys as a string

Valid Modifiers:
/S - Returns a comma-separated string rather than an array.
/R - Reverses the order of the elements returned.

GETMODE - Get the table mode and put it in result <res>.
SYNTAX: TABLE <tab> GETMODE <res>

GETSIZE - Gets the size of the table (i.e. the number of keys in the table).
SYNTAX: TABLE <tab> GETSIZE <res>

LIST - LIST the contents of a table
SYNTAX: TABLE <tab> LIST

LOAD - Loads a table from a .tbl file.
SYNTAX: TABLE <tabout> LOAD <filename>

Note1: Without the /T or /X the file name MUST end in .tbl or .xml.
This works the same as "results t:<tabout> <filename>".
(When accessing a URL or a file without an extension the /T
or /X modifier must be used.)
Note2: This forces output Table to KeyVector mode to allow
duplicates for legacy/backwards compatibility reasons (by
enabling the "Add" flag, but will only remove that flag on
the main table but not on sub-tables.)
To force using default Table mode and flags output table and
it sub-tables, use LOAD/T along with /FLAGS=default or
/FLAGS=<desired table flags> switch.

Warning: The OPAL table import feature is still experimental and not
all OPAL table constructs are supported.

Valid Modifiers:
/O - File is an Midas2K OPAL table (note this modifier uses a
capital 'o' not the number zero). [Since NeXtMidas 3.3.2]
/C - Ignore normal NeXtMidas rules and treat OPAL table keys as
case-sensitive (this will make the table difficult to use
within NeXtMidas). [Since NeXtMidas 3.3.2]
/T - File is a .tbl file even if does not end in .tbl.
/X - File is a .xml file even if does not end in .xml.

LOADCFG - Loads a table from a .cfg configuration file (such as those commonly
used on Windows systems and with Python). Since NeXtMidas 3.1.3.
SYNTAX: TABLE <tabout> LOADCFG <filename>

This will take a configuration file in the form:
[SECTION1]
KEY1=VAL1
KEY2=VAL2
...

[SECTION2]
KEY1=VAL1
KEY2=VAL2
...

And create a corresponding table of tables in the form:
{SECTION1={KEY1=VAL1,KEY2=VAL2,...},
SECTION2={KEY1=VAL1,KEY2=VAL2,...},
...}

Valid Modifiers:
/C - Ignore normal rules and treat the keys as case-sensitive
(does not apply to section names).
/D - If a default section is found, automatically merge any
default values in with the other section entries.

MERGE - Merges <table1> and <table2> by adding or replacing keys in <table1>
with those in <table2> and putting it into new <tabout> table. Inner
tables in <table1> are replaced with any matching keys (with table
values or otherwise) in <table2>.
SYNTAX: TABLE <tabout> MERGE <table1> <table2>

This is identical to the results syntax
nM> res t3 ^{table1}^{table2}

The MERGE/R function is particular useful to merge tables which
are similarly formed but may have missing keys.

Valid modifiers:
/L - Do legacy table merge. Since NeXtMidas 2.7.1 and DEPRECATED.
SYNTAX: TABLE <tab> MERGE/L <table1_and_tabout>
/R - Recursively merge the tables AND inner tables at the same
level and with the same key name in <table1> and <table2>.
Only valid without /L modifier. Since NeXtMidas 2.7.1.

See examples below.

SAVE - Saves a table to a .tbl file. Note that the file name should include
the .tbl extension.
SYNTAX: TABLE <tabin> SAVE <filename>

SAVECFG - Saves a table-of-tables to a Windows/Python type .cfg file. Note
that the file name should include the .cfg extension. This is
effectively the reverse of the LOADCFG function. Since 3.1.3.
SYNTAX: TABLE <tabin> SAVECFG <filename>

SETDEFAULTFLAGS - Set the default table flags.
SYNTAX: TABLE ,, SETDEFAULTFLAGS <mode>

Note that this can also be set using the ENVIRONMENT command (both
commands provide identical functionality).

SETDEFAULTMODE - Set the default table mode (note: can not set default table
mode to FB).
SYNTAX: TABLE ,, SETDEFAULTMODE <mode>

Note that this can also be set using the ENVIRONMENT command (both
commands provide identical functionality).

SETFLAGS - Set the table flags specified in <flags> for <tab>.
Since NeXtMidas 2.9.3.
SYNTAX: TABLE <tab> SETFLAGS <flags>

SETKEY - Set a <value> at <key> in Table <tab>. Quote the <key> name to put
a mixed or all lower case key into the Table. Since NeXtMidas 2.7.1.
SYNTAX: TABLE <tab> SETKEY <key> <value>

SETMODE - Set the table mode (note: can not set table mode to FB).
SYNTAX: TABLE <tab> SETMODE <mode>

SORT/K - Sorts a table alphabetically based on the keys. This updates the
specified <tab> and will change it's mode to KeyVector. Since
NeXtMidas 2.7.0, it is significantly faster to create and use a
table with TreeMap (TM) or STM mode.
SYNTAX: TABLE <tab> SORT/K

Valid Modifiers:
/R - After sorting, reverse the sort.

SORT/V - Sorts a table based on the values in the table. This assumes that
all of the values in the table are both homogeneous and Keyable.
This updates the specified <tab>. (NOT IMPLEMENTED)
SYNTAX: TABLE <tab> SORT/V <tags>

The <tags> parameter is a is a list of the records to sort by in
order of importance. Each tag may be prefixed with '+' (sort
ascending, the default) or '-' (sort descending). For example
"+FREQ|-MAG" sorts the table by FREQ (ascending) and sorts any
records with matching FREQ values by MAG (descending).

Escape Sequences in Table:
--------------------------
Since NeXtMidas 2.9.0, escape sequences can be used when creating Tables from
Strings.

nM> nM> res t:queen {a="\u265b"}
nM> res queen
T: QUEEN = Table of 1 entries
1S: A = ♛

The escape sequence is the escape character, which is a backslash ("\"),
followed by a sequence. generally, any escape sequence that can be used in Java
can be used here. This includes the Unicode escape sequences, which is "\u"
followed by 4 hex digits, such as the example above, which has the Unicode
escape sequence for the black queen chess piece.

If you have the backslash in your String and wish to preserve it, use another
backslash in front of it, just like in Java (and many other languages).

There is a Table flag to disable escape character translation, if desired. This
flag is able to be set as one of the default Table flags in the ENV table:

nM> env set DEFAULTTABLEFLAGS +DisableEscapeSequences

Examples:
1. Make a copy of an existing table
nM> res tab {x=1,y=2}
nM> tab tab COPY tab2

2. Create a new empty table
nM> table tab CREATE

3. Sort a table based on its keys.
nM> table tab SORT/K ! Alphabetical order
nM> table tab SORT/K/R ! Reverse alphabetical order

4. Compare two tables
nM> table tab1 EQUALS tab2 equal
nM> res equal

5. Merge two tables without recursion (replace keys at first level)
nM> set tbl1 {A=1,B=2,INNER={X=1,Y=1}}
nM> set tbl2 {B=20,C=30,D=40,INNER={X=2,Z=2}}
nM> table tbl3 MERGE tbl1 tbl2
nM> res/all tbl3
T: TBL3 = Table of 5 entries
L: A = 1
L: B = 20
T: INNER = Table of 2 entries
L: X = 2
L: Z = 2
L: C = 30
L: D = 40

Merge the same two tables WITH recursion (replace keys containing tables at
all levels):
nM> table tbl3 MERGE/R tbl1 tbl2
nM> res/all tbl3
T: TBL3 = Table of 5 entries
L: A = 1
L: B = 20
T: INNER = Table of 2 entries
L: X = 2
L: Z = 2
L: C = 30
L: D = 40

6. Set/Get a mixed-case key from a table
nM> res t:tab1 {}
nM> table tab1 SETKEY "mixedCaseKey" "value for mixed case key"
nM> table tab1 GETKEY "mixedCaseKey" value
nM> res value
24S: VALUE = value for mixed case key
nM> table tab1 GETKEY "mixedCaseKey"
24S: mixedCaseKey = value for mixed case key

7. Save a table to a Windows/Python type .cfg file
nM> res tmpTable {SECTION1={KEY1="VAL1",KEY2="VAL2"},&
SECTION2={KEY3="Val3",KEY4="val4"}}
nM> TABLE tmpTable SAVECFG test_table_cfg.cfg

8. Load a Windows/Python type .cfg file (table of tables) into a table
nM> res tmpTable2 {}
nM> TABLE tmpTable2 LOADCFG test_table_cfg.cfg

A FEW NOTES REGARDING NEXTMIDAS TABLES:
=======================================
Syntax
------------
When defined on the command line or in a macro a table is specified in the
following format:

nM> res t:mytable {TAG1=VAL1,TAG2=VAL2,...,TAGn=VALn}

Where the table is defined within a pair of braces ({...}) where each element
of the table is specified by a TAG=VALUE syntax with a comma (,) separating
subsequent elements.

Note that no translation is done on the values of a table unless they are
escaped with a caret:
nM> res ten 10
nM> res t:mytab {A=ten}
nM> res mytab.a
3S: MYTAB.A = TEN
nM> res t:mytab {A=^ten}
nM> res mytab.a
L: MYTAB.A = 1

Invalid Syntax
------------
Since NeXtMidas 3.3.0 (via the Table flag CheckMalformed, which is off by
default), if you fail to follow the syntax above, you will get an exception
in the following circumstances:
nM> res tab {X=""Y=2} ! missing comma
nM> res t:tab {X=1,Y=2 ! missing '}' at end of String

Valid Tag Names
---------------
All tags in a table must be one or more characters in length and may contain
only numbers, upper-case letters and the underscore. This is basically the
same as identifiers in C/C++ except that it may begin with a number and is
case-insensitive. Tags may be specified on the command line or in a macro with
lower-case letters since they will be automatically converted to upper case by
the NeXtMidas shell.
Regular Expression for Tag Names: [0-9A-Z_]+

There is no upper limit to the length of a tag name but users are likely to
find that tag names longer than 16 characters are unwieldy to use. Note: Older
versions of NeXtMidas allowed tag names to contain dashes (-). This behavior
has been deprecated since it caused confusion as to whether a statement like
^MYTBL.FOO-9 represented a single table reference (^{MYTBL.FOO-9}) or a table
reference followed by a subtraction (^{MYTBL.FOO}-9).

Types in a Table
----------------
When a table is specified on the command line its type is automatically
assigned by the NeXtMidas shell. For example:

nM> res t:mytable {ONE=1.0,TWO=2.0,TEN=10.0}

Would create a table with three doubles in it. It is possible to specify the
type of the value by giving the appropriate type specifier before the tag name
(see Typecasting in a Macro for the list of valid type specifiers). For
example:

nM> res t:mytable {B:ONE=1.0,B:TWO=2.0,F:TEN=10.0}

Would create a table with two bytes and a float.

Inside a table, strings should be quoted, numbers should not be quoted, and
subtables should be within a pair of braces. For example:

nM> res t:mytable {NUMBER=1,STRING="My String",SUBTABLE={ONE=1,TWO=2}}

One exception is that single-word strings with no special characters will be
accepted even if they are not quoted (though this practice is discouraged).

Translation
-----------
When translating a table reference translation always goes to the end unless
the translation is done using the caret-brace (^{...}) syntax. Here are a few
examples:

1: nM> res mytable {A=1,B=2,C=3,D={A=10,B=20,C=30},X=9,Y=99,Z=999}
2: nM> res a d
3: nM> res b {a="a",b="b",c="c",D={A="x",B="y",C="z",D="z"}}
4: nM> res mytable.d.a
L: MYTABLE.D.A = 10
5: nM> res mytable.^a.a
ERROR: ...
6: nM> res mytable.^{a}.a
L: MYTABLE.D.A = 10
7: nM> res mytable.^{b.a}.d
D: MYTABLE.A.D = 1.0
8: nM> res mytable.^{b.^a.d}
ERROR: ...
9: nM> res mytable.^{b.^{a}.d}
L: MYTABLE.Z = 999
10: nM> res mytable {X=5,Y=2}{X=1,Z=3}
is the same as
nM> res mytable {X=1,Y=2,Z=3}

Line 5 causes an error because it tried to access mytable.^{a.a} but a is a
string ("d") not a table. Similarly line 8 attempts to access a.d. To avoid
situations such as this the caret-brace syntax is always recommended when
dealing with translation in the middle of a table name.

While the above examples may seem a bit contrived, many applications use the
values within one table as the keys used to access another table. It is not
uncommon to see the nesting of these two or three levels deep.

Altering a Table
----------------
Once a table has been defined the the values in the table can be easily
modified or new values added using the RESULT or SET commands. For example:

nM> res mytable.a 7.0

This will set the value A in my table to be the double value 7.0. To set it to
a byte use the following:

nM> res b:mytable.a 7.0

Note that before adding values to a subtable the subtable must first be
inserted:

nM> res mytable {ONE=1,TWO=2}
nM> res mytable.SUBTABLE.a 3
ERROR: Table Entry SUBTABLE not found for put of A
nM> res mytable.SUBTABLE { }
nM> res mytable.SUBTABLE.a 3
nM> res mytable.SUBTABLE.a
L: MYTABLE.SUBTABLE.A = 3

Values in a table can be removed using the REMOVE command. For example:

nM> rem mytable.a

This will remove A from MYTABLE.

Table Modes and Sorting
-----------------------
As listed above, table can be set to use one of several internal modes. For
comparison purposes, three are described below:

Mode Underlying Storage Mechanism Nominal Order of Operation Sortable
Get Add Remove
---- ---------------------------- --- --- ------ --------
KV NeXtMidas KeyVector O(n) O(1) O(n) YES
SLHM* Java Synchronized HashMap O(1) O(1) O(1) NO
FB** Byte Buffer n/a n/a n/a n/a

* SLHM is the default mode.
** FB mode is used only for serialization of the table. A table in FB mode
will automatically switch to KV mode on first use.

The tradeoff between KV mode and SLHM mode is sortability versus access time.
The TABLE command provides functions that can change the mode of a table and
sort a table.

View the Table javadocs for a complete comparison of modes.

CLONE -vs- REFERENCE
--------------------
A table can be cloned (copied) such that the copy is stored in a separate
memory location from the original. A change to the clone will not affect the
original. When two tables need to refer to the same result a reference
can be created such that a change using either reference variable changes
the value seen by both. A results table example illustrates these
relationships:

nM> set origtable {x=1,y=2} (create a table and assign some values)
nM> set table1 origtable (create a reference to the original table)
nM> set table2 ^origtable (clone/copy the original table)
nM> set table1.x 9 (modify x value in table 1)
nM> res table1.x (x=9 because table 1 value was modified)
nM> res table2.x (x=1 because the copy remains unchanged)
nM> res origtable.x (x=9 table1 references same memory as origtable)

Switches:
/FLAGS= - Table flags mask/settings to use for LOAD/SAVE functions.
Use LOAD/T function along with this switch to ensure that the output
table and all it's sub-tables use specified flags. The SAVE function
only uses the DisableEscapeSequences and SerializeTimeAsTColon flags.
Mask Options are:
Add,AddRoot,AutoReadPrev,AutoWritePrev,
CaseSensitive,DisableEscapeSequences,ExceptionOnNull,
Force,OnlyDotIntoSubTables,ReadOnly,CheckMalformed,
SerializeTimeAsTColon
(Since 3.1.2 for LOAD and 3.5.3 for SAVE/SerializeTimeAsTColon)

See Also: ENVIRONMENT, nxm.sys.lib.Table

</pre>

[[Category:NXM_Explain]]

TABLE - Revision history

ConvertBot: Perform various table related functions