By examining a simple query through flame graph (see issue), one interesting point is that I find there are too many calls using reflection, which is not performant, although the total overhead is less than 1%, I still spend some time trying to improve. Most invocations are rooted down to ReflectiveVisitDispatcher, the current implementation creates new instance whenever needed, and looking up methods by reflection per instance, I think by caching methods globally, as the methods count is countable to 68 possible places, different ReflectiveVisitDispatcher in different thread is able to reuse. The fundamental change will benefit other likewise invocations as well.

see:
https://issues.apache.org/jira/browse/CALCITE-3737 https://issues.apache.org/jira/browse/CALCITE-3780

Some highlights on this PR:

1. support HOP as a table function.
2. support SESSION as a table function.
3. rename "table-valued function" to "table function" to improve naming.

High level speaking, this PR adds the following support:

SELECT * FROM TABLE(Tumble( TABLE ORDERS , DESCRIPTOR(ROWTIME) , INTERVAL '1' MINUTES))

This PR adds TUMBLE as table value function and also adds stream.iq along with Enumerable implementation. This is a big PR that actually is also related to the following JIRAs:

https://jira.apache.org/jira/browse/CALCITE-3340 https://jira.apache.org/jira/browse/CALCITE-3501 https://jira.apache.org/jira/browse/CALCITE-3499 https://jira.apache.org/jira/browse/CALCITE-3418 https://jira.apache.org/jira/browse/CALCITE-3339

Note that DESCRIPTOR support is also included in this PR.

Add an adapter to expose Kafka topics as STREAM tables.

KafkaTableFactory is used here so end users need to specify table-topic mapping one-by-one.

JIRA: https://issues.apache.org/jira/browse/CALCITE-2913

CC: @danny0405

JSON_LENGTH(**json_doc**[, *path*])

Returns the length of a JSON document, or, if a path argument is given, the length of the value within the document identified by the path. Returns NULL if any argument is NULL or the path argument does not identify a value in the document. An error occurs if the json_doc argument is not a valid JSON document or the path argument is not a valid path expression or contains a {} or }}{{* wildcard.

The length of a document is determined as follows:

• The length of a scalar is 1.

• The length of an array is the number of array elements.

• The length of an object is the number of object members.

• The length does not count the length of nested arrays or objects.

Example Sql:

SELECT JSON_LENGTH(v) AS c1
,JSON_LENGTH(v, 'lax $.a') AS c2
,JSON_LENGTH(v, 'strict $.a[0]') AS c3
,JSON_LENGTH(v, 'strict $.a[1]') AS c4
FROM (VALUES ('{"a": [10, true]}')) AS t(v)
LIMIT 10;

Result:

| c1 | c2 | c3 | c4 | | ---- | ---- | ---- | ---- | | 1 | 2 | 1 | 1 |

Fix ISSUE #2601

mysql

mysql> SELECT REVERSE('hello');
+------------------+
| REVERSE('hello') |
+------------------+
| olleh            |
+------------------+
1 row in set (0.00 sec)

sql server

DECLARE @str NVARCHAR(100) 

SET @str='ABCD'

SELECT REVERSE(@str)

pg

testdb=# SELECT REVERSE('abcd');
 reverse
---------
 dcba
(1 row)

oracle

SQL> select reverse('12345') from dual;
REVER

54321

doc:

https://dev.mysql.com/doc/refman/5.7/en/string-functions.html#function_reverse

Usually O_INPUTS are only applied for groups with physical convention. But when enabling AbstractConverter, the input of AbstractConverter might be a group with NONE convention. In that case, no need to apply O_INPUTS. Otherwise, it might throw an exception due to impossible transformation( physical convention -> none convention).

Other change:

• some cosmetic fix-ups
• print the upper bound of the RelSubSet

Replace the use of reflection/dynamic proxies with the AnnotationProcessor provided by Immutables

NOTE: This is an initial patch that only changes one ImmutableBean to Immutables to show what the changes look like

This commit provides an implementation for:

• Statement.addBatch(String)
• PreparedStatement.addBatch()
• PreparedStatement.executeBatch()

The implementation is fairly straightforward except for the addition of a new server interface: ProtobufMeta. This is a new interface which the Meta implementation can choose to also implement to provide a "native" implementation on top of Protobuf objects instead of the Avatica POJOs.

During the investigations Avatica performance pre-1.7.0, it was found that converting protobufs to POJOs was a very hot code path. This short-circuit helps us avoid extra objects on the heap and computation to create them in what should be a very hot code path for write-workloads.

Observed behavior is described here: https://issues.apache.org/jira/browse/CALCITE-1386

Below is the description of this patch:

• Modify MethodImplementor to cast return value to desired return type when necessary
• Change ItemImplementor to use NullPolicy.ANY since ITEM can still return null even though both operands are not null
• Fix Types.castIfNecessary to handle RecordType as an exceptional case (toClass() doesn't handle RecordType and throws Exception)
• Change Csv tests to test its behavior

Please let me know if I'm encouraged to do additional works. Thanks in advance!

This PR upgrades embedded Elasticsearch version from 7.0.1 to 7.15.2.

Description:

• New dependencies: org.codelibs.elasticsearch.module:scripting-painless-spi, as module "org.elasticsearch.painless.spi" is removed after ES 7.15.0 in lang-painless
• Third maven repo: org.codelibs.elasticsearch.module:lang-painless is no longer maintained after ES 7.10.2, which is migrated to https://maven.codelibs.org/
• RestClient Upgrading: the low level rest client in ES has good compatibilities(just http request) among 7.X, which is also upgraded to 7.15.2

Self-verification: I've run some tests locally to make sure new feature can be applied(not added in unit test).

• Supported: new features like RareTerms、minimun_interval in auto_date_histogram can be successfully applied, which are not supported in ES 7.0.1
• Not supported: top_metrics、multi_terms、rate and other features in x-pack are not supported currently. Those features can be registered when AnalyticsPlugin is loaded to build ES Node(test environment), however, dependency org.elasticsearch.plugin:x-pack-analytics cannot be reached through maven central at present

Add LENGTH() as a library function as an alias for the standard CHAR_LENGTH(). Some dependencies for soon-to-be deprecated standard functions refactored to avoid null pointer exceptions as a result of circular dependencies between the standard and library operators. This decision was made with the help of @mkou .

Add support for BigQuery's DATE_SUB, TIME_SUB, and TIMESTAMP_SUB functions. Create MINUS_DATE2 operator to handle subtracting an interval from a timestamp, time, or date expression. This differs from the standard MINUS_DATE operator which takes 3 arguments which is designed to subtract one time expression from another. Add week and quarter as valid time units for intervals in the parser.