1. CONNECT now supports expression value
2. Updated the WSE spec test scripts to use the new feature

Pending: Unit tests. Meanwhile, I need some feedback on the design changes I made.

x-kaazing-handshake protocol as defined in HTTPX specification must include an empty WebSocket binary frame at the end of extended handshake response. The empty WebSocket binary frame from the server looks as shown below:

0x82 0x00

Since the server may break the extended handshake response into multiple WebSocket frames, an empty WebSocket binary frame is used to indicate the end of the extended handshake response.

This means we have to update the specification as well as the existing spec scripts to include the empty WebSocket binary frame.

Note: this requires a backwards incompatible change in the K3PO grammar. Note: upgrade script provided in tools/upgrade.sed

Added an extensible TypeSystem used by the parser to support custom language elements not present in the core, such as http:method, http:version http:header, and http:status.

accept "http://localhost:8080/path?query"

accepted
connected

read http:method "GET"
read http:version "HTTP/1.1"
read http:header "Host" "localhost:8080"
read http:header "Upgrade" /(?i:websocket)/
read http:header "Connection" /(?i:Upgrade)/
read http:header "Sec-WebSocket-Key" /(?<key>[a-zA-Z0-9+\/=]{24})/
read http:header "Sec-WebSocket-Version" "13"

write http:status "101" "Switching Protocols"
write http:version "HTTP/1.1"
write http:header "Upgrade" "websocket"
write http:header "Connection" "Upgrade"
write http:header "Sec-WebSocket-Accept" ${ws:handshakeHash(key)}
write flush

# connection established

Added an extensible BehaviorSystem used by driver to support runtime enforcement of custom language elements.

New transport schemes can now be implemented outside the K3PO project and be discovered via ServiceLoader by placing the artifact on the <dependency> classpath of the k3po-maven-plugin.

Allows

    @Specification("test.barrier.passing.from.test.framework")
    @Test
    public void testPassingBarriers() throws Exception {
        k3po.awaitBarrier("NOTIFYING_BARRIER");
        k3po.notifyBarrier("AWAITING_BARRIER");
        k3po.finish();
    }

Adds NOTIFY and AWAIT commands to the control protocol, and the NOTIFIED, each of which has a header barrier:barrierName.

Additionally, the list of barriers is returned on the PREPARED event, by adding multiple barrier:barrierNames headers. This was intended to be backwards compatible as control protocols should ignore unrecognized headers. However, the control used by junit was not following that and required a fix. Based on that I recommend bumping the patch version on the next release.

Support syntax for scripting interactions with Agrona ring buffer and broadcast buffer shared memory communication data structures.

For example, an implementation acting as an Agrona client can use the following script to emulate server behavior.

property newLayout ${agronaIT:layoutInit("target/agrona-itest", 1024, 1024)}

accept agrona://stream/bidirectional
       option reader ${agrona:manyToOneReader(newLayout.ring)}
       option writer ${agrona:broadcastTransmitter(newLayout.broadcast)}

accepted
connected

read [0x00 0x00 0x00 0x01]
read ([0..32] :server32)

write [0x00 0x00 0x00 0x02]
write ${server32}

close
closed

...and, an implementation acting as an Agrona server can use the following script to emulate client behavior.

property layout ${agronaIT:layout("target/agrona-itest", 1024, 1024)}
property client32 ${agronaIT:randomBytes(32)}

connect agrona://stream/bidirectional
        option reader ${agrona:broadcastReceiver(layout.broadcast)}
        option writer ${agrona:manyToOneWriter(layout.ring)}

connected

write [0x00 0x00 0x00 0x01]
write ${client32}
write flush

read [0x00 0x00 0x00 0x02]
read ${client32}

close
closed

Agrona data structures require the first 4 bytes of each message represent a non-negative integer, and any successive writes combine to make up the message payload. When the writes are flushed, the message is delivered to the shared memory data structure.

When K3po driver receives an abort it asks for a hard stop of all pipelines. Any handler such has read handlers that have not processed their ChannelFuture (i.e. it is in a incomplete state) throws a ScriptProgress exception. I have debugged a few tests today where the bytes to a readHandler had arrived on the wire but were still being processed, causing an abort to fail the test when it shouldn't of. These tests where running with small junit timeouts (1 to 1.5 secs) and where sending either large number of bytes (65535 for example) or having numerous read handlers. The easy work around to this is to set the timeout to be larger. But it would be optimal if k3po allowed all buffered data on the upstream to be flushed before checking for script progress.

There are many different ways to send upstream data without violating the WSE specification. For example, if a client application creates a WebSocket connection, then sends data twice, then closes the WebSocket, this may legitimately be expressed as everything from one to three distinct upstream HTTP requests, and indeed what happens in practice will just depend on timing, for example, when using the JavaScript client it all depends when the upstream XMLHttpRequest enters ready state 4. The two sends will be enqueued then the CLOSE will be enqueued, and all will be sent in a single upstream request if ready state 4 is reached after they have all been put on the queue. But there will be multiple upstream requests if ready state 4 is reached before all the items have been placed on the queue.

This means that in practice it will be difficult for client tests (and WsebConnector tests) to use those response scripts which involve upstream data, for example org/kaazing/specification/wse/data/binary/echo.payload.length.127/response.rpt

There are several test scenarios where client needs to close the underlying tcp connection after websocket upgrade without doing a websocket close handshake. This is mandated by RFC 6455 for cases like invalid websocket handshake response or unrecognized extensions in websocket handshake response. I have not been able to find a way to make scripts work in this case. Client "close closed" and server "closed" does not work, test fails with actual having no "close closed" or "closed". Test case (added to HttpIT.java in driver project) is:

    @Test
    @TestSpecification({
        "http.accept.websocket.handshake.then.client.hard.close",
        "http.connect.websocket.handshake.then.client.hard.close" })
    public void shouldAcceptWebsocketHandshakeThenClientHardClose() throws Exception {
        k3po.finish();
    }

with scripts:

http.accept.websocket.handshake.then.client.hard.close.rpt | uploaded via ZenHub

http.connect.websocket.handshake.then.client.hard.close.rpt | uploaded via ZenHub

NOT ready for merge or review.

@jfallows @cmebarrow Seeking feedback on grammar changes. So review the following files:

1. Robot.g4
2. file.connect.write.rpt

Forcing a premature tcp close from an http stream is required to test some of the requirements of RFC-7230 and RFC-7231 (http). Authoring these tests in straight tcp semantics makes them brittle, as different implementations may order them in different ways.

Update the sent message to start with a letter other than ones of valid HTTP methods to bypass gateway validation. In a real scenario, the request would time-out.

Bumps jackson-databind from 2.9.10.6 to 2.12.7.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps commons-io from 2.1 to 2.7.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

A write close when using a transport for tls will result in a TLS completion, but not a half close on the underlying transport.

For example, a test as shown below will not result in a hal close for the nukleus transport

connect await ROUTED_SERVER
        "tls://localhost:9090"
  option tls:transport "nukleus://tls/streams/source"
  option tls:trustStoreFile "src/test/democa/cacerts"
  option tls:trustStorePassword "generated"
  option nukleus:route ${newServerAcceptRef}
  option nukleus:transmission "duplex"
connected

write close

This is a proposal to allow the client and server to agree to enhance behavior for the x-kaazing-idle-timeout extension. This PR is not yet complete, it just contains the proposed changes to the extension specification for now.

Currently it is possible for one JUnit test method to influence the behavior of the following one because K3poRule does not wait for the robot process to signal it has completed (DISPOSED message) at the end of . Instead, disposal is awaited when the next test method executes. As an example of the issue see the comment on PR https://github.com/reaktivity/nukleus-http-cache.java/pull/46.

The evaluate method in SpecificationStatement would need to be changed to fix this, probably by waiting to receive the DISPOSED message from the k3po process.

WORKAROUND

Make the k3po rule the outermost rule. That will guarantee the k3po process has disposed from the previous execution by the time any inner rules execute.

Currently there is no way to verify (assert) that a connection fails. This ticket is an enhancement request to support syntax connect aborted in the same spirit as write aborted, as in the following example:

connect "nukleus://server/streams/client"
        option nukleus:route ${newRouteRef}
        option nukleus:transmission "duplex"
        option nukleus:authorization 0x0002_000000000000L
        option nukleus:window 8192
connect aborted