Problem is that init file is overwriting some parameters which we expect to set them in configuration file. e.g. LevelDB home folder. After setting leveldb.home = /some/where/else/data in configuration file, our Warp10 instance failed. After investigation we found out that this parameter is overwritten in init file. But since I'm not sure what is the process and order of running LevelDB, I couldn't decide what should be the general solution.

In our case we introduced another parameter in init file to have the same value as configuration there.

If only by having these kind of parameters in configuration file, Warp10 platform can work properly, then they should be not set in init file again, when the configuration file exists.

We have Warp10 version 2.5 deployed directories (sharded and replicated). They load around 130 millions of series. I noticed that we do get trouble to restart them while series deletion is occuring. In the logs, we got: pool-301-thread-29 ERROR store.Directory - java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:63) at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) at io.warp10.continuum.store.Directory$DirectoryConsumer.run(Directory.java:1459) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

And in terms of Metrics it's related to a lot of "warp.directory.kafka.shutdowns".

The rebooted directory do not consume Kafka.

Our workaround at the moment is to forbid deletes while rebooting directories.

Let me know if this issue was already fix in new Warp10 version

With a bucketized dataset of:

• a list of 1600 GTS named A with only one point inside, having two labels ("host" and "disk").
• another list of GTS, same conditions, but named B. Every combination of host + disk in labels makes the gts unique (in A and B).

I want to add each unique combination of gts A to B: [ $a $b [ 'host' 'disk' ] op.add ] APPLY

It works, but it's very slow on a recent warp: On a old warp 1.2.22-88, it took less than 1000ms. On a recent warp 2.0.2-162, the APPLY alone takes more than 4500ms every time.

I tested to divide each combination by myself like this:

$a [ 'host' 'disk' ] PARTITION 'pa' STORE
$b [ 'host' 'disk' ] PARTITION
[] SWAP
<%
   'series' STORE
   'labels' STORE
   [ 
      $series
      $pa $labels GET
      [] 
      op.add 
   ] APPLY
   APPEND
%>
FOREACH

... and it took ~100ms.

I am a bit worried here about the low performance with the APPLY function. I am searching about what could be responsible for it. What would be the best way to perform this operation?

PS: All time measurements has been done in the WarpScript with NOW time subtractions around the APPLY.

Fix the fate of boolean GTS operation in Warp 10 2.0 : allow not, and, or, xor on boolean GTS. Tried to optimize NOT with flip operation, but I didn't test performances.

I choose to implement and or xor bitwise operators, not condition ones... I can change.

recently, we discover the need of being able to delegate a subpart of the authorisation of a token, like we do in macaroon.

I'm not confident about W10Token evolution without breaking things, and for a various reason get proper macaroons can be a good thing. But macaroons are way longer than w10tk. That's why I think a warp10 cluster will profit of multiple authentication methods.

In the warp script we will change nothing, get token as usual but prefix with an alternative when it's not a warp10 token (ie "macaroon:", "userpassword:"...). This way we can provide a various way to connect to the w10 cluster.

What do you think?

Do we want to be able to build auth extensions also outside of the w10 codebase? With a dynamic classloader plugin system? Or built in in the core?

Where do I have to plug this?

Here's an example:

NEWGTS "GTS1" RENAME 
{ 'label0' '42' 'env' 'prod' } RELABEL

NEWGTS "GTS2" RENAME 
{ 'label0' '43' } RELABEL

2 ->LIST

[ SWAP [] { 'env' 'prod' } filter.bylabels ] FILTER

At the end of the script, I have the right result, but in this case, having the the filtered elements during FILTER could be interesting.

Is it possible to have (for example) a new type of FILTER framework , which will push back 2 lists in the stack:

• the remaining elements after filtering
• the filtered elements

Currently the URLFETCH fonction only support GET method. The signatures are:

url:STRING headers:MAP<STRING> URLFETCH
urls:LIST<STRING> headers:MAP<STRING> URLFETCH

with headers being optional.

To support additional methods, I suggest:

request:MAP<STRING> headers:MAP<STRING> URLFETCH
requests:LIST<MAP<STRING>> headers:MAP<STRING> URLFETCH

where request shall have the fields 'url', 'method' and optionally 'message', and headers are still being optional.

Is that signature good ?

To know if a serie is still active or not, you can fetch the last available datapoint and check its date matching your criteria.

If data auto eviction is enabled, then you won't even be able to lookup this last datapoint.

It would be very useful to have a last_seen hint per GTS that give the timestamp (second is enough) corresponding to the last update on a GTS.

There is two possibilities to implements this :

• Ingress
• Store

Store component is CPU bound and uses less memory than Ingress, since it maintains the meta caching, so in case of distributed deployment, it would be better implement this on the Store side.

It would maintain a structure, like a concurrent hash map, setting the last timestamp as a value for a key corresponding to the TS ID.

Data could be sampled, and produced in best effort to either directory (standalone) or a dedicated Kafka topic.

The struct IndexSpec could have the last_seen field so that we can at the end perform a LASTSEEN with an optional parameter :

[ 'RTOKEN' 'class_pattern' { labels } ] LASTSEEN

result would be :

[{
		"c": "class",
		"l": {
			"label0": "value0",
			"label1": "value1"
		},
		"a": {
			"attr0": "value0"
		},
		"v": [
			[0, last_seen],
		]
	}
]

This way, we can leverage all frameworks to manipulate the result (FILTER, ...) and easily get series older than n days.

This would be very helpful to manage the Directory.

Many errors have the same return code :

• Exceeded MADS
• Invalid Token
• Exec Limit raised
• Exceeded DDP

It would be nice to have a fine grained error status codes so that we can better catch the source of the issue.

This new function allow to reschedule a runner from the runner itself.

To test, generate a token with this capability:

  'attributes' {
    '.cap:runner.reschedule.min.period' '5000'
  }

Use it with CAPADD inside your runner. From the runner, you can schedule the next iteration to "start of the script" + the period you want: 10 s RUNNERNEXT

It will fail if the requested period is less than the one defined in capability.

The function have no effect on a standalone with remote runner execution, or on a distributed instance.

Hi, When starting Warp 10, I get the following error:

  ___       __                           ____________
  __ |     / /_____ _______________      __<  /_  __ \
  __ | /| / /_  __ `/_  ___/__  __ \     __  /_  / / /
  __ |/ |/ / / /_/ /_  /   __  /_/ /     _  / / /_/ /
  ____/|__/  \__,_/ /_/    _  .___/      /_/  \____/
                           /_/

  Revision 2.9.0

2022-01-19 18:54:11.714:INFO:iwsjoejs.AbstractConnector:Started [email protected]:37147
2022-01-19T18:54:11,716 main INFO  store.Constants - ########[ Initialized with 1000 time units per millisecond ]###
#####
2022-01-19T18:54:11,725 main WARN  script.WarpFleetMacroRepository - No validator macro, default macro will reject a
ll URLs.
REPORT secret not set, using '1c77f05b-8aee-4328-a503-6d2363fefcf0'.
Exception in thread "main" java.lang.RuntimeException: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO e
rror: /srv/warp10/leveldb/14939849.sst: Trop de fichiers ouverts
        at org.fusesource.leveldbjni.internal.JniDBIterator.seek(JniDBIterator.java:68)
        at io.warp10.standalone.WarpDB$WarpIterator.seek(WarpDB.java:133)
        at io.warp10.standalone.StandaloneDirectoryClient.<init>(StandaloneDirectoryClient.java:193)
        at io.warp10.standalone.Warp.main(Warp.java:379)
Caused by: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /srv/warp10/leveldb/14939849.sst: Trop
 de fichiers ouverts
        at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:200)
        at org.fusesource.leveldbjni.internal.NativeIterator.checkStatus(NativeIterator.java:121)
        at org.fusesource.leveldbjni.internal.NativeIterator.seek(NativeIterator.java:151)
        at org.fusesource.leveldbjni.internal.NativeIterator.seek(NativeIterator.java:145)
        at org.fusesource.leveldbjni.internal.NativeIterator.seek(NativeIterator.java:138)
        at org.fusesource.leveldbjni.internal.JniDBIterator.seek(JniDBIterator.java:63)
        ... 3 more
2022-01-19T18:54:52,999 main INFO  store.Constants - ########[ Initialized with 1000 time units per millisecond ]########
2022-01-19T18:54:53,009 main WARN  script.WarpFleetMacroRepository - No validator macro, default macro will reject all URLs.
2022-01-19 18:54:54.427:INFO:iwsjoejs.Server:jetty-8.y.z-SNAPSHOT

It says that I have too many files open. However I set my leveldb.maxopenfiles (nothing, 1k, 100k…), Warp 10 opens 524287 files and then dies (found in a pretty dirty way: while true; do for pid in [0-9]*; do ls /proc/$pid/fd/ | wc -l; done | grep ......; done). My soft ulimit is 1024 and my hard ulimit is 1048576 (twice the number of files after which Warp 10 crashes). What can I do to properly start Warp 10?

Thanks in advance

When pushing data to external services (using HTTP or WEBCALL), we might need to respect a minimum period between requests. Current solution to do so is an infinite loop bounded by TIMEBOX, quite CPU intensive.

Adds sleep.maxtime capability.

In case of a concurrent execution of MAP, sorting the outputTicks will lead to a concurrent modification exception.

So, I just check if the input is correctly sorted instead of always calling Collections.sort()

As specified in the documentation: "Secret configured via token.secret. This parameter should not be specified when calling TOKENGEN from Worf." But the warp10-standalone.sh script in version 2.10.1 is still using this possibility, So as a result, if we define a token.secret in the configuration file (which is our case), the bootstrap fails with a stacktrace. (and the initialtokens file is not generated) This is probably not an issue but the code could be easily updated (and the default warp10-tokengen.mc2 with the {{secret}}template variable ...

cheers