Feeding huge data (4G) through serving layer gives crash of speed layer:
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: acks = 1
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: batch.size = 16384
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: bootstrap.servers = [127.0.0.1:9092]
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: buffer.memory = 33554432
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: client.id =
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: compression.type = gzip
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: connections.max.idle.ms = 540000
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: enable.idempotence = false
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: interceptor.classes = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: key.serializer = class org.apache.kafka.common.serialization.StringSerializer
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: linger.ms = 1000
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: max.block.ms = 60000
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: max.in.flight.requests.per.connection = 5
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: max.request.size = 67108864
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: metadata.max.age.ms = 300000
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: metric.reporters = []
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: metrics.num.samples = 2
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: metrics.recording.level = INFO
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: metrics.sample.window.ms = 30000
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: receive.buffer.bytes = 32768
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: reconnect.backoff.max.ms = 1000
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: reconnect.backoff.ms = 50
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: request.timeout.ms = 30000
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: retries = 0
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: retry.backoff.ms = 100
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.jaas.config = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.kerberos.kinit.cmd = /usr/bin/kinit
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.kerberos.min.time.before.relogin = 60000
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.kerberos.service.name = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.kerberos.ticket.renew.jitter = 0.05
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.kerberos.ticket.renew.window.factor = 0.8
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: sasl.mechanism = GSSAPI
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: security.protocol = PLAINTEXT
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: send.buffer.bytes = 131072
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.cipher.suites = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.endpoint.identification.algorithm = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.key.password = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.keymanager.algorithm = SunX509
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.keystore.location = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.keystore.password = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.keystore.type = JKS
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.protocol = TLS
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.provider = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.secure.random.implementation = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.trustmanager.algorithm = PKIX
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.truststore.location = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.truststore.password = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: ssl.truststore.type = JKS
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: transaction.timeout.ms = 60000
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: transactional.id = null
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: value.serializer = class org.apache.kafka.common.serialization.StringSerializer
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:42:59 INFO AppInfoParser:83 - Kafka version : 0.11.0.1
Sep 03 09:42:59 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:42:59 INFO AppInfoParser:84 - Kafka commitId : c2a0d5f9b1f45bf5
Sep 03 09:43:03 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:03 INFO ContextCleaner:54 - Cleaned shuffle 9
Sep 03 09:43:03 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:03 INFO ContextCleaner:54 - Cleaned shuffle 8
Sep 03 09:43:03 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:03 INFO KafkaProducer:1017 - Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO JobScheduler:54 - Finished job streaming job 1535956800000 ms.0 from job set of time 1535956800000 ms
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO JobScheduler:54 - Starting job streaming job 1535956800000 ms.1 from job set of time 1535956800000 ms
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2477630, OryxInput,1=2478826, OryxInput,2=2477739, OryxInput,3=2478777}
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@401052bb
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZkEventThread:65 - Starting ZkClient event thread.
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZkClient:936 - Waiting for keeper state SyncConnected
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a790136, negotiated timeout = 30000
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZkClient:713 - zookeeper state changed (SyncConnected)
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZkEventThread:83 - Terminate ZkClient event thread.
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ZooKeeper:684 - Session: 0x10160d98a790136 closed
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ClientCnxn:512 - EventThread shut down
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO JobScheduler:54 - Finished job streaming job 1535956800000 ms.1 from job set of time 1535956800000 ms
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO JobScheduler:54 - Total delay: 184.087 s for time 1535956800000 ms (execution: 161.298 s)
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO MapPartitionsRDD:54 - Removing RDD 293 from persistence list
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO BlockManager:54 - Removing RDD 293
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO KafkaRDD:54 - Removing RDD 292 from persistence list
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO BlockManager:54 - Removing RDD 292
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO ReceivedBlockTracker:54 - Deleting batches:
Sep 03 09:43:04 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:43:04 INFO InputInfoTracker:54 - remove old batch metadata: 1535956200000 ms
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Added jobs for time 1535957100000 ms
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Starting job streaming job 1535957100000 ms.0 from job set of time 1535957100000 ms
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Got job 264 (isEmpty at SpeedLayerUpdate.java:53) with 1 output partitions
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Final stage: ResultStage 274 (isEmpty at SpeedLayerUpdate.java:53)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Parents of final stage: List()
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Missing parents: List()
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Submitting ResultStage 274 (MapPartitionsRDD[321] at mapToPair at SpeedLayer.java:97), which has no missing parents
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO MemoryStore:54 - Block broadcast_274 stored as values in memory (estimated size 3.8 KB, free 912.3 MB)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO MemoryStore:54 - Block broadcast_274_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManagerInfo:54 - Added broadcast_274_piece0 in memory on htm-psycho-401.zxz.su:45401 (size: 2.4 KB, free: 912.3 MB)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO SparkContext:54 - Created broadcast 274 from broadcast at DAGScheduler.scala:1039
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 274 (MapPartitionsRDD[321] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(0))
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO YarnScheduler:54 - Adding task set 274.0 with 1 tasks
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Starting task 0.0 in stage 274.0 (TID 586, htm-psycho-401.zxz.su, executor 1, partition 0, PROCESS_LOCAL, 7746 bytes)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManagerInfo:54 - Added broadcast_274_piece0 in memory on htm-psycho-401.zxz.su:45269 (size: 2.4 KB, free: 912.3 MB)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Finished task 0.0 in stage 274.0 (TID 586) in 38 ms on htm-psycho-401.zxz.su (executor 1) (1/1)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO YarnScheduler:54 - Removed TaskSet 274.0, whose tasks have all completed, from pool
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - ResultStage 274 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.049 s
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Job 264 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.052823 s
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Got job 265 (isEmpty at SpeedLayerUpdate.java:53) with 3 output partitions
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Final stage: ResultStage 275 (isEmpty at SpeedLayerUpdate.java:53)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Parents of final stage: List()
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Missing parents: List()
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Submitting ResultStage 275 (MapPartitionsRDD[321] at mapToPair at SpeedLayer.java:97), which has no missing parents
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO MemoryStore:54 - Block broadcast_275 stored as values in memory (estimated size 3.8 KB, free 912.3 MB)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO MemoryStore:54 - Block broadcast_275_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManagerInfo:54 - Added broadcast_275_piece0 in memory on htm-psycho-401.zxz.su:45401 (size: 2.4 KB, free: 912.3 MB)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO SparkContext:54 - Created broadcast 275 from broadcast at DAGScheduler.scala:1039
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Submitting 3 missing tasks from ResultStage 275 (MapPartitionsRDD[321] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(1, 2, 3))
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO YarnScheduler:54 - Adding task set 275.0 with 3 tasks
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Starting task 0.0 in stage 275.0 (TID 587, htm-psycho-401.zxz.su, executor 1, partition 1, PROCESS_LOCAL, 7746 bytes)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManagerInfo:54 - Added broadcast_275_piece0 in memory on htm-psycho-401.zxz.su:45269 (size: 2.4 KB, free: 912.3 MB)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Starting task 1.0 in stage 275.0 (TID 588, htm-psycho-401.zxz.su, executor 1, partition 2, PROCESS_LOCAL, 7746 bytes)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Finished task 0.0 in stage 275.0 (TID 587) in 17 ms on htm-psycho-401.zxz.su (executor 1) (1/3)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Starting task 2.0 in stage 275.0 (TID 589, htm-psycho-401.zxz.su, executor 1, partition 3, PROCESS_LOCAL, 7746 bytes)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Finished task 1.0 in stage 275.0 (TID 588) in 8 ms on htm-psycho-401.zxz.su (executor 1) (2/3)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO TaskSetManager:54 - Finished task 2.0 in stage 275.0 (TID 589) in 7 ms on htm-psycho-401.zxz.su (executor 1) (3/3)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO YarnScheduler:54 - Removed TaskSet 275.0, whose tasks have all completed, from pool
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - ResultStage 275 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.047 s
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO DAGScheduler:54 - Job 265 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.050518 s
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Finished job streaming job 1535957100000 ms.0 from job set of time 1535957100000 ms
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Starting job streaming job 1535957100000 ms.1 from job set of time 1535957100000 ms
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2477630, OryxInput,1=2478826, OryxInput,2=2477739, OryxInput,3=2478777}
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@382c4378
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZkEventThread:65 - Starting ZkClient event thread.
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZkClient:936 - Waiting for keeper state SyncConnected
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a790137, negotiated timeout = 30000
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZkClient:713 - zookeeper state changed (SyncConnected)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZkEventThread:83 - Terminate ZkClient event thread.
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ZooKeeper:684 - Session: 0x10160d98a790137 closed
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ClientCnxn:512 - EventThread shut down
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Finished job streaming job 1535957100000 ms.1 from job set of time 1535957100000 ms
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO JobScheduler:54 - Total delay: 0.695 s for time 1535957100000 ms (execution: 0.189 s)
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO MapPartitionsRDD:54 - Removing RDD 307 from persistence list
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO KafkaRDD:54 - Removing RDD 306 from persistence list
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManager:54 - Removing RDD 307
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO ReceivedBlockTracker:54 - Deleting batches:
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO InputInfoTracker:54 - remove old batch metadata: 1535956500000 ms
Sep 03 09:45:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:45:00 INFO BlockManager:54 - Removing RDD 306
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO RecurringTimer:54 - Stopped timer for JobGenerator after time 1535957400000
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Added jobs for time 1535957400000 ms
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Starting job streaming job 1535957400000 ms.0 from job set of time 1535957400000 ms
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Got job 266 (isEmpty at SpeedLayerUpdate.java:53) with 1 output partitions
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Final stage: ResultStage 276 (isEmpty at SpeedLayerUpdate.java:53)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Parents of final stage: List()
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Missing parents: List()
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Submitting ResultStage 276 (MapPartitionsRDD[323] at mapToPair at SpeedLayer.java:97), which has no missing parents
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MemoryStore:54 - Block broadcast_276 stored as values in memory (estimated size 3.8 KB, free 912.3 MB)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobGenerator:54 - Stopped generation timer
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobGenerator:54 - Waiting for jobs to be processed and checkpoints to be written
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 WARN JobGenerator:66 - Timed out while stopping the job generator (timeout = 300000)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobGenerator:54 - Waited for jobs to be processed and checkpoints to be written
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobGenerator:54 - Stopped JobGenerator
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MemoryStore:54 - Block broadcast_276_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManagerInfo:54 - Added broadcast_276_piece0 in memory on htm-psycho-401.zxz.su:45401 (size: 2.4 KB, free: 912.3 MB)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkContext:54 - Created broadcast 276 from broadcast at DAGScheduler.scala:1039
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Submitting 1 missing tasks from ResultStage 276 (MapPartitionsRDD[323] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(0))
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnScheduler:54 - Adding task set 276.0 with 1 tasks
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Starting task 0.0 in stage 276.0 (TID 590, htm-psycho-401.zxz.su, executor 1, partition 0, PROCESS_LOCAL, 7746 bytes)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManagerInfo:54 - Added broadcast_276_piece0 in memory on htm-psycho-401.zxz.su:45269 (size: 2.4 KB, free: 912.3 MB)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Finished task 0.0 in stage 276.0 (TID 590) in 26 ms on htm-psycho-401.zxz.su (executor 1) (1/1)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnScheduler:54 - Removed TaskSet 276.0, whose tasks have all completed, from pool
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - ResultStage 276 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.036 s
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Job 266 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.040172 s
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkContext:54 - Starting job: isEmpty at SpeedLayerUpdate.java:53
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Got job 267 (isEmpty at SpeedLayerUpdate.java:53) with 3 output partitions
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Final stage: ResultStage 277 (isEmpty at SpeedLayerUpdate.java:53)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Parents of final stage: List()
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Missing parents: List()
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Submitting ResultStage 277 (MapPartitionsRDD[323] at mapToPair at SpeedLayer.java:97), which has no missing parents
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MemoryStore:54 - Block broadcast_277 stored as values in memory (estimated size 3.8 KB, free 912.3 MB)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MemoryStore:54 - Block broadcast_277_piece0 stored as bytes in memory (estimated size 2.4 KB, free 912.3 MB)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManagerInfo:54 - Added broadcast_277_piece0 in memory on htm-psycho-401.zxz.su:45401 (size: 2.4 KB, free: 912.3 MB)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkContext:54 - Created broadcast 277 from broadcast at DAGScheduler.scala:1039
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Submitting 3 missing tasks from ResultStage 277 (MapPartitionsRDD[323] at mapToPair at SpeedLayer.java:97) (first 15 tasks are for partitions Vector(1, 2, 3))
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnScheduler:54 - Adding task set 277.0 with 3 tasks
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Starting task 0.0 in stage 277.0 (TID 591, htm-psycho-401.zxz.su, executor 1, partition 1, PROCESS_LOCAL, 7746 bytes)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManagerInfo:54 - Added broadcast_277_piece0 in memory on htm-psycho-401.zxz.su:45269 (size: 2.4 KB, free: 912.3 MB)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Starting task 1.0 in stage 277.0 (TID 592, htm-psycho-401.zxz.su, executor 1, partition 2, PROCESS_LOCAL, 7746 bytes)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Finished task 0.0 in stage 277.0 (TID 591) in 13 ms on htm-psycho-401.zxz.su (executor 1) (1/3)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Starting task 2.0 in stage 277.0 (TID 593, htm-psycho-401.zxz.su, executor 1, partition 3, PROCESS_LOCAL, 7746 bytes)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Finished task 1.0 in stage 277.0 (TID 592) in 6 ms on htm-psycho-401.zxz.su (executor 1) (2/3)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO TaskSetManager:54 - Finished task 2.0 in stage 277.0 (TID 593) in 8 ms on htm-psycho-401.zxz.su (executor 1) (3/3)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnScheduler:54 - Removed TaskSet 277.0, whose tasks have all completed, from pool
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - ResultStage 277 (isEmpty at SpeedLayerUpdate.java:53) finished in 0.033 s
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO DAGScheduler:54 - Job 267 finished: isEmpty at SpeedLayerUpdate.java:53, took 0.035632 s
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Finished job streaming job 1535957400000 ms.0 from job set of time 1535957400000 ms
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Starting job streaming job 1535957400000 ms.1 from job set of time 1535957400000 ms
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO UpdateOffsetsFn:60 - Updating offsets: {OryxInput,0=2477630, OryxInput,1=2478826, OryxInput,2=2477739, OryxInput,3=2478777}
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZooKeeper:438 - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@30c667e9
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZkEventThread:65 - Starting ZkClient event thread.
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZkClient:936 - Waiting for keeper state SyncConnected
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ClientCnxn:975 - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ClientCnxn:852 - Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ClientCnxn:1235 - Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10160d98a790139, negotiated timeout = 30000
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZkClient:713 - zookeeper state changed (SyncConnected)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZkEventThread:83 - Terminate ZkClient event thread.
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ZooKeeper:684 - Session: 0x10160d98a790139 closed
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Finished job streaming job 1535957400000 ms.1 from job set of time 1535957400000 ms
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ClientCnxn:512 - EventThread shut down
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Total delay: 0.605 s for time 1535957400000 ms (execution: 0.099 s)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO JobScheduler:54 - Stopped JobScheduler
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ContextHandler:910 - Stopped o.s.j.s.ServletContextHandler@5a06904{/streaming,null,UNAVAILABLE,@Spark}
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ContextHandler:910 - Stopped o.s.j.s.ServletContextHandler@286866cb{/streaming/batch,null,UNAVAILABLE,@Spark}
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ContextHandler:910 - Stopped o.s.j.s.ServletContextHandler@14b95942{/static/streaming,null,UNAVAILABLE,@Spark}
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO StreamingContext:54 - StreamingContext stopped successfully
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO AbstractConnector:318 - Stopped Spark@18324f97{HTTP/1.1,[http/1.1]}{0.0.0.0:4041}
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkUI:54 - Stopped Spark web UI at http://htm-psycho-401.zxz.su:4041
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnClientSchedulerBackend:54 - Interrupting monitor thread
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnClientSchedulerBackend:54 - Shutting down all executors
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnSchedulerBackend$YarnDriverEndpoint:54 - Asking each executor to shut down
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SchedulerExtensionServices:54 - Stopping SchedulerExtensionServices
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: (serviceOption=None,
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: services=List(),
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: started=false)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO YarnClientSchedulerBackend:54 - Stopped
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MapOutputTrackerMasterEndpoint:54 - MapOutputTrackerMasterEndpoint stopped!
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO MemoryStore:54 - MemoryStore cleared
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManager:54 - BlockManager stopped
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO BlockManagerMaster:54 - BlockManagerMaster stopped
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped!
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO SparkContext:54 - Successfully stopped SparkContext
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 268.0 failed 4 times, most recent failure: Lost task 0.3 in stage 268.0 (TID 567, htm-psycho-401.zxz.su, executor 1): org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow. Available: 1, required: 4. To avoid this, increase spark.kryoserializer.buffer.max value.
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:350)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:393)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.lang.Thread.run(Thread.java:748)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: Caused by: com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 1, required: 4
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.require(Output.java:163)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.writeInt(Output.java:254)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.writeFloat(Output.java:496)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.UnsafeCacheFields$UnsafeFloatField.write(UnsafeCacheFields.java:80)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:366)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:307)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:347)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: ... 4 more
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: Driver stacktrace:
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1599)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1587)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1586)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1586)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at scala.Option.foreach(Option.scala:257)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:831)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1820)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1769)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1758)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:642)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2027)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2048)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2067)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.SparkContext.runJob(SparkContext.scala:2092)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:939)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.rdd.RDD.collect(RDD.scala:938)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.api.java.JavaRDDLike$class.collect(JavaRDDLike.scala:361)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.api.java.AbstractJavaRDDLike.collect(JavaRDDLike.scala:45)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.cloudera.oryx.app.speed.als.ALSSpeedModelManager.buildUpdates(ALSSpeedModelManager.java:182)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:56)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.cloudera.oryx.lambda.speed.SpeedLayerUpdate.call(SpeedLayerUpdate.java:37)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:628)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:416)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at scala.util.Try$.apply(Try.scala:192)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:257)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:257)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:256)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at java.lang.Thread.run(Thread.java:748)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: Caused by: org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow. Available: 1, required: 4. To avoid this, increase spark.kryoserializer.buffer.max value.
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:350)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:393)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: ... 3 more
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: Caused by: com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 1, required: 4
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.require(Output.java:163)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.writeInt(Output.java:254)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.io.Output.writeFloat(Output.java:496)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.UnsafeCacheFields$UnsafeFloatField.write(UnsafeCacheFields.java:80)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:366)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:307)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: at org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:347)
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: ... 4 more
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ShutdownHookManager:54 - Shutdown hook called
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ShutdownHookManager:54 - Deleting directory /tmp/spark-4d5c56ac-ccfc-4c15-ba08-6411e35a1e5e
Sep 03 09:50:00 htm-psycho-401.zxz.su bash[31144]: 2018-09-03 09:50:00 INFO ShutdownHookManager:54 - Deleting directory /tmp/spark-fab670a2-b75d-4494-a3c1-d49046e1c765
Sep 03 09:50:01 htm-psycho-401.zxz.su systemd[1]: oryx-speed.service: main process exited, code=exited, status=1/FAILURE
Sep 03 09:50:01 htm-psycho-401.zxz.su systemd[1]: Unit oryx-speed.service entered failed state.
Sep 03 09:50:01 htm-psycho-401.zxz.su systemd[1]: oryx-speed.service failed.
how can I fix it?