A distributed in-memory data store for the cloud

Related tags

Database EVCache
Overview

EVCache

Join the chat at https://gitter.im/Netflix/EVCache

EVCache is a memcached & spymemcached based caching solution that is mainly used for AWS EC2 infrastructure for caching frequently used data.

EVCache is an abbreviation for:

  • Ephemeral - The data stored is for a short duration as specified by its TTL (Time To Live).
  • Volatile - The data can disappear any time (Evicted).
  • Cache - An in-memory key-value store.

ScreenShot

Comments
  • EvCache - Eureka

    EvCache - Eureka

    I would like to ask question related with the EvCache and Eureka setup. I spent quite a while checking EvCache - testing and reading about different setup's. At this moment, I am not sure what is the best implementation with Eureka.

    Basically, sample deployment on wiki page explains how to run this on localhost (or on ec2 instances) with 2 replicas (with 2 shards each). But with hard coded memcached replicas/shards endpoint.

    In case I want to use dynamic method of discovering servers with memcached, what I need to do in order to initialize EvCache client?

    Right now, for example in Sample deployment we are creating instance of the EvCache using following: deploymentDescriptor = "SERVERGROUP1=localhost:11211;SERVERGROUP2=localhost:11212"; System.setProperty("EVCACHE_APP1-NODES", deploymentDescriptor); evCache = new EVCache.Builder().setAppName("EVCACHE_APP1").build();

    In case of the Eureka, do we need to use discovery service manually and then recreate instance of evCache or this is done automatically? Is there any kind of technical wiki document which explains this setup little bit more detailed?

    opened by mersed 11
  • Difference in hashing algorithm for evache-client v1.0.5 vs v4.38.0

    Difference in hashing algorithm for evache-client v1.0.5 vs v4.38.0

    Hello, After trying to upgrade evcache-client from v 1.0.5 to v 4.38.0 we faced found increase in cache misses during Blue-Green deployment, on environments using more then one cache nodes. Performing test where number of SETs performed from old client and GETs from new client, we can see that in ~50% of cases new client misses values. Backward test ( SETs from new and GETs from old) show almost same results. On environments with single cache node given tests pass 100% correctly. Both clients using default - KETAMA hashing algorithm.

    Assuming, this we think that something in hashing algorithm implementation have been changed. Is this known issue, and are there some workaround?

    opened by akhitev 9
  • tomcat 7/8 crashes when using the evcache with much data

    tomcat 7/8 crashes when using the evcache with much data

    We use the evcache as the memcached client (one note). Recently, the Tomcat 7 (we upgrade to tomcat 8.5) down several times every day. I have spent much time on it. But I only find few clue in the tomcat log (there isn't hs_err_pid file. I think the jvm crashed before it wrote the log file).

    Could you help me on this issue?

    1. java runtime error.

    A fatal error has been detected by the Java Runtime Environment:

    SIGSEGV (0xb) at pc=0x00007f9959e43e10, pid=15465, tid=0x00007f9829cec700

    JRE version: Java(TM) SE Runtime Environment (8.0_151-b12) (build 1.8.0_151-b12)

    Java VM: Java HotSpot(TM) 64-Bit Server VM (25.151-b12 mixed mode linux-amd64 compressed oops)

    Problematic frame:

    C [libc.so.6+0x81e10]2017-11-20 23:10:39.453 INFO com.netflix.evcache.EVCacheTranscoder: Compression increased the size of [B from 136 to 151

    [ timer expired, abort... ]

    1. Sometimes, tomcat crashes when the evcache reconnecting. Here is the log. 2017-11-20 20:58:18.970 INFO net.spy.memcached.EVCacheConnection: Reconnecting due to exception on {QA sa=m-bp175bd9d8160bc4.memcache.rds.aliyuncs.com/10.50.117.204:11211, #Rops=0, #Wops=0, #iq=54, topRop=null, topWop=null, toWrite=0, interested=1} java.lang.IllegalStateException: No read operation. at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:842) at net.spy.memcached.MemcachedConnection.handleReadsAndWrites(MemcachedConnection.java:732) at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:697) at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:450) at net.spy.memcached.EVCacheConnection.run(EVCacheConnection.java:50) 2017-11-20 20:58:18.972 WARN net.spy.memcached.EVCacheConnection: Closing, and reopening {QA sa=m-bp175bd9d8160bc4.memcache.rds.aliyuncs.com/10.50.117.204:11211, #Rops=0, #Wops=0, #iq=87, topRop=null, topWop=null, toWrite=0, interested=1}, attempt 0.
    opened by shengang1978 6
  • EVCache Touch method throw NoSuchMethodError exception

    EVCache Touch method throw NoSuchMethodError exception

    Here are the jar files version: EVCache Client: 4.104 spymemcached: 2.12.3

    I use the EVCache.touch(key, 120000) to refresh the time. But it throws the below exception. java.lang.NoSuchMethodError: net.spy.memcached.OperationFactory.touch(Ljava/lang/String;ILnet/spy/memcached/ops/OperationCallback;)Lnet/spy/memcached/ops/KeyedOperation; net.spy.memcached.EVCacheMemcachedClient.touch(EVCacheMemcachedClient.java:322) com.netflix.evcache.pool.EVCacheClient.touch(EVCacheClient.java:1099) com.netflix.evcache.EVCacheImpl.touchData(EVCacheImpl.java:755) com.netflix.evcache.EVCacheImpl.touch(EVCacheImpl.java:712) com.netflix.evcache.EVCacheImpl.touch(EVCacheImpl.java:663) ...

    Do I use the wrong jar files?

    opened by shengang1978 5
  • Using Memcached with Prana

    Using Memcached with Prana

    I am facing difficulty trying to configure healthurl "prana.host.healthcheck.url" for memcached . I have one instance running on port 11211 for memcached and prana is running on 8078. My understanding is this url should be reachable but memcached doesn't have any direct url that can be polled from prana. Please suggest.

    opened by userswat 5
  • Large amount of PoolRefresher threads

    Large amount of PoolRefresher threads

    Hello, We are using v4.76.0 with stand alone (non-eureka mode) and have two nodes memcached cluster. We faced situation when several thousands of PoolRefresher threads remain in BLOCKED state

    at com.netflix.evcache.pool.EVCacheClientPool.refresh(EVCacheClientPool.java:680) at com.netflix.evcache.pool.EVCacheClientPool.access$200(EVCacheClientPool.java:47) at com.netflix.evcache.pool.EVCacheClientPool$7.run(EVCacheClientPool.java:859)

    waiting for lock for com.netflix.evcache.pool.EVCacheClientPool owned by single PoolRefresher. Comparing samples we can see that it's not deadlock situation and PoolRefresher which owns the lock is changing constantly. All PoolRefresher related to single memcached node. Netstat shows that connections to this node in SYN_SENT status. Are there workaround for this situation ? From code I can see that it is result of EVCacheClientPool#refreshAsync. Would it make sense to limit number of concurrent refreshAsync operations?

    opened by akhitev 4
  • Cache calls taking 10 seconds

    Cache calls taking 10 seconds

    I have deployed evcache to a highish load production environment and we appeared to have some kind of network glitch where cache calls ended up taking 10 seconds per call (or multiples of 10 i.e. 20 or 30 seconds). Even though I had sent the max timeout to 100ms, this resulted in threads backing up and server grinding to a hault.

    As it was production we had very limited logging and I after extensive testing in QA I was unable to replicate the issue. So would anyone have any idea what could cause this?

    evcache version:1.0.5 spymemcache version:2.7.3

    ConfigurationManager.getConfigInstance(); EVCacheClientPoolManager.getInstance().initEVCache(applicationName);

    evCache = (new EVCache.Builder()).setAppName(applicationName).enableZoneFallback().build();

    Config EC2_AVAILABILITY_ZONE=zone1 ${evcache.pool.provider}=com.netflix.evcache.pool.standalone.ZoneClusteredEVCacheClientPoolImpl ${evcache.application.name}.${evcache.server.zone.a.name}.EVCacheClientPool.hosts">${evcache.server.zone.a.hostnames}=server1,server2,server3 ${evcache.application.name}.${evcache.server.zone.b.name}.EVCacheClientPool.hosts">${evcache.server.zone.b.hostnames}=server4,server5,server6 ${evcache.application.name}.EVCacheClientPool.poolSize">${evcache.connection.poolsize}=1 ${evcache.application.name}.EVCacheClientPool.readTimeout">${evcache.read.timeout}=100

    Logs Before problem We recieved a number of these warnings from a heartbeat thread we had running MemcachedConnection] - Could not redistribute to another node, retrying primary node for key11111111

    The about 1 minute later (where there were a number of successful cache calls) we started having the issue.

    Logs around problem time [ n.s.m.MemcachedConnection] - Closing, and reopening {QA sa=/serverip_1:serverport, #Rops=2, #Wops=162, #iq=0, topRop=Cmd: -1 Opaque: -1 Keys: , topWop=Cmd: 0 Opaque: 1865615 Key: Anotherkey_V11, toWrite=0, interested=5}, attempt 0. [ n.s.m.p.b.BinaryMemcachedNodeImpl] - Discarding partially completed op: Cmd: -1 Opaque: -1 Keys: [5ad1c8ab56d9445fa5aa4b0034c8c93d n.s.m.MemcachedConnection] - Could not redistribute to another node, retrying primary node for Anotherkey2_V11. [ n.s.m.MemcachedConnection] - Closing, and reopening {QA sa=/serverip_2:serverport, #Rops=2, #Wops=220, #iq=0, topRop=Cmd: 1 Opaque: 1865530 Key: cckey_V11 Cas: 0 Exp: 0 Flags: 1 Data Length: 222, topWop=Cmd: 0 Opaque: 1865532 Key: cckey_V11, toWrite=0, interested=8}, attempt 0. [ n.s.m.p.b.BinaryMemcachedNodeImpl] - Discarding partially completed op: Cmd: 1 Opaque: 1865530 Key: cckey_V11 Cas: 0 Exp: 0 Flags: 1 Data Length: 222 [ n.s.m.MemcachedConnection] - Closing, and reopening {QA sa=/serverip_3:serverport, #Rops=2, #Wops=180, #iq=0, topRop=Cmd: -1 Opaque: -1 Keys: , topWop=Cmd: 0 Opaque: 1865589 Key: cckey2_V11, toWrite=0, interested=5}, attempt 0. [ n.s.m.p.b.BinaryMemcachedNodeImpl] - Discarding partially completed op: Cmd: -1 Opaque: -1 Keys:

    Also I noticed the value DEFAULT_OP_QUEUE_MAX_BLOCK_TIME in the spymemcache code which coincidentally = 10 seconds. Which is quite strange, anyone know what this value is for?

    DefaultConnectionFactory public static final long DEFAULT_OP_QUEUE_MAX_BLOCK_TIME = TimeUnit.SECONDS.toMillis(10);

    Any ideas or help would be most welcome.

    opened by pat-humphreys 4
  • MultiZone PoolSize

    MultiZone PoolSize

    Can anyone recommend a poolsize to use for an application that would have around 1000 interactions per second with memcache? I had this set to 5, but I was seeing some very unusual results, almost looking like a threading issue. I am still trying to collect more data around that, but a poolsize recommendation would be great.

    opened by pat-humphreys 4
  • Possible bug - Multi-Zone fallback

    Possible bug - Multi-Zone fallback

    evcache-client-1.0.5

    I am using the ZoneClusteredEVCacheClientPoolImpl with 1 client instance and 6 memcache servers (3 in each zone). If I write a key to both zones and then kill the primary memcache the next read for that key goes to the primary first.

    Then realizes that the primary is down, throws an exception which is caught in EVCacheImpl.get(String key, EVCacheTranscoder tc) where it does the following check

    Line 244 catch (Exception ex) { //TODO: Add counter for exception ? if (log.isDebugEnabled()) { log.debug("Exception while getting data for key : " + key, ex); } if (!_throwException.get()) { return null; } throw new EVCacheException("Exception getting data for key : " + key, ex); } it then returns a null.

    The next request for this key goes to the secondary and succeeds, should it not try the secondary rather than returning a null?

    opened by pat-humphreys 4
  • Spymemcached client version 2.8 and above is not compatible

    Spymemcached client version 2.8 and above is not compatible

    I found out through testing, that i managed to start up the EvCache client using spymemcached client version 2.7.3. Everything from 2.8 and up breaks the contract of the HashAlgorithm interface.

    It would be nice if this could work this latest version present in Maven central. They only uploaded versions 2.9 and later.

    opened by pparth 4
  • EvCache + Eureka

    EvCache + Eureka

    Hello, We already integrate many Netflix components to our architecture and Eureka is, of course, one of them. So, i'm particularly interested in the EvCache setup using Eureka. Some questions:

    1. In the relevant setup example, you keep mentioning about a single instance of an EvCache Client per Zone. What exactly do you mean by this? That it is mandatory for the Client to be inside a Service app that acts as a proxy for all cache calls of our architecture and only one instance of this Service app is allowed per zone? Because, i was thinking of setting up a Hystrix wrapper over an EvCache Client instance and use it from every service that needs to get data from the cache. So, in my setup, if we have 4 Domain Service types and 3 instances of each of this service type spanned across multiple zones, can i use an EvCache Client embedded inside each one of these, in order to get data from the cache?
    2. Are you planning to release the EvCache Server any time soon?
    3. Due to the fact that EvCache Server is not yet released, we opted to go with ElastiCache. How do you imagine that a Eureka-based EvCache client setup can work with ElastiCache? I guess that the problem is, how do we report to Eureka about the ups and downs of the ElastiCache instances? Or maybe the Eureka-based setup does not really work with ElastiCache and we need to wait for EvCache Server to get released? Or maybe we can use this sidecar Java app you mention? Can it work for ElastiCache instances somehow or is strictly specific to EvCache Server?
    opened by pparth 4
  • Sort Client (Read/Write ASG followed by Write only) for add API

    Sort Client (Read/Write ASG followed by Write only) for add API

    Context: For ADD api we skip adding keys to the cluster if the key is present in the first client. Now if a new cluster is warmed with fewer keys, the ADDS might not go through as it doesn't respect read/write mode and this might result in drop in hit rate.

    Fix: Sorting the client (read/write ASG first then followed by write-only ASG). So now the ADD API with check if the key is present in ASG which is actually serving the traffic rather than write-only ASG.

    Please let me know if you have any questions. Please review and let me know your comments. Thanks!

    opened by sriram-rangarajan 0
  • Improve GRADLE build Performance

    Improve GRADLE build Performance

    Parallel builds. This project contains multiple modules. Parallel builds can improve the build speed by executing tasks in parallel. We can enable this feature by setting org.gradle.parallel=true.

    File system watching. Since Gradle 6.5, File system watching was introduced which can help to avoid unnecessary I/O. This feature is the default since 7.0. For an older version, we can enable this feature by setting org.gradle.vfs.watch=true.

    Configuration on demand. Configuration on demand tells Gradle to configure modules that only are relevant to the requested tasks instead of configuring all of them. We can enable this feature by setting org.gradle.configureondemand=true.

    gradle caching. Shared caches can reduce the number of tasks you need to execute by reusing outputs already generated elsewhere. This can significantly decrease build times. We can enable this feature by setting org.gradle.caching=true.

    ===================== If there are any inappropriate modifications in this PR, please give me a reply and I will change them.

    opened by shisheng-1 0
  • Asynchronous Gets

    Asynchronous Gets

    Hello! I was looking at the evcache-client to use as a memcached client. When playing around with it, I noticed that there is an asynchronous get method, but that its deprecated. I see that there are set methods that return futures, but the get does not. I was wondering, using this library, is it still possible to perform get operations asynchronously (without using the deprecated method), and if so, what is the recommended way to do that? Thanks!

    opened by bruuuuuuuce 0
  • Add a metric to track

    Add a metric to track "Wrong key returned" errors

    This change adds "wrongKeyReturned" as a metric and increments it everytime we see it in action.

    This change was added as we noticed some of these error messages in the logs, and we'd like to track the frequence of these errors.

    Note that even prior to this change, we did not return the wrong key back to the calling application but instead count it as a miss.

    Print stack trace when we notice a Wrong Key error

    Consolidate duplicate code under checkWrongKeyReturned()

    Address review comments for wrong key returned metric

    • Better logging to catch if returned key is from a different host
    • Make sure stack trace goes to logs instead of stdout
    • Naming nits addressed
    opened by smukil 0
  • Add a metric to track

    Add a metric to track "Wrong key returned" errors

    This change adds "wrongKeyReturned" as a metric and increments it everytime we see it in action.

    This change was added as we noticed some of these error messages in the logs, and we'd like to track the frequence of these errors.

    Note that even prior to this change, we did not return the wrong key back to the calling application but instead count it as a miss.

    opened by smukil 1
Releases(v5.19.1-rc.1)
Owner
Netflix, Inc.
Netflix Open Source Platform
Netflix, Inc.
CrateDB is a distributed SQL database that makes it simple to store and analyze massive amounts of machine data in real-time.

About CrateDB is a distributed SQL database that makes it simple to store and analyze massive amounts of machine data in real-time. CrateDB offers the

Crate.io 3.6k Jan 2, 2023
Java implementation of Condensation - a zero-trust distributed database that ensures data ownership and data security

Java implementation of Condensation About Condensation enables to build modern applications while ensuring data ownership and security. It's a one sto

CondensationDB 43 Oct 19, 2022
New-fangled Timeseries Data Store

Newts Newts is a time-series data store based on Apache Cassandra. Features High throughput Newts is built upon Apache Cassandra, a write-optimized, f

OpenNMS 190 Oct 3, 2022
Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

Trino is a fast distributed SQL query engine for big data analytics. See the User Manual for deployment instructions and end user documentation. Devel

Trino 6.9k Dec 31, 2022
The official home of the Presto distributed SQL query engine for big data

Presto Presto is a distributed SQL query engine for big data. See the User Manual for deployment instructions and end user documentation. Requirements

Presto 14.3k Dec 30, 2022
Apache Drill is a distributed MPP query layer for self describing data

Apache Drill Apache Drill is a distributed MPP query layer that supports SQL and alternative query languages against NoSQL and Hadoop data storage sys

The Apache Software Foundation 1.8k Jan 7, 2023
Replicate your Key Value Store across your network, with consistency, persistance and performance.

Chronicle Map Version Overview Chronicle Map is a super-fast, in-memory, non-blocking, key-value store, designed for low-latency, and/or multi-process

Chronicle Software : Open Source 2.5k Dec 29, 2022
MapDB provides concurrent Maps, Sets and Queues backed by disk storage or off-heap-memory. It is a fast and easy to use embedded Java database engine.

MapDB: database engine MapDB combines embedded database engine and Java collections. It is free under Apache 2 license. MapDB is flexible and can be u

Jan Kotek 4.6k Dec 30, 2022
MapDB provides concurrent Maps, Sets and Queues backed by disk storage or off-heap-memory. It is a fast and easy to use embedded Java database engine.

MapDB: database engine MapDB combines embedded database engine and Java collections. It is free under Apache 2 license. MapDB is flexible and can be u

Jan Kotek 4.6k Jan 1, 2023
IoTDB (Internet of Things Database) is a data management system for time series data

English | 中文 IoTDB Overview IoTDB (Internet of Things Database) is a data management system for time series data, which can provide users specific ser

The Apache Software Foundation 3k Jan 1, 2023
Distributed ID Generate Service

Leaf There are no two identical leaves in the world. — Leibnitz 中文文档 | English Document Introduction Leaf refers to some common ID generation schemes

美团 5.7k Dec 29, 2022
A scalable, distributed Time Series Database.

___ _____ ____ ____ ____ / _ \ _ __ ___ _ _|_ _/ ___|| _ \| __ ) | | | | '_ \ / _ \ '_ \| | \___ \| | | | _ \

OpenTSDB 4.8k Dec 26, 2022
Apache Pinot - A realtime distributed OLAP datastore

What is Apache Pinot? Features When should I use Pinot? Building Pinot Deploying Pinot to Kubernetes Join the Community Documentation License What is

The Apache Software Foundation 4.4k Dec 30, 2022
HurricaneDB a real-time distributed OLAP engine, powered by Apache Pinot

HurricaneDB is a real-time distributed OLAP datastore, built to deliver scalable real-time analytics with low latency. It can ingest from batch data sources (such as Hadoop HDFS, Amazon S3, Azure ADLS, Google Cloud Storage) as well as stream data sources (such as Apache Kafka).

GuinsooLab 4 Dec 28, 2022
Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

Copyright Debezium Authors. Licensed under the Apache License, Version 2.0. The Antlr grammars within the debezium-ddl-parser module are licensed unde

Debezium 7.9k Dec 26, 2022
jdbi is designed to provide convenient tabular data access in Java; including templated SQL, parameterized and strongly typed queries, and Streams integration

The Jdbi library provides convenient, idiomatic access to relational databases in Java. Jdbi is built on top of JDBC. If your database has a JDBC driv

null 1.7k Dec 27, 2022