Elastic cache with Cloud

We use embedded mode in graph db for performance, however any possibility of HA/clustering of dB might not be present…?

We avoid clustering in causal mode. All the data that we need to distribute is inserted in elastic search.For fast access we pull it in-memory.

Does this mean whenever failover happens we restore dB from elastic cache ?

We should be  n-way active, so there should not br any  concept of failover/failback. All instances of a given microservice should be  active with no standby.

There is no such data structure which needs to be “replicated” across a cluster in whole.

Multiple cases can exist –

A. When we re-start a given instance of a microservice – we recreate our in-memory context (graph/maps etc) from elastic search.

B. Any Microservice will persist  data in ES when it receives an incoming API call for  provisioning data. Suppose there were 4 active instances of this service.

In case a new instance of this service is spawned, it will read the data from elastic and re-create its state.

C. In case a data is modified, then the modification API call is serviced by one of the multiple service instances, and the changes persisted to Elastic (common data store). The ELB will load balance in round robin all incoming API calls.

Once the elastic persistence is done, rather than replicating anything to other instances, an event will be fired. This event will be subscribed to by all  service instances.

Once the event router delivers this event to all Microservices, they will read the updated data from ES, and update their internal data-structure.

We avoid “over the wire” clusters as they impede our ability to scale in a n-way active configuration. The number of active instances get limited beyond a point.

And with above approach it won’t be consistent. Or consistency is handled at service level ?

Data in elastic is consistent, as we so not concurrently modify elastic documents. There is a uniqueness criteria for each ES document.

Eg. Every alarm has a unique id, and every data has a unique id.

Means we are having sticky sessions for client requests ?

⁠⁠⁠⁠⁠Round Robin. As there is no statefullness – if there are 10 API calls, they can be distributed amongst 5 services (2 each).

There is no need to create HTTP sessions.

Lets say my service instance1 got some data persistent request and neo4j has stored that on instance1 only. Now my another request to read same data can go to instance2. So how data is fetched ?…. From instance1 to elastic search and then to instance2 ?

Suppose some data is created on instance-1.This data is stored in-memory in Neo4j and persisted to elastic.

Now a read request is sent to instance-2.Instance-2 checks whether it has the data in-memory.If it does not find the data (cache miss) it will query elastic search cluster and store if in the graph and use the data.

If it finds the data in-memory (cache hit), then the elastic query will not be fired.

Neo4j is “embedded” inside all instances, while the ES cluster is common for all and external to the microservices.

It may result in cache miss in most cases ?

It is rare if  most of this data is not so dynamically changing . Every service is reading from ES to cache as soon as it comes up.All updates get sync’d as explained above

So all our write to neo4j should be write through i.e. commit is successfully only when we wrote data to both neo4j and elastic search.

Yes,  Neo4j is like an in-memory library with a java API – just like u maintain maps. So transactions may not be needed there.

For elastic writes, we spawn a new worker thread to persist – so that the calling thread does not “block”.

Elastic takes care of write failures and inconsistencies.