Adding Retry Logics and Failover in Voting Services

in steem •  7 months ago 

I haven't revisited the code of voting services for a long long time since most of the time - it works. Today, there are some problems of the nodes, and that is why I decided to take some time to improve/fix it.

The voting services consist of several components but as you can imagine, it is decoupled, it works as something like this:

  1. there are several processes (components) that monitor the posts of delegators and good authors. Once criteras are met, the posts are pushed to database.
  2. the reader process checks periodically the database for new posts to vote, and then vote them one by one.

The problem is that sometimes the node API call failed, like this:

image.png

The error is Internal Error which doesn't give enough information of what went wrong here.
** getDiscussionsByAuthorBeforeDate Error [RPCError]: Internal Error **

Since this is scheduled to run every 10 minutes (as a crontab job), usually a next retry would work. But if a node is down for some time, then it will cause the problem.

Thus, i have added the retry - and also failover logics, so it will choose next RPC node and then try again 3 times before skipping.

for (let tc = 1; tc <= 3; tc ++) {
    console.log(`Try process counter ${tc}...`);
    if (await processForUser(user, totalDelegated)) {
        break;
     }
     failover();
     await sleep(3000);
}

It is not perfect, but it is better, and it works for now.

Steem to the Moon🚀!

steem steemit voting delegators programming bugfixes steemitdev algorithms
7 months ago by
99.16 BLURT
  • Past Payouts 99.16 BLURT
  • - Author 49.58 BP
  • - Curators 49.58 BP
12 votes $0.14
0
Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE BLURT!