The System Design of Steem Blockchain Bots

Yesterday, the ChatGPT has been integrated to Steem Blockchain, but the initial design has problems and so other bot commands e.g. !bing, !price, !info and !thumbup. The applications are written in NodeJs and run by pm2 manager.

The Design Flaws

Previously:

1. process `blockchain` listens the comments posted to steem blockchain on real time - and push the known commands to MySQL database.
2. process `ask` or other commands take the corresponding record from MySQL and process them immediately - and post to steem blockchain (synchronously) in the same process.

There is a fault - there could be multiple bots dealing with the message and posting to steem blockchain at the same time under the same account aka @witnesstools. As you may know, a steem account can post a comment every 3 seconds, and thus, there is going to be a problem for a large scale simutaneous processing.

We can have a retry at each process when the final comment posting fails - but this increases overhead and complexity. And also, this can cause code duplication for each bot.

The New Design

Thus, with a refactor and redesign - I have added a comments process which does one thing: read comments from MySQL, post them and mark them done, every 3 seconds.

Here is the overall design for the components:

The System Design Diagram of the Steem Blockchain Bots

The comments are pushed to the MySQL for each bot when they finish processing, for example, when ChatGPT API returns the answer, instead of posting it directly to the steem blockchain, it will be stored in the MySQL, and be picked up by comments process.

This has several advantages:

• Decouple the components: the `blockchain` reads transactions, the `ask` takes messages and calls ChatGPT, pushes the answer to database. The `comments` reads the comments and post them to the chain.
• Code reuse: all processing bots share the same code except the process part. e.g. `ask` calls ChatGPT, `thumbup` retrieves a random thumbup gif, `bing` returns a random wallpaper etc.
• Less bugs: previously, the records are marked processed only after the comments are posted, but this has delays which may trigger another round of processing meaning the same message may be processed in parallel more than once.

With this design, the message taken out will be marked processed immediately, the comments will be posted separately without being affected by this status.

Anyway, the following shows the processes running happily by pm2 manager.

PM2 Manager shows running Steem Blockchain Processes

Retry Logics/Strategy

Currently, the system does not have a retry strategy. For example, posting on Steem Blockchain may trigger an error - due to Out of Resource Unit, or simply the network issues. When that occurs, we can re-queue the record back to the comments table - or simply update the timestamp so that it can be picked up later (in the order of First-In-First-Out so that other new messages can be picked before the "retrying" messages).

Conclusion

This post describes the current system design of the Bots (commands) on steem blockchain. The Database (MySQL) is used for both purposes of persistence and Message Queue. If the performance is an issue, we may need to consider High Performant Message Queue instead of Database.

The current design overcomes the limitation of posting 1 comment on steem blockchain every 3 seconds, and allows multiple bots processing at the same time.

Post: The System Design of Steem Blockchain Bots

---

Steem to the Moon!

• You can swap the STEEM/SBD to USDT (TRC-20) via Steem2USDT!
• Register a free STEEM account at SteemYY!