Data structures store tied to simple a TCP server, how cool is that!!

While Redis is fairly easy to use via a set of commands, issues start arising when we try to implement complex operations that comprise multiple Redis commands.

For example, let's consider an operation that performs the following steps:

1. Pushes a list of integers to a Redis list mylist.
2. Update the value of key sum that stores the sum of all integers in list mylist.

Lets try to implement this; Im using redis-py, a Python Redis client.

import redisr = redis.Redis(host='localhost', port=6379, db=0)new_list = [2,1,2]# Pushing the new_list to the redis listr.lpush('mylist', *new_list)# Calculating the sum of the new_listsum_of_new_list = sum(new_list)# ... here we can have multiple application-level operations # ... before calling the next Redis command# Incrementing the valuof of the redis key "sum" by the sum_of_new_listr.incrby('sum', sum_of_new_list)

While at first glance the above implementation looks fine, but it is vulnerable to the following problems:

1. The atomicity. Ah, a good old problem. Redis commands r.lpush('mylist', *new_list) and r.incrby('sum', sum_of_new_list) should be atomic, i.e either both of them should execute or none of them should execute. Atomicity is not guaranteed in the above implementation.
2. The application (python script in this case) is handling data-level logic.

Using Lua script

This is where Lua script comes handy. Redis includes server-side scripting with the Lua programming language. This lets you perform a variety of operations inside Redis, which can both simplify your code and increase performance.

EVAL command is used to evaluate scripts using the Lua interpreter built into Redis.

The first argument of EVAL is a Lua 5.1 script. The script does not need to define a Lua function (and should not). It is just a Lua program that will run in the context of the Redis server.

The second argument of EVAL is the number of arguments that follows the script (starting from the third argument) that represent Redis key names. The arguments can be accessed by Lua using the KEYS global variable in the form of a one-based array (so KEYS[1], KEYS[2], ...).

All the additional arguments should not represent key names and can be accessed by Lua using the ARGV global variable, very similar to what happens with keys (so ARGV[1], ARGV[2], ...).

We can call Redis commands from a Lua script using the function redis.call().

Lets try to write a command using the above information:

Redis return values are converted into Lua data types when Lua calls a Redis command using call(). Similarly, Lua data types are converted into the Redis protocol when calling a Redis command and when a Lua script returns a value so that scripts can control what EVAL will return to the client.

Redis uses the same Lua interpreter to run all the commands. Also, Redis guarantees that a script is executed in an atomic way: no other script or Redis command will be executed while a script is being executed.

Loading Lua scripts into Redis

Instead of providing Lua script every time to Redis, we can load the Lua script once using SCRIPT LOAD command and then subsequently run it using the EVALSHA command.

The first and the only argument of SCRIPT LOAD is a Lua 5.1 script. The script does not need to define a Lua function (and should not). It is just a Lua program that will run in the context of the Redis server.

After executing SCRIPT LOAD command, a SHA1 digest of the script will be returned. We can call the script by providing SHA1 digest to the EVALSHA command along with needed KEYS and ARGV

Lets try to write a command using the above information.

Powerful, isnt it? The script is guaranteed to stay in the script cache forever (unless SCRIPT FLUSH is called).

Now that we have understood how to use Lua scripts with Redis, lets try to implement the same operation using Lua Scripts. Im using redis-py, a python redis client.

First off let's create the script.lua file that will hold the Lua Script.

-- script.lualocal sum_of_new_list = 0-- Pushing the new_list to the redis list -- Also calculating the sum of the new_listfor key,value in ipairs(ARGV) do    sum_of_new_list = sum_of_new_list + tonumber(value)    redis.call('LPUSH', KEYS[1], value)end-- Incrementing the value of the redis key "sum" by the sum_of_new_listlocal total_sum = redis.call('INCRBY', KEYS[2], sum_of_new_list)-- Finally returning the total_sumreturn total_sum

Now lets implement main.py where we will import script.lua and run it on the Lua interpreter of Redis.

# main.pyimport redisr = redis.Redis(host='localhost', port=6379, db=0)# Reading script.lua file into a stringf = open("script.lua", "r")lua = f.read()new_list = [2,1,2]# Registering the script on the redis serveroperation = r.register_script(lua)# Executing the scriptop_return = operation(keys=['mylist', 'sum'], args=new_list)print(op_return)

Awesome!, we have successfully implemented the needed operation using Lua script. The application-level logic (main.py) and data-level logic (script.lua) are separated. Our Lua script is Atomic and ensures that keys mylist and sum are always in sync.

Well, thats it for now! Peace 🍻

Although it is open-source, the cloud version is not offered free of cost. This blog will guide you to self-host commento using Nginx and Docker.

Before moving forward, lets compare Commento with other products such as Disqus, Facebook Comments, etc.

Why Commento?

Most other products in this space do not respect your privacy; showing adverts is their primary business model and that nearly always comes at the users cost. There is no free lunch. Commento is also orders of magnitude lighter than alternatives while Disqus and Facebook take megabytes of download to load, Commento is just 11 kB.

Prerequisites

Before proceeding make sure you keep following things ready:

• An Ubuntu host with Public IP.

• If your domain name is example.com, it would be a good idea to use subdomain commento.example.com. So add DNS A Record which points the subdomain to the hosts IP Address.

• A terminal session into your target host for running Commento.

  example.com is just a placeholder replace it with your domain name

Nginx Configuration

We will use Nginx as a reverse proxy which will pass traffic from commento.example.com to localhost:8080 where commento will be running.

First off install Nginx using apt.

sudo apt install nginx

We will also generate SSL certificate for commento.example.com. This is where Lets Encrypt comes handy.

Install certbot and its nginx plugin

sudo apt install python3-certbot python3-certbot-nginx

Add the following server block configuration to /etc/nginx/sites-enabled/default

server {    listen 80;    # Replace this with your domain    server_name commento.example.com;    location / {        proxy_pass http://127.0.0.1:8080;    }}

Installing SSL Certificate

Now that we have configured our reverse proxy, lets add SSL certificate to it

sudo certbot --nginx

Enter your email, subdomain and other details which are asked.

Certbot will create a SSL certificate and then it will automatically integrate it with Nginx.

I recommend running the following line, which will add a cron job to the default crontab. This will ensure that the SSL certificate is automatically renewed.

echo "0 0,12 * * * root python -c 'import random; import time; time.sleep(random.random() * 3600)' && certbot renew -q" | sudo tee -a /etc/crontab > /dev/null

And thats it! You have successfully configured Nginx with SSL Certificate. Now lets install Commento using docker.

Installing Commento

We will be using docker and docker-compose to install Commento. If you dont have docker and docker-compose installed, first install them and then follow these steps

Commento uses PostgreSQL as its database, for simplicity we will also use docker-compose to run postgres. In production environment you may use managed database service such as AWS RDS.

Create a docker-compose.yml file and add the following lines

version: '3'services:  server:    image: registry.gitlab.com/commento/commento    ports:      - 8080:8080    environment:      COMMENTO_ORIGIN: https://commento.example.me      COMMENTO_PORT: 8080      COMMENTO_POSTGRES: postgres://postgres:postgres@db:5432/commento?sslmode=disable          depends_on:      - db  db:    image: postgres    environment:      POSTGRES_DB: commento      POSTGRES_USER: postgres      POSTGRES_PASSWORD: postgres    volumes:      - postgres_data_volume:/var/lib/postgresql/datavolumes:  postgres_data_volume:

The above configuration pulls and sets up Commento and PostgreSQL in two separate containers with a persistent database volume. Modify the values of COMMENTO_ORIGIN, COMMENTO_PORT, COMMENTO_POSTGRES and other environment variables as per you need.

Save the docker-compose.yml file somewhere and run the following command to start the services.

docker-compose up

This will start both services and will map commento to port 8080 of the host.

Now visit https://commento.example.com and you will be greeted with Commento's login page.

At this point, you have successfully self-hosted commento. If you want to configure SMTP Server and setup OAuth move ahead.

Configure SMTP Server

Commento will use SMTP Server to send mails to notify users and moderators about relevant events. To configure SMTP Server you need to provide the credentials and other information using environment variables listed below

COMMENTO_SMTP_HOST=smtp.gmail.comCOMMENTO_SMTP_PORT=587COMMENTO_SMTP_USERNAME=example@gmail.comCOMMENTO_SMTP_PASSWORD=hunter2COMMENTO_SMTP_FROM_ADDRESS=no-reply@example.com

All these variables should be appended in docker-compose.yml file under environment section of server service.

Setup OAuth

Setting up OAuth is very easy with commento. All you need to do is provide credentials in environment variables. We will configure Google and Github OAuth.

Google OAuth

First off obtain the API Key and Secret by creating a new project at Google API Console. Then add those values to following environment variables.

COMMENTO_GOOGLE_KEY=961031300431-0fe76kc72xvo0otts78ug2aqmi4is067.apps.googleusercontent.com COMMENTO_GOOGLE_SECRET=XmaKz7gRkWw3MQgoAHmApuwb

While creating a project at Google API Console you will be asked for Callback URL, So replace example.com with your domain name and provide this URL [https://commento.example.com/api/oauth/google/callback](https://commento.example.com/api/oauth/google/callback)

Github OAuth

Create a new OAuth app in GitHub developer settings to generate a set of credentials.

COMMENTO_GITHUB_KEY=uk3juvzyyejgxhbym1sqkn3t COMMENTO_GITHUB_SECRET=2fbdc6bdbb7c02119fd8fa70b7bdcfa7af8e2c

Callback URL: [https://commento.example.com/api/oauth/github/callback](https://commento.example.com/api/oauth/github/callback)

Conclusion

Congratulations, you have successfully deployed Commento with SSL. We have also configured SMTP Server and OAuth. If you want to dive deeper in commento, you can refer the documentation