Migrate Redis to OCI Cache on Redis

You can migrate your Redis DB to OCI Cache on Redis using the following steps.

You need to use a data dump tools such as pyredis-dump for getting serialised dump from your Redis DB. 

This tool can be used to take a serialised dump from any Redis node running in any cloud environments like AWS ElastiCache. So far, we are testing only single master Redis cluster.

 

[root@jay ~]# git clone https://github.com/tkote/pyredis-dump.git

Cloning into 'pyredis-dump'...

remote: Enumerating objects: 51, done.

remote: Counting objects: 100% (40/40), done.

remote: Compressing objects: 100% (23/23), done.

remote: Total 51 (delta 22), reused 28 (delta 17), pack-reused 11

Receiving objects: 100% (51/51), 15.89 KiB | 15.89 MiB/s, done.

Resolving deltas: 100% (22/22), done.

[root@jay ~]#

 

Using pyredis-dump, you can create a backup of your dataset from OCI Redis as well for safe keeping of your data.

 

You can use the backup you have taken to move the Redis node to a different Redis node in another region/VCN as per your requirement.

 

The pyredis-dump has the following options:

[root@jay pyredis-dump]# python3 pyredis-dump.py -h

Usage: pyredis-dump.py [options] dump|restore|dblist

 

Options:

  -h, --help            show this help message and exit

  -H HOST, --host=HOST  connect to HOST (default localhost)

  -P PORT, --port=PORT  connect to PORT (default 6379)

  -s SOCKET, --socket=SOCKET

                        connect to SOCKET

  -d DB, --db=DB        database

  -w PASSWORD, --password=PASSWORD

                        connect with PASSWORD

  -p PATTERN, --pattern=PATTERN

                        pattern

  -o OUTFILE, --outfile=OUTFILE

                        write to OUTFILE

  -i INFILE, --infile=INFILE

                        read from INFILE

  -t                    use ttl when in restore mode

  -b BULK, --bulk=BULK  dump/restore bulk size

  -S, --ssl             use tls connection

  -k, --watch           watch key value change while dumping

[root@jay pyredis-dump]#

 

Take a serialised backup of the Redis.

Syntax:

# python3 pyredis-dump.py <Options> -H <Hostname/IP> dump -o /path/to/backupfile

 

[root@jay pyredis-dump]# python3 pyredis-dump.py -S -H amaaaaaa2ne4d2aaxhlt47oluwtlrvualroXXocvb3xmg546a-p.redis.eu-frankfurt-1.oci.oraclecloud.com dump -o /opt/redisdump/rdump1

dumping to '/opt/redisdump/rdump1'

connecting to {'db': 0, 'host': amaaaaaa2ne4d2aaxhlt47oluwtlrvualroXXocvb3xmg546a-p.redis.eu-frankfurt-1.oci.oraclecloud.com', 'port': 6379, 'ssl': True}

5 keys, bulk_size: 1000, watch_keys: False

dumped 5 records

elapsed time: 0.029 seconds

[root@jay pyredis-dump]#

 

Here we have taken a backup of the OCI Redis cluster. However, you may run the same command to take backup of any Redis clusters in any environments.

 

In the above command, we used the option  ‘-S’ as we are connecting to an OCI Redis cluster and it mandates secure TLS connection. For your Redis environment you might not need to use this option.

 

The pyredis-dump command creates a dump file in the path you specified. The time it takes to create a dump depends on your data size.

 

Once you get the dump file, you can move it to the location or an OCI instance from which you can access your new OCI Redis cluster.

 

 

Restore to OCI Redis Cache

You may use the following command to restore the data to the new cluster.

 

Syntax:

# python3 pyredis-dump.py <Options> -H <Hostname/IP> restore -i /path/of/backup

 

[root@jay pyredis-dump]# python3 pyredis-dump.py -S -H amaaaaaa2ne4d2aaxhlt4XXXrvualrociwb3xmg546a-p.redis.eu-frankfurt-1.oci.oraclecloud.com restore -i /opt/redisdump/rdump1

restore from '/opt/redisdump/rdump1'

connecting to {'db': 0, 'host': amaaaaaa2ne4d2aaxhlt4XXXrvualrociwb3xmg546a-p.redis.eu-frankfurt-1.oci.oraclecloud.com', 'port': 6379, 'ssl': True}

restored 5 records

elapsed time: 0.022 seconds

[root@jay pyredis-dump]#

 

 

 

Some catches:

If you get the following error when using pyredis-dump, install redis for python.

 

[root@jay pyredis-dump]# python3 pyredis-dump.py -h

Traceback (most recent call last):

  File "pyredis-dump.py", line 7, in <module>

    from redis import StrictRedis as Redis

ModuleNotFoundError: No module named 'redis'

[root@jay pyredis-dump]#

 

Install redis in python using pip3.

 

[root@jay pyredis-dump]# pip3 install redis

WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.

Collecting redis

  Downloading https://files.pythonhosted.org/packages/d6/f6/19237b28c632935c7359bddf703395ba13bbd134fc5e2eb297c4c120398c/redis-4.3.6-py3-none-any.whl (248kB)

    100% |████████████████████████████████| 256kB 5.7MB/s

Collecting importlib-metadata>=1.0; python_version < "3.8" (from redis)

  Downloading https://files.pythonhosted.org/packages/a0/a1/b153a0a4caf7a7e3f15c2cd56c7702e2cf3d89b1b359d1f1c5e59d68f4ce/importlib_metadata-4.8.3-py3-none-any.whl

Collecting async-timeout>=4.0.2 (from redis)

  Downloading https://files.pythonhosted.org/packages/d6/c1/8991e7c5385b897b8c020cdaad718c5b087a6626d1d11a23e1ea87e325a7/async_timeout-4.0.2-py3-none-any.whl

Collecting packaging>=20.4 (from redis)

  Downloading https://files.pythonhosted.org/packages/05/8e/8de486cbd03baba4deef4142bd643a3e7bbe954a784dc1bb17142572d127/packaging-21.3-py3-none-any.whl (40kB)

    100% |████████████████████████████████| 40kB 11.4MB/s

Collecting typing-extensions; python_version < "3.8" (from redis)

  Downloading https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl

Collecting zipp>=0.5 (from importlib-metadata>=1.0; python_version < "3.8"->redis)

  Downloading https://files.pythonhosted.org/packages/bd/df/d4a4974a3e3957fd1c1fa3082366d7fff6e428ddb55f074bf64876f8e8ad/zipp-3.6.0-py3-none-any.whl

Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/lib/python3.6/site-packages (from packaging>=20.4->redis)

Installing collected packages: zipp, typing-extensions, importlib-metadata, async-timeout, packaging, redis

Successfully installed async-timeout-4.0.2 importlib-metadata-4.8.3 packaging-21.3 redis-4.3.6 typing-extensions-4.1.1 zipp-3.6.0

[root@jay pyredis-dump]#

 

If you get a ConnectionError exception as follows, its mostly due to OCI Redis cache needing a secure TLS connection.

Error: redis.exceptions.ConnectionError: Error while reading from <Hostname>:6379 (104, 'Connection reset by peer')

[root@jay pyredis-dump]# python3 pyredis-dump.py -H amaaaaaa2ne4d2aaxhlt47xxxktdy77fovocvb3xmg546a-p.redis.eu-frankfurt-1.oci.oraclecloud.com dump -o rdump1

dumping to 'rdump1'

connecting to {'db': 0, 'host': amaaaaaa2ne4d2aaxhlt47xxxktdy77fovocvb3xmg546a-p.redis.eu-frankfurt-1.oci.oraclecloud.com', 'port': 6379, 'ssl': None}

[..]

redis.exceptions.ConnectionError: Error while reading from amaaaaaa2ne4d2aaxhlt47xxxktdy77fovocvb3xmg546a-p.redis.eu-frankfurt-1.oci.oraclecloud.com:6379 : (104, 'Connection reset by peer')

[root@jay pyredis-dump]#

 

OCI Redis cache requires secure TLS connection. The pyredis-dump supports TLS connection using switch ‘-S’. Use this option to the connection string.