Friday, March 2, 2018

Execute local bash scripts in remote system

Why do we need this? 

We normally login to the remote system and execute the scripts located in the remote system. What if we execute the local scripts in local system so that we need not copy the scripts into remote system which makes it easy to test also. The use case for this problem is when we have a job that has local tasks and remote tasks and they are dependent each other. While backing up remote files locally, first we need to create backup files and save those files in the remote system. After backup creation is successful, we can go ahead and copy those files into local system.

So, what do we need? Because I have successfully solved this problem and implemented in my company. So I am gonna write my achievement here.

1) Secure Login

This is the most important part because we are connecting from local system to remote system which can be located anywhere in the world. That means your traffic goes outside of your company and proper security mechanism should be implemented. We never send plain text! We always encrypt the text and we use secure shell communication and the traffic is always encrypted, no one can understand.

Because we automate this backup job, i.e. it do not need any human interaction. Normal ssh login need user-name and password to login, but we instead create SSH keys and install those in both systems so that we can carry out secure communications between two systems.

Creating ssh keys:

$ ssh-keygen -t rsa -b 2048

 It creates two files (private and public keys). If already created, then we do not need to create.

Copy these keys to the remote host.

$ssh-copy-id root@remote.frietec.com


Now we can connect(ssh) without need  of password.

$ssh root@remote.frietec.com

2) Executing scripts into remote system

$ssh root@remote.frietec.com 'bash -s' < SCRIPT_TO_RUN 

The script runs on the remote system. We can send the parameters if we need after the script. 

Until now, we executed local script into the remote system. In our case, the script creates backup files and these files are in remote system.



3) Transfer backup files into your local system

This is important because we need to securely transfer our files into our local system. So, we use secure copy tool (scp) to securely download the files. 

$scp root@remote.frietec.com:/opt/backups/*.bkp /backups/

Thats it. We have to implement the combined script as a cronjob, then we can get periodic remote backup into your local system. We can implement notification system also.

Notes

1) There are several possibilities to carry out this task. Here you have more control over your work, but there is another tool called rsync which is also promising which can replace scp tool which we have used here.

2) Since we have implement autologin to the remote system, you have to be sure the the locale system is secure enough, otherwise, anybody can reach to the remote system.

 3) If you have Jenkins, then it is quite easier to define task and run it periodically. And we can do more using jenkins.