AlaaSalama Seeking for the missing pixels!

How to copy files between two Amazon instances ?

I decided to launch a new Amazon instance with CentOS 7 and install PHP 7 on it, to test  some WordPress themes/plugins on the new PHP version, however, I got stuck a little bit about how to copy my files from the old Amazon instance to the new one, in this blog I will record what exactly I did to get this job done, so in case anyone had the same situation…

  • While logged in the old instances, I generated RSA pair of keys, using this command:
    ssh-keygen -t rsa

    then specifying the location of the keys to be saved, this will generate two keys, key_name and key_name.pub

  • After that you will need to copy the content of key_name.pub file to the authorized ssh keys file in second instance, to allow connection between both of them, to do that, simply run this command on the first instance:
    sudo cat /path_to_key_name.pub

    Then copy the file content exactly, and proceed to the next step.

  • On the second instance, you will need to edit the ssh authorized keys file, by entering this command:
    sudo nano /home/your_user/.ssh/authorized_keys

    and paste the exact key_name.pub file content copied int he previous step.

  • After that you can test the ssh connection from your first instance to the second one, by entering this command on the first instance:
    sudo ssh -i /path_to_key_name your_user@second_instance_public_ip

    You should now be connected to the second instance, simply, type logout to exit and proceed to the next step.

  • From the first instance, we will use SCP command to copy files to the second instances, by entering something like this:
    sudo scp -i "/path_to_key_name" -r /path_to_what_you_want_to_copy your_user@second_instance_public_ip:/path_to_where_to_save_copied_files

     

  • Another faster way than SCP is using tar command as the following:
    sudo tar -c /path_to_what_you_want_to_copy | ssh -i "path_to_key_name" your_user@second_instance_public_ip 'tar -xvf - -C /path_to_where_to_save_copied_files'

     

That’s it, everything should work fine by following the same steps, however, there is something to take care about, which is the folder permissions on the second instances must allow others to write permissions, you can enable that temporarily by chmod 757 or 777 till you finish the copying process.

 

Linux terminal commands that I use!

I used to deal with some debian-based linux distributions including (Ubuntu, Biolinux, Linux-mint) and one of the most powerful features that had been always found fascinating  me in linux is the terminal.

I missed this feeling in Windows, I missed this black screen and little blinking cursor, after using Linux for a while I thought I’m a genius just because I wrote my first command in the Terminal, especially when you know it was:

ls -alh

:D, anyway, in this post I decided to collect most of these commands that I use frequently, because searching Google for them every time has turned really exhausting, let’s get started!

1- I want to delete this directory and its sub files/folders!

I don’t need to say that this line saved my time a lot:

rm -rf directory-name-here

2- Copy all files and folders from directory to another

With also some good parameters like updating the old files with new ones, verbose mode, recursively go inside every sub-directory, use this magic:

cp -fiRuv /source/* /destination/

One useful hint there, you can use this copy command generator.

3- Useful trick for a VPS!

When I purchased my own VPS, I thought how I could get benefit of this really fast internet connection speed that Digitalocean has, after lots of search I came out with this idea, using it as a VPN, and as I have two small droplets one in San Francisco and the other one in  Amsterdam, it means that I have two different static IP addresses that I can surf the internet anonymously with.

First, I followed this guide to install openvpn on centos server, then to initiate the whole thing, I must enter this line in my terminal:

sudo openvpn --config ~/path/to/client.ovpn

4- Downloading torrents to my VPS

Again, let’s take advantages from this fast internet connection (I’m from Egypt if you know what I mean -_-) to download torrents on VPS, then download them on my pc using ssh or even sftp with the maximum of my internet connection speed, no delays now for looking for seeds/peers and having irregular download rate.

I followed this guide to install transmission on centos server, then just navigating to your-server-ip:9091 will initiate the web service for transmission client, of course 9091 is the default port number for transmission which can be changed.

5- Encoding video on the fly

Well, handbrake is an amazing application which is intended for encoding videos and the most amazing part of it is that it’s free 🙂

I remember one of my friends had a 2.4GB video file in wmv on his dropbox, and he wanted to get it on his local PC in just 5 minutes, although this can be done with a fast internet connection, but again it’s really hard to get such one here..

So, I figured out this trick, converting the video file on the fly to a smaller mp4 file size will definitely decrease the time needed to download this video without a significant lose of the video quality, so that was what we have done:

  • Downloading the video from my friend’s dropbox to my VPS (takes only minutes)

  • Running Handbrake on my server using this line:

/HandBrakeCLI -i ~/Desktop/my_original_file.wmv -o ~/Desktop/my_new_file.mp4 --encoder x264 --vb 1800 --ab 128 --maxWidth 640 --maxHeight 480 --two-pass --optimize;

 

and voila! it really did the trick, and for more information about this command parameters, it is really worthwhile to check Jay Robinson’s blog post.

6- Transferring files/folders using resync from different servers

This line will do the trick, even with a custom SSH port:

rsync -avz -e "ssh -p XXXX" user@server.com:/path /local_path

Another command using tar (much more faster):

tar -c /path_to files_you_need_to_copy | ssh -i "/path_to_rsa_key" your_user@your_new_host 'tar -xvf - -C /path_to_the_new_host'

 

P.S. this post will be updated every time I find any useful terminal command.

Photo by xmodulo

Copyright © 2016 AlaaSalama

Theme by Anders NorenUp ↑