Create Desktop Environment in Suse Linux on AWS

Having a Desktop environment on a Cloud Instance is helpful in many ways. You can troubleshoot application connectivity, take proper HAR files and so on. Even having a desktop is cool!

Here is how you can install GNOME on any SUSE Linux instances in any Cloud Environments. Remember, once you install GNOME (or KDE or any desktop environment as a matter of fact), you need to use VNC to connect to it.

The same steps can be used on any Cloud environments like Oracle Cloud (OCI), AWS, Azure, GCP and so on.

 

Requirements:

- SSH client that allows X11 forwarding

- TightVNC Server and Client

* Here are the steps I took to install GNOME desktop:

1. ssh into the instance with root username

2. type 'yast2' to get into YaST2 Control Center

3. Select "Software" on the left side bar, select "Online Update" on the right side bar, and then hint Enter key. This step is to update the repository of the system

4. Select "Software" on the left side bar, select "software Management" on the right side bar, and then hint Enter key.

5. In the "Search Phrase" textbox in the Filter Search session, type "gnome", and then hint Enter key

6. Install everything that listed on the right side bar, if the error page about "Package Dependencies"pops up, select the first option under "possible solutions", and then click "OK -- Try Again"

7. Select "Accept" on the bottom right of the page, hint Enter key. It will install all the packages you selected.

8. After installing the packages, click "F9" key twice to exit out YaST2 Control Center

Here are the steps to install and configure VNCServer:

1. Open TCP port 5901 in the security group that the instance belongs.

2. In the instance, type "zypper install vnc"

3. After installing VNCServer, type "vncpasswd" to set the access password

4. type "vncserver :1" to start a vnc session

5. sudo vim /root/.vnc/xstartup

6. comment out the "twm &" by typing # in front of the phrase, and then add "/usr/bin/gnome &" to the next line

7. save and exit out the xstartup file

8. type "vncserver -kill :1"

9. type "vncserver :1" to start a new session to load the modified xstartup file

10. In your local host, download and install tightvnc: http://www.tightvnc.com/download.php

11. Open "TightVNC Viewer"

12. For the Remote Host, type your DNS for the instance, and then add "::5901" at the end of the line

13. Click "Connect"

14. Type your password you set by vncpasswd

15. Now you can access to your instance via VNC connection

 

Hope this helps.

 

How to create 1000s of small files in Linux for troubleshooting

 

You can use the following one liner to create thousands of smaller random size files on local block volume.

 

 

In this example we are creating 21099 smaller files of multiple KBs.

for n in {1..21099}; do dd if=/dev/urandom of=file$( printf %03d "$n" ).txt bs=1 count=$(( RANDOM + 1024 )); done

How to do faster copy and delete operations on EFS file systems

 

Issue: How to do faster copy and delete operations on EFS file systems.


Environment:

Amazon Linux
Ubuntu Server
Amazon EFS


Solution:


To optimize copy and delete operations on EFS file systems, you can use the GNU Parallel shell tool for executing jobs in parallel. By doing this you will be able to complete these tasks faster than using the normal serial method.


1.a. Install the NFS utilities and the GNU parallel package on Amazon Linux.

[ec2-user ~]$ sudo yum install nfs-utils -y
[ec2-user ~]$ sudo yum install parallel
 

1.b. Install the NFS utilities and the GNU parallel package on Ubuntu Server.
[ubuntu ~]$ sudo apt-get install nfs-common parallel -y


1.c. Install from source:

[ec2-user ~]$ cd /tmp; wget http://ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2
[ec2-user ~]$ tar -xvf parallel-20120122.tar.bz2; cd parallel-20170822
[ec2-user ~]$ sudo yum groupinstall 'development tools' -y
[ec2-user ~]$ make; ./configure && sudo make install


2. Create a temporary directory and mount the EFS filesystem.
[ec2-user ~]$ sudo mkdir /mnt/efs; sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-XXXXXXXX.efs.REGION.amazonaws.com:/ /mnt/efs


3. Create ten thousand small files local in your instance.
[ec2-user ~]$ mkdir /tmp/efs; for each in $(seq 1 10000); do SUFFIX=$(mktemp -u _XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX); sudo dd if=/dev/zero of=/tmp/efs/${SUFFIX} bs=64k count=1; done


4. Copy the files from your instance to the EFS file system using the parallel method.
[ec2-user ~]$ cd /tmp/efs; find -L . -maxdepth 1 -type f | sudo parallel rsync -avR {} /mnt/efs/

5. Delete the files from the EFS file system using the parallel method.
[ec2-user ~]$ cd /mnt/efs; find -L . -maxdepth 1 -type f | sudo parallel rm -rfv {}

 

Test:
The following output is from my tests, using an EFS file system (General Purpose) mounted on a t2.micro instance type:


1. Copy ten thousand files from my EC2 instance to my EFS file system, using the normal serial method.
[ec2-user ~]$ cd /tmp/efs; time sudo find -L . -maxdepth 1 -type f -exec rsync -avR '{}' /mnt/efs/ \;

real 20m8.947s
user 0m0.060s
sys 0m0.980s

2. Copy ten thousand files from my EC2 instance to my EFS file system, using the parallel method.
[ec2-user ~]$ cd /tmp/efs; time find -L . -maxdepth 1 -type f | sudo parallel rsync -avR {} /mnt/efs/
real 5m34.264s
user 0m8.308s
sys 0m6.904s


3. Delete ten thousand files from my EFS file system, using the normal serial method.
[ec2-user ~]$ cd /mnt/efs; time sudo find -L . -maxdepth 1 -exec rm -rfv {} \;
real 2m24.799s
user 0m0.124s
sys 0m1.240s


4. Delete ten thousand files from my EFS file system, using the parallel method.
[ec2-user ~]$ cd /mnt/efs; find -L . -maxdepth 1 -type f | sudo parallel rm -rfv {}
real 1m55.153s
user 0m7.988s
sys 0m6.972s

Recursively copy :
To add to this amazing article as most us are using this as a sample for our customers I would like to mention that the examples above will copy all files from SRC only, not recursively into /SRC. If you need recursively copy you will have two option with rsync:

Loop:
find /SRC/ -type d | while read -r c; do cd "$c"; find -L . -maxdepth 1 -type f | parallel rsync --avR {} /DST; done

As the above allows you the possibility to run parallel copy and recursively copy the data from SRC/ to DST it also introduces a performance penalty as the loop have to read to each folder recursively and fire parallel copies only of the content of that location.

List Creation:
Create List
rsync -avR --dry-run /SRC /DST > lsit.log
 

Run the command:
cat list.log | parallel --will-cite -j 100 rsync -avR {} /DST/


The above is a much simpler approach, what basically do is to create a list of all files/folders recursively on /SRC and fire 100 parallel copies reading the path of the files to copy from the list. This allows the copies to be much more efficient as having less over the head.


[2] http://www.gnu.org/software/parallel/man.html#EXAMPLE:-Parallelizing-rsync