Sunday, April 8, 2018

Run programs as services in Ubuntu System

BRIEF INTRODUCTION

Running jobs from bash terminal is really easy. But what if we want to reduce user interaction to implement automation, we define services which run on its own. We do not need to run or click to start the program. Once we define the service and enable it, then the program runs when operating system boots. The service programs in the background, and they do not terminate when user logs out. That means, these service programs are running in the background and users do not notice them.  And they are quite handy to start and stop from remote system, or from terminal. After the job is started, we can safely disconnect remote system or close the terminal.


PROCEDURE

Now, we start creating a service that runs on the background. The perfect example would be running tomcat as a service because it need it always running, at the same time, we need to start or restart from time to time. Also we need to auto-start when system is rebooted.

So, we create a service that starts and stops tomcat server. To implement that, we first of all install Tomcat Server. I will not talk about tomcat installation here, it really straight forward. Just download packaged tomcat installer and extract the files into /opt/tomcat.

We could manually start and stop from the command in bin directory of tomcat folder. But that is not what we want. Basically we create a service and configure it so that it starts automatically when system reboots.  So, first task we do is, to create a tomcat.service file

The file looks something like this:

File: tomcat.service

[Unit]
Description=Tomcat Service
After=network.target

[Service]
Type=forking
ExecStart=/opt/tomcat/bin/catalina.sh start
ExecStop=/opt/tomcat/bin/catalina.sh stop
RestartSec=10
Restart=always

[Install]
WantedBy=multi-user.target
               

So, after this file is created, we copy this file into /lib/systemd/system/ directory and load daemon

sudo systemctl  daemon-reload

TESTING

So,  to just start service, we use the command

sudo systemctl start tomcat (starts  service)
sudo systemctl status tomcat (gets status)
sudo systemctl is-active tomcat
sudo systemctl is-enabled tomcat
sudo systectl enable tomcat (!Enables the service)

And we can simply start or stop service  in traditional way too as follows:

sudo service tomcat start
sudo service tomcat stop
sudo service tomcat status


In case I need to start service after sytem boot, I enable the service. Otherwise, I just use the command start and stop to start and stop service.

The file uses the basic configuration, we can extend it and add more configuration.



References:

https://wiki.ubuntu.com/SystemdForUpstartUsers

Friday, April 6, 2018

Spring with Vaadin

Spring and Vaadin Integration and a sample code 

Friday, March 2, 2018

Execute local bash scripts in remote system

Why do we need this? 

We normally login to the remote system and execute the scripts located in the remote system. What if we execute the local scripts in local system so that we need not copy the scripts into remote system which makes it easy to test also. The use case for this problem is when we have a job that has local tasks and remote tasks and they are dependent each other. While backing up remote files locally, first we need to create backup files and save those files in the remote system. After backup creation is successful, we can go ahead and copy those files into local system.

So, what do we need? Because I have successfully solved this problem and implemented in my company. So I am gonna write my achievement here.

1) Secure Login

This is the most important part because we are connecting from local system to remote system which can be located anywhere in the world. That means your traffic goes outside of your company and proper security mechanism should be implemented. We never send plain text! We always encrypt the text and we use secure shell communication and the traffic is always encrypted, no one can understand.

Because we automate this backup job, i.e. it do not need any human interaction. Normal ssh login need user-name and password to login, but we instead create SSH keys and install those in both systems so that we can carry out secure communications between two systems.

Creating ssh keys:

$ ssh-keygen -t rsa -b 2048

 It creates two files (private and public keys). If already created, then we do not need to create.

Copy these keys to the remote host.

$ssh-copy-id root@remote.frietec.com


Now we can connect(ssh) without need  of password.

$ssh root@remote.frietec.com

2) Executing scripts into remote system

$ssh root@remote.frietec.com 'bash -s' < SCRIPT_TO_RUN 

The script runs on the remote system. We can send the parameters if we need after the script. 

Until now, we executed local script into the remote system. In our case, the script creates backup files and these files are in remote system.



3) Transfer backup files into your local system

This is important because we need to securely transfer our files into our local system. So, we use secure copy tool (scp) to securely download the files. 

$scp root@remote.frietec.com:/opt/backups/*.bkp /backups/

Thats it. We have to implement the combined script as a cronjob, then we can get periodic remote backup into your local system. We can implement notification system also.

Notes

1) There are several possibilities to carry out this task. Here you have more control over your work, but there is another tool called rsync which is also promising which can replace scp tool which we have used here.

2) Since we have implement autologin to the remote system, you have to be sure the the locale system is secure enough, otherwise, anybody can reach to the remote system.

 3) If you have Jenkins, then it is quite easier to define task and run it periodically. And we can do more using jenkins.
 






Wednesday, October 25, 2017

hashCode and equals, why they are needed in Java?

All object in java are inherited from Object class, this Object has two methods hashCode() and equals(Object obj). We normally do not care about why these are made and what is the significance of these methods to derived objects.

To begin with, if we have Java objects and wanted to compare those object whether they are same or not. Primitive type values can be directly compared for equality with == operator. What if we use the same operator to check equality to Java object, then we are messed up. That is because the == operator compares the reference values of the objects. That means if two objects reference to the same memory address, they are equal. To clearify this, lets take example:

int a=10;
int b=10;

a==b  //true

Integer a=new Integer(10);
Integer b=new Integer(10);

a==b //false

First example give true because we are comparing values and they are same. In case example, they are object and the references of the objects are compared, although the values are same, they are NOT equal!!!


In many problem scenario, we need some mechanism so that the objects are evaluated based on their properties. In the above example, the evaluation of a and b should give the same result because they represent same, although they are different objects. For that we need to use equals() method of Object.


public boolean equals(Object object)

So, we override and implement this method to our object so that we can evaluate objects for equality based on their contents. So, two objects are evaluated as equal if this method return true.  

public native int hashCode()

When we override equals method, we MUST override hashCode method also. Why?

[Because a violation of the general contract for Object.hashCode will occur, which can have unexpected repercussions when your class is in conjunction with all hash-based collections.]

So, the rule is if equals() returns true, then their hashCode should ALWAYS be equal. But the other way might not be true. That is, if equals() returns false, it is not necessary that the two objects have different hashCodes.


How to create hash code?


How to implement equals method?



Wednesday, September 27, 2017

Enabling SSL in Tomcat


To install and start tomcat server is a really straight forward, but to run it securely needs some extra configuration. In this article I am going to describe the steps needed to enable encryption in tomcat server so that the communication between client and server is being carried by encrypting the data traffic, and nobody in between client and server can read the information.

Creation of KeyStore

The first and foremost requirement to implement SSL is creation of keystore file. The documentation says only three formats are supported  (JKS, PKCS11 or PKCS12) and I am gonna use JKS format because it is java standard keystore and can be created using keytool commands that comes with Java installation. 

So, lets create keystore. Just execute the command, it creates a jks file with private key and certificate. 

keytool -genkey -alias tomcat -keyalg RSA -keystore tomcat.jks -storepass ***** -validity 3650

Please the not keystore password used while creation. This is needed in tomcat configuration. Yes, tomcat.jks should be placed in a very secured location in the server. 


Configuration

After creation of keystore file, the next step is to copy this file to the server. It is best practice to copy it in conf folder of tomcat installation directory. 

So, we go to tomcat installation directory. In conf folder there, we open server.xml file where can enable SSL and provide the keystore file location along with keystore password. 


So, basically, we add the following connector element in  service element:
<Service name="Catalina">
.
.
<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
        maxThreads="150" SSLEnabled="true" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" keystoreFile="conf/tomcat.jks" keystorePass="****" /> 
.
.
.
</Service>


Limiting SSL Usage

Obviously, we want to disable plain text communication after enabling SSL. So far we have configured, supports bot encrypted and plain communication. So, we disable plain text communication. 

Now, we add the following lines at the end of the file inside tags. 

   
    <security-constraint>
    <web-resource-collection>
        <web-resource-name>secure-tomcat-app</web-resource-name>
        <url-pattern>/*</url-pattern>
    </web-resource-collection>
    <user-data-constraint>
        <transport-guarantee>CONFIDENTIAL</transport-guarantee>
    </user-data-constraint>
</security-constraint>

   


Restart Tomcat server and now the connection to the tomcat server is always secure. 



Monday, September 18, 2017

Ubuntu Basics

In this article, I am going to write some basic Ubuntu operations which we need day to day. I am gonna describe everything as a list and this list goes on updated.

Font Installation

One of the frequently facing problem we get is the required fonts are not installed and we have to install by ourselves. Yes, there are several ways to install fonts in Ubuntu system. First of all, we have to know where the fonts are located and what is the purpose of the fonts.

First of all, we have to know about user defined fonts i.e. every every defines their fonts in their home directory:

~/.fonts

The fonts in this directory are only for the specific user and not available globally.

If we have to make the fonts globally available, then we have to copy the fonts into other locations.The locations can be defined in

/etc/fonts/fonts.conf 

The default directories are
/usr/share/fonts, 
/usr/local/share/fonts 
and
~/.fonts

So, if we copy directly into the /usr/share/fonts or /usr/local/share/fonts to make the font available for all users. Of course, you have to be administrator to copy the fonts into the above-mentioned directories.

Here is the sample fonts to test. 

Sample Fonts


After copying into the corresponding directories, we have to run the following commands:

sudo fc-cache -fv

If system is rebooted, we do not need to execute above command, fonts are loaded automatically.

After installation is complete, we check if the fonts have been successfully installed.

sudo fc-list |grep verdana

If the font is successfully installed, then it show the newly installed font.

Note: we need to restart the application which are using the font to reflect the newly installed fonts.

Localization

This is one of the common problems I have faced. Basically, when using the German alphabets with umlauts, they are not properly displayed because of unicode related problems.

Here, I will try to explain as simple as possible to work around with that:

  • Check the current local settings:
      $ locale

  • See the available locales
      $ locale -a  

  • If locale is not in the list, then it should be generated(installed)
     $ locale-gen fr_FR.UTF-8


  • To regenarate locales, 
     $ locale-gen  
  • The default settings are stored in /etc/default/locale file. 
      We can directly change the contents of this file. Or we can use the command update-locale.
      $ update-locale LANG=de_DE.UTF-8

Note: the supported locales are located in the file /usr/share/i18n/SUPPORTED.

Shorcut Method:  

From terminal, run the following command, and select the required locales. That does everything we needed!

$ sudo dpkg-reconfigure locales


Yes, it is recommended to restart the system to properly load the locales.

Quickly Test USB Boot

  • Install qumu

            sudo apt install qemu

  • Test ISO
      qemu-system-x86_64 -cdrom filename.iso
  • Test USB
      qemu-system-x86_64 -hda /dev/sdx



Date Time Settings


This section describe how can we set date and time in Ubuntu system from command terminal. The auto update of date time is carried out through NTP server, the configuration of datetime sync server is a different topic. We simply set the date and time here.

First of all, there are two clocks: 1) System clock, 2) Hardware clock

Here, the date time set in hardware clock is what we see the time in Bios, and if Bios time is not correct, then the system time could be also incorrect because when system boots, it gets time from Hardware clock.

So, if hardware clock is wrong by any chance, the system time also gets wrong.

1) See system date and time
$ date
2) Set system date and time
$ sudo date -s '2017-10-04 16:31:32'
3) See hardware clock time
$ sudo hwclock
4) Set hardware clock time from system time
$sudo hwclock -w
5) Set system time from hardware clock time
$sudo hwclock -s

In the above commands, -w can be replaced with --systohc and -s can be replaced with --hctosys.

Enable Remote Desktop in Ubuntu Server  

If we install standalone Ubuntu server, and want it to be accessible via remote desktop, we have to do some extra task. Since, ubuntu-server comes without any desktop application, i.e. no GUI possible, only terminal. Thats cool if you are familiar with command line terminal. If you still want to make your server available via remote desktop, we have to install the desktop application in the server. The program we need for remote desktop is xrdp.

So, we install it using terminal as follows:

sudo apt update
sudo apt upgrade
sudo apt install xrdp
sudo apt install ubuntu-mate-core ubuntu-mate-desktop 
echo mate-session >~/.xsession
sudo service xrdp restart


Then we are ready to connect using rdesktop from Linux and remote desktop from windows based systems.





Saturday, August 19, 2017

More on Software Testing

In this article, I am going to write more about software testing. The quality control of a software product is carried out in different software testing methodologies. The software testing is a phase that should not be neglected, because if a defect is found at the time of software delivery, the cost will be increased by 10 times, and will be 20 times more at the maintenance phase. So, it is recommended to carry out testing when the software development begins.

Software Phases

There are 5 main phases of software development:

1) Documentation: Requirement analysis, design document, test documents
2) Coding/Execution: The development phase
3) Testing: Different testing methodologies
4) Deployment : The software is delivered to the customer
5) Maintenance: After deployment, if any failure is detected.

Software Testing

The software testing can be broadly categorized into two categories:

A. Blackbox Testing: Tested overall functionalities of the software without knowing the details of the implementation, or design.
B. Whitebox Testing: White box testing also considers the implementation details, software design, database design.


Defect & Failure: If a defect reached to end-user, it is failure. 60% defects are prone to exist in design phase and 40% defects in development phase.

Testing Process

The testing process

Unit Testing => Integration Testing => System Testing => Acceptance Testing

Acceptance testing is carried out alpha-testing(carried out in development site) and beta-testing(tested by actual customer).


When a bug is found or some module is changed/added, we have to carry out
1) Confirmation Testing: Confirm that the bug/defect is fixed.
2) Regression Testing: Testing if all other parts of software( or module) are working

More...

We must be also concerned with

1) Defect cascading: A defect can be propagated to other modules and affects those modules too, this is called cascading.
2) Cohabiting Software: When software is installed in actual end-user machine, there might be other software installed which using the same shared libraries or resources. In this scenario, the other software need to be carried out testing.