Friday, October 30, 2015

Enabling User Password Recovery for WSO2 API Manager with WSO2 Identity Server - Part 1

Any of the WSO2 API Manager versions released so far does not have the capability of user password recovery. If the API Manager users completely forget their password, they do not have a way to recover it. However WSO2 Identity Server have this feature implemented as a service called UserInformationRecoveryService. This blog post will explain how to use this password recovery feature of WSO2 IS in WSO2 API Manager. I will demonstrate the service calls through soap UI here.

User Information Recovery API

WSO2 Identity Server has given an API to recover forgotten/lost password in several ways. Those are as follows.

1. Recover with notification : password recovery by using email based notifications. An email is sent to the user's email account through axis2 mail transport.
2. Recover with Secret Questions : By obtaining answers for some challenge questions by invoking UserIdentityManagementAdminService.
3. User Account Recovery : helps to recover username of the account if the user has forgotten it.

All the above recovery processes are secured with captcha verification. In this blog post I will describe how the first option can be enabled in WSO2 API Manager 1.8.0.

Installing WSO2 Identity Server Feature in API Manager

First of all if we want to have the password recovery capability in WSO2 API Manager you have to install the Account Recovery and Credential Management Feature of WSO2 Identity Server in WSO2 API Manager. Please follow the below steps to install the feature.

Steps : 
1. Start the Server and log in to the Management Console of WSO2 API Manager

For more information on downloading WSO2 API Manager, installing it and about installation prerequisites please refer WSO2 API Manager Documentation.

2. Navigate to the Configuration menu and go to Features --> Repository Management and add repository with a relevant name.  e.g. credential-mgt

3. Give the following as the repository URL This is the related P2 repository for the feature.

4. Click on Find the features and select Account Recovery and Credential Management 4.2.2 and click install.

5. Accept Licence Agreement and finish installing the feature.

Configuring WSO2 API Manager for User Information Recovery

After installing the feature follow the documentation on Recover with Notification in WSO2 Identity Server and configure WSO2 API Manager for User Information Recovery. Consider <IS_HOME> mentioned in the documentation as <APIM_HOME> which is the installation directory of product API Manager.

Now WSO2 API Manager has the capability of calling the UserInformationRecoveryService.

Getting the WSDL of UserInformationRecoveryService

Steps : 
1. Stop the Server.
2. Go to <APIM_HOME>/repositiory/conf/carbon.xml and make <HideAdminServiceWSDLs> property false as follows.

2. Start the Server and access the WSDL of UserInformationRecoveryService through following link.

Now what you have to do is calling the relevant service methods in order. Follow the service methods mentioned in documentation and call the service methods.

To use this feature as an existing feature in WSO2 API Manager either you have to implement a jaggery module in API Manager Store or you have to create a separate web app to be redirected from within API Manager Store. however you can check how this is working using Soap UI.

Please go through my next blog post to see how you can call the UserInformationRecoveryService through Soap UI and change the password of a user.

Thursday, July 2, 2015

WSO2 API Manager Distributed Architecture : Configuration Tips (Ubuntu)

Hello all....!

I thought to write another blog on WSO2 API Manager. Well, you may have a basic understanding on what is WSO2 API Manager. If not you can go through my previous blog post or refer to the WSO2 API Manager documentation.

When I am writing this blog it is only two weeks after the WSO2 API Manager 1.9.0 release and now we are working on releasing the next versions of WSO2 API Manager. In the  version 1.9.0  I could work closely on the distributed Architecture of WSO2 API Manager.

There are lot of configurations to do when creating a distributed setup for WSO2 API Manager and you will get the chance to use several other WSO2 Products as well. Other than that you have to fulfill some prerequisites before installing the products.

The intention of this blog post is not to describe how to create an API Manager Distributed setup. To get that information you can refer to the WSO2 Documentation. My intention is to guide you in some configuration steps of the distributed setup to save your time finding here and there what to do and how to do by pointing out where are the places you might do mistakes and  how to work with the remote machines when configuring the instances.

First of all lets get a basic understanding on the distributed architecture of WSO2 API Manager.

API Deployment Distributed Architecture

There are four main components in the WSO2 API Manager distributed setup. They may be one instance per each or more than one instance  (clustered/not clustered) according to the requirement.

1. API Gateway  - This is the component which manage the API calls by securing them and scaling them.

2. Key Manager - Responsible for the Key-related security operations

3. Publisher - Instance which is used by API providers to publish APIs, share documents, provision keys and gather feedback on the features

4. Store - Instance which is used by the consumers to do self sign up,  to subscribe the APIs, invoke them and interact with the API publishers.

Other than that you need a DBMS (e.g. MySQL Server) to create the three databases described in following.

Databases :

There are three databases we need to have for the cluster deployment architecture. Those are,

  • API Manager Database (Information about APIs and API sbscription details)
  • User Management Database  (Information about users and user roles)
  • Registry Database (Shared information between Publisher and Store)

Single Sign On Login with WSO2 Identity Server (WSO2 IS)

To enhance the efficiency, accessibility and for a better user experience you are encouraged to Configure the Publisher and Store in your distributed setup. You can follow the WSO2 documentation for SSO configuration to accomplish this task after setting up the cluster. For this we use another WSO2 Product : WSO2 Identity Server which manage the identities across internal, Shared an SaaS services.

In this particular situation, as an example this allows the authenticated users to access API Manager Store without repeatedly authenticating if the user is authenticated in API Manager Publisher in WSO2 API Manager.

Publishing Runtime Statistics with WSO2 Business Activity Monitor (WSO2 BAM)

To collect the runtime statistics of WSO2 API Manager and to analyse them you are encouraged to use another WSO2 Product which has developed to aggregate, analyse data and present information about business activities. WSO2 API Manager use this product to configure publishing the run time statistics about API related activities. Follow the WSO2 Documentation to Configure WSO2 API Manager with WSO2 BAM to publish API runtime statistics.

Load Balancing with nginx

For load balancing we used WSO2 Elastic Load Balancer (ELB) earlier. Now we encourage the customer to use Nginx for load balancing. To do Nginx configuration follow the WSO2 Documentation.

Now lets move into the important tips.

1. Access remote instances 

When configuring a distributed setup for WSO2 API Manager you need to access the remote instances through terminal. To access the remote machine you may have a key included in a .pem file. First of all you have to give the read and write rights to the .pem file

<path_to_pem_file>/chmod 600

Then you access the remote instance using the following command.

ssh -i <pem_file_name>.pem <remote_host_name>@<remote_host_ip> 
e.g. ssh -i permission.pem

You will log into the remote machine and you can work with it using terminal. 

2. Coping product binaries to remote machines 

Since we are using remote machines and we have to use several software products it is required to download those files to the machines or trasfer them. You can download the products to the remote machines through terminal using the following command.

wget "<download_link_to_the_file>"

wget ""

But this is a time consuming task. Therefore the proper solution is to copy the files to the remote machine through the SSH connection you built. To do that we use secure copy commands which allow coping files through an SSH connection. For that also you need the .pem file.

scp -i <path_to_pem_file>/<pem_file_name>.pem  <path_to_file_in_your_machine>/<file_to_copy> <remote_host_name>@<remote_host_ip>:<location_to_be_copied>

e.g. scp -i /home/API-deployment-Architecture/permission.pem

3. Installing prerequisites 

For each of the WSO2 product to run there is a set of prerequisites to be installed in the Machine which it is running. In this setup we are using WSO2 API Manager, WSO2 Business Activity Monitor and WSO2 Identity Server. To get more information on prerequisits got through installation prerequisites.

You need to install all these through the terminal. Following are the steps to install them.

Install  Java : 
Follow my this blog post to install Java in the remote machine.

Installing Maven : 
sudo apt-get install maven

By running above command you can install maven in the remote machine. To check the version execute the following command.

mvn -version

installing ActiveMQ : 

download ActiveMQ :

tar -zxvf apache-activemq-5.6.0-bin.tar.gz

go to bin folder of ActiveMQ and run get the read write permission.
chmod 755 activemq

start ActiveMQ :
sudo sh activemq start

test the installation
netstat -an|grep 61616

Since you are installing the binary distribution not building the source you do not need to install SVN client and if you do not compile and run the product samples in the remote machine you do not need to install Apache Ant. You need to install the SVN client in the remote machine where the deployment synchronization is going to be configured. Use the following command for that.

sudo apt-get install subversion

4. Installing MySQL Server 

Since you have to configure the three databases described above you have to install a database server. Assuming that you will choose MySQL server I will give the configuration tips. You can follow my previous post to install MySQL Server in your remote machines.

5. Port Offset

In this cluster setup you will have to run more than one WSO2 product in the same cluster and also more than one WSO2 product in the same machine. To avoid the port conflicts you have to make sure setting the port offset correctly by editing the <PRODUCT_HOME>/repository/conf/carbon.xml file.


According to the value that you set it increment the ports used by the server starting from the 9443 which is the default port.

6. Who need which ?

Another thing you need to understand when setting up this distributed architecture is that which component need which component to be connected together to perform. We configure WSO2 BAM to publish run time statistics. It is required by Gateway, Publisher, KeyManager and Store. We configure Single Sign On which is required or affected by Publisher and Store. We need load balancing of we are going to balance the load in the gateway cluster which manage the API calls.
Considering the databases, API Manager database is needed by Publisher, Store and KeyManager. But gateway do not need it. User Manager database and Registry database is needed by all the four components in the cluster. Likewise when doing the configuration we should have it in mind who need which.

7. Configuring databases  - username, password, database names, coping mysql files - bam do not need, but creating the database is needed for all, apim and registry db shoud run the query. proper db name in both xml and db

8. Configuring API Manager for Stats

properly setting the IP addresses in the API Manager configuration in stat configuration is needed to avoid errors. According to that Event Receiver configurations and Data Analyser Configurations should have the URL with IP address where the WSO2 BAM instance is running. Statistics summary datasource should be the URL of the Statistical database and the username password should be the username and password of your MySQL server.

9. Locating Database Driver 

Another main important point that you may have missed is that locate/save the MySQL JDBC driver in the product. According to the setup describing here since we ar eusing MySQL server we need connector to perform the database calls. Therefore in every WSO2 product you are using here you need to place MySQL JDBC Driver jar file in <PRODUCT_HOME>/repository/components/lib folder.

12. SVN based deployment synchronization

In the distributed setup we used to cluster the Gateway nodes (some times the key Manager as well) to be in one internal cluster domain with a Manager node and a set of worker nodes. It leads to set up the load balancing configurations in the gateway. For that enabling deployment synchronization in the manager node is essential. Registry-based Deployment Synchronizer does not work for WSO2 products based on Carbon 4.2.0 onwards. We are using SVN here. The deployment synchronizer uses this configuration to identify the manager and synchronize deployment artifacts across the nodes of a cluster. WSO2 Documentation on configuring SVN based deployment synchronizer will follow you through this process.

13. Updating known hosts

Make sure to update the host IP addresses with the host names in the machines that you wish to access the instances of the cluster setup. To that open the "hosts" file in the etc folder in the Ubuntu machine you are using  add the IP addresses and the host names as indicated in the following image.

vi /etc/hosts

For that you must log into the system root.

14. Configuring the databases: 

There are three main databases in the cluster deployment setup that we need to configure. Those are API Manager database, Registry database and User Management database. If you are using MySQL server you should create the databases and run the mysql script on it to create the schema of the databases.

mysql> \. mysql.sql

API Manager database : <APIM_HOME>/dbscripts/apimgt/mysql.sql
Registry database : <APIM_HOME>/dbscripts/mysql.sql
User Management database : <APIM_HOME>/dbscripts/mysql.sql

Other than that you need to configure the Statistical database for publishing statistics where you only need to create the database. When running WSO2 BAM instance it execute the relevant queries and create the database schema.

MySQL remote settings:

For all the instances in the cluster setup there need to be one API Manager database, one Registry database and one User Management database. Therefore we should the set the configuration by binding the address where the database is located in each of the instance.

For that open the my.cnf file in /etc/myql directory and give the IP where the database is located as the bind address. Then log into the mysql root and grant the permission to the database users to read and write to the database.

Grant permission :


You can check the access from the remote host with this command.

mysql -u root -h -p

This will give the remote access to the databases in the cluster setup.

Other than that, make sure the database names, URLs and the username and password are correctly configured in the master-datasources.xml file where you add the XML configurations for the databases.

Concentrating on the above mentioned points you can reduce the time taken to do the configuration and can get better understanding about the cluster set up that you are implementing.

Tuesday, June 30, 2015

Install and Uninstall MySQL Server in Ubuntu LTS 14.04 Terminal

Installing MySQL Server

Here are the quick steps to download and install MySQL Server in Ubuntu through terminal.

step 1 : Installation
OpenUp a terminal and enter the following command.

sudo apt-get install mysql-server

It will install MySQL server and will ask a password during the process.

You can give a refereed password and re-enter it in the next window.

step 2 : Verification
After the above step MySQL should start automatically. To verify that enter the following command.

sudo netstat -tap | grep mysql

it will show the listening port of the Server and so on.

step 3 : Restarting in case

If the Server is not restarting properly restart the server eneringthe following command.

sudo /etc/init.d/mysql restart

Access MySQL Server

You can access the MySQL server through your terminal by entering following command.

mysql -u root -p

After entering your password the terminal switch into mysql> then you can use that interface to create databases and do other database operations.

Uninstall MySQL Server from Ubuntu

At times you may need to reinstall MySQL Server. For that you need complete removing it from the linux system. Enter the following commands one after the other to completely remove the MySQL Server from the system.

sudo apt-get remove --purge mysql-server mysql-client mysql-common

sudo apt-get autoremove

sudo apt-get autoclean

These will remove the MySQL Server Completely from the system. If you have issues still after these steps remove the mysql directory in the lib folder by running following command.

sudo rm -rf /var/lib/mysql

After that if you want to re install MySQL Server you can follow the above mentioned steps again.

Ting! :)

Saturday, May 9, 2015

Installing and Configuring Java in Ubuntu completely through terminal -Part 2 : Set JAVA_HOME

My previous post explained how to install java completely through terminal. After installing java you need to set the java path in your machine as a variable since various kinds of applications and servers require and request the JAVA_HOME in your machine. Therefore the path to JAVA should be correctly set.

I start this assuming that you have correctly installed java in your machine. If not have a look at part 1 post to correctly install java in your machine or in a remote server. 

To do this task through terminal, you will need vi or vim editor installed in your machine. If not do the following in the terminal.

1.First type apt-get command to install vim as following.

     sudo apt-get install vim

Give password credentials and get installed vim.

Now you can use vi commands in your terminal. 

The only thing you have to do to set the java path is editing .bashrc file and set JAVA_HOME.

2. Type the following command to open up .bashrc file.

     vi .bashrc

3. Go down to the bottom of the file and press the key "i" to ge the file editable.

4. Type following lines at the bottom of the file.

      #JAVA HOME
      export JAVA_HOME
      export PATH

These lines will add JAVA_HOME variable to the system properties. 
/usr/lib/jvm/java-6-oracle is the path to JAVA in my machine. This may vary according to the version that you have installed in your machine.

Set the PATH to the bin folder of JAVA. Export both variables.

5. press Esc to go back from the edit mode of the .bashrc file

6. press :w for changes to be written. Then the changes to the file will be saved.

7. pres :q! to quit the fie.

8. Now type the following command in the terminal to verify (print) the JAVA_HOME.


 It will print JAVA_HOME.

9. Type the following command in the terminal to verify (print) the PATH.

echo $PATH

It will print PATH.


You have successfully configured the java path in your machine now.

Installing and Configuring Java in Ubuntu completely through terminal -Part 1

This post will describe how to install java in Ubuntu OS completely through terminal. It is quick and easy. The importance is that you can use the same way to install and configure java in a remote machine as well if you have the privilege to access it.

Ok. Let's start.

I am using Ubuntu 14.04 LTS here. First of all log in to your machine and open up a terminal window with Ctrl +Alt + T. If you are going to install java in a remote machine login to that machine using ssh client.

Type the following line and press enter.

sudo apt-add-repository ppa:webupd8team/java

Then you will get following output. Through this you can install java 6, 7, 8 or later according to your preference.

Press Enter to continue.

Now you should download the package lists from the repositories and update them to get information on the newest versions of packages and their dependencies.

For that type : sudo apt-get update in the terminal and press enter. It will do this for all repositories and PPAs.

When it is done you are ready to install your preferred version of java.

For that type following.

sudo apt-get install oracle-java<version_number>-installer

e.g. sudo apt-get install oracle-java6-installer

The version number is 6, 7 or 8. According to that it will start the installation of java version.

Press Y to continue.
Then you will lead to the package configuration wizard. According to the java version that you have selected to install.

Agree the licence by pressing ok.

Press OK.
 If you want you can go to page and read the licence before this step.

Press Yes.

Then it will continue the installation.

Once it is done it will direct to the directory you were in the terminal.

Done. Now you can test whether java has installed in the machine using java -version command as shown in the following image.

If you have successfully installed JAVA in your machine you will receive the shown output.
It will show java version that you have installed, Java SE Run-time Environment built and the VM built with the mode.

Lets see how you can set the java path (JAVA_HOME) in your machine in Part 2 of this post.

See you in Installing and Configuring Java in Ubuntu completely through terminal  -Part 2.

Saturday, May 2, 2015

Invoking an API published in WSO2 API Manager using Chrome Advanced REST Client

This blog post describes how Chrome Advanced REST Client can be used to invoke an API published in API published in WSO2 API Manager.

WSO2 API Manager

WSO2 API Manager is one of the major open source products of WSO2. It facilitates the API providers with various features such as API creation, publication, versioning, monetization, governance, security etc. It provides a user friendly user interface as well as performance rich back-end. Customers can subscribe to the APIs published by the providers to consume them. You can find more details of WSO2 Manager from here and in API Manager documentation.

Get the latest version of WSO2 API Manager from here. Start the pack running the .bat file (for windows) or .sh file (for Unix and Mac os x) in the bin directory of the extracted pack. After the the server started you will get the URLs for Management console, publisher API and, Store API respectively. WSO2 API Manager documentation  will guide you to create user roles and log in to each of them.

Following is an example terminal output of running the file in a linux machine.

Publishing an API 

In WSO2 Manager 1.9.0 publisher API you are facilitated to Add an existing API by giving a swagger file or a swagger URL, or giving a SOAP endpoint (WSDL), or if you want design a new API.

You can follow the WSO2 API Manager documentation to add an API. Here I am using an API created to expose the Phone Validation Service by the Cdyne services provider. It has SOAP and REST interfaces. The endpoint is

This Phone Validation Service has two operations. Those are CheckPhoneNumber and CheckPhoneNumbers. In the following example I am using CheckPhoneNumber operation.

The API is created in three stages. Those are design stage, implementation stage and manage stage. At the third stage, you can save and publish it. Then the status of the API become published. if in case you want to change the state of the API, you can go to API Life cycle and change to one of the status provided there.

Invoking the API

After publish the API, log into API store. There you can see the published API in the store.

Go to the My Applications tab and create an application. You can specify the throttling tier you want. You can find more information in API throttling tiers in WSO2 API Manager from here

Now you have to subscribe to the API. Click on the Phone Verification API and go to the subscription page. 

There you can specify your created application and the throttling tier you want to and subscribe to the API by clicking on the subscribe button.

When clicking on it you will be asked to "Go to My subscriptions". Click on that button and go to My Subscriptions Tag and generate the keys for the application. For that click on the "Generate Keys" button. 

Now you have taken an Access token for the application to invoke the API. Now click on the API and go to the API console Tab in the UI. There you can see the set Request Header you got.

Well. Now it is the time to start Chrome Advanced REST Client. 
Go to the following link and Launch the app. 

Now you can see the Following UI of the REST Client.

Give following details and click send. Here I am going to send a request with GET method.

URL : https://<your IP>:<port>/phoneverify/<version>?PhoneNumber=<sample_phoneNumber>&LicenseKey=<0_or_licence_key_you_have> 
example : https://<your IP>:9443/phoneverify/1.0.0?PhoneNumber=0112344567&LicenseKey=0 

Key : Authorization  (select from the drop down list)

Method: GET

Value: Bearer<space><Access_token>
           example : Bearer 52cc104b661dc27f76e14aa1f739e7ba

After sending the request on success you will get the following response. 

It gives the response by invoking the Phone Verification API. Validity of the phone number, if not valid the original phone number, Clean number, Whether it is wireless or not are the information provided here.


You have invoked the Phone Verification API published in WSO2 API Manager.

For a unit time period (minute) API Manager allow to send a particular number of requests predefined by the throttling tiers. 
  • Bronze      - one request per minute
  • Silver        - 5 requests per minute
  • Gold          - 20 requests per minute
  • Unlimited  - unlimited requests per minute

If you exceed the number of requests send you will get the following message in the Chrome Advanced REST Client interface.

<amt:message>Message Throttled Out</amt:message>
<amt:description>You have exceeded your quota</amt:description>

Then you can wait for that particular time period and send number of requests permitted in a minute until the validity time of the access token expires.

Thats All!