Sunday, June 19, 2016

How to use Non-English Languages with WSO2 API Manager

Hello All,
When I am writing this blog WSO2 API Manager 2.0.0 is about to release and we have already released our beta version. I thought to let you know about one interesting feature of it that comes with this release. That is non- English character support in WSO2 API Manager Store and Publisher. 

Specifically, now with WSO2 API Manager 2.0.0 you will be able to use your own language to create APIs and their descriptions etc shown in following image. 

API published which has created with non-English characters


You can sign up with non-English usenames, create applications in API Store with any of the languages you want and can generate the application access tokens. No worries !!!

Application with non-English

Self Signup with non-English

Login with Non-English username 

Now let's see what you need to configure to get this feature in your API Manager environment. 

Embeeded H2


With the embedded H2 database that comes with WSO2 API Manager you can do it without a hassle.  That means there you have nothing to configure. Just create the databases. Run your API Manager product with -Dsetup and you got it.

MySQL


With MySQL versions below 5.7, when creating the databases you need to give the character set as UTF-8. Assume that you are using two databases here; one is for apimgt database (WSO2AM_DB) and other is for registry database (WSO2CARBON_DB). So, the following steps show how you can do this.

1. Start MySQL Server and login with the credentials.
2.Create the apimgt database with UTF-8 character set and collation as follows.

                create database apimgtdb CHARACTER SET utf8 COLLATE utf8_Unicode_ci;

3. Create Registry database with UTF-8 character set and collation as follows.

               create database regdb CHARACTER SET utf8 COLLATE utf8_Unicode_ci;


4. Then Configure your databases with WSO2 API Manager. 

      i. Open <AM_HOME>/repository/conf/datasources/master-datasources.xml file and configure your database as shown bellow replacing the default configuration comes for the WSO2AM_DB and WSO2CARBON_DB.


WSO2CARBON_DB

     <datasource>
       <name>WSO2_CARBON_DB</name>
       <description>The datasource used for registry and user manager</description>
       <jndiConfig>
           <name>jdbc/WSO2CarbonDB</name>
       </jndiConfig>
       <definition type="RDBMS">
           <configuration>
               <url>jdbc:mysql://localhost:3306/regdb?characterEncoding=UTF-8</url>
               <username>mysql_username</username>
               <password>mysql_password</password>
               <driverClassName>com.mysql.jdbc.Driver</driverClassName>
               <maxActive>80</maxActive>
               <maxWait>60000</maxWait>
               <minIdle>5</minIdle>
               <testOnBorrow>true</testOnBorrow>
               <validationQuery>SELECT 1</validationQuery>
               <validationInterval>30000</validationInterval>
           </configuration>
       </definition>
     </datasource>


There make sure to give the character Encoding as UTF-8 as highlighted above in the connection URL to support your database connection for Unicode character encoding.

WSO2AM_DB

     <datasource>
            <name>WSO2AM_DB</name>
            <description>The datasource used for API Manager database</description>
            <jndiConfig>
                <name>jdbc/WSO2AM_DB</name>
            </jndiConfig>
            <definition type="RDBMS">
           <configuration>
               <url>jdbc:mysql://localhost:3306/apimgtdb?characterEncoding=UTF-8</url>
               <username>mysql_username</username>
               <password>mysql_password</password>
               <driverClassName>com.mysql.jdbc.Driver</driverClassName>
               <maxActive>80</maxActive>
               <maxWait>60000</maxWait>
               <minIdle>5</minIdle>
               <testOnBorrow>true</testOnBorrow>
               <validationQuery>SELECT 1</validationQuery>
               <validationInterval>30000</validationInterval>
           </configuration>
       </definition>
     </datasource>


Now you have done with the configuration. Start WSO2 API Manager with -Dsetup. 
Well, Now you can create APIs with non-English characters and Publish to the API Store like in above picture. And also you can create applications, get self sign up etc with using your own language.

How can you do this for your existing database ?

The steps that I have shown above are for a situation where you are beginning with API Manager 2.0.0 database scripts. Well, what if you have the database with data already ? or you want to migrate with existing data ? Ok, I have a solution for that as well. 

Steps :

1. Run the following alter scripts against your WSO2AM_DB.

ALTER TABLE AM_APPLICATION CHANGE UUID UUID VARCHAR(254);
ALTER TABLE AM_SUBSCRIPTION CHANGE UUID UUID VARCHAR(254);
ALTER TABLE AM_API CHANGE API_TIER API_TIER VARCHAR(254);
ALTER TABLE AM_SUBSCRIPTION_KEY_MAPPING CHANGE KEY_TYPE KEY_TYPE VARCHAR(255);
ALTER TABLE AM_APPLICATION_KEY_MAPPING CHANGE KEY_TYPE KEY_TYPE VARCHAR(255);

2. Get a back up of the database.

3. Create a new database using the following command.

create database <new_database_name> CHARACTER SET utf8 COLLATE utf8_Unicode_ci; 

4. Restore the database backup which has previously taken to the newly created database.

5. Point the new database name in the master-datasources.xml file in <AM_HOME>/repository/conf/datasource directory under particular DB'connection URL.

6. Repeat the steps 2,3,4,5 against the WSO2CARBON_DB also.

7. Start the server with -Dsetup.

Ex. In Ubuntu :         sh wso2server.sh -Dsetup
           
You are done.

MySQL 5.7+


When using MySQL 5.6 or previous versions, you need to keep the VARCHAR length of the data types limited below 256 which are used as PRIMARY keys, FOREIGN Keys or UNIQUE Keys. That is due to a limitation exists in MySQL  product. But with MySQL 5.7 and above versions this issue has fixed. Therefore you can have your database with non-English stuff just by creation the database with UTF-8 encoding type.
If you are using MySQL 5.7+, Follow the below steps to get non-English character support.

1. Start MySQL Server and login with the credentials.

2.Create the apimgt database with UTF-8 character set and collation as follows.

                create database apimgtdb CHARACTER SET utf8 COLLATE utf8_Unicode_ci;

3. Create Registry database with UTF-8 character set and collation as follows.

               create database regdb CHARACTER SET utf8 COLLATE utf8_Unicode_ci;


4. Configure your master datasources.xml file same as described in above configuration under MySQL.

5. Start the server with -Dsetup.

If you are using an existing database, you do not need to run alter scripts against the data as like in MySQL 5.6 and previous versions. You can follow same steps described here and backup your database into the new database created.

PostgreSQL


By default PostgreSQL supports UTF-8 character encoding. Therefore if you are using PostgreSQL you need not to do anything. You already have the support for non-English in your API Manager.


Oracle and Oracle RAC


Although you need to configure Oracle for UTF-8 character support for normal databases you use of you haven't done it in it's schema, the script for Oracle database which comes with WSO2 API Manager supports it by having the VARCHAR2 as the character datatype. Therefore for your existing database and also for new databases that you create with Oracle with API Manager 2.0.0, you have the chance to use non-English languages. Same like in PostgreSQL or H2 database you just need to create the database according to the documentation.

In this article I focused on elaborating the configuration procedure that you have to follow to get the non-English support feature. Hope you feel much more familiar with WSO2 API Manager with your own language. With new localization support feature that comes with WSO2 API Manager 2.0.0, it will be a more fantastic experience.

ok then, Until we meet again...