Generate REST API from mysql database in less than 5 minutes

Representation State Transfer is the new industry standard of building technology neutral API’s. REST API are essentially web services to read or transfer data. REST API’s can be build on top of most relational databases. Mysql being the most popular database around. If you have nodejs and mysql already installed, you can build a REST API for your mysql db in less than 2 minutes 🙂

Before we begin we must have nodejs and mysql running on the server. If you don’t have nodejs, you can check my other article on how to install node.js on Cent OS Linux : https://easyoradba.com/2018/05/08/install-nodejs-on-centos-6-7/

And to quickly install mysql refer to my another of my blog posts 🙂 https://easyoradba.com/2018/05/18/install-mysql-database-along-with-phpadmin-on-centos-6-7/

Once you have mysql and nodejs setup. You can install and spin up  REST API’s doing the below steps

1.Install xmysql using node package manager

$ npm install -g xmysql

2. Create a new user, a new test database and a new table with few records

mysql -u root -p

mysql> GRANT ALL PRIVILEGES ON *.* TO ‘test’@’localhost’ IDENTIFIED BY ‘test123’;

mysql> CREATE DATABASE easyoradba;

mysql> USE easyoradba;

mysql> CREATE TABLE emp ( id smallint unsigned not null auto_increment, name varchar(20) not null, constraint pk_example primary key (id) );

mysql> INSERT INTO emp ( id, name ) VALUES ( null, ‘SHADAB’ );

mysql> select * from emp;
+—-+——–+
| id | name |
+—-+——–+
| 1 | SHADAB |
+—-+——–+
1 row in set (0.00 sec)

mysql> exit;

3. Generate the REST API for db ‘easyoradba’ using xmysql

$ xmysql -h localhost -u test -p test123 -d easyoradba -n 8899 -r 192.168.1.46

Generating REST APIs at the speed of your thought..

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – –

Database : easyoradba
Number of Tables : 1

REST APIs Generated : 19

Xmysql took : 0.3 seconds

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – –

-h hostname of your mysql DB server

-u username of mysql

-p password for mysql user

-d DB name for which API needs to be generated

-n port number to create the REST service

-r your server IP, if left blank it will create with default value localhost

4. Check the REST API service and Call it from Postman

Go to browser and check the ip with defined port number : http://192.168.1.46:8899/

[{“resource”:”emp”,”routes”:[{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/describe”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/count”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/groupby”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/distinct”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/ugroupby”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/chart”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/aggregate”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/findOne”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/autoChart”},{“httpType”:”post”,”routeUrl”:”http://192.168.1.46:8899/api/emp”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp”},{“httpType”:”post”,”routeUrl”:”http://192.168.1.46:8899/api/emp/bulk”},{“httpType”:”delete”,”routeUrl”:”http://192.168.1.46:8899/api/emp/bulk”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/bulk”},{“httpType”:”put”,”routeUrl”:”http://192.168.1.46:8899/api/emp”},{“httpType”:”patch”,”routeUrl”:”http://192.168.1.46:8899/api/emp/:id”},{“httpType”:”delete”,”routeUrl”:”http://192.168.1.46:8899/api/emp/:id”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/:id”},{“httpType”:”get”,”routeUrl”:”http://192.168.1.46:8899/api/emp/:id/exists”}]},”http://192.168.1.46:8899/api/tables”,”http://192.168.1.46:8899/api/xjoin”,”http://192.168.1.46:8899/api/dynamic”,”/upload”,”/uploads”,”/download”]

Open Postman and call the URL with GET method to fetch the data

Screen Shot 2018-05-18 at 2.17.30 PM

 

And voila ! There you have it an REST API running on top of a mysql database in less than 5 minutes :)))

 

 

 

Install mysql database along with phpadmin on CENTOS 6 7

Mysql is the most popular open database and is the M in the LAMP stack (others being Linux-Apache-PHP)

If your on CentOS and your linux server is connected to the internet installing mysql is just a few terminal commands

1. Install the LAMP stack via yum

$ yum install httpd mysql-server php php-mysql

2. Configure the LAMP Stack

$ service httpd start

Now you should have a webserver up and running on http://localhost/

3. Start and secure your mysql database by executing below script

$ service mysqld start

$ mysql_secure_installation

Securing your database is very important, specially if your deploying a productions database

4. Install phpmyadmin

Install the EPEL Repo, if your on CentOS 6

$ wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
$ rpm -ivh epel-release-6-8.noarch.rpm

if your on CentOS 7

$ wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
$ rpm -ivh epel-release-latest-7.noarch.rpm

Now install phpmyadmin

$ yum install phpmyadmin

If you want access to phpmyadmin from other computers edit below file and add the host ip’s

$ vi /etc/httpd/conf.d/phpMyAdmin.conf

Require ip <other workstation ip>

Allow from <other workstation ip>

Restart httpd services

$ service httpd restart

You can now access using below URL

http://localhost/phpmyadmin

Enter your root username and password and you now have a nice GUI to manage your mysql instance

Install Oracle Apex 5.1.x on ORDS 3,17.4, 18.1

Oracle Apex is a modern framework to quickly develop and deploy web apps sitting on top of Oracle Database. Apex can be run directly from the database engine itself using a component called Mod PL/SQL. It is one of the easiest way to run Apex, but it is not a production ready setup. In a production scenario your database server should not be internet facing. The ideal design is a reverse proxy server facing the internet and a webserver sitting behind it and the DB server behind a firewall isolated from any public network.

Oracle has a modern gateway to do this, it is called ORDS( previously called Apex Listener). It stands for Oracle Rest Data Services. Not only it can help you host your apex instance in a Java web container like Tomcat, Glassfish, Weblogic but also it gives your developer access to build enterprise scale data access API’s. It can help you build your front-end in the modern JavaScript instead of the traditional programming languages. You can read about it more here : https://blogs.oracle.com/newgendbaccess/why-use-rest-and-ords-to-transform-your-oracle-database-into-a-restful-api-service

architecture big

Now to setup ORDS to host your Apex instance first we need to make sure you have 3 components

1. Oracle APEX itself, though from ORDS 17.1 it is not compulsory to have Apex installed, but i always prefer having it. Download the latest version from here : http://www.oracle.com/technetwork/developer-tools/apex/downloads/index.html

2. A Servlet web container like Tomcat, Glassfish or Weblogic. I prefer to use Tomcat since it is the easiest to setup, is open source and is quite lightweight. Apache tomcat can be downloaded from here  : https://tomcat.apache.org/download-90.cgi

3. ORDS, this is the most important component in this setup. Oracle has recently changed it’s numbering scheme. So it jumped from ORDS 3 to ORDS 17.4 and ORDS 18.1. It is now done using YY.MM naming convention. It can be downloaded from here : http://www.oracle.com/technetwork/developer-tools/rest-data-services/downloads/index.html

Let’s  Start with Installing Apache tomcat first on you server

                             1.  INSTALL APACHE TOMCAT

1. Create a user tomcat on the linux server which will be your webserver

useradd tomcat

2. Untar the file in the home directory or create a separate directory. To have a well defined structure i always prefer to have a directory structure like /u01/tomcat

tar xvf apache-tomcat-9.0.8.tar.gz

This will create the directory ‘apache-tomcat-9.0.8’ inside

go to the .bash_profile of tomcat user and add below for the home directory

PATH=$PATH:$HOME/bin:$CATALINA_HOME/bin
CATALINA_HOME=/u01/tomcat/apache-tomcat-9.0.8
export CATALINA_HOME
export PATH

. .bash_profile

3. Start Stop Apache Tomcat 9 snd enable Gui Access

sh $CATALINA_HOME/bin/shutdown.sh

sh $CATALINA_HOME/bin/startup.sh

You can now access the console of tomcat using http://localhost:8080/

To enable access to manager app and manager app gui. Add below to lines to file tomcat-users.xml

vi $CATALINA_HOME/conf/tomcat-users.xml

<role rolename=”manager-gui”/>
<user username=”tomcat” password=”Abc1234$#” roles=”manager-gui”/>
<role rolename=”admin-gui”/>
<user username=”tomcat” password=”Abc1234$#” roles=”admin-gui”/>

Save and restart Tomcat

                        2.   INSTALL ORACLE APEX 5.1.x

This installation has to be done on the DB Server. Copy your Apex zip file to Oracle Home Directory. Remove the existing Apex directory and unzip the fil

1.Install Apex Binary and Create Tablespace and Schema

cd $ORACLE_HOME

rm -rf apex/

unzip apex_5_0_1_en.zip

cd $ORACLE_HOME/apex

sqlplus “/as sysdba”

SQL> CREATE TABLESPACE APEX  DATAFILE ‘/u01/oradata/orcl/datafiles/apex01.dbf’ SIZE 1G AUTOEXTEND ON NEXT 10M;

SQL> @apexins.sql APEX APEX TEMP /i/

APEX – Tablespace to hold Apex Schema and its associated files

TEMP –  Temporary Tablespace

/i/ – It is  your image directory

Change your Admin password

SQL> @apxchpwd.sql

Create APEX_LISTENER and  APEX_REST_PUBLIC_USER

SQL> @apex_rest_config.sql

If you want to run the PL/SQL gateway continue to the next step else you can skip this and go to Step 3. directly

SQL> @apex_epg_config.sql /u01/app/oracle/product/12.2.0/dbhome_1

‘/u01/app/oracle/product/12.2.0/dbhome_1’ is the home directory of Oracle Database

SQL> ALTER USER ANONYMOUS ACCOUNT UNLOCK;

Check the HTTP Port,

SQL> SELECT DBMS_XDB.gethttpport FROM DUAL;

If this is 0 and you need to to install Apex with ORDS then goto Step 3. else continue and get the port

SQL> EXEC DBMS_XDB.sethttpport(8080);

SQL> SELECT DBMS_XDB.gethttpport FROM DUAL;

8080

The URL for Apex application now should be accessible from Web Browser http://localhost:8080/apex/apex_admin

              3. ORACLE REST DATA SERVICES (ORDS) CONFIGURATION & INSTALLATION

After downloading the ORDS to the server, go to the server where you installed Tomcat earlier

1.Unzip the ords zip file

cd /u01

mkdir -p /u01/ords

mkdir -p /u01/ords/conf

unzip ords.18.1.1.95.1251.zip

2. Login to DB Server SQL*PLUS and unlock the Apex  listener usernames and password to ensure ORDS installation goes smoothly

SQL> ALTER USER SYS IDENTIFIED BY abc123 ACCOUNT UNLOCK;

SQL> ALTER USER APEX_LISTENER IDENTIFIED BY abc123 ACCOUNT UNLOCK;
SQL> ALTER USER APEX_PUBLIC_USER IDENTIFIED BY abc123 ACCOUNT UNLOCK;
SQL> ALTER USER APEX_REST_PUBLIC_USER IDENTIFIED BY abc123 ACCOUNT UNLOCK;

SQL>ALTER USER ORDS_PUBLIC_USER IDENTIFIED BY abc123 ACCOUNT UNLOCK;

If last SQL command fails, it is because there is no ORDS schema yet.

3. Configure ORDS for Database connectivity. Go back to WebServer and there will be a ords.war file in /u01/ords

Edit the /u01/ords/params/ords_params.properties and set the parameters of your DB server and instance. Make sure there is network connectivity between DB Server and Web Server

cd /u01/ords

java -jar ords.war configdir /u01/ords/conf

java -jar ords.war

Don’t select Standalone option in the last, just press 2 and exit. Standlone server is a quick Jetty webserver build in ORDS. But its not the best production setup, hosting on Apache Tomcat is better.

mkdir $CATALINA_HOME/webapps/i/

Copy the image directory from the Apex installation in DB Server to Tomcat image directory.

cp -R $ORACLE_HOME/apex/images/* $CATALINA_HOME/webapps/i/

Copy the ords.war file from /u01/ords to Webapps directory of tomcat

cd /u01/ords
cp ords.war $CATALINA_HOME/webapps/

ORDS should now be accessible on URL http://localhost:8080/ords

Remember to remove the port 8080 configuration from Embedded PL/SQL gateway, if you did Step 2. and are moving to ORDS from PL/SQL Gateway

SQL> EXEC DBMS_XDB.sethttpport(0);

SQL> SELECT DBMS_XDB.gethttpport FROM DUAL;

0

And try URL again : http://localhost:8080/ords

Error 1 :

Oracle Rest Data Services 404 Not Found When Running APEX – PL/SQL Gateway Configured

Check APEX_PUBLIC_USER account and unlock it, stop and start Apache tomcat and try again. Or maybe an incorrect APEX_PUBLIC_USER password was entered during ORDS configuration. reconfigure the ords.war using below command:

java -jar ords.war setup

And copy new war file to webapps directory again and access URL again

http://localhost:8080/ords

Reference : Oracle Rest Data Services 404 Not Found When Running APEX – PL/SQL Gateway Configured (Doc ID 2048493.1)

Error 2:

Server isn’t redirecting oracle apex ords on Firefox

This page isn’t working
<hostname> redirected you too many times.

Try clearing the cookies and try again. Most likely close the browser windows and open and again and the error will go away

http://localhost:8080/ords

Reference: Problem Accessing APEX Application Using ORDS 3.0.9.348.07.16 (Doc ID 2280694.1)

Summary :

1.Steps to do when installing only PL/SQL gateway

1 > 2

2. Steps when upgrading from PL/SQL Gateway to ORDS

1 > 3

3. Steps when doing fresh new install of ORDS

1 > 2 > 3

Install NodeJS on Mac OS

To install NodeJS on a Mac the first thing you have to do is download Brew package manager for mac. Though you can install NodeJS via other methods but doing it with Brew is the easiest and least error prone method

1. Install Brew on your mac if you dont have it already. Open the terminal app and paste the below command (make sure you have the password for the user)

ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)”

2. Now the Brew package manager is installed. Open terminal and type below command

brew install node

3. Check Versions of Node and NPM

node -v

npm -v

 

4. Create a Demo Web server and Check. Open terminal and type below

vim demo_server.js

— Paste below Code in the file —

var http = require(‘http’);
http.createServer(function (req, res) {
res.writeHead(200, {‘Content-Type’: ‘text/plain’});
res.end(‘Welcome To EasyOraDBA’);
}).listen(3005, “127.0.0.1”);
console.log(‘Server is running at http://127.0.0.1:3005/’);

 

node demo_server.js

— Now open a Web browser and paste the URL http://127.0.0.1:3005/ And voila you have a NodeJS Web Server running —

 

Install NodeJS on CentOS 6 – 7

Install Latest NodeJS on CentOS 6 or 7
————————————–

1. Add Repo and Install NodeJS

For Stable Release:-

$ yum install -y gcc-c++ make
$ curl -sL https://rpm.nodesource.com/setup_8.x | sudo -E bash –

$ yum install nodejs

 

2. Check the version for Node and NPM

$ node -v

v8.11.1

$ npm -v

5.6.0

 

3. Create a Demo Server

$ vim demo_server.js

add below lines

var http = require(‘http’);
http.createServer(function (req, res) {
res.writeHead(200, {‘Content-Type’: ‘text/plain’});
res.end(‘Welcome To EasyOraDBA’);
}).listen(3005, “127.0.0.1”);
console.log(‘Server is running at http://127.0.0.1:3005/’);

4. Start the Node Server

[root@easyoradba CordovaApps]# node demo_server.js
Server is running at http://127.0.0.1:3005/

Go to your Browser and Paste the URL, your WebServer has been started on port 3005

Quick and Easy Integration with Twitter with Apex 18.1

Quick Integration with Twitter in Oracle Apex 18.1

https://blogs.oracle.com/apex/quick-and-easy-twitter-api-with-apex-181

 

ORA-01103: database name ‘MW’ in control file is not ‘MWTST’

This error most likely occurred when you duplicated a database and the DB_NAME parameter in spfile is not matching the one in the control file. Sometimes it so happened that after duplicating the database name in SPFILE with which you started the instance with is not matching the one in the control file.

Explanation:
=====================

The database name in the controlfile is not consistent with the db_name
parameter in the init.ora file.

Solution Step 1:

Change the DB_NAME parameter in spfile and bounce the database

STARTUP NOMOUNT

ALTER SYSTEM SET DB_NAME=ORIGINAL_DBNAME SCOPE=spfile;

SHUTDOWN IMMEDIATE;

STARTUP MOUNT;

ALTER DATABASE OPEN RESETLOGS;

If you get like below error

ORA-19838: Cannot use this control file to open database then have to recreate the control file from trace . Go to Solution 2:

See Oracle support Doc ID 1906009.1

 

Solution Step 2:

 

Now the only way to bring up the cloned DB is to re-create controlfile from trace. Login to the source database and run below commands.

SQL> alter database backup controlfile to trace;

Check location of controlfile which is created to trace; checking in ALERT LOG

ALTER DATABASE BACKUP CONTROLFILE TO TRACE
Backup controlfile written to trace file /u02/app/oracle/diag/rdbms/mw/mw1/trace/mw1_ora_146366.trc

Then Copy/Edit the Details of ControlFile from the block like below.

[code language=”sql”]

CREATE CONTROLFILE SET DATABASE “MW” RESETLOGS FORCE LOGGING ARCHIVELOG
MAXLOGFILES 192
MAXLOGMEMBERS 3
MAXDATAFILES 1024
MAXINSTANCES 32
MAXLOGHISTORY 9344
LOGFILE
GROUP 1 (
‘+DATAC1/mwtst1/onlinelog/group_1.366.854619221’,
‘+RECOC1/mwtst1/onlinelog/group_1.504.854619221’
) SIZE 100M BLOCKSIZE 512,
GROUP 2 (
‘+DATAC1/mwtst1/onlinelog/group_2.367.854619389’,
‘+RECOC1/mwtst1/onlinelog/group_2.505.854619389’
) SIZE 100M BLOCKSIZE 512,
GROUP 3 (
‘+DATAC1/mwtst1/onlinelog/group_3.368.854619411’,
‘+RECOC1/mwtst1/onlinelog/group_3.506.854619411’
) SIZE 100M BLOCKSIZE 512,
GROUP 4 (
‘+DATAC1/mwtst1/onlinelog/group_4.365.854619175’,
‘+RECOC1/mwtst1/onlinelog/group_4.507.854619175’
) SIZE 100M BLOCKSIZE 512
DATAFILE
‘+DATAC1/mwtst1/datafile/SYSTEM.467.971185463’,
‘+DATAC1/mwtst1/datafile/SYSAUX.439.971266529’,
‘+DATAC1/mwtst1/datafile/USERS.473.971266529’,
‘+DATAC1/mwtst1/datafile/UNDOTBS1.468.971185463’,
‘+DATAC1/mwtst1/datafile/UNDOTBS2.466.971185463’,
‘+DATAC1/mwtst1/datafile/MQ.357.971266521’,
‘+RECOC1/mwtst1/datafile/AUDIT_TBS.1787.971185463′
CHARACTER SET AL32UTF8
;

[/code]

Save it as control_file.sql

Ensure you check the Datafile location and name, it should be same as in what resides on your filesystem. This is specially true when you using ASM and OMF together.

 

Now start the instance

 

SQL> startup nomount

SQL> @control_file.sql

Control file created.

SQL> alter database open resetlogs;

Database altered.

SQL> select open_mode from v$database;

OPEN_MODE
——————–
READ WRITE

SQL> select open_mode,name from v$database;

OPEN_MODE NAME
——————– ———
READ WRITE MW

SQL> select * from v$controlfile;

STATUS
——-
NAME
——————————————————————————–
IS_ BLOCK_SIZE FILE_SIZE_BLKS
— ———- ————–

+RECOC1/mw/controlfile/current.2156.971360621
YES 16384 1878

Now change the database name in control file using nid utility

nid TARGET=system/****** DBNAME=MWTST SETNAME=Y

 

DBNEWID: Release 11.2.0.4.0 – Production on Wed Mar 21 14:27:18 2018

Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.

Connected to database MW (DBID=1312370394)

Connected to server version 11.2.0

Control Files in database:
+RECOC1/mw/controlfile/current.2156.971360621

Change database name of database MW to MWTST? (Y/[N]) => Y

Proceeding with operation
Changing database name from MW to MWTST
Control File +RECOC1/mw/controlfile/current.2156.971360621 – modified
Datafile +DATAC1/mwtst1/datafile/system.467.97118546 – wrote new name
Datafile +DATAC1/mwtst1/datafile/sysaux.439.97126652 – wrote new name
Datafile +DATAC1/mwtst1/datafile/undotbs1.468.97118546 – wrote new name
Datafile +DATAC1/mwtst1/datafile/users.473.97126652 – wrote new name
Datafile +DATAC1/mwtst1/datafile/undotbs2.466.97118546 – wrote new name
Datafile +DATAC1/mwtst1/datafile/mq.357.97126652 – wrote new name
Datafile +RECOC1/mwtst1/datafile/audit_tbs.1787.97118546 – wrote new name
Control File +RECOC1/mw/controlfile/current.2156.971360621 – wrote new name
Instance shut down

Database name changed to MWTST.
Modify parameter file and generate a new password file before restarting.
Succesfully changed database name.
DBNEWID – Completed succesfully.

 

Change PARAMETER in PFILE or in Memory

SQL> alter system set db_name=’MWTST’ scope=spfile;

System altered.

SQL> show parameter cluster

NAME TYPE VALUE
———————————— ———– ——————————
cluster_database boolean FALSE
cluster_database_instances integer 1
cluster_interconnects string

 

SQL> alter system set db_unique_name=’MWTST’ scope=spfile;

SQL> shutdown immediate;

SQL> startup mount

SQL> alter database open;

Database altered.

SQL> select open_mode,name from v$database;

OPEN_MODE NAME
——————– ———
READ WRITE MWTST

 

 

 

 

 

Playing with oracle DB 18c on-premises before official release