Here in this post I am going to discuss how to create a simple dashboard using WSO2 BAM gadget generation tool and package that dashboard in a BAM Toolbox.
CREATE EXTERNAL TABLE IF NOT EXISTS PhoneSalesTable
(orderID STRING, brandName STRING, userName STRING, quantity INT, version STRING) STORED BY 'org.apache.hadoop.hive.cassandra.CassandraStorageHandler' WITH SERDEPROPERTIES (
"wso2.carbon.datasource.name" = "WSO2BAM_CASSANDRA_DATASOURCE",
"cassandra.cf.name" = "org_wso2_bam_phone_retail_store_kpi" ,
"cassandra.columns.mapping" =
":key,payload_brand, payload_user, payload_quantity, Version" );
CREATE EXTERNAL TABLE IF NOT EXISTS PhonebrandTable(brand STRING, totalOrders INT, totalQuantity INT) STORED BY 'org.wso2.carbon.hadoop.hive.jdbc.storage.JDBCStorageHandler'
TBLPROPERTIES (
'wso2.carbon.datasource.name'='WSO2BAM_DATASOURCE',
'hive.jdbc.update.on.duplicate' = 'true' ,
'hive.jdbc.primary.key.fields' = 'brand' ,
'hive.jdbc.table.create.query' =
'CREATE TABLE brandSummary (brand VARCHAR(100) NOT NULL PRIMARY KEY, totalOrders INT, totalQuantity INT)' );
insert overwrite table PhonebrandTable select brandName, count(DISTINCT orderID), sum(quantity) from PhoneSalesTable where version= "1.0.0" group by brandName;
Note : Read here for more information about BAM analytics framework.
Let's generate a couple of BAM dashboard gadgets according to the instruction given here.
JDBC URL : jdbc:h2:repository/database/samples/BAM_STATS_DB;AUTO_SERVER=TRUE
Driver Class Name : org.h2.Driver
User Name : wso2carbon
Password : wso2carbon
2. In the next step, add the following SQL script.
Publishing Data
First we should publish some data to BAM. Here, I am going to use KPI sample to publish data. Open a command prompt (or a shell in Linux) and go to the Go to <BAM_HOME>/samples/kpi-definition directory. Then run the ant command to publish some sample data.Analyzing Data
Now run below hive script to summarize the data that has been published. Instructions on adding and executing a hive script can be found in here.CREATE EXTERNAL TABLE IF NOT EXISTS PhoneSalesTable
(orderID STRING, brandName STRING, userName STRING, quantity INT, version STRING) STORED BY 'org.apache.hadoop.hive.cassandra.CassandraStorageHandler' WITH SERDEPROPERTIES (
"wso2.carbon.datasource.name" = "WSO2BAM_CASSANDRA_DATASOURCE",
"cassandra.cf.name" = "org_wso2_bam_phone_retail_store_kpi" ,
"cassandra.columns.mapping" =
":key,payload_brand, payload_user, payload_quantity, Version" );
CREATE EXTERNAL TABLE IF NOT EXISTS PhonebrandTable(brand STRING, totalOrders INT, totalQuantity INT) STORED BY 'org.wso2.carbon.hadoop.hive.jdbc.storage.JDBCStorageHandler'
TBLPROPERTIES (
'wso2.carbon.datasource.name'='WSO2BAM_DATASOURCE',
'hive.jdbc.update.on.duplicate' = 'true' ,
'hive.jdbc.primary.key.fields' = 'brand' ,
'hive.jdbc.table.create.query' =
'CREATE TABLE brandSummary (brand VARCHAR(100) NOT NULL PRIMARY KEY, totalOrders INT, totalQuantity INT)' );
insert overwrite table PhonebrandTable select brandName, count(DISTINCT orderID), sum(quantity) from PhoneSalesTable where version= "1.0.0" group by brandName;
Note : Read here for more information about BAM analytics framework.
Creating Gadgets
As you can see in the above Hive script, the summarized data is stored in the table named 'brandSummary' in summary database(RDBMS). Since H2 is the default packed summary database that comes with WSO2 BAM, the table named 'brandSummary' should be created in the H2 database.Let's generate a couple of BAM dashboard gadgets according to the instruction given here.
1. Go to the Tools → Gadget Gen Tool in BAM Management Console. Then enter the values given below, which are related to the data source 'WSO2BAM_DATASOURCE'.
Driver Class Name : org.h2.Driver
User Name : wso2carbon
Password : wso2carbon
2. In the next step, add the following SQL script.
select * from brandSummary
3. Enter as follows in the next step.
Pick UI Element : Bar Graph
Chart Title : Phone Brand Vs Quantity
Y-Axis Label : Quantity
Y-Axis Column : TOTALQUANTITY
X-Axis Label : Phone Brand
X-Axis Column : BRAND
4. In the next step, enter as follows.
Gadget Title : Phone Brand Vs Quantity
Gadget File Name : brand_vs_qty
Refresh Rate (in Seconds) : 60
Gadget File Name : brand_vs_qty
Refresh Rate (in Seconds) : 60
Then click on the Generate button and follow the steps mentioned in 'Add gadget to dashboard' section of the doc here to add the gadget to dashboard.
5. Create another gadget by selecting 'Table' UI Element (in above given Step 3) as follows. Use same details given in Step 1 and Step 2.
5. Create another gadget by selecting 'Table' UI Element (in above given Step 3) as follows. Use same details given in Step 1 and Step 2.
Generate the gadget and add it to the same tab of the dashboard. Your final dashboard will looks like follows.
There are lots of information about hadoop have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get to the next level in big data. Thanks for sharing this.
ReplyDeleteBig Data Training
Big Data Course in Chennai
The content provided here is vital in increasing one's knowledge regarding hadoop, the way you have presented here is simply awesome. Thanks for sharing this. The uniqueness I see in your content made me to comment on this. Keep sharing article like this. Thanks :)
ReplyDeleteHadoop Training Chennai | Big Data Training Chennai | Big data training in Chennai