Importing and Analyzing Data in Datameer

Overview

Datameer, an end-to-end big data analytics platform, is built on Apache Hadoop to perform integration, analysis, and visualization of massive volumes of both structured and unstructured data. It can be rapidly integrated with any data sources such as new and existing data sources to deliver an easy-to-use, cost-effective, and sophisticated solution for big data analytics.

It simplifies data extraction, data transformation, data loading, and real-time data retrieval. It helps to gain actionable insights from complex organizational data through data preparation and analytics. In this blog, let us discuss about importing, analyzing, and visualizing large volume of financial or bank data in Datameer.

Pre-requisite

Download and install Datameer 6.1.14 from the below link:
https://www.datameer.com/direct/

Use Case

The financial data file such as CSV, Excel, and so on is considered for importing into Datameer before starting data analysis. A workbook is created to associate with the data. A database connection is established to link the data with the database.

Importing Data into Datameer

In this section, let us discuss about importing the data into Datameer.

Uploading Files

To upload a file, perform the following steps:

  • Open Datameer.
  • In the left panel, click FileUploads –> Create new –> File upload to upload a file into Datameer.
  • Click Browse and upload the required file.
  • Choose File Type and click Next.
  • Enter Data Details and Define Fields in the subsequent tabs.
  • Configure the file and Save it.

Adding Data to Workbook

To add data into a workbook, right-click on the uploaded file and choose Add Data To New Workbook.

Establishing Database Connection

You can create a connection with any type of databases such as DB2, MySQL, or Oracle. To establish a database connection, add appropriate database drivers to Datameer installation.

Adding Database Connection

To add a database connection, perform the following steps:

  • In the left panel, click Connections –> Connection
  • Choose the required Type of database.
  • Provide Connection Details and Save it.

Adding Jar File

To add a jar file, perform the following steps:

  • Click View –> Admin Tab.
  • In the left panel, click Database Drivers –> New.
  • Provide database driver details to add a new database driver.
  • Click Save to save the details. The new database driver will be added and will be listed in the Database Drivers tab.

Fetching Data from Database

To fetch data from the database, perform the following steps:

  • In the left panel, click FileUploads –> Create New –> Import Job.

You will be redirected to the New Import Job tab.

  • Choose the Connection by clicking Select Connection.
  • Select the required connection and click Next.
  • Provide Data Details and click Next.
  • Select the required Data Fields.
  • Click Next.
  • Provide Schedule details to schedule the data import and click Next.
  • Provide the required location to Save.

Analyzing Data in Datameer

In this section, let us discuss about analyzing the data in Datameer.

Setting up Data for Analysis

To set up the data for analysis, Datameer has provided the following four capabilities:

  • Formulas
  • Filtering
  • Joining
  • Sorting

Using the above capabilities, you can locate numbers, trends, or other information needed for analysis. In this section, let us discuss about formulas and joining capabilities in Datameer. To set up the data for analysis using formulas, perform the following steps:

  • Log in to Datameer using your login credentials.
  • In the left pane, click Connection –> Workbook.
  • Open the required workbook. A popup window with Formula Builder tab will be opened.

Setting up Data using Formulas

Formulas – Grouping Records with GROUPBY

This function is used to create groups of records based on the column selected. In the left pane of Formula Builder, select Grouping and choose GROUPBY in the relevant right pane to group the records in a column.

Formulas – Counting Records with GROUPCOUNT

This function is used to count the records in a group. In the left pane of Formula Builder, select Grouping and choose GROUPCOUNT in the relevant right pane to count the records in a group.

Formulas – Comparing Records with COMPARISON

This function is used to compare records in two different columns. In the left pane of Formula Builder, select Comparison and choose COMPARE in the relevant right pane to compare the records in the selected two columns.

Setting up Data using Data Joins

To join data from two columns, perform the following steps:

  • Open the saved workbook.
  • Click Join to start joining data from two different sheets.
  • Click join type to join data.

Visualizing Data

After setting up the data, visualization can be easily created in the form of graphs and charts for performing analysis. To visualize data, click Add Tab icon and choose Infographic to visualize the data.

Conclusion

In this blog, we discussed about importing data into Datameer, setting up data for analysis, and visualizing data in Datameer.

References