Data Management (ETL & DWH)

Expertise in setting up multiple Data warehousing, Data migration, Data Integrations & Data Analysis projects.

ETL Project

Step 1) Extraction

Data is extracted from the source system into the staging area. Transformations if any are done in staging area so that performance of source system in not degraded. Also, if corrupted data is copied directly from the source into Data warehouse database, rollback will be a challenge. Staging area gives an opportunity to validate extracted data before it moves into the Data warehouse.

Three Data Extraction methods:

  1. Full Extraction
  2. Partial Extraction- without update notification.
  3. Partial Extraction- with update notification

Irrespective of the method used, extraction should not affect performance and response time of the source systems. These source systems are live production databases. Any slowdown or locking could affect company’s bottom line.

Some validations are done during Extraction:

  • Reconcile records with the source data
  • Make sure that no spam/unwanted data loaded
  • Data type check
  • Remove all types of duplicate/fragmented data
  • Check whether all the keys are in place or not

Step 2) Transformation

Data extracted from source server is raw and not usable in its original form. Therefore, it needs to be cleansed, mapped and transformed. In fact, this is the key step where ETL process adds value and changes data such that insightful BI reports can be generated.

In this step, you apply a set of functions on extracted data. Data that does not require any transformation is called as direct move or pass through data.

Validations are done during this stage

  • Filtering – Select only certain columns to load
  • Using rules and lookup tables for Data standardization
  • Character Set Conversion and encoding handling
  • Conversion of Units of Measurements like Date Time Conversion, currency conversions, numerical conversions, etc.
  • Data threshold validation check. For example, age cannot be more than two digits.
  • Data flow validation from the staging area to the intermediate tables.
  • Required fields should not be left blank.
  • Cleaning ( for example, mapping NULL to 0 or Gender Male to “M” and Female to “F” etc.)
  • Split a column into multiples and merging multiple columns into a single column.
  • Transposing rows and columns,
  • Use lookups to merge data
  • Using any complex data validation (e.g., if the first two columns in a row are empty then it automatically rejects the row from processing)

Step 3) Loading

Loading data into the target datawarehouse database is the last step of the ETL process. In a typical Data warehouse, huge volume of data needs to be loaded in a relatively short period (nights). Hence, load process should be optimized for performance.

Types of Loading:

  • Initial Load — populating all the Data Warehouse tables
  • Incremental Load — applying ongoing changes as when needed periodically.
  • Full Refresh —erasing the contents of one or more tables and reloading with fresh data.


Planning, Budgeting & Forecasting

Create and analyze sophisticated plans, budgets and forecasts based on business process with small to large data sets. Integrated scorecards and strategy management – Model metrics to measure progress toward objectives and link them dynamically to actions and forecasts.

Business Intelligence Reports & Dashboards

Business Intelligence integrates reporting, modeling, analysis, dashboards, stories, and event management so you can understand your organization’s data, and make effective business decisions.

Financial Consolidation

Financial Consolidation & Reporting supports the close, consolidation and reporting process with the agility and affordability of an integrated solution.It also helps finance teams deliver financial results, create informative financial and management reports and provide the Chief Financial Officer (CFO) with an enterprise view of key financial ratios and metrics.

Data Management (ETL & DWH)

A data warehouse, or enterprise data warehouse (EDW), is a system that aggregates data from different sources into a single, central, consistent data store to support data analysis, data mining, artificial intelligence (AI), and machine learning. Simplifies ongoing integration tasks and new app development. Ensure consistent master information across transactional and analytical systems. Addresses key issues such as latency and data quality feedback proactively rather than “after the fact” in the data warehouse. Data warehouses are solely intended to perform queries and analysis and often contain large amounts of historical data.

Financial and Operational KPI’s

KPI is a quantifiable value expressing the business performance in a shorter time-frame level. They are used in different industries to track organizational processes, improve efficiency and help businesses to understand and reflect on the outcomes. When a business is measuring the effectiveness of a process, often metrics and KPIs are established to perform the evaluation and analysis.

Performance Management

Gain value with the most complete cloud: Comprehensively address your needs with functional breadth and depth across financial and operational planning, consolidation and close, master data management, and more.