$12.0 per Hour
3.6+ years of experience in the IT industry as a Data Engineer.
3.6+ years of Data Analytics, Data Engineering and Big data exposure.
Experience in Data Analytics, Cloud based Leveraging, Data warehousing, & Big Data Analytics.
Hands-on experience with Python, Spark, Azure, AWS, Advanced Excel, PowerBI, SQL, SSMS.
Developed scalable pipelines, Data Warehouses, proven data analytics using Python.
Subject matter expert for Azure & AWS.
Have worked in HDFS and Pyspark as the processing for Data transformation from client server to OLAP system for Data analysis and model designing.
Developed statistical and various machine learning models for predictions.
Delivering end to end Data Engineering solutions with time bounded situations
Strong & proven experience in SQL, Advanced Excel, Pyspark, Azure, AWS, PowerBI, Analytics.
Gained experience in requirement analysis, design, development, deployment, testing of products.
Gained expertise in report writings, documentation, present findings.
Ability to handle different file formats like CSV, Json, XML, Parquet, ORC, etc
Orchestrate the metadata driven pipeline & data flow using Cloud services, Databases management, and product development.
Experience in working effectively, productive, managing teams in remote working conditions.
Tech Stack Expertise
Microsoft SQL Server
AWS,AWS S3,AWS Glue,AWS Athena0 Years
- January 2019 - November 2022 - 3 Year
- January 2018 - October 2018 - 10 Months
- Designed, Developed, & Implemented Kappa architecture based DEP platform for Fortune 500 client.
- Implemented Data streaming(Kafka), Analytics(Synapse), Data Lake(ADLS, Hadoop),
- Visualization & Data warehousing(SSMS, ADF , Azure Blob), Consumption (Apps, PowerBI, SSRS).
Recruitment Process Automation
- June 2019 - January 2020 - 8 Months
- Recruitment Process Automation with Azure
- Orchestrate Data Warehouse for an Early Stage Startup’s Recruitment process
- Provided the Recruitment solution with Data which helped to reduce the manual work by 80%.
- Automated process with parametrized storage, pipeline, & scheduled executions to get latest updates.
- Created Unified Data Flow (Azure) & Dashboard (PowerBI) for Real-Time Monitoring.
Nike – Data Analytics Platform
- January 2020 - October 2020 - 10 Months
- Designed a Data analytics platform to handle, and analyze the sales, and marketing data in order to drive the business with the help of data analytics
- Integrate the data from various resources and create regulatory storage for unified data leveraging AWS S3 storage.
- Process the raw data, rejecting duplicates, and null values making it useful for data analytics using Pyspark.
- Applied ETL transformations using AWS Glue, converting data into dimensions and facts and creating a data warehouse along with management of databases.Querying the data, analyzing the data patterns using the Visualization tool – PowerBI, converting the findings into reports along with the logic used, and finally converting it into the hidden insights out of the raw data to enhance the business.
- April 2021 - October 2021 - 7 Months
- Movie Recommendation system with Data Engineering.
- (Performed ETL operation to gain business insights out of Raw Data)
- Data Extraction (Kaggle API), EC2(Computing), Data Transformation (Spark, Hadoop), Orchestration (Apache Airflow), Data Warehousing(Redshift), containerization(Docker), storage (S3).
- Performed Data modeling, Ingestion, ETL development & Containerization to gain business insights.
- Bridged the gap between Data & Business with Data Engineering.
Data Platform - EazyPG
- February 2022 - May 2022 - 4 Months
- Data Analytics & Engineering, EazyPG
- Developed & managed data warehouse & lakes with SCD logics using S3, Redshift, MySQL, and Hadoop.
- Built web platforms & applications based on RestAPI, Batch processing, Hive, Airflow, and MongoDB, Github.
- Built, orchestrated, & automated pipelines using Airflow, S3, Python & data batch processing with Amazon SQS.
Computer Science Engineering in B.EDelhi Institute
- January 2016 - June 2018