Talent's Information
-
Location
Pune, India
-
Rate
$10.0 per Hour
-
Experience
3.7 Year
-
Languages Known
English,Hindi
Available for
About Sahil
Developed and maintained Data Pipelines and ETL processes using Python, Spark, Kafka, Hive and AWS services.
Provided end-to-end data solutions to the business and analytics team ensuring end-to-end encryption by leveraging AWS cloud services and native python scripting.
Designed and architected solutions to load multipart files which can't rely on a scheduled run and must be event-driven, leveraging AWS SNS, SQS, Lambda, and Glue.
Designed a solution to handle AWS Glue Timeout issue, automated reruns, and notification process.
Engineered data pipelines using internal ETL tools, and native python scripts with Spark SQL and AWS services like Glue (Pyspark) and Lambda ensuring data encryption at rest and in transit.
Worked on QA support activities, test data creation, and Unit testing activities.
Actively participated in daily stand-up, biweekly SCRUM, and project meetings
Tech Stack Expertise
-
Python
Python
4 Years -
Microsoft SQL Server
SSMS
0 Years
Work Experience
Data Engineer
- January 2020 - March 2023 - 3 Year
- India
Projects
Financial Domain
- October 2017 - January 2022 - 52 Months
-
Led the migration of Proc workflow (SQL) into PySpark, while applying optimization techniques to reduce the time and generate the expected reports.
Engage with BA to understand the requirement clearly and develop missing scenarios and clarified them with BA.
Involved in working on Spark SQL code as an alternative approach for faster data processing and better performance.
Inject and process a large amount of data from various structured and semi-structured sources into S3 (AWS Cloud).
Deployed the development code after the client's approval.
Collaborate on QA support activities, test data creation, and Unit testing activities.
Telecom Domain
- February 2021 - September 2021 - 8 Months
-
Analyzed customer requirement, data and generated reports in Power Bi Desktop to meet specific business needs
Developed highly interactive reports and dashboards using Power Bi Desktop.
Generated daily, weekly and monthly performance reports for managers to monitor performance and productivity.
Performed data cleaning and transformation using Power Query and DAX.
Published and shared reports and dashboards with stakeholders using Power Bi.
Implementing dynamic RLS.
Wrote calculated columns, and Measures queries on the power bi desktop to show good data analysis techniques.
Filtered data and performed multiple operations to get meaningful insides that help our client in decision-making and analysis
Retail Domain
- December 2021 - January 2022 - 2 Months
Responsible for applying data cleaning techniques to correct and remove in-accurate and corrupted data, to improve data quality which significantly improves productivity.
Responsible for applying multiple performance tuning techniques to get faster performance.
Develop and Maintain ETL flows, data models, reports, and dashboards for Supply Chains departments Inventory Data modeling, Complexity model, and products analysis.
Solved performance issues in scripts with the understanding of Join, Group, and Aggregation
Wrote calculated columns, and Measures queries on the power bi desktop to show good data analysis techniques. Filtered data and performed multiple operations to get meaningful insides that help our client in decision-making and analysis.
Working in Parallel with QA to solve bugs raised on modules.
Soft Skills
Industry Expertise
Education
in B.E
Maharashtra University- October 2021 - January 2022
