Aneeque Zubair

As a Data Engineer, my primary focus is on building and maintaining the infrastructure necessary for storing, processing, and analyzing large volumes of data. I have a deep understanding of database technologies, data warehousing, and data processing frameworks, and I am constantly exploring new technologies and techniques to improve my skills and keep up with the rapidly evolving field of data engineering.

I am experienced in designing and implementing data pipelines that can efficiently extract, transform, and load data from various sources, including relational databases, NoSQL databases, and file systems. I am also skilled in data modeling and schema design, ensuring that data is properly structured and organized for easy access and analysis.

In addition to my technical skills, I have a strong understanding of business requirements and data needs. I collaborate with data analysts and other stakeholders to ensure that the data infrastructure meets their needs and enables them to gain valuable insights from the data. I am also proficient in scripting and automation, which allows me to streamline data processing workflows and reduce manual intervention.

Overall, I am a highly skilled and motivated Data Engineer with a passion for leveraging data to drive business outcomes. I am always seeking new challenges and opportunities to learn and grow in this dynamic field.

The first project involved implementing Change Data Capture/Replication using PySpark in AWS, creating RDS, S3 Bucket, DMS, Lambda Function, and using Glue Job service with Data bricks. The ETL pipeline was loaded in AWS cloud and trigger checks were conducted using cloud watch service.

The second project involved building a solution architecture for data engineering using Azure Databricks, Azure Data Lake Gen2, and Azure Data Factory. The pipelines included ingestion of CSV, Text, simple & complex JSON files into the data lake as parquet files/tables. The data was transformed using operations like filter, join, aggregation, group by, and window functions. The pipelines were scheduled to execute at regular intervals using data factory triggers. Overall, these projects demonstrate proficiency in designing, implementing, and managing complex data pipelines using a variety of tools and services in both AWS and Azure environments.

Data Enthusiast with a demonstrated history of working in the internet industry. Skilled in Data Engineering, Data Analysis, Python, Java, JavaScript, Azure, SAP BTP, and SQL. Strong research & development professional with a Master of Science