support@recruitmentforagencies.com ✉
+00 123 456 789 RFA, Madison Road, USA 5504
Get Started

Data Engineer

  • Staff Augmentation
  • 3-5 years
  • Full-time
  • Noida
  • Ref WNT46

Design and maintain scalable data pipelines that turn complex data into reliable insights for reporting, analytics and business decisions.

Data Pipeline · ETL/ELT
    Processed 0 Pending 0 Pipeline processing is active.

    Role Overview

    The Data Engineer will design, develop and maintain scalable data pipelines and ETL/ELT workflows. The role focuses on building data ingestion and transformation processes from multiple sources, working with cloud-based data platforms and storage solutions, and supporting reporting and analytics through reliable data models.

    You will integrate data from APIs, databases, files and third-party platforms, while monitoring pipeline performance, troubleshooting data-related issues and optimizing workflows for performance and scalability.

    The role also involves collaboration with Data Scientists, BI Analysts, Software Engineers and business stakeholders, while following best practices for data security, governance, documentation and reliability.

    Key Responsibilities

    • Design, develop and maintain scalable data pipelines and ETL/ELT workflows.
    • Build data ingestion and transformation processes from multiple sources.
    • Develop and optimize SQL queries, stored procedures and data processing workflows.
    • Work with cloud-based data platforms and data storage solutions.
    • Perform data cleansing, transformation, validation and quality checks.
    • Develop and maintain data models to support reporting and analytics.
    • Integrate data from APIs, databases, files and third-party platforms.
    • Monitor pipeline performance and troubleshoot data-related issues.
    • Optimize existing pipelines and queries for improved performance and scalability.
    • Collaborate with Data Scientists, BI Analysts, Software Engineers and business stakeholders.
    • Implement best practices around data security, governance, documentation and reliability.
    • Participate in the design and implementation of scalable data architecture.

    Required Skills

    • Strong proficiency in Python and SQL.
    • Hands-on experience with ETL/ELT pipeline development.
    • Experience working with relational and/or cloud databases.
    • Good understanding of data warehousing and data modeling concepts.
    • Experience with data integration and API-based data ingestion.
    • Familiarity with cloud platforms such as AWS, Azure or GCP.
    • Experience with workflow orchestration tools such as Apache Airflow.
    • Strong understanding of data quality, validation and pipeline monitoring.
    • Good problem-solving and analytical skills.

    Core Technologies

    • Python and SQL for data processing and query development.
    • ETL/ELT workflows for data ingestion and transformation.
    • Relational databases and cloud-based data storage solutions.
    • AWS, Azure or GCP cloud platforms.
    • Apache Airflow for workflow orchestration.
    • APIs, databases, files and third-party platforms for data integration.

    Engineering Focus Areas

    • Data cleansing, transformation, validation and quality checks.
    • Data modeling for reporting and analytics.
    • Pipeline performance monitoring and troubleshooting.
    • Performance and scalability optimization.
    • Data security, governance, documentation and reliability.
    • Scalable data architecture design and implementation.
    DATA ENGINEERING WORKFLOW

    From Raw Data to Reliable Insights

    Build dependable data workflows through ingestion, transformation, validation and continuous optimization.

    01

    Data Ingestion

    Build data ingestion processes that bring information together from APIs, databases, files and third-party platforms.

    APIs Databases Data Sources
    02

    ETL / ELT Transformation

    Develop scalable ETL/ELT workflows, transform datasets and optimize SQL queries and data processing workflows.

    Python SQL ETL / ELT
    03

    Data Quality & Modeling

    Cleanse, validate and model data to support reliable reporting and analytics while maintaining data quality.

    Validation Data Quality Data Modeling
    04

    Monitoring & Optimization

    Monitor pipeline performance, troubleshoot issues and optimize workflows for improved performance, reliability and scalability.

    Apache Airflow Cloud Platforms Monitoring
    GOT QUESTIONS?

    Questions candidates ask

    Anything else? Our consultants are happy to talk it through and help you understand the role.

    Contact us Apply

    Strong proficiency in Python and SQL is required. The role also involves developing and optimizing SQL queries, stored procedures and data processing workflows.

    You will design, develop and maintain scalable data pipelines and ETL/ELT workflows, including data ingestion and transformation processes from multiple sources.

    Familiarity with cloud platforms such as AWS, Microsoft Azure or Google Cloud Platform (GCP) is listed among the required skills.

    The required skillset includes experience with workflow orchestration tools such as Apache Airflow.

    Yes. The responsibilities include integrating data from APIs, databases, files and third-party platforms, as well as building ingestion and transformation processes from multiple sources.

    You will collaborate with Data Scientists, BI Analysts, Software Engineers and business stakeholders.