Big Data Engineer

3 weeks ago

NCR, India Qcentrio Full time

We are seeking an experienced and driven Data Engineer with 5+ years of hands-on experience in building scalable data infrastructure and systems. You will play a key role in designing and developing robust, high-performance ETL pipelines and managing large-scale datasets to support critical business functions. This role requires deep technical expertise, strong problem-solving skills, and the ability to thrive in a fast-paced, evolving environment.

Key Responsibilities :

  • Design, develop, and maintain scalable and reliable ETL/ELT pipelines for processing large volumes of data (terabytes and beyond).

  • Model and structure data for performance, scalability, and usability.

  • Work with cloud infrastructure (preferably Azure) to build and optimize data workflows.

  • Leverage distributed computing frameworks like Apache Spark and Hadoop for large-scale data processing.

  • Build and manage data lake/lakehouse architectures in alignment with best practices.

  • Optimize ETL performance and manage cost-effective data operations.

  • Collaborate closely with cross-functional teams including data science, analytics, and software engineering.

  • Ensure data quality, integrity, and security across all stages of the data lifecycle.

Required Skills & Qualifications :

  • 7 to 10 years of relevant experience in bigdata engineering.

  • Advanced proficiency in Python,

  • Strong skills in SQL for complex data manipulation and analysis.

  • Hands-on experience with Apache Spark, Hadoop, or similar distributed systems.

  • Proven track record of handling large-scale datasets (TBs) in production environments.

  • Cloud development experience with Azure (preferred), AWS, or GCP.

  • Solid understanding of data lake and data lakehouse architectures.

  • Expertise in ETL performance tuning and cost optimization techniques.

  • Knowledge of data structures, algorithms, and modern software engineering practices.

Soft Skills :

  • Strong communication skills with the ability to explain complex technical concepts clearly and concisely.

  • Self-starter who learns quickly and takes ownership.

  • High attention to detail with a strong sense of data quality and reliability.

  • Comfortable working in an agile, fast-changing environment with incomplete requirements.

Preferred Qualifications :

  • Experience with tools like Apache Airflow, Azure Data Factory, or similar.

  • Familiarity with CI/CD and DevOps in the context of data engineering.

  • Knowledge of data governance, cataloging, and access control principles.

Skills : Python,Sql,Aws,Azure, Hadoop

  • Big Data Engineer

    3 weeks ago

    NCR, India Qcentrio Full time

    Work Location : Pan India Experience : 6+ Years Notice Period : Immediate - 30 days Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud JD and required Skills & Responsibilities : Actively participate in all phases of the software development lifecycle, including requirements gathering, functional and technical design,...

  • Big Data Engineer

    2 weeks ago

    Delhi, NCR, India Qcentrio Full time US$ 1,50,000 - US$ 2,00,000 per year

    Work Location : Pan India Experience : 6+ Years Notice Period : Immediate - 30 days Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud JD and required Skills & Responsibilities : Actively participate in all phases of the software development lifecycle, including requirements gathering, functional and technical design, development,...

  • Azure & Big Data Engineer

    6 days ago

    Delhi, NCR, Pune, India Iris Software Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job description:Iris Software has been a trusted software engineering partner to several Fortune 500 companies for over three decades. We help clients realize the full potential of technology-enabled transformation by bringing together a unique blend of domain knowledge, best-of-breed technologies, and experience executing essential and critical application...

  • Senior Data Engineer

    3 weeks ago

    NCR, India MchooversPacific Data Integrators Full time

    Job Title: Senior Data Engineer (IDMC-CDI)Primary Skills: SQL, IDMC (CDI), ETL ConceptsMode of Work: Remote/WFHExperience: 5-10 Years About Us We are PDI (Pacific Data Integrators), a leading provider of Data Management, Analytics, and Cloud solutions. As an Informatica Platinum Partner and a reseller of various market-leading products, we help enterprises...

  • Azure & Big Data - Lead

    2 weeks ago

    Delhi, NCR, Pune, India Iris Software Full time ₹ 1,80,000 - ₹ 3,60,000 per year

    Job description:Iris Software has been a trusted software engineering partner to several Fortune 500 companies for over three decades. We help clients realize the full potential of technology-enabled transformation by bringing together a unique blend of domain knowledge, best-of-breed technologies, and experience executing essential and critical application...

  • Senior Data Engineer

    2 weeks ago

    Delhi, NCR, India MchooversPacific Data Integrators Full time US$ 1,00,000 - US$ 1,50,000 per year

    Job Title: Senior Data Engineer (IDMC-CDI)Primary Skills: SQL, IDMC (CDI), ETL ConceptsMode of Work: Remote/WFHExperience: 5-10 YearsAbout UsWe are PDI (Pacific Data Integrators), a leading provider of Data Management, Analytics, and Cloud solutions. As an Informatica Platinum Partner and a reseller of various market-leading products, we help enterprises...

  • Senior Data Engineer

    1 week ago

    Delhi, NCR, India MchooversPacific Data Integrators Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Title: Senior Data Engineer (IDMC-CDI)Primary Skills: IDMC (CDI), ETL ConceptsMode of Work: Remote/WFHExperience: 8-15 yearsShift Timing: Open to work in EST shift (5PM to 2AM IST)About UsWe are PDI (Pacific Data Integrators), a leading provider of Data Management, Analytics, and Cloud solutions. As an Informatica Platinum Partner and a reseller of various...

  • Data Engineer

    4 days ago

    Delhi, NCR, India Planet Ecom Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are seeking a skilled Data Engineer to design, build and maintain scalable data pipelines and architectures. The ideal candidate will have strong expertise in data modeling, ETL processes and big data technologies, ensuring high-quality, reliable.

  • Freelancer Data Engineer

    3 weeks ago

    NCR, India Khaitan Education Centre Full time

    Working hours : 5-6 hours per week @Rs.400/500 per hour About the Role We are seeking a highly skilled and motivated Data Engineer to join our team for a Data Warehouse (DWH) project. The successful candidate will be responsible for designing, developing, and maintaining data pipelines and workflows to support the data warehouse's operations, ensuring data...

  • GCP Data Engineer

    7 hours ago

    Delhi, NCR, Pune, India Tata Consultancy Services Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title : GCP Data EngineerJob Location Chennai / Hyderabd / Bangalore / Pune / Gurgoan/ Noida / NCRExperience: 5 to 10 Years of experience in IT industry in Planning, deploying, and configuring GCP based solutions.Requirement:Mandatory to have knowledge of Big Data Architecture Patterns and experience in delivery of BigData and Hadoop Ecosystems.Strong...