Data Engineer 2

9 hours ago

Hyderabad, Telangana, India Phenom People Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Job Summary

Phenom People is looking for an experienced Data Engineer to optimize and enhance our data pipelines that power insights across the organization. This role focuses on improving performance, ensuring data quality, automating pipeline development, and enabling data sharing through scalable and secure mechanisms.

The ideal candidate will have strong expertise in PySpark, SQL optimization, and experience with data sharing platforms such as DataHub. Youll collaborate closely with data engineering, platform, and analytics teams to modernize our data ecosystem built on Airflow, Livy, EMR, Flink, Iceberg, and Snowflake.

If you're passionate about performance tuning, automation, and building resilient data systems — we'd love to hear from you.

What You'll Do

  • Optimize existing data pipelines built using PySpark for performance and scalability.
  • Review and tune SQL queries to improve execution time and resource utilization.
  • Design and develop new batch data pipelines that meet evolving business needs (real-time decisioning is not required; systems can tolerate up to 6 hours of delay).
  • Automate pipeline generation and metadata management to streamline development.
  • Implement data quality and validation frameworks to ensure reliability and trust in data assets.
  • Collaborate with teams to integrate and share data via DataHub, leveraging connectors such as SFTP and Snowflake Data Share.
  • Enhance observability by integrating and monitoring pipelines through Grafana and other telemetry tools.
  • Work closely with other engineers to improve coding practices and leverage AI-powered developer assistants like Cursor, Copilot, or Windsurf for productivity.
  • Participate in design reviews, documentation, and operational support for production systems.

What You've Done

  • 5–8 years of experience as a Data Engineer working with large-scale distributed data systems.
  • Strong proficiency in Python (especially PySpark) and hands-on experience optimizing Spark jobs on EMR or similar environments.
  • Deep knowledge of SQL tuning and query optimization across analytical databases (preferably Snowflake).
  • Experience working with Airflow for workflow orchestration and Livy for Spark job management.
  • Exposure to Flink and Apache Iceberg for batch or incremental data processing.
  • Strong understanding of data quality frameworks, testing, and validation techniques.
  • Familiarity with DataHub or similar data catalog and sharing platforms.
  • Experience setting up dashboards and alerts in Grafana or equivalent monitoring tools.
  • Exposure to AI coding assistants (Cursor, GitHub Copilot, Windsurf) for accelerating development is a plus.
  • Passionate about automation, continuous improvement, and building scalable, maintainable systems.

  • Data Engineer 2

    4 days ago

    Hyderabad, Telangana, India Careeco Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Job Title : Mid-Level Data Engineer Healthcare DomainLocation: Remote OptionExperience: 2 to 3 YearsEmployment Type: Full-TimeAbout the RoleWe are seeking a motivated and detail-oriented Mid-Level Data Engineer with 2 to 3 years of experience in designing, developing, and optimizing data pipelines within the healthcare domain. The ideal candidate will have...

  • Azure Data Engineer

    14 hours ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Data Engineer 2

    5 days ago

    Hyderabad, Telangana, India Microsoft Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    Microsoft is a company where passionate innovators come to collaborate, envision what can be and take their careers further. This is a world of more possibilities, more innovation, more openness, and the sky is the limit thinking in a cloud-enabled world.  Microsoft's Azure Data engineering team is leading the transformation of analytics in the...

  • Azure Data Engineer With Snowflake

    3 days ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Azure Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Data Engineer 2

    14 hours ago

    Hyderabad, Telangana, India The Hacker Central Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Job Title: Data EngineerLocation: HyderabadExperience: 4 to 7 yrsResponsibilities:Design, develop & maintain data pipelines using Python, PySpark & SQL.Optimize performance through Spark & SCALA.

  • Data Engineer Advisor

    1 week ago

    Hyderabad, Telangana, India NTT DATA Full time ₹ 1,00,00,000 - ₹ 2,00,00,000 per year

    Ensure audit trail is in place for data migration and migrated data is traceable back to the cleanup / data enrichment activities and a clear line of sight to the source legacy systems Work closely with SAP S4 HANA implementation teams to ensure blueprint is effectively understood and translated into data migration requirements / data concept documents 10+...

  • Data Engineer 2

    1 week ago

    Hyderabad, Telangana, India Phenom People Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    About PHENOM:PHENOM is a global HR tech company that is revolutionizing the talent experience. Our AI-powered platform helps companies hire faster, develop better, and retain longer. Were a passionate team-building solutions that empower candidates, recruiters, managers, and employees with meaningful, data-driven insights.Phenom People is looking for bright...

  • Senior Data Engineer

    21 hours ago

    Hyderabad, Telangana, India The Modern Data Company Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job InformationDate Opened11/07/2025IndustrySoftware ProductJob TypeFull timeCityHyderabadState/ProvinceTelanganaCountryIndiaZip/Postal Code500081About UsThe Modern Data Company is redefining data management for the AI era, transforming data from a technical challenge into a company's most powerful business asset. Modern's DataOS platform is the world's...

  • Senior Data Engineer

    1 week ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture, Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 29th 2025)Translate business rules into technical specifications and implement scalable data solutions.Manage a...