Data Engineer

3 weeks ago

Hyderabad, Telangana, India Navajna Technologies Full time

Role Summary :

We are looking for an experienced Data Engineer to design, build, and optimize large-scale data processing systems.

The ideal candidate will have strong expertise in Python programming, Apache Spark for big data processing, and Apache Airflow for workflow orchestration.

This role requires a deep understanding of data architecture principles and best practices to enable scalable, reliable, and efficient data pipelines that support advanced analytics and business intelligence.

Key Responsibilities :

- Design and implement scalable data architectures and pipelines leveraging Apache Spark and Python to process large datasets efficiently.

- Develop and maintain workflow orchestration solutions using Apache Airflow to automate complex data processing tasks.

- Collaborate with data engineering, data science, and analytics teams to understand data requirements and translate them into robust architectural solutions.

- Define data standards, governance, and best practices to ensure data quality, security, and compliance.

- Optimize data storage and retrieval strategies across data lakes, warehouses, and streaming platforms.

- Evaluate new tools, frameworks, and technologies to improve data platform capabilities.

- Monitor and troubleshoot data pipeline performance issues, ensuring high availability and reliability.

- Document architectural designs, data flows, and operational processes.

- Mentor and guide junior data engineers and analysts on architectural best practices and technical skills.

- Work closely with cloud platform teams to integrate and deploy data solutions (e., AWS, Azure, or GCP).

Required Skills & Experience :

- Strong programming skills in Python for data processing, scripting, and automation.

- Hands-on expertise in Apache Spark for batch and stream processing of large-scale datasets.

- Experience designing and managing workflows with Apache Airflow, including DAG development and monitoring.

- Solid understanding of data architecture principles, ETL/ELT design patterns, and distributed computing.

- Experience with relational and NoSQL databases, data lakes, and data warehouses.

- Familiarity with cloud data services and infrastructure (AWS, Azure, or GCP).

- Proficient in SQL and data modeling techniques.

- Knowledge of data governance, security practices, and compliance standards.

- Strong analytical, problem-solving, and communication skills.

- Ability to work collaboratively in cross-functional teams and mentor junior members.

Preferred Qualifications :

- Bachelors or Masters degree in Computer Science, Engineering, or related field.

- Experience with containerization and orchestration tools (Docker, Kubernetes).

- Familiarity with other data processing frameworks (Kafka, Flink).

- Exposure to machine learning pipelines and MLOps practices.

- Certifications related to Big Data, Cloud, or Data Architecture

(ref:hirist.tech)

  • Senior Data Engineer

    13 hours ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 15th 2025)Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and...

  • Data Engineer

    12 hours ago

    Hyderabad, Telangana, India ICE Data Services Full time US$ 1,25,000 - US$ 1,75,000 per year

    Job PurposeAt Intercontinental Exchange (ICE), we engineer technology, exchanges and clearing houses that connect companies around the world to global capital and derivative markets. With a leading-edge approach to developing technology platforms, we have built market infrastructure in all major trading centers, offering customers the ability to manage risk...

  • Azure Data Engineer ll

    14 hours ago

    Hyderabad, Telangana, India Enable Data Full time US$ 90,000 - US$ 1,20,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Senior Data Engineer

    16 hours ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and Databricks. Gather and analyse data requirements from business stakeholders and identify opportunities for data-driven insights. Build and optimize data pipelines for data ingestion, processing, and integration using Spark and Databricks. Ensure data...

  • Azure Data Engineer

    1 week ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Senior Data Engineer

    15 hours ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 15th 2025)Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and...

  • Lead/Senior Data Engineer

    14 hours ago

    Hyderabad, Telangana, India Data Economy Full time US$ 1,50,000 - US$ 2,00,000 per year

    We are seeking a Lead/Senior Data Engineer with 7-12 years of experience to architect, develop, and optimize data solutions in a cloud-native environment. The role requires strong expertise in AWS Glue, PySpark, and Python with a proven ability to design scalable data pipelines and frameworks for large-scale enterprise systems. Prior exposure to financial...

  • Aws Data Engineer

    15 hours ago

    Hyderabad, Telangana, India Data Economy Full time US$ 1,50,000 - US$ 2,00,000 per year

    We are seeking a highly skilled and experienced Senior Data Engineer to lead the end-to-end development of complex models for compliance and supervision. The ideal candidate will have deep expertise in cloud-based infrastructure, ETL pipeline development, and financial domains, with a strong focus on creating robust, scalable, and efficient solutions.Key...

  • Lead/Sr. ETL Data Engineer

    14 hours ago

    Hyderabad, Telangana, India Data Economy Full time US$ 90,000 - US$ 1,20,000 per year

    We are seeking a skilled ETL Data Engineer to design, build, and maintain efficient and reliable ETL pipelines, ensuring seamless data integration, transformation, and delivery to support business intelligence and analytics. The ideal candidate should have hands-on experience with ETL tools like Talend, strong database knowledge, and familiarity with AWS...

  • Lead Cloud Data Engineer

    16 hours ago

    Hyderabad, Telangana, India Data Economy Full time US$ 1,20,000 - US$ 2,00,000 per year

    Key Responsibilities:Engage directly with clients to understand requirements, provide solution design, and drive successful project delivery.Lead cloud migration initiatives, specifically moving on-premise applications and databases to AWS cloud platforms.Design, develop, and maintain scalable, reliable, and secure data applications in a cloud...