Data Engineer

3 days ago

Hyderabad, Telangana, India CORTEX Consultants Full time

Job Title : Data Engineer Kafka, Python, and Azure Databricks

Location : Bangalore, Noida, Hyderabad (Hybrid 2 days in office per week)

Experience : 5 to 8 Years

Notice Period : Immediate to 15 Days (Immediate Joiners Preferred)

About the Role :


We're looking for a seasoned Data Engineer to join our dynamic healthcare data engineering team. You'll be instrumental in designing, developing, and managing robust and scalable data pipelines that support our critical business functions. This role is a perfect fit for a proactive and skilled professional who thrives in a collaborative environment and has a passion for building innovative data solutions. You'll work with cutting-edge technologies to process and analyze large volumes of healthcare data, ensuring high performance, reliability, and security.

Key Responsibilities :


- Design and Development : Architect, build, and maintain real-time and batch data pipelines using Kafka, Azure Databricks, and Python. You'll be responsible for the entire lifecycle, from data ingestion to processing and storage.

- Performance Optimization : Optimize and tune data pipelines and queries for maximum performance and efficiency. You'll troubleshoot and resolve data pipeline issues, ensuring data quality and integrity.

- Collaboration : Work closely with data scientists, analysts, and other engineering teams to understand data requirements and deliver solutions that meet business needs. You'll contribute to a culture of collaboration and continuous improvement.

- System Architecture : Contribute to the design and implementation of large-scale, cloud-based data solutions on the Microsoft Azure platform.

- Technical Leadership : Mentor junior engineers and champion best practices in data engineering, including coding standards, design patterns, and testing methodologies.

- Data Governance : Ensure all data solutions adhere to healthcare data standards and compliance regulations, maintaining a strong focus on data security and privacy.

Required Skills and Qualifications :


- Experience : A minimum of 5 years of hands-on experience as a Data Engineer, with a focus on building data pipelines.

- Kafka : Advanced expertise in Kafka, including Kafka Streams, Kafka Connect, and Schema Registry. You should be comfortable designing and managing Kafka topics and clusters for real-time data processing.

- Python : Strong programming and scripting skills in Python. You should be proficient in using Python for data manipulation, automation, and building data-centric applications.

- Azure Databricks : Proven experience in building and optimizing data pipelines using Azure Databricks.

- PySpark : Proficiency with PySpark for distributed data processing and transformation.

- SQL & Data Modeling : Strong knowledge of SQL and solid experience in data modeling, including designing relational and dimensional models for large datasets.

- Cloud Platforms : Hands-on experience with Microsoft Azure services, specifically Azure Data Lake, Azure Synapse, and Azure Functions.

Desirable Skills :


- Experience with other data streaming technologies like Spark Streaming or Flink.

- Knowledge of containerization technologies (Docker, Kubernetes).

- Familiarity with CI/CD pipelines (Azure DevOps, GitHub Actions).

- Experience in the healthcare domain and understanding of standards like HL7 or FHIR. (ref:hirist.tech)

  • Senior Data Engineer

    2 days ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 15th 2025)Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and...

  • Azure Data Engineer ll

    2 days ago

    Hyderabad, Telangana, India Enable Data Full time US$ 90,000 - US$ 1,20,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Data Engineer

    2 days ago

    Hyderabad, Telangana, India ICE Data Services Full time US$ 1,25,000 - US$ 1,75,000 per year

    Job PurposeAt Intercontinental Exchange (ICE), we engineer technology, exchanges and clearing houses that connect companies around the world to global capital and derivative markets. With a leading-edge approach to developing technology platforms, we have built market infrastructure in all major trading centers, offering customers the ability to manage risk...

  • Senior Data Engineer

    2 days ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and Databricks. Gather and analyse data requirements from business stakeholders and identify opportunities for data-driven insights. Build and optimize data pipelines for data ingestion, processing, and integration using Spark and Databricks. Ensure data...

  • Azure Data Engineer

    1 week ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Senior Data Engineer

    2 days ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 15th 2025)Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and...

  • Lead/Senior Data Engineer

    2 days ago

    Hyderabad, Telangana, India Data Economy Full time US$ 1,50,000 - US$ 2,00,000 per year

    We are seeking a Lead/Senior Data Engineer with 7-12 years of experience to architect, develop, and optimize data solutions in a cloud-native environment. The role requires strong expertise in AWS Glue, PySpark, and Python with a proven ability to design scalable data pipelines and frameworks for large-scale enterprise systems. Prior exposure to financial...

  • Senior Data Engineer

    6 hours ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture, Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 29th 2025)- Translate business rules into technical specifications and implement scalable data solutions.- Manage...

  • Senior Data Engineer

    14 hours ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture, Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 29th 2025)Translate business rules into technical specifications and implement scalable data solutions. Manage a...

  • Aws Data Engineer

    2 days ago

    Hyderabad, Telangana, India Data Economy Full time US$ 1,50,000 - US$ 2,00,000 per year

    We are seeking a highly skilled and experienced Senior Data Engineer to lead the end-to-end development of complex models for compliance and supervision. The ideal candidate will have deep expertise in cloud-based infrastructure, ETL pipeline development, and financial domains, with a strong focus on creating robust, scalable, and efficient solutions.Key...