Senior Data Engineer

5 days ago

Hyderabad, India Sonata Software Full time

Role & responsibilities

We are seeking a highly skilled Senior Data Engineer with strong expertise in data ingestion, transformation, and pipeline development. The ideal candidate will be an individual contributor responsible for designing, implementing, and maintaining robust data engineering solutions using modern tools and platforms.

Key Responsibilities:

  • Install, configure, and administer data platforms: StreamSets, Apache Spark, Informatica, Apache NiFi
  • Develop real-time and batch data pipelines using PySpark (Scala is a plus)
  • Administer StreamSets Data Collector and Control Hub for pipeline deployment and monitoring
  • Automate data flows using Apache NiFi, including processor configuration and cluster coordination
  • Upgrade and migrate data ingestion tools with minimal downtime
  • Write scalable and efficient Python code for data processing and automation
  • Perform complex SQL transformations and optimize database interactions

  • Work with diverse databases:

  • MPP: Teradata, Greenplum

  • OLTP: Oracle, SQL Server, PostgreSQL
  • In-memory: SAP HANA, MemSQL

  • Big Data: Hive, HBase, Kudu

  • Integrate automated testing using CI/CD tools (GitLab preferred, Jenkins acceptable)

  • Implement deployment pipelines and version control strategies
  • Ensure platform security, including TLS hardening
  • Monitor production environments and perform root cause analysis
  • Enforce coding standards and data governance practices
  • Schedule and orchestrate workflows with error handling and retry mechanisms

Mandatory Skills:

  • Data Ingestion & Transformation Tools
  • StreamSets, Spark (Python preferred), Informatica, Apache NiFi
  • Strong proficiency in Python and SQL

Secondary Skills:

  • CI/CD tools: GitLab, Jenkins
  • Database technologies: Teradata, Greenplum, Oracle, SQL Server, PostgreSQL, SAP HANA, MemSQL
  • Big Data platforms: Hive, HBase, Kudu
  • Knowledge of data governance, lineage tracking, and validation frameworks

  • Lead/Senior Data Engineer

    5 days ago

    Hyderabad, India Data Economy Full time

    We are seeking a Lead/Senior Data Engineer with 7-12 years of experience to architect, develop, and optimize data solutions in a cloud-native environment. The role requires strong expertise in AWS Glue, PySpark, and Python with a proven ability to design scalable data pipelines and frameworks for large-scale enterprise systems. Prior exposure to financial...

  • Senior Data Engineer

    5 days ago

    Hyderabad, India Enable Data Incorporated Full time

    Experience Required: 8+Years Mode of work: Remote Skills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, Spark Notice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 15th 2025) Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and...

  • Data Engineer Senior Consultant

    3 weeks ago

    Hyderabad, Telangana, India NTT Data Full time

    Job DescriptionReq ID:330296NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer Senior Consultant to join our team in Hyderabad, Telangana (IN-TG), India (IN).- Understand...

  • Property Data Engineer

    5 days ago

    Hyderabad, India ICE Data Services Full time

    Job Purpose The Property Data Engineeris responsible for developing and maintaining data conversion programs that transform raw property assessment data into standardized formats based on specifications by Property Data Analyst and Senior Analysts. This role requires not only advanced programming and ETL skills but also a deep understanding of the structure,...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture, Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 29th 2025)- Translate business rules into technical specifications and implement scalable data solutions.- Manage...

  • Senior Data Engineer/Architect

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, Spark,Copilot Studio , Databricks Lakehouse Platform, its architecture, and its capabilities.Notice Period : Immediate Joiners/ Permanent/Contract role (Can join within 15th September...

  • Aws Data Engineer

    5 days ago

    Hyderabad, India Data Economy Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to lead the end-to-end development of complex models for compliance and supervision. The ideal candidate will have deep expertise in cloud-based infrastructure, ETL pipeline development, and financial domains, with a strong focus on creating robust, scalable, and efficient solutions. Key...

  • Azure Data Engineer

    10 hours ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataEngineering Full time ₹ 35,00,000 - ₹ 40,00,000

    Unlock Data Potential as a Senior Data EngineerAs a seasoned Senior Data Engineer, you will be responsible for crafting and implementing cutting-edge data solutions to drive business growth. Your expertise in designing and building data foundations, end-to-end solutions, and mentoring junior engineers will be invaluable to our organization.Key...

  • Senior Data Engineer

    5 days ago

    Hyderabad, India Unify Technologies Full time

    Company Description Unify Technologies is a leading digital engineering company committed to helping global clients build high-quality software products quickly and affordably. With extensive experience in software product engineering practices, Unify Technologies delivers custom software solutions. We collaborate with industry leaders across multiple...