Senior Data Engineer

6 days ago

Hyderabad, India Grid Dynamics Full time
  • Proven experience leading data engineering teams, including distributed teams across multiple geographies and time zones.
  • Effective in managing cross-team collaboration with architects, product managers, and operations.
  • Scala and Python
  • Apache Spark (batch & streaming) – must
  • Deep knowledge of HDFS internals and migration strategies.
  • Experience with Apache Iceberg (or similar table formats like Delta Lake / Apache Hudi) for schema evolution, ACID transactions, and time travel.
  • Running Spark and/or Flink jobs on Kubernetes (e.g., Spark-on-K8s operator, Flink-on-K8s).
  • Experience with distributed blob storages like Ceph or AWS S3 and similar
  • Building ingestion, transformation, and enrichment pipelines for large-scale datasets.
  • Infrastructure-as-Code (Terraform, Helm) for provisioning data infrastructure.
  • Strong communication skills

Essential Functions

  • Lead and mentor a team of data engineers, providing technical direction and career guidance.
  • Define the target data platform architecture for migrating from on-prem HDFS/Hive to Cloud Object Storage (e.g., AWS S3, Azure Data Lake Storage, or GCP Cloud Storage).
  • Select and integrate cloud-based compute and query engines (e.g., Spark).
  • Lead the design of ingestion, transformation, and storage patterns optimized for scalability, cost-efficiency, and performance in the cloud.
  • Define security, encryption, and compliance controls for sensitive enterprise data in the cloud.
  • Develop and own the migration roadmap, including phased transition from on-prem to cloud while minimizing business disruption.
  • Oversee data migration strategies (bulk historical loads, incremental sync, and cutover).
  • Define and enforce coding standards, CI/CD pipelines, and automated testing for data pipelines.
  • Partner with Data Architects, Cloud Engineers, and Security teams to align platform design with enterprise standards.

Qualifications

  • Strong hands-on experience in Scala and Python.
  • Expertise in Apache Spark (batch and streaming).
  • Familiarity with Cloud - AWS
  • Solid understanding of HDFS internals and migration strategies.
  • Experience with Apache Iceberg, Delta Lake, or Apache Hudi.
  • Familiarity with distributed blob storage (e.g., S3, Ceph).
  • Proficiency in Infrastructure-as-Code tools like Terraform and Helm.

Would be a plus

  • Experience with Apache Flink
  • Apple experience preferred (to enable him/her to get up to speed on our tooling set quickly and more independently)

We offer

  • Opportunity to work on bleeding-edge projects
  • Work with a highly motivated and dedicated team
  • Competitive salary
  • Flexible schedule
  • Benefits package - medical insurance, sports
  • Corporate social events
  • Professional development opportunities
  • Well-equipped office

About Us
Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. Fusing technical vision with business acumen, we solve the most pressing technical challenges and enable positive business outcomes for enterprise companies undergoing business transformation. A key differentiator for Grid Dynamics is our 8 years of experience and leadership in enterprise AI, supported by profound expertise and ongoing investment in data, analytics, cloud & DevOps, application modernization and customer experience. Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the Americas, Europe, and India.

  • Lead/Senior Data Engineer

    6 days ago

    Hyderabad, India Data Economy Full time

    We are seeking a Lead/Senior Data Engineer with 7-12 years of experience to architect, develop, and optimize data solutions in a cloud-native environment. The role requires strong expertise in AWS Glue, PySpark, and Python with a proven ability to design scalable data pipelines and frameworks for large-scale enterprise systems. Prior exposure to financial...

  • Senior Data Engineer

    6 days ago

    Hyderabad, India Enable Data Incorporated Full time

    Experience Required: 8+Years Mode of work: Remote Skills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, Spark Notice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 15th 2025) Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and...

  • Data Engineer Senior Consultant

    3 weeks ago

    Hyderabad, Telangana, India NTT Data Full time

    Job DescriptionReq ID:330296NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer Senior Consultant to join our team in Hyderabad, Telangana (IN-TG), India (IN).- Understand...

  • Property Data Engineer

    6 days ago

    Hyderabad, India ICE Data Services Full time

    Job Purpose The Property Data Engineeris responsible for developing and maintaining data conversion programs that transform raw property assessment data into standardized formats based on specifications by Property Data Analyst and Senior Analysts. This role requires not only advanced programming and ETL skills but also a deep understanding of the structure,...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture, Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 29th 2025)- Translate business rules into technical specifications and implement scalable data solutions.- Manage...

  • Senior Data Engineer/Architect

    2 weeks ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, Spark,Copilot Studio , Databricks Lakehouse Platform, its architecture, and its capabilities.Notice Period : Immediate Joiners/ Permanent/Contract role (Can join within 15th September...

  • Aws Data Engineer

    6 days ago

    Hyderabad, India Data Economy Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to lead the end-to-end development of complex models for compliance and supervision. The ideal candidate will have deep expertise in cloud-based infrastructure, ETL pipeline development, and financial domains, with a strong focus on creating robust, scalable, and efficient solutions. Key...

  • Azure Data Engineer

    18 hours ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Senior Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataEngineering Full time ₹ 35,00,000 - ₹ 40,00,000

    Unlock Data Potential as a Senior Data EngineerAs a seasoned Senior Data Engineer, you will be responsible for crafting and implementing cutting-edge data solutions to drive business growth. Your expertise in designing and building data foundations, end-to-end solutions, and mentoring junior engineers will be invaluable to our organization.Key...

  • Senior Data Engineer

    6 days ago

    Hyderabad, India Unify Technologies Full time

    Company Description Unify Technologies is a leading digital engineering company committed to helping global clients build high-quality software products quickly and affordably. With extensive experience in software product engineering practices, Unify Technologies delivers custom software solutions. We collaborate with industry leaders across multiple...