Senior Data Engineer – Python

5 days ago

Hyderabad, Telangana, India Egen Full time ₹ 12,00,000 - ₹ 24,00,000 per year

Job Overview:

We are looking for an experienced Senior Data Engineer to design, build, and optimize scalable, high-performance data platforms using AWS cloud services and Python. The ideal candidate will play a key role in architecting end-to-end data pipelines, driving automation, ensuring data quality, and enabling analytics and AI workloads across the organization.

This role requires deep technical expertise in AWS data services, modern data architecture, and a passion for delivering reliable, high-quality data solutions at scale.

Key Responsibilities
  • Architect and implement scalable, fault-tolerant data pipelines using AWS Glue, Lambda, EMR, Step Functions, and Redshift
  • Build and optimize data lakes and data warehouses on Amazon S3, Redshift, and Athena
  • Develop Python-based ETL/ELT frameworks and reusable data transformation modules
  • Integrate multiple data sources (RDBMS, APIs, Kafka/Kinesis, SaaS systems) into unified data models
  • Lead efforts in data modeling, schema design, and partitioning strategies for performance and cost optimization
  • Drive data quality, observability, and lineage using AWS Data Catalog, Glue Data Quality, or third-party tools
  • Define and enforce data governance, security, and compliance best practices (IAM policies, encryption, access control)
  • Collaborate with cross-functional teams (Data Science, Analytics, Product, DevOps) to support analytical and ML workloads
  • Implement CI/CD pipelines for data workflows using AWS CodePipeline, GitHub Actions, or Cloud Build
  • Provide technical leadership, code reviews, and mentoring to junior engineers
  • Monitor data infrastructure performance, troubleshoot issues, and lead capacity planning
Required Skills & Qualifications
  • Bachelor's or Master's degree in Computer Science, Information Systems, or related field
  • 5–10 years of hands-on experience in data engineering or data platform development
  • Expert-level proficiency in Python (pandas, PySpark, boto3, SQLAlchemy)
Advanced experience with AWS Data Services, including:
  • AWS Glue, Lambda, EMR, Step Functions, DynamoDB, EDW Redshift, Athena, S3, Kinesis, Amazon Quicksight.
  • IAM, CloudWatch, CloudFormation / Terraform (for infrastructure automation)
  • Strong experience in SQL, data modeling, and performance tuning
  • Proven ability to design and deploy data lakes, data warehouses, and streaming solutions
  • Solid understanding of ETL best practices, partitioning, error handling, and data validation
  • Hands-on experience in version control (Git) and CI/CD for data pipelines
  • Knowledge of containerization (Docker/Kubernetes) and DevOps concepts
  • Excellent analytical, debugging, and communication skills
Preferred Skills
  • Experience with Apache Spark or PySpark on AWS EMR or Glue
  • Familiarity with Airflow, dbt, or Dagster for workflow orchestration
  • Exposure to real-time data streaming (Kafka, Kinesis Data Streams, or Firehose)
  • Knowledge of Lake Formation, Glue Studio, or DataBrew
  • Experience integrating with machine learning and analytics platforms (SageMaker, QuickSight)
  • Certification: AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect
Soft Skills
  • Strong ownership mindset with focus on reliability and automation
  • Ability to mentor and guide data engineering teams
  • Effective communication with both technical and non-technical stakeholders

  • Senior Data Engineer

    4 days ago

    Hyderabad, Telangana, India ShimentoX Technologies Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Title: Senior Data EngineerLocation: Hyderabad and BangaloreJob SummaryWe are seeking a highly skilled and experienced Senior Data Engineer to join our growing data engineering team. This role will be instrumental in designing, building, and maintaining scalable data pipelines and architectures to support advanced analytics, data science, and business...

  • Senior Data Engineer

    1 day ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture, Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 29th 2025)Translate business rules into technical specifications and implement scalable data solutions.Manage a...

  • Senior Data Engineer

    1 week ago

    Hyderabad, Telangana, India Straive Full time

    Job SummaryWe're looking for aSenior Data Engineerwith 5-8 years of experience to build and maintain scalable, production-grade data pipelines. The ideal candidate is a strong software engineer with hands-on experience in Spark (3.x), Scala, SQL, and Python. You'll be responsible for designing and implementing ETL/ELT solutions, collaborating with teams to...

  • Senior Backend Engineer – Python

    2 weeks ago

    Hyderabad, Telangana, India Indiglobe IT Solutions Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Role : Senior Backend EngineerExp : YearsLocation : Hyderabad.Mode of Hire : Contract-to-HirePayroll : Indiglobe IT Solutions.Mandatory Skills : Python programmingBackend/API developmentCloud experience (AWS or GCP)CI/CD, Docker, LinuxMicroservices architectureDetailed JD: Backend engineer with the following skills -Good programming knowledge in...

  • Azure Data Engineer

    5 days ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Enable Data Incorporated is currently seeking a skilled and experienced Azure Data Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, Enable Data has developed deep expertise across various industries. We work closely with our customers to leverage modern solutions and technologies to drive...

  • Senior Data Engineer

    5 days ago

    Hyderabad, Telangana, India WalkingTree Technologies Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Company DescriptionFounded in 2008, WalkingTree Technologies is a pioneering IT services company specializing in digital and data solutions. We collaborate with our clients to shape and execute their data and digital strategies, unlocking their full potential. Our core strengths in Product Engineering, Digital Transformation, and Modernization create a...

  • Senior Python Developer

    3 days ago

    Hyderabad, Telangana, India Photonx technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Senior Python Developer (4–8+ Years) | [PhotonX Technologies]About the Role We are seeking a Senior Python Developer to join our engineering team and lead the design, development, and scaling of high-performance, distributed, and cloud-native systems. This is a senior-most role that combines hands-on coding with system architecture, technical...

  • Data Engineer – Python

    5 days ago

    Hyderabad, Telangana, India Egen Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Job Overview:We are seeking a highly skilled Data Engineer with strong expertise in AWS cloud services and Python programming. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines, ensuring data availability, quality, and performance across enterprise systems. You will collaborate closely with data...

  • Python Developer

    2 weeks ago

    Hyderabad, Telangana, India Data Unveil Full time

    DescriptionAbout us : Data Unveil is the next generation provider of best-in-class data aggregation services. Our solutions and methodologies have been honed over the past 10 years by experienced practitioners who have collectively supported 100+ brands, 100+ data partners and 25+ new drug launches.Our integrated capabilities create a differentiated value...

  • Senior Data Engineers

    5 days ago

    Hyderabad, Telangana, India SID Information Technologies Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    hiring #Hiring #hyderabad #hyderabadjob #HyderabadJobs #immediatejoiner #opportunity #CareerOpportunity #Senior #TechJobs #dataengineer #DataEngineering #DataArchitect #DataEngineeringJobs #HealthcareDataCloudComputing #BigData #DataAnalytics #PlatformEngineering #Terraform #Kubernetes #SQL #Python #Scala #apachespark #pyspark #ETLpipeline #PySpark...