Lead Data Engineer

4 days ago

Pune, Maharashtra, India beBeeCloudEngineering Full time ₹ 20,00,000 - ₹ 25,00,000
Job Title: Cloud Engineering Lead

This is a leadership role where you will lead the design, development, and management of scalable ETL pipelines using AWS Glue, Databricks, Apache Spark, and Python to process structured and unstructured data from diverse sources.

Main Responsibilities:
  1. Design, develop, and manage scalable ETL pipelines for data processing and analysis.
  2. Build and orchestrate data workflows integrating with services such as AWS Lambda, Step Functions, S3, and Redshift.
  3. Optimize Spark jobs for performance and cost-efficiency across Databricks and AWS Glue environments.
  4. Maintain secure data solutions in AWS, leveraging IAM roles, KMS encryption, and VPC-based security to meet compliance and governance standards.
  5. Migrate legacy ETL jobs and data from on-prem systems to cloud-native architectures on AWS Glue, Redshift, and DynamoDB.
Qualifications We Seek:
  • Experience in designing, implementing data pipelines, building data applications, and data migration on AWS.
  • Strong experience of implementing data lake using AWS services like Glue, Lambda, Step, and Redshift.
  • Experience of Databricks will be an added advantage.
  • Strong experience in Python and SQL.
  • Proven expertise in AWS services such as S3, Lambda, Glue, EMR, and Redshift.
  • Advanced programming skills in Python for data processing and automation.
  • Hands-on experience with Apache Spark for large-scale data processing.
  • Proficiency in SQL for data querying and transformation.
  • Strong understanding of security principles and best practices for cloud-based environments.
  • Experience with monitoring tools and implementing proactive measures to ensure system availability and performance.
  • Excellent problem-solving skills and ability to troubleshoot complex issues in a distributed, cloud-based environment.
  • Strong communication and collaboration skills to work effectively with cross-functional teams.

  • Data Engineering Lead

    6 days ago

    Pune, Maharashtra, India Ascentt Full time US$ 1,25,000 - US$ 1,75,000 per year

    Job Title: Data Engineering LeadJob Summary:We are seeking an experienced Data Engineering Lead to oversee and guide our organization's data engineering function. The ideal candidate will combine deep technical expertise in big data and data engineering with strong leadership skills to drive innovation, best practices, and excellence across our data...

  • Lead Data Engineer

    39 minutes ago

    Pune, Maharashtra, India Rearc Full time US$ 1,04,000 - US$ 1,30,878 per year

    About RearcFounded in 2016, we pride ourselves on fostering an environment where creativity flourishes, bureaucracy is non-existent, and individuals are encouraged to challenge the status quo. We're not just a company; we're a community of problem-solvers dedicated to improving the lives of fellow software engineers.Our commitment is simple - finding the...

  • Lead Data Engineer

    1 week ago

    Pune, Maharashtra, India beBeeEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineering LeaderA seasoned Data Engineer to spearhead our data strategy and drive business outcomes.Job SummaryWe are seeking an experienced Data Engineer to lead our team in driving the development, implementation, and maintenance of data engineering solutions. The successful candidate will have a strong background in data engineering principles,...

  • Lead Data Engineer

    7 days ago

    Pune, Maharashtra, India Mastercard Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Job Title:Lead Data Engineer Overview:OverviewWe are the global technology company behind the world's fastest payments processing network. We are a vehicle for commerce, a connection to financial systems for the previously excluded, a technology innovation lab, and the home of Priceless. We ensure every employee has the opportunity to be a part of something...

  • Data Engineering Lead

    5 days ago

    Pune, Maharashtra, India beBeeDataEngineering Full time US$ 1,30,000 - US$ 1,70,000

    Job Title: Data Engineering LeadWe are seeking a highly skilled and experienced data engineering lead to design, implement, and manage large-scale data solutions using Azure technologies. The ideal candidate will have a proven track record of designing, implementing, and managing complex system integration processes focused primarily on the use of Azure SQL,...

  • Data Operations Engineer

    2 hours ago

    Pune, Maharashtra, India Mars Data Insights Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Title: Data Operations Engineer & RUN SupportSkills:Data Operations Engineering, data manipulation, Python, Talend, GCP, Bigquery, DataIku, ITSM/ticketing tools, Helix, Jira, task management, data pipelines,  RUN Service, data infrastructure, data quality Job Location:Pune Job Type:Fulltime/Hybrid Work Experience:5+ years We are seeking a highly...

  • Lead Data Engineer

    1 week ago

    Pune, Maharashtra, India Mastercard Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...

  • Data Engineering Lead

    3 days ago

    Pune, Maharashtra, India beBeeBusinessDevelopment Full time US$ 1,50,000 - US$ 2,00,000

    Job DescriptionWe are seeking a Head of Business Development with a strong background in data engineering and the pharma domain. This role will be responsible for driving new business, deepening client relationships, and leading our go-to-market strategies across key territories.Drive business growth by identifying and converting new opportunities in the US...

  • Lead Data Engineer

    2 hours ago

    Pune, Maharashtra, India Diggi Growth Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Experience Level: 8+ years (with 3+ in Snowflake)HYBRID MODELRole Overview:We are looking for an experienced Snowflake Team Lead to oversee both development and production support operations within our managed services model. This is a dual-role leadership position requiring strong technical capabilities in Snowflake as well as expertise in managing ongoing...

  • Lead Data Engineer

    6 days ago

    Pune, Maharashtra, India Mastercard Full time US$ 1,25,000 - US$ 1,75,000 per year

    Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...