Sr. Data Engineer

4 days ago

Pune, Maharashtra, India Systems Plus Full time ₹ 6,00,000 - ₹ 18,00,000 per year

Location: Pune

Employment Type: full-time

Designation: Sr. Data Engineer

Job Details

Role Overview

The ideal candidate will have strong hands-on experience working with Azure Databricks, Spark (PySpark), Delta Lake, Azure Data Factory, and Python as well as a solid understanding of data engineering best practices, governance, and CI/CD automation.

Key Responsibilities

Data Engineering & Pipeline Development

  • Design, develop, and optimize ETL/ELT pipelines using Azure Databricks (PySpark, SQL, Delta Lake)
  • Use Python extensively for scripting, pipeline orchestration, data transformation, and custom logic implementation
  • Develop metadata-driven ingestion frameworks across sources such as SQL Server, APIs, and file-based feeds (CSV, JSON, etc.)
  • Design and maintain Bronze, Silver, Gold layers in ADLS Gen2
  • Apply advanced Spark optimization techniques: partitioning, broadcast joins, caching, memory tuning

SQL Development & Data Modeling

  • Write optimized, scalable SQL queries for large datasets
  • Support dimensional modeling, data marts, and curated datasets for analytics use cases

Governance & Security

  • Implement Unity Catalog for access control, lineage, and data auditing
  • Work with Azure AD, Key Vault, and policies for securing access to data and secrets

DevOps & CI/CD Automation

  • Collaborate on CI/CD pipeline setup for Databricks notebooks, jobs, and ADF pipelines using Azure DevOps
  • Apply Git-based version control practices and deployment automation for data workloads
  • Contribute to infrastructure-as-code using Terraform or Bicep

Must-Have Qualifications

  • 5+ years of hands-on experience in data engineering roles
  • Strong experience with Azure Databricks and Apache Spark (PySpark)
  • Strong proficiency in Python for data engineering and scripting tasks
  • Experience building pipelines in Azure Data Factory or Synapse Pipelines
  • Proficiency with SQL, Delta Lake, ADLS Gen2, and Databricks Notebooks
  • Solid understanding of medallion architecture and data lake principles
  • Experience working with structured and semi-structured data
  • Exposure to CI/CD tools (Azure DevOps, Git) and Infrastructure-as-Code
  • Strong problem-solving, debugging, and optimization skills

Nice-to-Have Skills

  • Experience with Unity Catalog, Delta Live Tables (DLT), or Delta Sharing
  • Familiarity with Azure Synapse SQL Pools (Dedicated/Serverless)
  • Understanding of network security (Private Link, Key Vault, AAD integration)
  • Working knowledge of Terraform or ARM templates
  • Experience in data governance and compliance frameworks

Soft Skills

  • Proactive communication with remote/distributed teams
  • Ability to work under deadlines and adapt in a fast-paced environment
  • Strong documentation and collaboration practices

  • Sr. Data Engineer

    2 weeks ago

    Pune, Maharashtra, India ImmersiveData Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Description: Sr. Data EngineerLocation: Unit No.506, 5th Floor Solitaire Business Hub, Survey Number: 27/1, Balewadi high Street, Baner,Pune-411045Experience: 4+ yearsPosition:Sr. Data EngineerAbout Immersive At Immersive , we are building next-generation AI and data-driven platforms that help enterprises unlock the true power of their data. We...

  • Sr. Data Integration Engineer

    2 weeks ago

    Pune, Maharashtra, India Orbia Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    We are currently looking for an experiencedSr. Data Integration Engineer (SAP BODS)to join a fast-growing product-based organisation in Pune (Baner).The Data Integration Developer will focus on hands-on development, including table replications, extractors, and transformations, ensuring smooth and accurate data flow between systems such as SAP, data...

  • Sr. Data Engineer

    7 days ago

    Pune, Maharashtra, India Arting Digital Private Limited Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Posting title: Sr. Data Engineer (Databricks)Experience: 5+ YearsLocation: Pune/ AhmedabadWork mode: HybridPrimary skills: Databricks, Data Modelling/ Data Lakehouse, SQL, Python/Scala, Azure Data Factory (ADF)/AWS GlueQualification: Any Technical DegreeRoles & Responsibilities:Data Architecture & Development: Lead the design, development, and optimization...

  • Sr Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Mastercard Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...

  • Sr Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Mastercard Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...

  • Sr. Big Data Engineer

    1 week ago

    Pune, Maharashtra, India DV Consultancy Full time ₹ 42,00,000 per year

    Job profile - Sr Big Data EngineerJob Location - PuneExperience required yearsCTC - 35 LPALooking for immediate joiners.For detailed JD, kindly send your resume at Job Type: Full-timePay: Up to ₹3,500,000.00 per yearApplication Question(s):How many years of experience do you have?What is your current CTC?What is your expected CTC?Work Location: In person

  • Sr. Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Barclays Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Join us as a Sr. Data Engineer role at Barclays, where you'll spearhead the evolution of our digital landscape, driving innovation and excellence. You'll harness cutting-edge technology to revolutionise our digital offerings, ensuring unapparelled customer experiences .You may be assessed on the key critical skills relevant for success in role, such as...

  • Data Engineer

    1 week ago

    Pune, Maharashtra, India Data Axle Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    About Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and leveraging proprietary...

  • Sr. Data Engineer

    1 week ago

    Pune, Maharashtra, India Nts Automation Labs Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    We are seeking a highly skilled Data Architect with expertise in AWS cloud services and data integration to join our team. The ideal candidate will be responsible for architecting, designing, and implementing scalable, secure, and high-performance data solutions. This role requires close collaboration with cross-functional teams and leadership in driving...

  • Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Jash Data Sciences Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Do you love solving real-world data problems with the latest and best techniques? And having fun while solving them in a team Then come join our high-energy team of passionate data people. Jash Data Sciences is the right place for you.We are a cutting-edge Data Sciences and Data Engineering startup based in Pune, India.We believe in continuous learning and...