Infocepts - Senior Data Engineer - Azure Databricks

2 days ago

Pune, Maharashtra, India InfoCepts Technologies Pvt. Ltd. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Description :

Key Responsibilities :

Data Pipeline Development :

- Design and implement robust batch and real-time streaming data pipelines leveraging Azure Databricks and Apache Spark.

- Build scalable and maintainable ETL/ELT workflows to support data ingestion, transformation, and integration from diverse sources.

Data Architecture Implementation :

- Apply the Medallion Architecture to organize data into distinct layers raw, enriched, and curated ensuring clarity, quality, and usability of data assets.

- Collaborate in designing data models and schemas that support analytics and business intelligence needs.

Data Quality & Governance :

- Define and enforce data quality rules and principles to maintain data accuracy and consistency across the pipeline.

- Implement data governance policies and manage metadata using tools like Azure Purview and Unity Catalog.

- Monitor data lineage and ensure compliance with organizational and regulatory standards.

Performance Optimization :

- Optimize Spark jobs, Delta Lake tables, and SQL queries for improved efficiency, reduced latency, and cost-effectiveness.

- Identify and resolve bottlenecks in data processing workflows to ensure high availability and reliability.

Collaboration & Delivery :

- Work closely with data analysts, solution architects, business stakeholders, and other engineering teams to deliver end-to-end data solutions aligned with business goals.

- Participate in Agile/Scrum ceremonies and contribute to continuous improvement initiatives within the team.

Required Qualifications :

- Bachelors or Masters degree in Computer Science, Engineering, Information Technology, or related fields.

- 5 years of experience in data engineering, with strong expertise in Azure Databricks and Apache Spark.

- Proficiency in data integration techniques, ETL/ELT processes, and data pipeline design.

- Experience implementing Medallion Architecture or similar data layering concepts.

- Knowledge of Delta Lake and experience managing large-scale data lakes.

- Familiarity with data quality frameworks and governance tools such as Azure Purview and Unity Catalog.

- Strong SQL skills and experience optimizing complex queries for performance.

- Experience with cloud data storage, processing, and orchestration tools within Azure ecosystem.

- Good understanding of data security, privacy, and compliance requirements.

- Excellent communication, collaboration, and problem-solving skills.

  • Data Engineer- Databricks

    2 days ago

    Pune, Maharashtra, India InfoCepts Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Position: Data Engineer- DatabricksPurpose of the Position:Develop, support and steer end-to-end business intelligence using Databricks.Location: Nagpur/ Pune/ Chennai/ BangaloreKey Responsibilities:Work with business analysts and data architects to translate requirements into technical implementations.Design, develop, implement, and maintain pyspark code...

  • Senior Azure Databricks Engineer

    2 weeks ago

    Pune, Maharashtra, India T&N Business Services Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We seeking a talented and motivated senior Azure Data Engineer. The role will be responsible for developing solutions for risk, regulatory, and compliance projects on AzureRole :Senior Azure Databricks EngineerExp :8 to 12 yrsLocation :Pune/IndoreNotice Period:Immediate to 30 DaysMust-have skills :Azure Databricks heavyYou will play a key role in...

  • Databricks Engineer

    1 day ago

    Pune, Maharashtra, India InfoCepts Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    Position: Data Engineer- DatabricksPurpose of the Position:Develop, support and steer end-to-end business intelligence using Databricks.Location: Nagpur/ Pune/ Chennai/ BangaloreKey Responsibilities:Work with business analysts and data architects to translate requirements into technical implementations.Design, develop, implement, and maintain pyspark code...

  • Azure Databricks Data Engineer

    5 days ago

    Pune, Maharashtra, India Reveille Technologies,Inc Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Hiring: Azure Databricks Data Engineer (FTE | Face-to-Face Interview) Location: Pune Experience: 9+ Years Notice Period: 0–30 Days (Serving candidates preferred)We're looking for an experienced Data Engineer with strong hands-on expertise in Azure Databricks and a solid data background. Key Skills:Primary: Azure, Databricks, ADF, PySpark/PythonSecondary:...

  • Infocepts - Senior Data Engineer - AWS

    1 week ago

    Pune, Maharashtra, India InfoCepts Technologies Pvt. Ltd. Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Description : - Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements. - Should be able to design complex Big Data use cases using AWS Services and Databricks, under guidance of Architect. - Should be able to assist the architect in reasoning about various architecture...

  • Azure Databricks Engineer

    2 weeks ago

    Pune, Maharashtra, India Savi Technologies Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Opening: Azure Databricks EngineerLocation: Pune, Maharashtra (Onsite)Experience Range: 5-6 YearsApply at: -Required Skills:Azure Data Factory (ADF)PySparkAzure Databricks (ADB)SQLJob Description:We're looking for an experienced Azure Databricks Engineer with a background in financial crime analytics and hands-on expertise in:Databricks engineering &...

  • Azure Databricks Developer

    2 weeks ago

    Pune, Maharashtra, India Newscape Consulting Full time ₹ 8,00,000 - ₹ 16,00,000 per year

    We're Hiring: Senior Databricks Engineer / Azure Data Engineer Location:Baner, Pune (Onsite Only) Work Mode:Full-time | Office-based Availability:Immediate Joiners / Notice Period ≤ 15 Days Note:Only local Pune candidates or those willing to relocate to Pune may apply.* L2 interviews will be conductedin person— apply only if you are open to face-to-face...

  • Azure Data Engineer – Databricks

    3 weeks ago

    Pune, Maharashtra, India, Maharashtra Tata Consultancy Services Full time

    Role: Azure Data Engineer – DatabricksLocation: PAN INDIAExperience: 6 to 8 YearsRequired Technical Skill Set**Data Lake architecture, Azure Services – ADLS, ADF, Azure Databricks, SynapseMust-Have**Build the solution for optimal extraction, transformation, and loading of data from a wide variety of data sources using Azure data ingestion and...

  • Senior Databricks Engineer

    2 weeks ago

    Pune, Maharashtra, India Newscape Consulting Full time US$ 80,000 - US$ 1,20,000 per year

    About the RoleWe are seeking a Senior Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions on the Databricks platform. The ideal candidate will be a hands-on expert in data engineering, cloud infrastructure, and distributed data processing, with a deep understanding of how to build reliable and performant...

  • Azure Data Engineer

    7 days ago

    Pune, Maharashtra, India Symphoni Hr Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Locations:Pune, Maharashtra, IndiaMinimum Experience:6Maximum Experience:12ctc lac lac maxSkill Set:Databricks Engineer, Azure Databricks, SQL,Skill to Evaluate:Databricks Engineer, Azure Databricks, SQL,Pyspark, Data warehousing, ADFExperience:5 to 12 YearsLocation:Pune, Maharashtra, IndiaJob Description:Job Title: Senior Data EngineerKey Responsibilities:-...