Infocepts - Senior Data Engineer - Azure Databricks

1 day ago

Pune, Maharashtra, India InfoCepts Technologies Pvt. Ltd. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Description :

Key Responsibilities :

Data Pipeline Development :

- Design and implement robust batch and real-time streaming data pipelines leveraging Azure Databricks and Apache Spark.

- Build scalable and maintainable ETL/ELT workflows to support data ingestion, transformation, and integration from diverse sources.

Data Architecture Implementation :

- Apply the Medallion Architecture to organize data into distinct layers raw, enriched, and curated ensuring clarity, quality, and usability of data assets.

- Collaborate in designing data models and schemas that support analytics and business intelligence needs.

Data Quality & Governance :

- Define and enforce data quality rules and principles to maintain data accuracy and consistency across the pipeline.

- Implement data governance policies and manage metadata using tools like Azure Purview and Unity Catalog.

- Monitor data lineage and ensure compliance with organizational and regulatory standards.

Performance Optimization :

- Optimize Spark jobs, Delta Lake tables, and SQL queries for improved efficiency, reduced latency, and cost-effectiveness.

- Identify and resolve bottlenecks in data processing workflows to ensure high availability and reliability.

Collaboration & Delivery :

- Work closely with data analysts, solution architects, business stakeholders, and other engineering teams to deliver end-to-end data solutions aligned with business goals.

- Participate in Agile/Scrum ceremonies and contribute to continuous improvement initiatives within the team.

Required Qualifications :

- Bachelors or Masters degree in Computer Science, Engineering, Information Technology, or related fields.

- 5 years of experience in data engineering, with strong expertise in Azure Databricks and Apache Spark.

- Proficiency in data integration techniques, ETL/ELT processes, and data pipeline design.

- Experience implementing Medallion Architecture or similar data layering concepts.

- Knowledge of Delta Lake and experience managing large-scale data lakes.

- Familiarity with data quality frameworks and governance tools such as Azure Purview and Unity Catalog.

- Strong SQL skills and experience optimizing complex queries for performance.

- Experience with cloud data storage, processing, and orchestration tools within Azure ecosystem.

- Good understanding of data security, privacy, and compliance requirements.

- Excellent communication, collaboration, and problem-solving skills.

  • Data Engineer- Databricks

    2 days ago

    Pune, Maharashtra, India InfoCepts Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Position: Data Engineer- DatabricksPurpose of the Position:Develop, support and steer end-to-end business intelligence using Databricks.Location: Nagpur/ Pune/ Chennai/ BangaloreKey Responsibilities:Work with business analysts and data architects to translate requirements into technical implementations.Design, develop, implement, and maintain pyspark code...

  • Senior Azure Databricks Engineer

    2 weeks ago

    Pune, Maharashtra, India T&N Business Services Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We seeking a talented and motivated senior Azure Data Engineer. The role will be responsible for developing solutions for risk, regulatory, and compliance projects on AzureRole :Senior Azure Databricks EngineerExp :8 to 12 yrsLocation :Pune/IndoreNotice Period:Immediate to 30 DaysMust-have skills :Azure Databricks heavyYou will play a key role in...

  • Databricks Engineer

    1 day ago

    Pune, Maharashtra, India InfoCepts Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    Position: Data Engineer- DatabricksPurpose of the Position:Develop, support and steer end-to-end business intelligence using Databricks.Location: Nagpur/ Pune/ Chennai/ BangaloreKey Responsibilities:Work with business analysts and data architects to translate requirements into technical implementations.Design, develop, implement, and maintain pyspark code...

  • Azure Data Engineer – Databricks

    12 hours ago

    Pune, Maharashtra, India, Maharashtra Tata Consultancy Services Full time

    Role: Azure Data Engineer – DatabricksLocation: PAN INDIAExperience: 6 to 8 YearsRequired Technical Skill Set**Data Lake architecture, Azure Services – ADLS, ADF, Azure Databricks, SynapseMust-Have**Build the solution for optimal extraction, transformation, and loading of data from a wide variety of data sources using Azure data ingestion and...

  • Azure Data Engineer

    7 days ago

    Pune, Maharashtra, India Symphoni Hr Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Locations:Pune, Maharashtra, IndiaMinimum Experience:6Maximum Experience:12ctc lac lac maxSkill Set:Databricks Engineer, Azure Databricks, SQL,Skill to Evaluate:Databricks Engineer, Azure Databricks, SQL,Pyspark, Data warehousing, ADFExperience:5 to 12 YearsLocation:Pune, Maharashtra, IndiaJob Description:Job Title: Senior Data EngineerKey Responsibilities:-...

  • Senior AWS Databricks Data Engineer

    2 days ago

    Pune, Maharashtra, India Sapphire Manpower Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Candidate should have experience in Data Engineering with AWS Databricks Please don't require Azure Databricks

  • Azure Data Engineer

    3 days ago

    Pune, Maharashtra, India TEKIT SOFTWARE SOLUTIONS PRIVATE LIMITED Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Job Title : Azure Data EngineerLocation : Pune, India (On-site only)Experience Required : 7-8 YearsJob Description : We are looking for an experienced Azure Data Engineer with strong proficiency in Azure Data Ingestion Services and Data Engineering to join our dynamic team. As a key member of our data engineering team, you will play a crucial role in...

  • Azure Data Engineer

    3 days ago

    Pune, Maharashtra, India Vidushi Infotech Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are seeking a highly skilled and experienced Azure Data Engineer to join our data team. The ideal candidate will have over five years of professional experience and possess deep expertise in building, managing, and optimizing scalable data pipelines and solutions within the Microsoft Azure ecosystem. This role requires a strong focus on Databricks,...

  • Azure data bricks engineer

    3 days ago

    Pune, Maharashtra, India Changeleaders Full time ₹ 40,00,000 - ₹ 1,20,00,000 per year

    Data Engineer having good experience on Azure Databricks and PythonMust Have Databricks Python AzureGood to have ADFCandidate must be proficient in Databricks

  • Azure Data Engineer

    2 days ago

    Pune, Maharashtra, India Neev Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are seeking a highly motivated and skilledAzure Data Engineerwith 6+ years of hands-on experience in designing, developing, and maintaining secure and scalable data pipelines. The ideal candidate will have strong expertise inAzure Data Factory, Synapse Analytics, and Databricks, along with proven skills inETL development, data modeling, and automation....