Data Engineer Specialist

11 hours ago

Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 16,80,000 - ₹ 2,01,20,000
Job Title: Data Engineer Specialist

We are seeking a skilled data engineer to develop and maintain automated data pipelines supporting business analytics initiatives.

Responsibilities:

• Design and Develop ETL/ELT Pipelines: Utilize PySpark, SQL, and Delta Lake to create production-ready workflows for ingestion, transformation, and curation of metadata.

• Implement Granular Data Quality Checks: Employ profiling logic to capture issue-level metadata, enabling targeted remediation and drill-down capabilities.

• Develop Governance Metrics Automation: Generate metrics for dashboards covering data quality, lineage, job monitoring, access & permissions, query cost, usage & consumption, retention & lifecycle, policy enforcement, sensitive data mapping, and governance KPIs.

• Integrate with Microsoft Purview: Automate asset onboarding, metadata enrichment, classification tagging, and lineage extraction for governance reporting.

• Ensure Data Retention & Policy Compliance: Implement logic for retention tracking and policy compliance monitoring (masking, RLS, exceptions).

• Monitor Job & Query Performance: Track job performance, SLA adherence, and query costs for cost and performance optimization.

• Optimize Metadata Storage: Maintain curated Delta tables for governance metrics, structured for efficient dashboard consumption.

• Collaborate with Teams: Work closely with the lead engineer, QA, and reporting teams to validate metrics and resolve data quality issues.

• Ensure Security & Compliance: Ensure all pipelines meet organizational governance, privacy, and security standards.

Qualifications:

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.

• 4+ years of hands-on data engineering experience, with Azure Databricks and Azure Data Lake.

• Proficiency in PySpark, SQL, and ETL/ELT pipeline design.

• Demonstrated experience building granular data quality checks and integrating governance logic into pipelines.

• Working knowledge of Microsoft Purview for metadata management, lineage capture, and classification.

• Experience with Azure Data Factory or equivalent orchestration tools.

• Understanding of data modeling, metadata structures, and data cataloging concepts.

• Strong debugging, performance tuning, and problem-solving skills.

• Ability to document pipeline logic and collaborate with cross-functional teams.

  • Data Engineering Specialist

    2 days ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    **Job Title:** Data Engineering SpecialistAs a seasoned data engineer, you will play a pivotal role in designing, developing, and maintaining high-performance data pipelines and ETL processes on Databricks. You will lead the architecture and implementation of scalable, secure, and efficient data solutions.

  • Digital Engineering Networking Engineer Specialist

    3 days ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Req ID 336414 NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Digital Engineering Networking Engineer Specialist to join our team in Bangalore Karn taka IN-KA India IN At...

  • Digital Engineering Networking Engineer Specialist

    2 weeks ago

    Bengaluru, Karnataka, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Networking Engineer Specialist to join our team in Bangalore, Karntaka (IN-KA), India (IN).At NTT...

  • Urgent Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - ETL to join our team in Bangalore, Karntaka (IN-KA), India (IN).Job Duties:. Migrate ETL workflows from...

  • Data Engineering Specialist

    5 days ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time

    Job Title: Data Engineering SpecialistJob Description:As a seasoned professional in data engineering, you will be responsible for designing, developing, and maintaining complex data pipelines using cutting-edge technologies like Google Cloud Platform (GCP). Your expertise in GCP services such as Dataflow, Dataproc, Pub/Sub, Big Query, Cloud Storage, and...

  • Senior Associate Data Engineer

    8 hours ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time

    Job DescriptionMake an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion its a place where you can grow, belong and thrive.Your day at NTT DATAThe...

  • Specialist - Data Engineering

    4 weeks ago

    Bengaluru, Karnataka, India Talpro Full time

    Job DescriptionSenior Specialist - Data EngineeringJob Summary : We are seeking a highly skilled Senior Specialist - Data Engineering with strong Spark Scala knowledge, mandatory for a significant and complex engagement. An AWS background is welcome. As a Big Data Full Stack Developer, you will work with various underlying tools such as Spark/Scala, Oozie,...

  • Senior Data Engineering Specialist

    11 hours ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Data Engineering SpecialistWe are seeking a highly skilled Senior Data Engineering Specialist to join our team. The ideal candidate will have extensive experience in designing, developing, and implementing large-scale data integration pipelines using Informatica BDM/DEI.Job Description:This is a full-time on-site role for a seasoned data engineer...

  • Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India NTT DATA North America Full time

    Job DescriptionReq ID: 330827NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Bengaluru, Karntaka (IN-KA), India (IN).Sr Developer - Data...

  • Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India ConcertAI Full time

    Job DescriptionJob RequirementsLooking for energetic, self-motivated and exceptional Data Engineer to work on extraordinary enterprise products based on AI and Big Data engineering. He/she will work with star team of Architects, Data Scientists/AI Specialists, Data Engineers, Integration Specialists and UX developers.Work ExperienceAdvanced working SQL...