Senior Data Engineer

4 days ago

Hubli, Karnataka, India beBeeEngineer Full time ₹ 1,50,00,000 - ₹ 2,01,00,000
About the Role

We are looking for a seasoned engineer to lead our data processing initiatives. This position requires strong expertise in Python and its applications in large-scale data pipelines, architecture design, and performance optimization.

Key Responsibilities:
  • Architecture and Design
    • Develop a shared component library for ingestion, parsing, extraction, validation, enrichment, and publishing of data.
    • Create templates and patterns for Apache Beam pipelines and Databricks jobs, standardizing configuration, packaging, versioning, CI/CD, and documentation.
    • Design pluggable interfaces for multiple teams to use without code rewrites.
    • Establish a repository strategy for each use case.
  • Performance and Reliability
    • Profiling and tuning: cProfile, py-spy, line_profiler, memory (tracemalloc), CPU vs I/O analysis.
    • Instrument services with Elastic APM and correlate traces/metrics with Splunk logs; build dashboards and runbooks.
    • Implement concurrency best practices: asyncio for I/O-bound, ThreadPool/ProcessPool for CPU-bound, batching, rate limiting, retries, etc.
    • Implement robust LLM API rate limiting/governance: enforce provider TPM and concurrency caps, request queueing/token budgeting, and emit APM/Splunk metrics (throttle rate, queue depth, cost per job) with alerts.
    • Establish Service Level Objectives (SLOs) for throughput, latency, error rates; set up Dead Letter Queues (DLQs) and recovery patterns.
  • Team Enablement
    • Mentor developers, lead design reviews, codify best practices, write clear documentation and examples.
    • Partner with Machine Learning engineers on the future LLM/SLM path (evaluation harness, safety/PII, cost/perf).

  • Senior Data Engineer

    6 days ago

    Hubli, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,20,00,000

    As a senior data engineer, you will be responsible for designing and implementing scalable, secure, and efficient solutions on the Databricks platform.The ideal candidate will have experience working with Databricks, including platform architecture, data engineering, and data analytics. Proficiency in languages such as Python, Scala, or Java is also...

  • Senior Data Specialist

    4 days ago

    Hubli, Karnataka, India beBeeData Full time US$ 1,00,000 - US$ 1,20,000

    Job Title:Key Responsibilities of a Senior Data SpecialistOverview:Design and implement data pipelines using Python and AWS services such as S3, Redshift, Lambda, and Glue.As a senior data specialist, you will be responsible for developing and maintaining large-scale data systems that support business analytics and reporting requirements.Requirements:7+...

  • Senior Data Engineer

    1 hour ago

    Hubli, Karnataka, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 22,00,000

    Job Title: Data Engineering Scrum MasterA senior data engineer with experience in scrum methodology is required to lead agile teams and ensure data quality.Responsibilities:Lead agile teams using scrum methodologyEnsure data integrity through documentation and processingCollaborate with cross-functional teams to drive business outcomesRequirements:6-8 years...

  • Data Engineer

    4 days ago

    Hubli, Karnataka, India Lenity Health Full time

    Role Overview Location: Hebbal, Bengaluru Work Mode: Monday to Friday | On-site Work Time: 2pm - 11pm Compensation: ₹15-20 LPA (negotiable based on experience) Experience Level: 3+ years About Lenity Health The healthcare industry operates on systems so outdated, that it clogs the flow of innovation and efficient care coordination. These...

  • Highly Skilled Data Engineering Professional

    4 days ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Key Role SummaryA Senior Data Engineer is required to design, develop and maintain large-scale data pipelines and ETL processes.The ideal candidate will have a strong background in data engineering, architecture and integration.

  • Ai Security Engineer

    2 days ago

    Hubli, Karnataka, India NTT DATA Full time

    Req ID 338875 NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a AI Security Engineer - Shared Security Services Engineering to join our team in Banaglore Karn taka IN-KA India ...

  • Data Engineering

    7 days ago

    Hubli, Karnataka, India beBeeInfrastructure Full time ₹ 15,00,000 - ₹ 20,00,000

    Senior Snowflake Developer & Infrastructure Integration ExpertWe're seeking a seasoned Snowflake Developer and Infrastructure Expert to lead full lifecycle implementations, from secure platform setup and cloud integration to high-performance data pipeline development.This role combines data engineering, infrastructure, and DevOps expertise to drive...

  • Senior Cloud Data Specialist

    5 days ago

    Hubli, Karnataka, India beBeeDatabase Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior Redshift Database AdministratorOverviewThis is a senior-level position responsible for overseeing and maintaining scalable cloud environments and data-driven applications.Administer and maintain cloud environments and data applicationsInstall, configure, and update software, services, and applicationsManage and maintain databases using AWS...

  • Highly Skilled Senior Data Architect

    5 days ago

    Hubli, Karnataka, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    ">Senior Data Engineer Opportunity">We're seeking a seasoned professional with exceptional technical expertise in ETL pipeline design, data modeling, and data infrastructure to spearhead the development of our cutting-edge data platform.

  • Senior Data Engineering Leader

    2 days ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,10,00,000

    Job DescriptionWe are seeking a highly skilled and accomplished Data Engineer to lead our Data Warehouse team.The ideal candidate will have 10+ years of experience in software development, with a focus on backend and data engineering.They should possess strong leadership skills, including the ability to lead a team of Sr. Data Engineers and communicate...