Expert Data Pipelines Developer

12 hours ago


Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

We are seeking a skilled and experienced Azure Databricks Engineer to build scalable data pipelines and streaming architectures using Azure-native technologies.

Key Responsibilities:
  • Design, develop, and optimize data pipelines using Databricks (PySpark) for batch and real-time data processing.
  • Implement CDC (Change Data Capture) and Delta Live Tables/Autoloader to support near-real-time ingestion.
  • Integrate various structured and semi-structured data sources using ADF, ADLS, and Kafka (Confluent).
  • Develop CI/CD pipelines for data engineering workflows using GitHub Actions or Azure DevOps.
  • Write efficient and reusable SQL and Python code for data transformations and validations.
  • Ensure data quality, lineage, governance, and security across all ingestion and transformation layers.
Required Skills & Qualifications:
  • Strong proficiency in SQL, Python, and PySpark.
  • Proven experience with Azure Databricks, including notebooks, jobs, clusters, and Delta Lake.
  • Experience with Azure Data Lake Storage (ADLS Gen2) and Azure Data Factory (ADF).
  • Hands-on with Confluent Kafka for streaming data integration.
  • Strong understanding of Autoloader, CDC mechanisms, and Delta Lake-based architecture.
  • Experience implementing CI/CD pipelines using GitHub and/or Azure DevOps.
  • Knowledge of data modeling, data warehousing, and data security best practices.
Benefits:

This role offers the opportunity to work on critical data assets and support regulatory, risk, and operational reporting use cases.

Preferred Qualifications:
  • Azure certifications (e.g., Azure Data Engineer Associate).
  • Experience with tools such as Delta Live Tables, Unity Catalog, and Lakehouse architecture.


  • Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 2,51,20,000

    Data Engineer OpportunityWe are seeking an experienced data engineer to join our team in a role that will involve designing, developing, testing, and supporting data pipelines and applications using Python, PySpark, and SQL on AWS.Key Responsibilities:Designing and developing scalable and efficient data pipelines using Python, PySpark, and SQL on...


  • Chennai, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job SummarySnowflake DBT Developer: Design and Develop Data PipelinesKey Responsibilities:Develop robust data pipelines using DBT, integrating Talend data, and ensuring high-quality data in big data environments.Write complex SQL queries against Snowflake and create scripts in Unix, Python, etc. to perform Extract, Load, and Transform operations.Maintain and...


  • Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data EngineerWe are seeking a highly skilled and motivated Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and managing scalable data pipelines.Key Responsibilities:Design and implement data pipelines using GCP services such as BigQuery, Dataflow, Pub/Sub, Cloud Storage, Composer, and...


  • Chennai, Tamil Nadu, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 25,00,000

    Senior Data Engineer Job OpportunityWe are looking for a highly skilled Senior Data Engineer to lead our team in developing efficient data pipelines, following best practices and standards.Key Responsibilities:Develop scalable ETL pipelines that meet business requirements and adhere to company standards.Perform thorough integration testing of created...

  • Data Pipeline Specialist

    43 minutes ago


    Chennai, Tamil Nadu, India beBeeDataOperator Full time ₹ 5,00,000 - ₹ 8,00,000

    Job Title">We are seeking a skilled Data Support Operator to join our team. As a Data Support Operator, you will play a critical role in ensuring the smooth operation of our data pipelines and analytics technologies.">Your primary responsibility will be to connect multiple open and closed sourced systems and transition raw data to predictive information for...


  • Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 27,00,000

    Senior Data EngineerYou will play a key role in driving the development and implementation of our data pipeline, working closely with DW/BI leads to understand new ETL pipeline requirements.With 5+ years of experience, you will collaborate with stakeholders to identify gaps in existing pipelines and develop data models for reporting needs.As a seasoned data...


  • Chennai, Tamil Nadu, India beBeeIntegration Full time ₹ 18,00,000 - ₹ 21,00,000

    System Integration SpecialistThe ideal candidate will possess a strong understanding of ITIL best practices and a proven track record in monitoring and maintaining data models, jobs, and schedules. They will be proficient in working with integration technologies such as APIs, REST, SOAP, ETL, and Cloud Data Platforms.Key Responsibilities:Monitor integration...


  • Chennai, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    \As a key member of our Data Engineering team, you will play a critical role in designing and developing scalable data infrastructure that supports business intelligence.\The ideal candidate has strong SQL skills, hands-on experience with cloud ETL tools, and a passion for building robust data pipelines.\Required Skills and Qualifications:\\5–8 years of...


  • Chennai, Tamil Nadu, India beBeePipeline Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OverviewWe are seeking a seasoned data pipeline expert to lead the design and implementation of complex data pipelines using Snaplogic.Key Responsibilities:Broadening expertise in Snap logic through building robust pipelines with error handling, scheduling tasks & alerts, Fault Tolerance, and Error Percolation.Familiarity with Expressions language for...


  • Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 8,00,000 - ₹ 15,00,000

    As a Data Engineer, you will play a crucial role in designing, developing and implementing scalable and high-performance data pipelines. Key Responsibilities:Data IngestionTranslate business requirements into technical specifications and solutions.Create and maintain documentation for data processes and flows.Perform data validation, profiling, cleansing and...