Data Pipeline Architect

5 days ago

Hubli, Karnataka, India beBeeAzure Full time ₹ 15,00,000 - ₹ 25,00,000
Senior Azure Databricks Engineer

We are seeking an expert Senior Azure Databricks Engineer to spearhead the design, development, and optimization of data ingestion pipelines for integrating multiple sources into Databricks.

Key Responsibilities:

  • Design and develop scalable data ingestion pipelines for integrating multiple sources into Databricks.
  • Implement and maintain Continuous Integration/Continuous Deployment (CI/CD) pipelines for data workflows.
  • Deploy and manage containerized applications using Kubernetes.
  • Evaluate data integrity, security, and compliance throughout the ingestion process.
  • Collaborate with Data Engineers to streamline data ingestion strategies.
  • Optimize ingestion pipelines for performance and scalability.

Required Skills & Qualifications:

  • Proven experience in data ingestion and pipeline development.
  • Hands-on experience with CI/CD tools such as GitHub Actions, Jenkins, or Azure DevOps.
  • Strong knowledge of Kubernetes and container orchestration.
  • Experience with Databricks, Spark, and data lake architectures.
  • Proficiency in Python, Scala, or SQL for data processing.
  • Familiarity with cloud platforms such as AWS, Azure, or GCP.
  • Strong analytical and problem-solving skills.

Preferred Qualifications:

  • Background in streaming data ingestion (Kafka, Kinesis, etc.).
  • Knowledge of infrastructure as code (Terraform, Helm, etc.).
  • Understanding of data governance and security best practices.

Job Benefits:

  • Opportunity to work with cutting-edge technologies.
  • Collaborative team environment.
  • Potential for career growth and professional development.

  • Highly Skilled Data Pipeline Architect

    5 days ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Senior Data EngineerAbout the RoleWe are seeking a highly skilled and experienced data engineer to play a critical role in designing, developing, and maintaining complex data pipelines.Key Responsibilities:Data Pipeline Development and MaintenanceUtilize various tools and technologies to design and implement efficient ETL pipelines.Data...

  • Chief Data Pipeline Architect

    13 hours ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Key Roles of a Data EngineerA Data Engineer at CodeVyasa plays a pivotal role in designing, developing and maintaining robust data pipelines for efficient ingestion, transformation and storage of large-scale data.Responsibilities include working with Big Data ecosystems (Hadoop, Spark, Hive) to process structured and unstructured data, developing efficient...

  • Data Pipeline Specialist

    2 days ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    As a Data Engineer, you will play a pivotal role in crafting and implementing scalable data pipelines to support our growing data needs. Your expertise in Bigdata, Pyspark, Python, Hadoop/HDFS, Spark, and GCP/any cloud will enable you to collaborate with analytics and business teams to refine data models and foster data-driven decision making across the...

  • Data Pipeline Expert

    3 days ago

    Hubli, Karnataka, India beBeeEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Databricks Engineer OpportunityWe are seeking a highly skilled Senior Databricks Data Engineer to join our dynamic data engineering team. The ideal candidate will be well-versed in PySpark, CI/CD pipelines, and Terraform for infrastructure as code.Key Responsibilities:Design, develop, and maintain scalable and high-performance data pipelines using...

  • Data Pipelines Engineer

    4 days ago

    Hubli, Karnataka, India beBeeData Full time ₹ 12,00,000 - ₹ 18,00,000

    Big Data SpecialistWe are seeking a highly skilled professional to design, develop, and maintain robust ETL/ELT data pipelines for our organization.Key Responsibilities:Design and develop efficient ETL/ELT data pipelines using AWS services such as Lambda, DMS, and Glue.Develop and optimize Snowflake stored procedures to implement complex business logic and...

  • Data Pipeline Specialist

    3 days ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 2,50,00,000

    About the RoleWe are seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will play a key role in designing and implementing scalable data pipelines using Azure Databricks, PySpark, and Delta Lake.ResponsibilitiesDesign and implement ETL/ELT pipelines using Azure Databricks and PySpark.Develop high-quality database programming...

  • Chief Data Architect

    5 days ago

    Hubli, Karnataka, India beBeeDataArchitecture Full time ₹ 18,00,000 - ₹ 20,00,000

    Job Summary:We are seeking a highly skilled Chief Data Architect to lead our data engineering efforts and drive business growth through strategic data management.About the Role:The ideal candidate will have extensive experience in designing and implementing scalable data pipelines, working with streaming data, and collaborating with cross-functional teams to...

  • Expert Data Pipeline Specialist

    2 days ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Data Engineer Job OpportunityThreatXIntel is a cyber security company focused on protecting businesses and organizations from cyber threats.We offer customized cloud security, web and mobile security testing, and DevSecOps services to meet the specific needs of our clients.A key requirement for this role is designing, developing, and optimizing data...

  • Expert Data Pipeline Designer

    2 days ago

    Hubli, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Freelance Data Engineer PositionWe are seeking a seasoned Freelance Data Engineer with exceptional skills in designing, building, and optimizing large-scale data pipelines. As part of our team, you will collaborate closely with internal teams and clients to deliver robust, scalable, and high-performance data solutions.Key Responsibilities:Develop ETL/ELT...

  • Chief Data Architect

    14 hours ago

    Hubli, Karnataka, India beBeeDataModeling Full time ₹ 15,00,000 - ₹ 22,50,000

    Transform Your Career with a Data Modeler RoleWe're seeking a highly skilled and experienced professional to design, develop, and optimize scalable data pipelines.Design and implement robust data warehousing solutions using cloud-native technologies.Collaborate with stakeholders and consultants to understand data requirements and translate them into...