Data Engineer

3 hours ago

Vijayawada, Andhra Pradesh, India beBeeIngestion Full time ₹ 15,00,000 - ₹ 20,00,000

Ingestion engineers are crucial for our organization's data strategy, responsible for crafting efficient data ingestion pipelines that integrate multiple sources into Databricks.

Key Responsibilities:

  • Data Ingestion Pipelines: Develop and optimize data ingestion pipelines for integrating multiple sources into Databricks.
  • CI/CD Pipeline Implementation: Implement and maintain CI/CD pipelines for data workflows.
  • Kubernetes Deployment: Deploy and manage containerized applications using Kubernetes.
  • Data Integrity and Security: Ensure data integrity, security, and compliance throughout the ingestion process.
  • Collaboration and Troubleshooting: Collaborate with Data Engineers and other stakeholders to streamline data ingestion strategies and troubleshoot and optimize ingestion pipelines for performance and scalability.

Required Skills & Qualifications:

  • Proven Experience: Proven experience in data ingestion and pipeline development.
  • CI/CD Tools: Hands-on experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps, etc.).
  • Kubernetes and Container Orchestration: Strong knowledge of Kubernetes and container orchestration.
  • Databricks and Data Lake Architectures: Experience with Databricks, Spark, and data lake architectures.
  • Programming Languages: Proficiency in Python, Scala, or SQL for data processing.
  • Cloud Platforms: Familiarity with cloud platforms such as AWS, Azure, or GCP.

Preferred Qualifications:

  • Infrastructure as Code: Experience with Infrastructure as Code (Terraform, Helm, etc.).
  • Streaming Data Ingestion: Background in streaming data ingestion (Kafka, Kinesis, etc.).
  • Data Governance and Security: Knowledge of data governance and security best practices.

  • Data Scientist/ML Engineer

    4 days ago

    Vijayawada, Andhra Pradesh, India Quant-data Full time

    We're Hiring: Machine Learning Engineer / Data Engineer (Remote | Full-Time) Build AI-powered credit decisioning systems on Microsoft AzureWe're looking for a Machine Learning Engineer / Data Engineer with 5+ years of experience to join our AI-driven credit lending platform team. In this role, you'll design and deploy scalable ML solutions that power loan...

  • Data Engineer

    3 hours ago

    Vijayawada, Andhra Pradesh, India VSquare Systems Pvt. Ltd. Full time

    Job Description:We are looking for a highly skilled Databricks PySpark Developer to join our data platform implementation team. In this role, you will be instrumental in designing, developing, and maintaining ETL processes to ensure efficient extraction, transformation, and loading of data from various sources into data lake and data warehouse. You will work...

  • Principal Data Engineer

    12 hours ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data Engineering ProfessionalOverview:Develop, optimize and maintain scalable data pipelines to support business intelligence and analytics initiatives.Description:Key ResponsibilitiesPipeline Development: Design, build and deploy robust ETL/ELT pipelines in Azure Databricks (PySpark, SQL, Delta Lake) to ingest, transform and curate governance and...

  • Cloud Data Engineering Leader

    2 days ago

    Vijayawada, Andhra Pradesh, India beBeeCloudEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Unlock your potential as a Cloud Data Engineering Leader at our innovative organization. We are seeking an experienced professional to spearhead the development of scalable data pipelines and analytics platforms in the cloud.The ideal candidate will lead a team of talented data engineers, guiding them to design and deliver high-performance data solutions...

  • Spatial Data Engineer

    11 hours ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    Job Title: Spatial Data EngineerWe are seeking an experienced Spatial Data Engineer to join our team. This role is crucial in the production and management of geospatial data across various projects.The position encompasses tasks such as digitization, geocoding, and conducting surveys when necessary.Key Responsibilities:Geospatial Data Production: Engage in...

  • Leading Data Systems Engineer

    12 hours ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 90,00,000 - ₹ 1,20,00,000

    Data Engineer RoleOur organization is seeking a seasoned Data Engineer to collaborate with our global engineering team on a multi-year project initiative.Key Responsibilities:Cultivate data engineering expertise within the project teamDevelop and maintain large-scale data systems utilizing Palantir Foundry, Workshop, PySpark, and TypescriptRequirements:5-8...

  • Senior Data Engineering Expert

    2 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time US$ 90,000 - US$ 1,20,000

    Job Title: Senior Data EngineerWe are looking for a skilled data engineer to join our team.The ideal candidate will have 5+ years of experience in ETL development and data engineering.In this role, you will be responsible for developing and maintaining data pipelines using FiveTran, DBT Cloud, and custom frameworks.You will work closely with the QC team to...

  • Senior Data Solutions Engineer

    2 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Key ResponsibilitiesDesign, develop and maintain data pipelines and ETL processes on Databricks to ensure high-quality data.Lead the architecture and implementation of scalable, secure and high-performance data solutions that meet business needs.Collaborate with cross-functional teams to gather requirements and deliver actionable insights through data-driven...

  • Cloud Data Engineering Specialist

    2 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Azure Data EngineerTransform business operations by leveraging cutting-edge data engineering techniques.Collaborate with cross-functional teams to drive innovation and growth through data-driven insights.Job Description:As a seasoned data engineer, you will play a pivotal role in designing, building, and maintaining large-scale data systems that...

  • Chief Data Engineer

    14 hours ago

    Vijayawada, Andhra Pradesh, India beBeeDataArchitect Full time ₹ 17,50,000 - ₹ 23,45,000

    Job Overview:We are seeking an experienced Data Architect to lead our data engineering initiatives on Google Cloud Platform (GCP). As a key member of our team, you will be responsible for designing and implementing end-to-end data pipelines and integration solutions for structured and unstructured data sources and targets.The ideal candidate will have...