
Data Pipeline Developer
1 week ago
Job Opportunity
We are seeking an Ingestion Specialist with expertise in data pipeline development and Kubernetes to fill a key role in our organization.
This position requires a strong understanding of data ingestion processes, focusing on integrating various data sources into Databricks.
The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support scalable and efficient ingestion workflows.
Key Responsibilities:
- Design, develop, and optimize data ingestion pipelines for integrating multiple sources into Databricks.
- Implement and maintain CI/CD pipelines for data workflows.
- Deploy and manage containerized applications using Kubernetes.
- Ensure data integrity, security, and compliance throughout the ingestion process.
- Collaborate with Data Engineers and other stakeholders to streamline data ingestion strategies.
- Troubleshoot and optimize ingestion pipelines for performance and scalability.
Required Skills & Qualifications:
- Proven experience in data ingestion and pipeline development.
- Hands-on experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps, etc.).
- Strong knowledge of Kubernetes and container orchestration.
- Experience with Databricks, Spark, and data lake architectures.
- Proficiency in Python, Scala, or SQL for data processing.
- Familiarity with cloud platforms such as AWS, Azure, or GCP.
- Strong problem-solving and analytical skills.
-
Lead Data Pipeline Developer
6 days ago
Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000Job DescriptionWe are seeking a skilled Data Engineer to join our team in Mumbai. The ideal candidate will have strong experience in PySpark and hands-on expertise in building scalable data pipelines.The successful candidate will work on designing, developing, and maintaining batch and streaming data pipelines using Spark SQL, DataFrame, and RDD APIs. They...
-
Data Pipeline Architect
1 week ago
Palakkad, Kerala, India beBeeETLDeveloper Full time ₹ 1,50,00,000 - ₹ 2,00,00,000We are seeking a skilled professional to fill the role of an ETL Specialist with expertise in designing and optimizing complex data pipelines using Talend Open Studio.Key qualifications for this position include 8 to 10 years of experience working with large datasets, proficiency in SQL and Python programming languages, as well as hands-on experience with...
-
Expert Data Pipeline Architect
2 days ago
Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000Job Title:A Data Engineer is needed to design, build, and maintain large-scale data pipelines using Google Cloud Platform.Responsibilities:Develop, construct, test, and maintain data acquisition pipelines for structured and unstructured data in GCP.Create complex datasets based on business requirements.Construct 'big data' pipeline architecture.Identify...
-
Chief Data Pipeline Architect
5 hours ago
Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000Data Engineer Role OverviewSeeking a skilled data engineer to design, develop, and maintain robust data pipelines. Responsibilities include extracting data from Oracle Symphony via APIs, processing and storing it in the Databricks Lakehouse platform, and integrating it into Oracle EPM.Key Responsibilities:Design and implement end-to-end data pipelines from...
-
Data Pipeline Specialist
2 days ago
Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000Data Engineering ProfessionalWe are seeking a skilled data engineer to join our analytics division.You will be part of our data analytics and business intelligence vertical, responsible for conceptualizing and owning the build out of problem-solving data marts for consumption by data science and business intelligence teams. You will evaluate design and...
-
Senior Data Engineer
5 days ago
Palakkad, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Job DescriptionWe are seeking a highly skilled professional to join our team as a Big Data Engineer. In this role, you will be responsible for designing, building, and maintaining robust data pipelines that handle high-volume financial data.The ideal candidate will have experience with data pipeline tools, cloud platforms, and data warehousing. They will...
-
Data Engineer for Scalable Data Pipelines
2 days ago
Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 19,00,000We are seeking a highly skilled Data Engineer to lead the design and development of scalable data pipelines. The ideal candidate will have expertise in data engineering principles, with a strong understanding of cloud-based data platforms.Key Responsibilities:Design and develop efficient data pipelines using Airflow and SnowflakeCollaborate with...
-
Senior Data Pipeline Engineer
5 days ago
Palakkad, Kerala, India beBeeETL Full time ₹ 18,00,000 - ₹ 25,40,000Job OpportunitySeeking a highly skilled ETL Developer to design, develop, and maintain data pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda for ingestion from varied sources.As an experienced ETL professional, you will be responsible for building and optimizing data flows for loading curated datasets into Snowflake, leveraging best practices for...
-
Senior Data Pipeline Specialist
2 days ago
Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000Cloud Data Engineer PositionLemongrass is seeking a skilled Cloud Data Engineer to design, develop, and maintain scalable data pipelines using cloud-native tools. The ideal candidate will have extensive experience with AWS, Azure, and GCP, as well as knowledge of data governance, security, and compliance best practices.">Key Responsibilities:Design, develop,...
-
Chief Data Pipeline Engineer
6 days ago
Palakkad, Kerala, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,00,00,000Key Responsibilities:We are seeking a skilled Data Architect to develop and maintain data pipelines using Python and AWS (S3, Redshift, Lambda, Glue).Develop scalable data architectures using AWS services.Implement and optimize ETL processes for large datasets using Python and AWS Glue.Collaborate with cross-functional teams to design and implement data...