
Data Pipeline Specialist
3 days ago
We are seeking a skilled Backend Developer to join our team. The ideal candidate will have a strong background in developing scalable data collection pipelines and implementing automated web data extraction solutions.
The successful candidate will be responsible for designing, developing, and maintaining large-scale data pipeline systems that process financial documents from diverse web sources. They will also work closely with cross-functional teams to ensure technical decisions align with business goals.
- Main Responsibilities:
- Design and develop scalable data collection pipelines
- Implement automated web data extraction solutions
- Collaborate with cross-functional teams
- Monitor and troubleshoot data pipelines
Required Skills:
- Technical Skills:
- Python programming language
- Experience with web scraping technologies (Selenium, BeautifulSoup, Scrapy)
- AWS services (Lambda, EC2, ECS, S3)
- ETL processes, data pipeline management, workflow tools (Apache Airflow)
- Docker for backend services and web data extraction scripts
- Proxy services
Benefits:
- Competitive salary package
- Top-notch equipment of your choice
- Wellness allowance
- Flexible work setup: work remotely or from our offices
Key Performance Indicators (KPIs):
- Data quality and integrity
- Pipeline efficiency and scalability
- Timely data delivery and availability
-
Data Pipeline Specialist
1 day ago
Bengaluru, Karnataka, India beBeeDataPipeline Full time ₹ 12,00,000 - ₹ 25,00,000Job Opportunity: Data Pipeline SpecialistWe are seeking a skilled data pipeline expert to validate and optimize our data transformations.ETL testing methodologies and best practicesDebugging data issues and resolving discrepanciesWorking with large datasets in cloud environmentsThe ideal candidate should have hands-on experience in Databricks, Python, SQL,...
-
Data Pipeline Specialist
2 days ago
Bengaluru, Karnataka, India beBeeDataPipeline Full timeJob Title: Data Pipeline SpecialistA leading IT consulting and software solutions provider is seeking a highly skilled professional to design, build, and maintain scalable and reliable data pipelines.This role involves developing ETL processes to ensure data quality, integrity, and consistency. The ideal candidate will collaborate with analysts, scientists,...
-
Data Pipeline Specialist
7 days ago
Bengaluru, Karnataka, India beBeeDataPipeline Full time US$ 1,00,000 - US$ 1,50,000Job Overview- We are seeking a highly skilled Data Pipeline Specialist to join our team. The ideal candidate will have experience building and productionizing data pipelines using industry-leading technologies.- A strong background in working with complex SQL datasets is required, as well as expertise in building efficient SQL scripts using BigQuery Syntax-...
-
Senior Data Pipeline Specialist
3 days ago
Bengaluru, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000We are seeking a highly skilled Senior Data Pipeline Specialist to join our team.As a Senior Data Pipeline Specialist, you will be responsible for designing and developing real-time and batch data pipelines using tools like Apache NiFi, Apache Kafka, and Apache Airflow.Design and develop scalable data pipelines that meet the needs of various business...
-
Data Pipeline Architect
6 days ago
Bengaluru, Karnataka, India beBeeAzure Full time ₹ 15,00,000 - ₹ 25,00,000Freelance Azure Data Factory SpecialistWe are seeking an experienced Freelance Azure Data Factory (ADF) Specialist to support our data integration and ETL needs.The ideal consultant will have strong hands-on expertise in Azure ADF pipelines, data orchestration, and transformations to ensure smooth, scalable, and secure data flows across multiple sources and...
-
Data Pipeline Engineer
7 days ago
Bengaluru, Karnataka, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,00,00,000The Role of Senior Data Integration SpecialistJob Overview:We are seeking an experienced Senior Data Integration Specialist to join our team. This is a key position that will play a vital role in the smooth operation of our enterprise data integration platform.As a Senior Data Integration Specialist, you will be responsible for designing, implementing, and...
-
Data Pipeline Specialist
2 weeks ago
Bengaluru, Karnataka, India beBeeDataEngineer Full timeJob Title: Azure Databricks EngineerAs a skilled Data Engineer, you will play a pivotal role in our dynamic team.High Impact Required: Immediate joiners only. Please note that candidates with less than 12 months of experience will not be considered.Key Responsibilities:Our ideal candidate will have strong expertise in building and optimizing data pipelines...
-
Data Pipelines Specialist
19 hours ago
Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000About Our Data Engineering PositionAt our company, we are seeking an experienced data engineer to join our team. The ideal candidate will have a strong background in building scalable data pipelines and working on modern cloud & big data platforms.Key Responsibilities:Design, develop, and maintain large-scale data systems using Python, PySpark, and SQL.Work...
-
Data Pipeline Specialist
3 days ago
Bengaluru, Karnataka, India beBeedata Full time ₹ 10,00,000 - ₹ 15,00,000Job Opportunity Description:A Data Engineer Associate/Jr Data Engineer position is available in our organization. The jobholder will be responsible for designing, building and managing scalable data pipelines to support business needs. They will also perform complex data transformations using SQL and Spark, conduct performance tuning to improve the...
-
Data Pipeline Specialist
5 days ago
Bengaluru, Karnataka, India beBeeDataEngineer Full time US$ 1,60,000 - US$ 2,00,000Job Title:Big Data EngineerWe are seeking a skilled and experienced Spark Scala Developer to fill this key role.Key Responsibilities:Design and develop scalable data pipelines using Apache Spark and ScalaOptimize and troubleshoot Spark jobs for performance (e.g. memory management, shuffles, skew)Work with massive datasets in on-prem Hadoop clusters or cloud...