
Data Engineer for Large-Scale Data Pipelines
1 day ago
At ThreatXIntel, we take a proactive approach to security by continuously monitoring and testing clients' digital environments to identify vulnerabilities before they can be exploited.
About the Role- We are seeking an experienced Freelance Data Engineer with expertise in Python, SQL, and AWS to design, build, and optimize large-scale data pipelines.
- Design, develop, and maintain Extract, Transform, Load (ETL)/Load, Transform, Extract (ELT) pipelines using PySpark, AWS, and Databricks.
- Perform data profiling, validation, and transformation using Python and SQL.
- Optimize and monitor workflows for performance, scalability, and cost efficiency.
- Build automation scripts using Linux and Bash for data pipeline orchestration.
- Collaborate with analytics teams to surface trustworthy Key Performance Indicators (KPIs) and metrics.
- Troubleshoot and resolve data inconsistencies, configuration issues, and pipeline errors.
- Work closely with cross-functional teams (analysts, product managers, engineers) to define and implement data requirements.
- 5+ years of experience in data engineering with a strong background in Python and SQL.
- Proven expertise in PySpark, AWS services, and Databricks.
- Proficiency in Linux/Bash scripting for automation and data operations.
- Hands-on experience with ETL/ELT pipelines and large-scale integrations.
- Familiarity with Agile and Continuous Integration/Continuous Deployment (CI/CD) practices.
- Strong problem-solving and communication skills, with experience in client-facing roles.
-
Senior Data Engineer
4 days ago
Surat, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Transforming Data into InsightsAbout This RoleWe are seeking an experienced data engineer to drive the development and maintenance of efficient data pipelines. The ideal candidate will possess a strong background in extract, transform, and load (ETL) processes, data engineering, and experience working with cloud-based data platforms.Data Pipeline...
-
High-Performance Data Pipeline Specialist
4 days ago
Surat, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Job Title: Senior Data EngineerJob Description:We are looking for a skilled Big Data Engineer to join our team. As a key member of the Professional Services department, you will design, build, and maintain high-performance data pipelines that process massive amounts of data.Key Responsibilities:Design and implement scalable batch processing systems using...
-
Senior Data Pipeline Developer
2 days ago
Surat, Gujarat, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000Our team is seeking a highly skilled Data Engineer to join our ranks. As a key member of our organization, you will be responsible for designing, developing, and maintaining complex data pipelines using PySpark.Key Responsibilities:The ideal candidate will havea strong background in PySpark, with hands-on experience in building scalable data...
-
AWS Cloud Data Engineer
2 days ago
Surat, Gujarat, India beBeeDataEngineer Full time ₹ 40,00,000 - ₹ 50,00,000Seeking a seasoned Data Architect to design, build, and deploy scalable data pipelines on AWS cloud infrastructure.This is an exciting opportunity for a talented individual who can collaborate with cross-functional teams to optimize data pipelines for performance, reliability, and cost-effectiveness.We are looking for someone with:8+ years of experience...
-
Big Data Solutions Engineer
2 days ago
Surat, Gujarat, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,00,00,000Job Title: Data ArchitectA company in the data engineering sector is looking for a highly skilled professional with expertise in Scala, Spark, Python, NiFi, and Apache Kafka to join its team.Key Responsibilities:Design, develop, and optimize large-scale data pipelines and distributed data processing systems.Work extensively with Scala, Spark (PySpark), and...
-
GCP Data Engineer Role
1 day ago
Surat, Gujarat, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Job Summary:We are seeking a skilled Data Engineer to design, develop and maintain large-scale data processing systems on GCP.Key Responsibilities:Data Engineering Expertise: Develop and implement efficient data pipelines, utilizing GCP's comprehensive suite of tools and services.Collaboration and Integration: Work closely with cross-functional teams to...
-
Senior Data Engineer
7 days ago
Surat, Gujarat, India beBeeData Full time ₹ 18,00,000 - ₹ 19,20,000We are modernizing our demand forecasting capabilities through a cutting-edge AI platform. We are seeking a highly skilled Data Scientist to design and operationalize advanced forecasting solutions at scale.Job Responsibilities:Create, build, and deploy production-grade prediction modelsDevelop end-to-end machine learning pipelines for data ingestion,...
-
Azure Data Engineer Position
3 days ago
Surat, Gujarat, India beBeeDataEngineering Full time ₹ 9,00,000 - ₹ 12,00,000Data Engineering SpecialistWe are seeking a highly skilled Data Engineering Specialist to join our team. The ideal candidate will have extensive experience in designing, implementing, and maintaining large-scale data processing systems on Azure.Key ResponsibilitiesDevelop scalable data pipelines using Azure Data Factory, Apache Kafka, and other relevant...
-
Data Pipeline Specialist
1 day ago
Surat, Gujarat, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000Job DescriptionWe are seeking a skilled Data Engineer with expertise in designing and implementing scalable, high-performance data pipelines. The role involves building and optimizing ETL/ELT workflows, managing big data environments, and ensuring seamless data availability for analytics, AI/ML, and business intelligence use cases.
-
Big Data Engineer
4 days ago
Surat, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000Job Title: Big Data EngineerOverview:We are seeking an experienced Big Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining large-scale data processing systems.Key Responsibilities:Design and develop scalable big data architectures using technologies like AWS, Python, and PySpark.Build and...