High-Performance Data Pipeline Specialist
5 days ago
This role is ideal for a skilled professional who can design, build, and maintain high-performance data pipelines.
Key Responsibilities:- Design and implement scalable batch processing systems using Python and big data technologies
- Optimize database performance, focusing on slow-running queries and latency improvements
- Use Python profilers and performance monitoring tools to identify bottlenecks
- Reduce P95 and P99 latency metrics across our data platform
- Build efficient ETL pipelines that can handle large-scale data processing
- Collaborate with data scientists and product teams to understand data requirements
- Monitor and troubleshoot data pipeline issues in production
- Implement data quality checks and validation mechanisms
- Document data architecture and engineering processes
- Stay current with emerging big data technologies and best practices
- Bachelor's degree in Computer Science, Engineering, or related technical field
- 4+ years of experience in data engineering roles
- Strong Python programming skills with focus on data processing libraries
- Experience with big data technologies (Spark, Hadoop, etc.)
- Proven experience optimizing database performance (SQL or NoSQL)
- Knowledge of data pipeline orchestration tools (Airflow, Luigi, etc.)
- Understanding of performance optimization techniques and profiling tools
- Experience with cloud platforms (AWS, GCP, or Azure)
- Ability to translate business requirements into technical specifications
- Ability to work effectively in a collaborative team environment
-
Data Pipeline Specialist
1 day ago
Shimoga, Karnataka, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000Job Title: Data Pipeline SpecialistAbout the Role:We are seeking a skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, building, and maintaining production-level data pipelines that handle large volumes of structured and unstructured data.Key Responsibilities:Design and implement end-to-end ETL/ELT processes to...
-
High Performance Computing Specialist
3 days ago
Shimoga, Karnataka, India beBeeGPU Full time ₹ 1,80,00,000 - ₹ 2,20,00,000Job Title: High-Performance Computing SpecialistWe are seeking an experienced engineer to develop, optimize and deploy cutting-edge solutions for high-performance machine learning workloads.Key Responsibilities:Develop, optimize and maintain components for machine learning pipelines using frameworks such as CUDA, HIP or OpenCL.Analyze and improve kernel...
-
Expert Azure Data Pipelines Consultant
2 days ago
Shimoga, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000At ThreatXIntel, we are dedicated to protecting businesses and organizations from cyber threats. Our proactive approach to security involves continuously monitoring and testing digital environments to identify vulnerabilities.We are seeking an experienced Azure Data Factory (ADF) Specialist to support our data integration and ETL needs. The ideal consultant...
-
Data Specialist
4 days ago
Shimoga, Karnataka, India beBeeData Full time US$ 80,000 - US$ 1,00,000Job Title: Data SpecialistWe are seeking an experienced data specialist to lead our data infrastructure initiative.Key Responsibilities:Design and develop scalable data pipelines using modern technologies.Develop high-quality code for efficient data processing and analysis.Utilize programming skills for building robust software applications.Requirements:At...
-
Data Development Specialist
3 days ago
Shimoga, Karnataka, India beBeeDataDevelopment Full time ₹ 15,00,000 - ₹ 20,00,000Senior Data Development SpecialistJob Description:We are seeking an accomplished data development specialist to take on the key challenge of developing and maintaining high-performance data pipelines, collaborating with analytics and business teams, and implementing best practices to monitor data quality.Develops and maintains scalable data pipelines to...
-
Database Data Pipeline Specialist
5 days ago
Shimoga, Karnataka, India beBeeDataEngineer Full time ₹ 25,17,500 - ₹ 25,45,000ETL/ Database Engineer / Tester Position">We are seeking an experienced professional to design, develop and implement ETL processes and data pipelines.The ideal candidate will have a strong understanding of ETL concepts and data warehouse testing methodologies. They should be proficient in RDBMS: MySQL, Oracle SQL, and have hands-on experience with SSIS (SQL...
-
Advanced Data Pipeline Specialist
3 days ago
Shimoga, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000Data Engineer Position", "We are seeking an experienced Data Engineer with strong expertise in Python and AWS services to design, develop, and optimize data pipelines.""
-
Shimoga, Karnataka, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000Unlock Next-Generation Analytics PlatformsDesign, scale, and implement high-performance analytics solutions in the Microsoft ecosystem.As a Fabric Architect, you will define architecture, guide implementation, and enable data teams to deliver insights faster with enterprise-grade governance.The ideal candidate will have a strong understanding of Lakehouse...
-
Shimoga, Karnataka, India beBeePerformance Full time ₹ 18,00,000 - ₹ 24,00,000Job OverviewWe are seeking a skilled High Performance Testing Specialist with 5+ years of experience who is passionate about coding, problem-solving, and innovation.Key Responsibilities:Develop and execute comprehensive performance testing plans to ensure optimal system scalability and reliability.Write and maintain custom test scripts using Apache JMeter...
-
High-Profile Data Architect
6 days ago
Shimoga, Karnataka, India beBeeDatabase Full time ₹ 19,44,000 - ₹ 25,12,000Senior Database Architect">We are seeking a skilled and proactive Senior Database Architect to support our data infrastructure. This role blends traditional DBA responsibilities with modern data engineering tasks to ensure optimal performance of critical database systems.">Key Responsibilities:">">Data Architecture & Scalability: Redesign and improve data...
