High-Performance Data Engineer

2 days ago

Pune, Maharashtra, India beBeeData Full time ₹ 17,95,200 - ₹ 24,74,800

As a seasoned data professional, you will design and implement high-performance data pipelines that process vast amounts of information. Your work will be critical in ensuring our data infrastructure can handle increasing volumes of data while maintaining exceptional performance standards.

Key Responsibilities

• Design and implement scalable batch processing systems using Python and big data technologies • Optimize database performance, focusing on slow-running queries and latency improvements • Use Python profilers and performance monitoring tools to identify bottlenecks

Responsibilities Continued

• Reduce P95 and P99 latency metrics across our data platform • Build efficient ETL pipelines that can handle large-scale data processing • Collaborate with data scientists and product teams to understand data requirements • Monitor and troubleshoot data pipeline issues in production • Implement data quality checks and validation mechanisms • Document data architecture and engineering processes • Stay current with emerging big data technologies and best practices

Qualifications Required

• Bachelor's degree in Computer Science, Engineering, or related technical field • 4+ years of experience in data engineering roles • Strong Python programming skills with focus on data processing libraries • Experience with big data technologies (Spark, Hadoop, etc.) • Proven experience optimizing database performance (SQL or NoSQL) • Knowledge of data pipeline orchestration tools (Airflow, Luigi, etc.) • Understanding of performance optimization techniques and profiling tools • Experience with cloud platforms (AWS, GCP, or Azure)

Bonus Qualifications

• Master's degree in Computer Science or related field • Experience with SEO data or web crawling systems • Experience with Clickhouse Database • Knowledge of distributed systems and microservices architecture • Familiarity with container orchestration (Kubernetes, Docker) • Experience with real-time data processing • Contributions to open-source projects • Experience with machine learning operations

Skills and Abilities

• Strong analytical thinking and problem-solving skills • Excellent attention to detail and commitment to data quality • Ability to work effectively in a collaborative team environment • Good communication skills to explain complex technical concepts • Self-motivated with the ability to work independently • Passion for performance optimization and efficiency

  • High-Performance Data Engineer

    7 days ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job OpeningWe are seeking a skilled PySpark Data Engineer to drive the development of robust data processing and transformation solutions within our data platform.Key Responsibilities:Data Engineering Development:Design, develop, and test PySpark-based applications to process, transform, and analyze large-scale datasets from various sources.Implement...

  • High-Performance Data Engineer

    4 days ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title: Data EngineerWe are seeking a skilled data engineer to take on a challenging role in designing, building, and maintaining large-scale data systems.Collaborate with team members and stakeholders to understand data requirementsPull data from various sources into a central data platformStructure datasets and calculate metrics based on...

  • High-Performance Data Architect

    1 hour ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Description:We're seeking a highly skilled Data Engineer to join our team. In this role, you'll work directly with researchers to accelerate experiments, develop new datasets, improve infrastructure efficiency, and enable key insights across our data assets.As a Data Engineer, you'll design, build, and operate scalable, fault-tolerant data infrastructure...

  • Data Engineer

    1 day ago

    Pune, Maharashtra, India Jash Data Sciences Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Do you love solving real-world data problems with the latest and best techniques? And having fun while solving them in a team Then come join our high-energy team of passionate data people. Jash Data Sciences is the right place for you.We are a cutting-edge Data Sciences and Data Engineering startup based in Pune, India.We believe in continuous learning and...

  • High-Performance Software Engineer

    7 days ago

    Pune, Maharashtra, India beBeeBackendDeveloper Full time ₹ 20,00,000 - ₹ 25,00,000

    Job DescriptionWe are looking for a highly skilled software engineer to design and develop scalable backend systems. Your primary responsibilities will include building and maintaining high-performance microservices, implementing custom algorithms and data models, and collaborating with product teams.You will use AI developer tools and LLMs to accelerate...

  • High Performance Engineering Lead

    20 hours ago

    Pune, Maharashtra, India beBeeEngineering Full time ₹ 30,00,000 - ₹ 60,00,000

    Senior Engineering Leadership RoleWe're seeking a highly skilled and experienced engineering leader to join our team. As a key member, you'll be responsible for leading a team of talented engineers in building a scalable and compliant platform.The AI Platform Team is a dynamic group focused on building centralized AI infrastructure and acceleration solutions...

  • High Salary Software Engineer

    2 days ago

    Pune, Maharashtra, India Data Axle Full time

    Data Axle Inc has been an industry leader in data marketing solutions sales and research for 50 years in the US Data Axle has set up a strategic global centre of excellence in Pune This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and leveraging proprietary business ...

  • High-Performance Data Strategist

    18 hours ago

    Pune, Maharashtra, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Data Analyst: A High-Performance Role">About the JobWe are seeking a seasoned data analyst with advanced expertise in data analysis, visualization, and reporting to join our team. As a senior data analyst, you will play a key role in developing and maintaining complex data-driven solutions that drive business growth and decision-making.">Data Analysis...

  • High Performance Engineer

    4 days ago

    Pune, Maharashtra, India beBeeEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: High Performance Engineer">This role requires a seasoned expert in software reliability and platform stability to drive improvements through automated build and deployment processes.Key Responsibilities:Design and implement tailored Continuous Integration and Continuous Deployment (CICD) pipelines to meet environment-specific needs.Proactively...

  • High Performing Software Engineer

    2 days ago

    Pune, Maharashtra, India beBeeEngineering Full time US$ 1,50,000 - US$ 2,00,000

    Unlock Your Potential as a High-Performing EngineerWe're seeking an experienced Staff Software Engineer to join our team and help drive innovation in building distributed, high-scale, and data-intensive integrations. You'll partner with other SaaS providers and cloud vendors to provide cutting-edge solutions to customers.Job Overview:Prioritize customer...