Senior Data Engineer

3 weeks ago

Cochin, Kerala, India BrightEdge Full time

Big Data Engineer - BrightEdge About Us BrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results. Our platform processes massive amounts of data to provide actionable insights to our clients. We're looking for a talented Big Data Engineer to join our Professional Services to help us scale and optimize our data processing capabilities.

Role Overview

As a Big Data Engineer at BrightEdge, you will design, build, and maintain high-performance data pipelines that process terabytes of data. You'll work on optimizing our existing systems, identifying and resolving performance bottlenecks, and implementing solutions that improve the overall efficiency of our platform. This role is critical in ensuring our data infrastructure can handle increasing volumes of data while maintaining exceptional performance standards.

Key Responsibilities

● Design and implement scalable batch processing systems using Python and big data technologies

● Optimize database performance, focusing on slow-running queries and latency improvements

● Use Python profilers and performance monitoring tools to identify bottlenecks

● Reduce P95 and P99 latency metrics across our data platform

● Build efficient ETL pipelines that can handle large-scale data processing

● Collaborate with data scientists and product teams to understand data requirements

● Monitor and troubleshoot data pipeline issues in production

● Implement data quality checks and validation mechanisms

● Document data architecture and engineering processes

● Stay current with emerging big data technologies and best practices

Qualifications Required

● Bachelor's degree in Computer Science, Engineering, or related technical field

● 4+ years of experience in data engineering roles

● Strong Python programming skills with focus on data processing libraries

● Experience with big data technologies (Spark, Hadoop, etc.)

● Proven experience optimizing database performance (SQL or NoSQL)

● Knowledge of data pipeline orchestration tools (Airflow, Luigi, etc.)

● Understanding of performance optimization techniques and profiling tools

● Experience with cloud platforms (AWS, GCP, or Azure)

● Ability to translate business requirements into technical specifications

● Ability to work in UK timezone (12 PM IST to 9:30 PM IST)

Preferred

● Master's degree in Computer Science or related field

● Experience with SEO data or web crawling systems

● Experience with Clickhouse Database

● Knowledge of distributed systems and microservices architecture

● Familiarity with container orchestration (Kubernetes, Docker)

● Experience with real-time data processing

● Contributions to open-source projects

● Experience with machine learning operations Skills and Abilities

● Strong analytical thinking and problem-solving skills

● Excellent attention to detail and commitment to data quality

● Ability to work effectively in a collaborative team environment

● Good communication skills to explain complex technical concepts

● Self-motivated with the ability to work independently

● Passion for performance optimization and efficiency

  • Senior Data Engineer

    2 weeks ago

    Cochin, Kerala, India Velodata Global Pvt Ltd Full time

    Job Title : Senior Data EngineerLocation : Kochi / Trivandrum / BangaloreExperience : 5+ yearsNotice period : Immediate JoinersJob Summary : We are looking for an experienced Senior Data Engineer to join our team and lead the design, development, and deployment of enterprise data solutions. The ideal candidate is proficient in MS SQL, SSIS, Azure Data...

  • Senior Data Engineer

    2 weeks ago

    Cochin, Kerala, India Velodata Global Pvt Ltd Full time

    Role: Lead/ Senior Data Engineer - AWS, Python, Pyspark Mandatory Skills: AWS, Python, Pyspark Notice Period: Immediate to 15 days Experience Range: 7+ Years Location: Kerala (Local candidates only – Kochi/Trivandrum) Budget: 23 LPA Job Overview We are seeking an experienced Senior Data Engineer to lead the development of a scalable data...

  • Senior Data engineer

    2 weeks ago

    Cochin, Kerala, India Milestone Technologies, Inc. Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    Role Overview: Seeking an experienced Senior Data Engineer to lead data ingestion, transformation, and optimization initiatives using the modern Apache and Azure data stack. The role involves working on scalable pipelines, large-scale distributed systems, and data lake management. Core Responsibilities:Build and manage high-volume data pipelines using...

  • Senior Data Engineer

    2 weeks ago

    Cochin, Kerala, India Velodata Global Pvt Ltd Full time

    About the job :Role : Lead/ Senior Data Engineer - AWS, Python, PysparkMandatory Skills : AWS, Python, PysparkNotice Period : Immediate to 15 daysExperience Range : 7+ YearsLocation : Kerala (Local candidates only Kochi/Trivandrum)Job Overview : We are seeking an experienced Senior Data Engineer to lead the development of a scalable data ingestion...

  • Expertise in GCP Data Engineering

    2 weeks ago

    Cochin, Kerala, India beBeeEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer Position OverviewThe role of a Senior Data Engineer entails expertise in Google Cloud Platform (GCP), specifically with Experience Data Transfer (ETL) tools.

  • Senior Data Visualization Engineer

    3 weeks ago

    Cochin, Kerala, India e-Hireo Full time

    Job DescriptionJOB DESCRIPTIONExperience : 5 - 8 YrsLocation : Kochi, KeealaDesignation : Senior Data Visualization EngineerKey Responsibilities:- Design, develop, and maintain interactive dashboards and reports using AWS QuickSight.- Integrate data from AWS Redshift and other data warehouse platforms for reporting purposes.- Write complex SQL queries for...

  • Data Engineering Manager

    2 weeks ago

    Cochin, Kerala, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 20,00,000

    Senior Data Engineering ManagerWe are seeking an experienced and skilled Senior Data Engineering Manager to lead our data engineering team.This is a leadership role where you will oversee the design, development, and implementation of large-scale data platforms and systems.You will be responsible for developing and maintaining comprehensive project plans,...

  • Cloud Data Engineer

    2 weeks ago

    Cochin, Kerala, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Senior Cloud Data EngineerWe are seeking a highly skilled Senior Cloud Data Engineer to join our team. As a key member of our engineering team, you will design and implement data ingestion, ETL pipelines, and cloud-based machine learning solutions using GCP AI/ML services.ResponsibilitiesDesign and implement scalable and efficient data pipelines on Google...

  • ai - Senior Data Engineer - ETL Pipeline

    3 weeks ago

    Cochin, Kerala, India DBiz Full time

    Position : Senior Data EngineerWe are seeking an experienced Senior Data Engineer to design and implement high-performance data pipelines, ensuring the smooth flow of large datasets across systems.In this role, you will leverage your expertise in Databricks, Pyspark, and ETL design to build efficient data solutions.You will also contribute to optimizing...

  • Senior Data Engineer

    2 weeks ago

    Cochin, Kerala, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Key Role: Data Engineering LeadWe are seeking a seasoned professional to build and maintain high-quality data ingestion and validation processes from multiple upstream systems, laying the foundation for AI-driven anomaly detection, business insight extraction, and scalable reporting solutions.Responsibilities:Data Quality ManagementDesign and implement...