Big Data Engineer- Professional Services

2 weeks ago

Hyderabad, Telangana, India BrightEdge Full time ₹ 15,00,000 - ₹ 20,00,000 per year

About Us

BrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results. Our platform processes massive amounts of data to provide actionable insights to our clients. We're looking for a talented Big Data Engineer to join our Professional Services to help us scale and optimize our data processing capabilities.

Role Overview

As a Big Data Engineer at BrightEdge, you will design, build, and maintain high-performance data pipelines that process terabytes of data. You'll work on optimizing our existing systems, identifying and resolving performance bottlenecks, and implementing solutions that improve the overall efficiency of our platform. This role is critical in ensuring our data infrastructure can handle increasing volumes of data while maintaining exceptional performance standards. 

Key Responsibilities
  • Design and implement scalable batch processing systems using Python and big data technologies
  • Optimize database performance, focusing on slow-running queries and latency improvements
  • Use Python profilers and performance monitoring tools to identify bottlenecks
  • Reduce P95 and P99 latency metrics across our data platform
  • Build efficient ETL pipelines that can handle large-scale data processing
  • Collaborate with data scientists and product teams to understand data requirements
  • Monitor and troubleshoot data pipeline issues in production
  • Implement data quality checks and validation mechanisms
  • Document data architecture and engineering processes
  • Stay current with emerging big data technologies and best practices
Qualifications
  • Required
  • Bachelor's degree in Computer Science, Engineering, or related technical field
  • 4+ years of experience in data engineering roles
  • Strong Python programming skills with focus on data processing libraries
  • Experience with big data technologies (Spark, Hadoop, etc.)
  • Proven experience optimizing database performance (SQL or NoSQL)
  • Knowledge of data pipeline orchestration tools (Airflow, Luigi, etc.)
  • Understanding of performance optimization techniques and profiling tool
Preferred
  • Master's degree in Computer Science or related field
  • Experience with SEO data or web crawling systems
  • Experience with Clickhouse Database
  • Knowledge of distributed systems and microservices architecture
  • Familiarity with container orchestration (Kubernetes, Docker)
  • Experience with real-time data processing
  • Contributions to open-source projects
  • Experience with machine learning operations

  • Big Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India RandomTrees Full time

    Job Title: Big Data Engineer Experience: 5–9 Years Location: Hyderabad-Hybrid Employment Type: Full-Time Job Summary: We are seeking a skilled Big Data Engineer with 5–9 years of experience in building and managing scalable data pipelines and analytics solutions. The ideal candidate will have strong expertise in Big Data, Hadoop, Apache Spark, SQL,...

  • Big Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India TECHOAKS IT SOLUTIONS PRIVATE LIMITED Full time

    Full-time with Info ServicesClient: Disney (Offshore)Job Description :Java with Big DataMust have skills :- 8+ years of total work experience is expected.- Java, Big Data, Spring, Kafka, AWS, Scala, and Spark: All these are needed to be strong.Roles & Responsibilities :- Design and Development: Architect, design, and develop high-performance, scalable, and...

  • Big Data Developer

    2 weeks ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    Role :Data Engineering/Big Data DevelopmentLocation: HyderabadExperience: 4 to 10 YearsMust Have skills :Bigdata Hadoop Platform -Cloudera

  • Big Data Developer

    1 week ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    Role :Data Engineering/Big Data Development Location: Hyderabad Experience: 4 to 10 Years Must Have skills :Bigdata Hadoop Platform -Cloudera

  • Big Data Developer

    2 weeks ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    Role :Data Engineering/Big Data Development Location: Hyderabad Experience: 4 to 10 Years Must Have skills :Bigdata Hadoop Platform -Cloudera

  • Big Data Cloud Engineer

    3 weeks ago

    Hyderabad, Telangana, India Nagarro Full time

    Role: Big Data Engineer Required Experience: 3 – 6 years Job Location: Hyderabad Job Description:Must have Skills: Spark, Python/ Scala, AWS/Azure, Snowflake, Databricks, SQL Server/NoSQL.Key Responsibilities:Design and implement data pipelines for batch and real-time data processing.Optimize data storage solutions for efficiency and...

  • Big Data Expert

    1 day ago

    Hyderabad, Telangana, India beBeeDataScience Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Big Data DeveloperWe are seeking a highly skilled and experienced big data expert to design, develop, and maintain scalable big data systems and data processing frameworks.Main Responsibilities:Design, develop, and maintain scalable and efficient big data architectures and data pipelines.Implement and optimize data processing frameworks using...

  • Big Data Support Engineer

    3 days ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    Greetings from TCSTCS is hiring for Big Data Support Engineer / Hadoop Support EngineerDesired Experience: 4 to 10 YearsJob Location: HyderabadRequire skillset: Bigdata Hadoop Platform -Cloudera, Sqoop, hive, spark, oozie and Flume- Cloudera (CDH)Job Description:Must Have Technical Qualifications:· 3 years exp on Sqoop, hive, spark, oozie and Flume-...

  • Big Data Expert

    19 hours ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time US$ 80,000 - US$ 1,20,000

    Senior Data EngineerWe're seeking a skilled professional to help implement an application for a large retailer. This role requires expertise in Telemetry, Big Data and backend engineering.Key Responsibilities:Design and develop data processing pipelines using Python and PysparkImplement telemetry solutions for IoT systemsWork closely with business...

  • Big Data Architect

    2 days ago

    Hyderabad, Telangana, India beBeeDataScientist Full time ₹ 12,00,000 - ₹ 20,00,000

    Job Title: Big Data ArchitectWe are seeking a highly skilled professional to lead the design and implementation of our big data infrastructure.Responsibilities:Develop and maintain high-performance big data applications using Python, PySpark, and Spark Dataframes.Design and implement scalable data architectures to process huge volumes of data.Collaborate...