Brightedge - Big Data Engineer - ETL/Python

3 weeks ago

Hyderabad, Telangana, India brightedge Full time

About Us

BrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results.

Our platform processes massive amounts of data to provide actionable insights to our clients.

We're looking for a talented Big Data Engineer to join our Professional Services to help us scale and optimize our data processing capabilities.

Role Overview :

As a Big Data Engineer at BrightEdge, you will design, build, and maintain high-performance data pipelines that process terabytes of data.

You'll work on optimizing our existing systems, identifying and resolving performance bottlenecks, and implementing solutions that improve the overall efficiency of our platform.

This role is critical in ensuring our data infrastructure can handle increasing volumes of data while maintaining exceptional performance standards.

Key Responsibilities :

- Design and implement scalable batch processing systems using Python and big data technologies.

- Optimize database performance, focusing on slow-running queries and latency improvements.

- Use Python profilers and performance monitoring tools to identify bottlenecks.

- Reduce P95 and P99 latency metrics across our data platform.

- Build efficient ETL pipelines that can handle large-scale data processing.

- Collaborate with data scientists and product teams to understand data requirements.

- Monitor and troubleshoot data pipeline issues in production.

- Implement data quality checks and validation mechanisms.

- Document data architecture and engineering processes.

- Stay current with emerging big data technologies and best practices.

Qualifications :

Required :

- Bachelor's degree in Computer Science, Engineering, or related technical field.

- 4+ years of experience in data engineering roles.

- Strong Python programming skills with focus on data processing libraries.

- Experience with big data technologies (Spark, Hadoop, etc.

- Proven experience optimizing database performance (SQL or NoSQL).

- Knowledge of data pipeline orchestration tools (Airflow, Luigi, etc.

- Understanding of performance optimization techniques and profiling tool.

Preferred :

- Master's degree in Computer Science or related field.

- Experience with SEO data or web crawling systems.

- Experience with Clickhouse Database.

- Knowledge of distributed systems and microservices architecture.

- Familiarity with container orchestration (Kubernetes, Docker).

- Experience with real-time data processing.

- Contributions to open-source projects.

- Experience with machine learning operations.

(ref:hirist.tech)

  • Big Data Engineer- Professional Services

    2 weeks ago

    Hyderabad, Telangana, India BrightEdge Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    About UsBrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results. Our platform processes massive amounts of data to provide actionable insights to our clients. We're looking for a talented Big Data Engineer to join our Professional Services to help us scale and optimize our data...

  • Advanced Big Data Professional

    2 weeks ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Big Data DeveloperWe are seeking an experienced Big Data Developer to join our team. As a key member of our engineering team, you will be responsible for developing and maintaining large-scale data processing pipelines using Hadoop ecosystem tools.The ideal candidate will have strong programming skills and a deep understanding of Big Data ecosystems. They...

  • Big Data Engineer

    1 day ago

    Hyderabad, Telangana, India PROGRESSIVE VIDEO PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Key Responsibilities : - Design, develop, and support robust ETL pipelines to extract, transform, and load data into analytical products that drive strategic organizational goals. - Develop and maintain data workflows on platforms like Databricks and Apache Spark using Python and Scala. - Create and support data visualizations using tools such as...

  • ETL Senior Engineer

    2 weeks ago

    Hyderabad, Telangana, India TESTQ Technologies Limited Full time

    About the Role : We are looking for a highly skilled ETL Senior Engineer with AWS expertise to design, build, and optimize robust data pipelines that power enterprise decision-making. This role involves working with modern cloud-native tools to build scalable, secure, and high-performance ETL workflows that integrate structured and unstructured data from...

  • Big Data Engineering Professional

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Data EngineerWe are seeking an experienced data engineer to develop and maintain large-scale data engineering projects.Job Summary:Develop scalable data pipelines and ETL processes using Databricks.Optimize Spark jobs for efficient data processing and storage.Collaborate with stakeholders to translate data requirements into technical...

  • Big Data Developer

    1 day ago

    Hyderabad, Telangana, India Softility, Inc. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Company DescriptionSoftility is a digital transformation technology services company specializing in innovative solutions across cloud adoption and modernization, data engineering and analytics, AI-driven operations, digital service management, and cybersecurity. Based in Herndon, Virginia, USA, with an Innovation & Development Center in Hyderabad, India, we...

  • Big Data engineer

    3 weeks ago

    Hyderabad, Telangana, India Cozzera Full time

    Job Title: Data Engineer Location: Hyderabad | Type: Full-time Description: We're hiring a Big Data Engineer to design and implement scalable data pipelines and streaming applications. You'll work with modern big data technologies and cloud platforms to power real-time data solutions. Key Skills: Strong software engineering with Java,...

  • Data Engineer – Python

    2 weeks ago

    Hyderabad, Telangana, India CBNsense Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    Job Title: Data Engineer – Python / PySparkNumber of Position:10Experience: 3 to 10 yearsLocation: Hyderabad (Client - USA)Work Model: Full-time, In-Office. This is an offsite opportunity supporting a US client. The role requires you to work from our Hyderabad client location daily.Job Type: Contract-to-HireClient Domain: Utilities / EnergyAbout...

  • Big Data Developer

    3 weeks ago

    Hyderabad, Telangana, India Alp Consulting Ltd. Full time

    Job Title: Big Data DeveloperWork Location: Hyderabad, Telangana (TG), IndiaExperience Range: 6–8 YearsSkill Area: Digital – Big Data and Hadoop EcosystemsJob Description:We are looking for an experienced and results-driven Big Data Developer to join our dynamic team in Hyderabad. The ideal candidate will have a strong background in designing,...

  • Python/Hadoop Developer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA, Inc. Full time ₹ 1,50,000 - ₹ 28,00,000 per year

    NTT DATA Services currently seeks Python Hadoop Developer to join our team inResponsibilitiesSkillset: Python, Hadoop, ETL, RDBMS, Unix, GCP Vertex AIResponsibilities:Data Ingestion Pipelines, AIML Model deployments, Data Engineering, ML Engineering with Python.