Big Data Engineer

1 day ago

Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000
Big Data Engineer

We are looking for a skilled Big Data Engineer to design, develop, and maintain large-scale data pipelines using PySpark and Databricks.

 

Key Responsibilities:

  • Design and implement scalable data architectures using PySpark and Databricks.
  • Develop and optimize Spark applications using Spark UI and performance tuning techniques.
  • Work with large datasets and implement best practices for partitioning, caching, join optimization, and cluster resource management.
  • Build efficient data ingestion, transformation, and aggregation logic from multiple sources using Kafka and other streaming technologies.
  • Leverage AWS services such as S3, EMR, Lambda, Glue, Redshift, etc., for cloud-based data processing and orchestration.
  • Implement workflows using tools such as Airflow or Databricks Workflows.
  • Utilize skills in distributed data processing, data formats like Parquet, Avro, and OyC, and exposure to orchestration tools (Airflow, Databricks Workflows).

 

Requirements:

  • 5+ years of experience in designing and developing big data solutions using PySpark and Databricks.
  • Strong knowledge of data engineering principles, including data pipelining, data warehousing, and ETL processes.
  • Experience with cloud-based data platforms, including AWS services.
  • Excellent problem-solving skills and ability to troubleshoot complex data processing issues.
  • Strong communication and collaboration skills.

 

Benefits:

  • Competitive salary and benefits package.
  • Ongoing training and professional development opportunities.
  • Collaborative and dynamic work environment.

  • Big Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India RandomTrees Full time

    Job Title: Big Data Engineer Experience: 5–9 Years Location: Hyderabad-Hybrid Employment Type: Full-Time Job Summary: We are seeking a skilled Big Data Engineer with 5–9 years of experience in building and managing scalable data pipelines and analytics solutions. The ideal candidate will have strong expertise in Big Data, Hadoop, Apache Spark, SQL,...

  • Big Data engineer

    4 days ago

    Hyderabad, Telangana, India Cozzera Full time

    Job Title: Data Engineer Location: Hyderabad | Type: Full-time Description: We're hiring a Big Data Engineer to design and implement scalable data pipelines and streaming applications. You'll work with modern big data technologies and cloud platforms to power real-time data solutions. Key Skills: Strong software engineering with Java,...

  • Big Data engineer

    12 hours ago

    Hyderabad, Telangana, India Cozzera Full time US$ 90,000 - US$ 1,20,000 per year

    Job Title:Data Engineer Location:Hyderabad | Type:Full-time Description:We're hiring aBig Data Engineerto design and implement scalable data pipelines and streaming applications. You'll work with modern big data technologies and cloud platforms to power real-time data solutions. Key Skills:Strong software engineering with Java, Scala, or PythonExpertise in...

  • Big Data Engineer

    15 hours ago

    Hyderabad, Telangana, India Allegis Group Full time US$ 1,20,000 - US$ 2,00,000 per year

    Role & responsibilitiesConsult on or participate in moderately complex initiatives and deliverables within Software Engineering and contribute to large-scale planning related to Software Engineering deliverables.Review and analyze moderately complex Software Engineering challenges that require an in-depth evaluation of variable factors.Contribute to the...

  • Big Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India TECHOAKS IT SOLUTIONS PRIVATE LIMITED Full time

    Full-time with Info ServicesClient: Disney (Offshore)Job Description :Java with Big DataMust have skills :- 8+ years of total work experience is expected.- Java, Big Data, Spring, Kafka, AWS, Scala, and Spark: All these are needed to be strong.Roles & Responsibilities :- Design and Development: Architect, design, and develop high-performance, scalable, and...

  • Big Data Engineer

    1 day ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Description:To design, develop and maintain large-scale data systems that meet business needs.You will create efficient pipelines against multiple data sources such as SQL, Big Data on cloud platforms. Large, complex datasets will be processed to meet both functional and non-functional business requirements.Existing data models, pipelines, related...

  • Senior Data Engineer

    12 hours ago

    Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 15th 2025)Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and...

  • Big Data Engineer

    4 days ago

    Hyderabad, Telangana, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Title: Data Engineering LeadMain responsibilities include designing, developing and implementing data pipelines using ADF for data ingestion and transformation.The ideal candidate will have hands-on experience with Azure services such as ADLS, Azure Databricks, Data Factory, Synapse, and Azure SQL DB. Familiarity with DevOps and CI/CD deployments is a...

  • Big Data Expert

    6 days ago

    Hyderabad, Telangana, India beBeeDataScience Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Big Data DeveloperWe are seeking a highly skilled and experienced big data expert to design, develop, and maintain scalable big data systems and data processing frameworks.Main Responsibilities:Design, develop, and maintain scalable and efficient big data architectures and data pipelines.Implement and optimize data processing frameworks using...

  • Big Data Cloud Engineer

    4 weeks ago

    Hyderabad, Telangana, India Nagarro Full time

    Role: Big Data Engineer Required Experience: 3 – 6 years Job Location: Hyderabad Job Description:Must have Skills: Spark, Python/ Scala, AWS/Azure, Snowflake, Databricks, SQL Server/NoSQL.Key Responsibilities:Design and implement data pipelines for batch and real-time data processing.Optimize data storage solutions for efficiency and...