Big Data Engineer

2 weeks ago

Hyderabad, Telangana, India PROGRESSIVE VIDEO PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Key Responsibilities :

- Design, develop, and support robust ETL pipelines to extract, transform, and load data into analytical products that drive strategic organizational goals.

- Develop and maintain data workflows on platforms like Databricks and Apache Spark using Python and Scala.

- Create and support data visualizations using tools such as MicroStrategy, Power BI, or Tableau, with a preference for MicroStrategy.

- Implement streaming data solutions utilizing frameworks like Kafka for real-time data processing.

- Collaborate with cross-functional teams to gather requirements, design solutions, and ensure smooth data operations.

- Manage data storage and processing in cloud environments, with strong experience in AWS cloud services.

- Use knowledge of data warehousing, data modeling, and SQL to optimize data flow and accessibility.

- Develop scripts and automation tools using Linux shell scripting and other languages as needed.

- Ensure continuous integration and continuous delivery (CI/CD) practices are followed for data pipeline deployments using containerization and orchestration technologies.

- Troubleshoot production issues, optimize system performance, and ensure data accuracy and integrity.

- Work effectively within Agile development teams and contribute to sprint planning, reviews, and retrospectives.


Required Skills & Experience :

- 7 years of experience in technology with a focus on application development and production support.

- At least 5 years of experience in developing ETL pipelines and data engineering workflows.

- Minimum 3 years hands-on experience in ETL development and support using Python/Scala on Databricks/Spark platforms.

- Strong experience with data visualization tools, preferably MicroStrategy, Power BI, or Tableau.

- Proficient in Python, Apache Spark, Hive, and SQL.

- Solid understanding of data warehousing concepts, data modeling techniques, and analytics tools.

- Experience working with streaming data frameworks such as Kafka.

- Working knowledge of Core Java, Linux, SQL, and at least one scripting language.

- Experience with relational databases, preferably Oracle.

- Hands-on experience with AWS cloud platform services related to data engineering.

- Familiarity with CI/CD pipelines, containerization, and orchestration tools (e.g., Docker, Kubernetes).

- Exposure to Agile development methodologies.

- Strong interpersonal, communication, and collaboration skills.

- Ability and eagerness to quickly learn and adapt to new technologies.


Preferred Qualifications :

- Bachelors or Masters degree in Computer Science, Information Technology, or related fields.

- Experience working in large-scale, enterprise data environments.

- Prior experience with cloud-native big data solutions and data governance best practices.

  • Big Data Engineer

    5 days ago

    Hyderabad, Telangana, India Sky Systems, Inc. (SkySys) Full time

    Role:Big Data EngineerPosition Type:Full-Time Contract (40hrs/week)Contract Duration:Long TermWork Schedule: 8 hours/day (Mon-Fri)Location: Hybrid - Hyderabad, India (3x days onsite/week)We are looking for a skilledBig Data Engineer (5+ years' experience)to join Experian's dynamic data engineering team. You will design and optimize scalable data pipelines,...

  • Big Data engineer

    7 days ago

    Hyderabad, Telangana, India Cozzera Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Title:Data Engineer Location:Hyderabad | Type:Full-time Description:We're hiring aBig Data Engineerto design and implement scalable data pipelines and streaming applications. You'll work with modern big data technologies and cloud platforms to power real-time data solutions. Key Skills:Strong software engineering with Java, Scala, or PythonExpertise in...

  • Big Data Engineer

    7 days ago

    Hyderabad, Telangana, India IDESLABS PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are looking for a skilled Big Data Engineer with 6 to 22 years of experience. The ideal candidate will have expertise in Pyspark, Azure Data Bricks, and workflows. This position is available as a contract role across Pan India.Roles and ResponsibilityDesign and develop scalable big data systems using Pyspark and Azure Data Bricks.Implement and manage...

  • Big Data Engineer

    5 days ago

    Hyderabad, Telangana, India Incedo Inc Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Data Engineers – Big Data Platform At Incedo, we're looking for experienced Big Data Engineers (5–8 years) to join our AI & Data team in Hyderabad — and be part of building cutting-edge data ecosystems for the Banking domain. What You'll Do • Design, develop & optimize large-scale data pipelines using Hadoop, Hive, HDFS, and Kafka •...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Pennant Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Title: Senior Big Data EngineerDepartment: Data & AnalyticsEmployment Type: Full‑timeRole OverviewWe are seeking an experienced Big Data Engineer to join our growing Data & Analytics team. In this role, you'll design, build, and optimize our modern data platform—enabling scalable pipelines, reliable data lakes, and robust ML operations. You'll work...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Softility, Inc. Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Company DescriptionSoftility is a digital transformation technology services company specializing in cloud adoption, data engineering, AI-driven operations, digital service management, and cybersecurity. We are the preferred partner for many Fortune 500 companies, SLED entities, healthcare institutions, major enterprises, and federal agencies. Headquartered...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Pennant Technologies Private Limited Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job DescriptionWe are looking for experienced Big Data Engineers to join our growing Data & Analytics team. In this role, you'll design, build, and optimize our modern data platform—enabling scalable pipelines, reliable data lakes, and robust ML operations. You'll work closely with data scientists, analysts, and stakeholders to turn raw data into...

  • Big Data Engineer

    7 days ago

    Hyderabad, Telangana, India Vaakruthi Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role & responsibilitiesJob Description : Big Data EngineerAnalyze, Code , Test and Implement Big Data Applications (API & Batch) Manage and mitigate incidentsRequired Qualifiactions:years of experience in analysis, coding, implementation and testing of Big Data Applications (API & Batch) on AWS Cloud using java/scala/kotlin, Cassandra,Python, Hadoop, HDFS,...

  • Big Data Engineer

    6 hours ago

    Hyderabad, Telangana, India Skilviu Soft Solutions Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Role: Big Data EngineerLocation: Hyderabad, India (Hybrid).Experience: 5+ YearsDuration: 1 year+Notice Period: Immediate -15 days.Key Responsibilities:Design, develop, and maintain scalable data processing pipelines using Spark/PySpark and Hadoop on AWS.Implement and optimize data workflows on AWS EMR, EC2, and ECS.Develop and integrate RESTful APIs and AWS...

  • Big Data Developer

    7 days ago

    Hyderabad, Telangana, India Alp Consulting Ltd. Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Job Title:Big Data DeveloperWork Location:Hyderabad, Telangana (TG), IndiaExperience Range:6–8 YearsSkill Area:Digital – Big Data and Hadoop EcosystemsJob Description:We are looking for an experienced and results-drivenBig Data Developerto join our dynamic team in Hyderabad. The ideal candidate will have a strong background in designing, developing, and...