Big Data Engineer

1 day ago

Hyderabad, Telangana, India PROGRESSIVE VIDEO PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Key Responsibilities :

- Design, develop, and support robust ETL pipelines to extract, transform, and load data into analytical products that drive strategic organizational goals.

- Develop and maintain data workflows on platforms like Databricks and Apache Spark using Python and Scala.

- Create and support data visualizations using tools such as MicroStrategy, Power BI, or Tableau, with a preference for MicroStrategy.

- Implement streaming data solutions utilizing frameworks like Kafka for real-time data processing.

- Collaborate with cross-functional teams to gather requirements, design solutions, and ensure smooth data operations.

- Manage data storage and processing in cloud environments, with strong experience in AWS cloud services.

- Use knowledge of data warehousing, data modeling, and SQL to optimize data flow and accessibility.

- Develop scripts and automation tools using Linux shell scripting and other languages as needed.

- Ensure continuous integration and continuous delivery (CI/CD) practices are followed for data pipeline deployments using containerization and orchestration technologies.

- Troubleshoot production issues, optimize system performance, and ensure data accuracy and integrity.

- Work effectively within Agile development teams and contribute to sprint planning, reviews, and retrospectives.


Required Skills & Experience :

- 7 years of experience in technology with a focus on application development and production support.

- At least 5 years of experience in developing ETL pipelines and data engineering workflows.

- Minimum 3 years hands-on experience in ETL development and support using Python/Scala on Databricks/Spark platforms.

- Strong experience with data visualization tools, preferably MicroStrategy, Power BI, or Tableau.

- Proficient in Python, Apache Spark, Hive, and SQL.

- Solid understanding of data warehousing concepts, data modeling techniques, and analytics tools.

- Experience working with streaming data frameworks such as Kafka.

- Working knowledge of Core Java, Linux, SQL, and at least one scripting language.

- Experience with relational databases, preferably Oracle.

- Hands-on experience with AWS cloud platform services related to data engineering.

- Familiarity with CI/CD pipelines, containerization, and orchestration tools (e.g., Docker, Kubernetes).

- Exposure to Agile development methodologies.

- Strong interpersonal, communication, and collaboration skills.

- Ability and eagerness to quickly learn and adapt to new technologies.


Preferred Qualifications :

- Bachelors or Masters degree in Computer Science, Information Technology, or related fields.

- Experience working in large-scale, enterprise data environments.

- Prior experience with cloud-native big data solutions and data governance best practices.

  • Big Data engineer

    3 weeks ago

    Hyderabad, Telangana, India Cozzera Full time

    Job Title: Data Engineer Location: Hyderabad | Type: Full-time Description: We're hiring a Big Data Engineer to design and implement scalable data pipelines and streaming applications. You'll work with modern big data technologies and cloud platforms to power real-time data solutions. Key Skills: Strong software engineering with Java,...

  • Senior Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Veltris Full time

    Position-Senior Big Data EngineerMust have Big Data analytics platform experience.• Key stacks: Spark, Druid, Drill, ClickHouse.• 8 years experience in Python/Java, CI/CD, infrastructure & cloud, Terraform, plus depth in:           o Big Data pipelines: Spark, Kafka, Glue, EMR, Hudi, Schema Registry, Data Lineage.           o Graph...

  • Big Data Engineer

    1 day ago

    Hyderabad, Telangana, India Softility, Inc. Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Company DescriptionSoftility is a digital transformation technology services company specializing in cloud adoption, data engineering, AI-driven operations, digital service management, and cybersecurity. We are the preferred partner for many Fortune 500 companies, SLED entities, healthcare institutions, major enterprises, and federal agencies. Headquartered...

  • Big Data Engineering Specialist

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataEngineering Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking a seasoned data engineering professional to spearhead the development and implementation of cutting-edge big data analytics platforms.Key Responsibilities:Design and implement robust big data pipelines utilizing Spark, Kafka, Glue, EMR, Hudi, Schema Registry, and Data Lineage.Develop expertise in graph databases such as Neo4j,...

  • Big Data Engineering Leader

    2 weeks ago

    Hyderabad, Telangana, India beBeeDataEngineering Full time ₹ 30,00,000 - ₹ 36,00,000

    Job Title:Big Data Engineering LeaderWe are seeking a Big Data Engineering Leader to join our team. As a key member of our engineering organization, you will be responsible for delivering roadmap features of our Enterprise TruRisk Platform.This platform helps customers Measure, Communicate and Eliminate Cyber Risks. You will work closely with...

  • Big Data Developer

    3 weeks ago

    Hyderabad, Telangana, India Alp Consulting Ltd. Full time

    Job Title: Big Data DeveloperWork Location: Hyderabad, Telangana (TG), IndiaExperience Range: 6–8 YearsSkill Area: Digital – Big Data and Hadoop EcosystemsJob Description:We are looking for an experienced and results-driven Big Data Developer to join our dynamic team in Hyderabad. The ideal candidate will have a strong background in designing,...

  • Big Data Developer

    3 weeks ago

    Hyderabad, Telangana, India Alp Consulting Ltd. Full time

    Job Title: Big Data Developer Work Location: Hyderabad, Telangana (TG), India Experience Range: 6–8 Years Skill Area: Digital – Big Data and Hadoop Ecosystems Job Description: We are looking for an experienced and results-driven Big Data Developer to join our dynamic team in Hyderabad. The ideal candidate will have a strong background in...

  • Big Data Developer

    2 weeks ago

    Hyderabad, Telangana, India Alp Consulting Ltd. Full time

    Job Title: Big Data DeveloperWork Location: Hyderabad, Telangana (TG), India Experience Range: 6–8 Years Skill Area: Digital – Big Data and Hadoop EcosystemsJob Description:We are looking for an experienced and results-driven Big Data Developer to join our dynamic team in Hyderabad. The ideal candidate will have a strong background in designing,...

  • Big Data Developer

    2 weeks ago

    Hyderabad, Telangana, India Kresta Softech Private Limited Full time

    Role Highlights: Position: Big Data Engineer Experience: 2+ years Location: Hyderabad, Chennai- Hybrid Notice Period: Immediate/7 days joiners mandate Job Overview: Must have skills- Big Data, Scala, AWS and Python or Java

  • Big Data Engineer- Professional Services

    2 weeks ago

    Hyderabad, Telangana, India BrightEdge Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    About UsBrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results. Our platform processes massive amounts of data to provide actionable insights to our clients. We're looking for a talented Big Data Engineer to join our Professional Services to help us scale and optimize our data...