Big Data Engineer

1 week ago

Bengaluru, India VidPro Consultancy Services Full time

Role : Big Data Engineer Location : Bangalore Job Type : Fulltime, Hybrid Skills : Python, Spark, SQL, Databricks, Hadoop, Hive, Spark, AWS, DSA Job Overview We are seeking a Junior Big Data Engineer specializing in AWS for a full-time role located in Bangalore Urban, Bangalore Rural, Bangalore. As a key contributor in this role, you will leverage your expertise in big data technologies to design, implement, and optimize complex data processing solutions. Your contribution will be pivotal in enhancing our clients' data architectures and ensuring robust data pipelines. Qualifications and Skills Proficiency in Python (Mandatory skill) is required to efficiently handle data processing tasks and pipeline creation. Experience with PySpark (Mandatory skill) for scalable data analytics and large data set processing is essential. Expertise in Databricks (Mandatory skill) to collaborate and transform data in a cloud-based environment is necessary. Strong knowledge of Amazon Redshift for creating and managing data warehouses and ensuring data storage efficiency. Experience with AWS Glue for developing and managing ETL jobs to prepare data for analytics. Familiarity with AWS Lambda for executing code in response to triggers, ensuring a flexible data processing environment. Experience with Hadoop frameworks for processing large volumes of data and managing data storage effectively. Advanced SQL skills for querying, managing, and structuring data stored in relational databases. Roles and Responsibilities Design and implement scalable data processing pipelines to handle large data sets and support business goals. Collaborate with clients and team members to gather data requirements and ensure accurate data structuring. Optimize data architectures for performance, reliability, and scalability within cloud environments. Utilize AWS services to develop innovative data solutions tailored to client needs. Conduct regular data monitoring and maintenance to ensure data integrity and security. Analyze and interpret complex data sets, providing actionable insights to stakeholders. Stay updated with emerging trends and technologies in big data engineering to drive continuous improvement. Participate in code reviews and contribute to maintaining coding standards and practices.

  • Big Data Engineer

    3 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    Job Title :- Data Engineer - Pyspark Experience: 5 to 8 Years Location: Pune/Hyderabad Job Description Required Skills: 5+ years of experience in Big data and pyspark Must-Have Good work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQL Good-to-Have Good Spark, Pyspark,Big Data experience Spark UI/Optimization/debugging techniques...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, India Whatjobs IN C2 Full time

    Job Title :- Data Engineer - Pyspark Experience: 5 to 8 Years Location: Pune/Hyderabad Job Description Required Skills: 5+ years of experience in Big data and pyspark Must-Have Good work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQL Good-to-Have Good Spark, Pyspark,Big Data experience Spark UI/Optimization/debugging techniques...

  • Big Data Engineer

    6 days ago

    Bengaluru, Karnataka, India Acme Services Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    We are looking for Big Data Engineer who has experience with:- 1) Big Query 2) Pyspark 3) GCP or AWS or Azure Required Candidate profileCandidate should have more than 5 Years of overall experience in Data Engineer. We are not looking for Data Analyst or Snowflake Developer.

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Akshya Patra Services Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    We are looking for a skilled Big Data Engineer with expertise in various Big Data tools and ecosystems. The ideal candidate should have hands-on experience with technologies such as Hadoop, Spark, Hive, and Impala. A strong proficiency in programming languages like Scala, Java, and Python is essential for designing and implementing data processing...

  • Data Engineer

    7 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    As a Data Engineer, your main responsibilities will involve: The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects. Analyze data models and derive logical conclusions. Processes modeling. Hands on development and monitoring of the Azure cloud Platform and various associated...

  • Big Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Brillio Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Key skills - Hive, SQL, Python, Spark, Hadoop and kafka , apache airflowJob Title Big Data EngineerRole OverviewWe are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves...

  • Senior data engineer(big data)

    2 weeks ago

    Bengaluru, India Andor Tech Full time

    Andor Tech is looking for a highly skilled Senior Big Data Engineer with 8–15 years of experience to join our dynamic team. If you're passionate about big data technologies and thrive in solving complex problems, we want to hear from you!Location : Deployment to a Product-Based Retail MNCImmediate Joiners PreferredKey Skills Required :Strong expertise in...

  • Senior data engineer(big data)

    2 weeks ago

    Bengaluru, India Andor Tech Full time

    Andor Tech is looking for a highly skilled Senior Big Data Engineer with 8–15 years of experience to join our dynamic team. If you're passionate about big data technologies and thrive in solving complex problems, we want to hear from you! ???? Location : Deployment to a Product-Based Retail MNC ???? Immediate Joiners Preferred ???? Key Skills Required :...

  • Senior data engineer(big data)

    4 days ago

    Bengaluru, India Andor Tech Full time

    Andor Tech is looking for a highly skilled Senior Big Data Engineer with 8–15 years of experience to join our dynamic team. If you're passionate about big data technologies and thrive in solving complex problems, we want to hear from you! ???? Location : Deployment to a Product-Based Retail MNC ???? Immediate Joiners Preferred ???? Key Skills Required :...

  • Big Data Engineer

    2 days ago

    Bengaluru, Karnataka, India Innova ESI Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role: Big Data EngineerExperience: 7+ YearsLocation: BangaloreNotice: Immediate Joiners OnlyJob Description:Azure Databricks, Azure Data Factory, Azure Function apps, Apache Spark, Scala, Java, Apache Kafka, event stream and Big DataOptional Skills: Airflow, PythonRoles & ResponsibilitiesOverall, 12 years of experience in IT industry, including 5 years in...