Big Data Engineer

3 weeks ago

Bengaluru, India Acme Services Full time

Job Description We are actively seeking a skilled Big Data Engineer to join our client's dynamic team through Acme Services. This pivotal role requires solid experience in ETL / Data Engineering / Data Warehousing principles. The ideal candidate will possess strong proficiency in Python, PySpark, and SQL or other ETL tools, along with hands-on experience in various database and big data technologies such as MySQL, SQL Server, MongoDB, Hadoop, Hive, Pig, and Spark. Experience with orchestration tools like AirFlow and container platforms like OpenShift would be a significant advantage. Key Responsibilities - Data Pipeline Development: Design, build, and maintain robust and scalable ETL (Extract, Transform, Load) pipelines to ingest, process, and transform large datasets from various sources into data warehouses or data lakes. - Big Data Technologies: Work extensively with Big Data platforms including Hadoop, Hive, Pig, and Spark, leveraging their capabilities for data processing and analytics. - Programming & Scripting: Utilize Python, PySpark, and SQL for data manipulation, scripting, and developing data engineering solutions. - Database Management: Interact with and manage data in various database systems such as MySQL, SQL Server, and MongoDB. - Data Warehousing: Contribute to the design, implementation, and optimization of Data Warehouses, ensuring data quality and accessibility for business intelligence and reporting. - Orchestration & Automation (Advantage): (If applicable) Use AirFlow for scheduling, monitoring, and orchestrating complex data workflows. - Containerization (Advantage): (If applicable) Leverage OpenShift for deploying and managing data applications in a containerized environment. - Troubleshooting & Optimization: Identify and resolve data-related issues, and optimize data processes for performance and efficiency. Skills - Solid experience in ETL / Data Engineering / Data Warehouse. - Strong proficiency in Python, PySpark, and SQL or other ETL tools. - Hands-on experience with databases including MySQL, SQL Server, MongoDB. - Expertise in Hadoop, Hive, Pig, and Spark. - Experience in AirFlow (big advantage). - Experience in OpenShift (big advantage). - Strong analytical and problem-solving skills. - Ability to work with large, complex datasets. Qualifications - Proven professional experience as a Big Data Engineer or in a similar data-focused role. - Demonstrable experience in building and managing data pipelines. - Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

  • Big Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Etaclam Techno Services Full time ₹ 12,00,000 - ₹ 14,40,000 per year

    Job Title: Big Data EngineerLocation: BengaluruExperience Required: 5 to 8 yearsJob Description:We are looking for a skilled Big Data Engineer with strong hands-on experience in PySpark and SQL. This is a hybrid role, requiring the candidate to work both remotely and from the office as needed. The ideal candidate should be comfortable working on large-scale...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, India Whatjobs IN C2 Full time

    Job Title :- Data Engineer - Pyspark Experience: 5 to 8 Years Location: Pune/Hyderabad Job Description Required Skills: 5+ years of experience in Big data and pyspark Must-Have Good work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQL Good-to-Have Good Spark, Pyspark,Big Data experience Spark UI/Optimization/debugging techniques...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    Job Title :- Data Engineer - Pyspark Experience: 5 to 8 Years Location: Pune/Hyderabad Job Description Required Skills: 5+ years of experience in Big data and pyspark Must-Have Good work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQL Good-to-Have Good Spark, Pyspark,Big Data experience Spark UI/Optimization/debugging techniques...

  • Big Data

    3 weeks ago

    Bengaluru, India Teamware Solutions Full time

    Job Description Teamware Solutions is seeking a talented and passionate Big Data Engineer with 2-4 years of experience to join our dynamic team. This role is crucial for designing, developing, implementing, and troubleshooting scalable Big Data solutions that process, store, and manage large volumes of diverse data. The successful candidate will contribute...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Deqode Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Profile: Big Data Engineer (System Design)Experience:5+ yearsLocation:BangaloreWork Mode:HybridAbout The RoleWe're looking for an experiencedBig Data Engineerwith system design expertise to architect and build scalable data pipelines and optimize big data solutions.Key ResponsibilitiesDesign, develop, and maintain data pipelines and ETL processes using...

  • Big Data Engineer

    6 days ago

    Bengaluru, Karnataka, India Akshya Patra Services Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    We are looking for a skilled Big Data Engineer with expertise in various Big Data tools and ecosystems. The ideal candidate should have hands-on experience with technologies such as Hadoop, Spark, Hive, and Impala. A strong proficiency in programming languages like Scala, Java, and Python is essential for designing and implementing data processing...

  • Big Data Engineer

    1 week ago

    Bengaluru, Delhi, Mumbai, NCR, India Suzva Software Technologies Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Big Data Engineer (Remote, Contract 6 Months+) ig Data ecosystem including Snowflake (Snowpark), Spark, MapReduce, Hadoop, and more.We are looking for a Senior Big Data Engineer with deep expertise in large-scale data processing technologies and frameworks. This is a remote, contract-based position suited for a data engineering expert with strong experience...

  • Big Data Engineer

    3 weeks ago

    Hyderabad, Bengaluru, India Fusion Plus Solutions Full time

    Job Description We are looking for a skilled Big Data Engineer with 5 to 7 years of hands-on experience in Big Data technologies. The ideal candidate should have strong expertise in Hadoop, HDFS, Hive, SQL, and Unix, and be able to work on large-scale data processing systems. Roles and Responsibilities: - Work on design, development, and optimization of Big...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Chennai, Hyderabad, India Net2source Llp Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Hiring: Big Data/Hadoop Engineer (PySpark, Hive) | 4 to 6 YearsWe are looking for talented Big Data/Hadoop Engineer for one of our premium clientsWork Locations: Bangalore (KA) | Chennai (TN) | Hyderabad (TG)Experience: 4 to 6 YearsRequired Skills:Strong hands-on experience in PySpark and HiveExpertise in Big Data & Hadoop Ecosystem toolsAbility to design...

  • Data Engineer

    12 hours ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    As a Data Engineer, your main responsibilities will involve: The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects. Analyze data models and derive logical conclusions. Processes modeling. Hands on development and monitoring of the Azure cloud Platform and various associated...