Big Data Engineer

1 day ago

Bengaluru, India Acme Services Full time

Job Description We are actively seeking a skilled Big Data Engineer to join our client's dynamic team through Acme Services. This pivotal role requires solid experience in ETL / Data Engineering / Data Warehousing principles. The ideal candidate will possess strong proficiency in Python, PySpark, and SQL or other ETL tools, along with hands-on experience in various database and big data technologies such as MySQL, SQL Server, MongoDB, Hadoop, Hive, Pig, and Spark. Experience with orchestration tools like AirFlow and container platforms like OpenShift would be a significant advantage. Key Responsibilities - Data Pipeline Development: Design, build, and maintain robust and scalable ETL (Extract, Transform, Load) pipelines to ingest, process, and transform large datasets from various sources into data warehouses or data lakes. - Big Data Technologies: Work extensively with Big Data platforms including Hadoop, Hive, Pig, and Spark, leveraging their capabilities for data processing and analytics. - Programming & Scripting: Utilize Python, PySpark, and SQL for data manipulation, scripting, and developing data engineering solutions. - Database Management: Interact with and manage data in various database systems such as MySQL, SQL Server, and MongoDB. - Data Warehousing: Contribute to the design, implementation, and optimization of Data Warehouses, ensuring data quality and accessibility for business intelligence and reporting. - Orchestration & Automation (Advantage): (If applicable) Use AirFlow for scheduling, monitoring, and orchestrating complex data workflows. - Containerization (Advantage): (If applicable) Leverage OpenShift for deploying and managing data applications in a containerized environment. - Troubleshooting & Optimization: Identify and resolve data-related issues, and optimize data processes for performance and efficiency. Skills - Solid experience in ETL / Data Engineering / Data Warehouse. - Strong proficiency in Python, PySpark, and SQL or other ETL tools. - Hands-on experience with databases including MySQL, SQL Server, MongoDB. - Expertise in Hadoop, Hive, Pig, and Spark. - Experience in AirFlow (big advantage). - Experience in OpenShift (big advantage). - Strong analytical and problem-solving skills. - Ability to work with large, complex datasets. Qualifications - Proven professional experience as a Big Data Engineer or in a similar data-focused role. - Demonstrable experience in building and managing data pipelines. - Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

  • Big Basket

    1 week ago

    Bengaluru, India big basket Full time

    What would you be doing/ Expected from this role ?- Collaborate with cross-functional teams including data scientists, engineers, and product managers to deliver AI-driven solutions.- Drive the architecture of machine learning pipelines from data collection and preprocessing to model training, evaluation, and deployment.- Implement scalable machine learning...

  • Big Data Engineer

    1 week ago

    Bengaluru, India Tata Consultancy Services Full time

    Job Title :- Data Engineer - PysparkExperience: 5 to 8 YearsLocation: Pune/HyderabadJob DescriptionRequired Skills:5+ years of experience in Big data and pysparkMust-HaveGood work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQLGood-to-HaveGood Spark, Pyspark,Big Data experienceSpark UI/Optimization/debugging techniquesGood...

  • Big Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Etaclam Techno Services Full time ₹ 12,00,000 - ₹ 14,40,000 per year

    Job Title: Big Data EngineerLocation: BengaluruExperience Required: 5 to 8 yearsJob Description:We are looking for a skilled Big Data Engineer with strong hands-on experience in PySpark and SQL. This is a hybrid role, requiring the candidate to work both remotely and from the office as needed. The ideal candidate should be comfortable working on large-scale...

  • Big Data

    1 day ago

    Bengaluru, India Teamware Solutions Full time

    Job Description Teamware Solutions is seeking a talented and passionate Big Data Engineer with 2-4 years of experience to join our dynamic team. This role is crucial for designing, developing, implementing, and troubleshooting scalable Big Data solutions that process, store, and manage large volumes of diverse data. The successful candidate will contribute...

  • Big Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Reveille Technologies,Inc Full time ₹ 16,00,000 - ₹ 24,00,000 per year

    We're Hiring: Big Data Engineer – Full-TimeLooking for a skilledBig Data Engineerto join our team inBangaloreRole: Big Data EngineerGrade: G4 |TSRLocation: Bangalore (Onsite/Hybrid – specify if needed)Budget: ₹13,39,200Experience: 4–6 yearsSkills:Azure Databricks (ADB)PySparkSQLSpark (Scala/Python)TerraformAirflow & KubernetesData Warehousing &...

  • Big Data Engineer

    4 days ago

    Bengaluru, Karnataka, India The IT Firm Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role OverviewAs a Big Data Engineer, you will be responsible for building and maintaining scalable data pipelines and architectures that support data analytics, reporting, and real-time processing. You will collaborate with data scientists, business analysts, and software engineers to design robust solutions that drive data-driven decisions.Mandatory Skill...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Deqode Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Profile: Big Data Engineer (System Design)Experience:5+ yearsLocation:BangaloreWork Mode:HybridAbout The RoleWe're looking for an experiencedBig Data Engineerwith system design expertise to architect and build scalable data pipelines and optimize big data solutions.Key ResponsibilitiesDesign, develop, and maintain data pipelines and ETL processes using...

  • Big Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Fractal Analytics Private Limited Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    BigData Engineer It's fun to work in a company where people truly BELIEVE in what they are doing We're committed to bringing passion and customer focus to the business.About the Role:We are seeking highly skilled Big Data Engineers/Senior Engineers with a strong background in building and supporting big data solutions. The ideal candidate will have...

  • Big Data Engineer

    1 week ago

    Bengaluru, Delhi, Mumbai, NCR, India Suzva Software Technologies Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Big Data Engineer (Remote, Contract 6 Months+) ig Data ecosystem including Snowflake (Snowpark), Spark, MapReduce, Hadoop, and more.We are looking for a Senior Big Data Engineer with deep expertise in large-scale data processing technologies and frameworks. This is a remote, contract-based position suited for a data engineering expert with strong experience...

  • Big Data Engineer

    1 day ago

    Hyderabad, Bengaluru, India Fusion Plus Solutions Full time

    Job Description We are looking for a skilled Big Data Engineer with 5 to 7 years of hands-on experience in Big Data technologies. The ideal candidate should have strong expertise in Hadoop, HDFS, Hive, SQL, and Unix, and be able to work on large-scale data processing systems. Roles and Responsibilities: - Work on design, development, and optimization of Big...