Data Engineer

6 days ago

Chennai, India Tata Communications Full time

Position OverviewAn experienced Senior Data Engineer with at least 5 years of hands-on experience in data engineering. The ideal candidate will have a solid understanding of big data technologies and be skilled in building scalable data infrastructure, designing ETL pipelines, and leveraging tools like Hadoop, PySpark, Kafka, and Apache NiFi.Key Responsibilities- Design, develop, and maintain large-scale, high-performance data systems and data pipelines using Python, PySpark, Hadoop, and Kafka.- Build, deploy, and optimize ETL workflows to process and transform large volumes of structured and unstructured data.- Collaborate with cross-functional teams to understand requirements and implement solutions that meet business needs.- Work with Apache NiFi for data ingestion, transformation, and flow management.- Write and optimize complex SQL queries for data manipulation and reporting.- Apply strong data structures and algorithms knowledge to solve complex technical problems.- Automate tasks and processes using Shell Script and Linux-based tools.- Participate in code reviews, design discussions.- Ensure adherence to best practices in software development, testing, and deployment.- Continuously improve software performance, scalability, and reliability.- Stay up-to-date with the latest developments in data engineering, big data technologies and incorporate them into the team's practices.Required Skills and Qualifications- Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.- At least 5 years of professional software development experience, with strong expertise in the following:- Python: Advanced proficiency in Python, including libraries like pandas, numpy, etc.- PySpark: Experience with distributed data processing using PySpark.- Hadoop: Familiarity with the Hadoop ecosystem, including HDFS, MapReduce, and related tools.- Kafka: Hands-on experience in building and maintaining Kafka-based messaging systems.- SQL: Strong knowledge of relational databases and advanced SQL querying.- Data Structures & Algorithms: Strong understanding and practical application of data structures and algorithms.- Data Engineering Best Practices: Deep understanding of data modeling, pipeline design, and data infrastructure architecture.- ETL Pipelines: Expertise in designing, building, and maintaining efficient ETL pipelines.- Apache NiFi: Knowledge of data flow management using Apache NiFi.- Shell Scripting: Proficiency in writing efficient shell scripts for task automation.- Linux: Strong knowledge of Linux systems and tools for development and deployment.- Experience with Agile development methodologies.- Excellent problem-solving skills and ability to troubleshoot complex technical issues.- Strong communication skills with the ability to work in a collaborative team environment.Preferred Qualifications- Experience with platforms like Cloudera, Databricks.- Familiarity with containerization technologies like Docker and Kubernetes.- Knowledge of data warehousing and data lakes.

  • Senior Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Full time

    Eucloid is looking for a skilled Data Engineer to join our team and contribute to the design, development, and optimization of data frameworks and pipelines using Python, Elasticsearch, SQL DB, and Queue System. The ideal candidate will have a strong technical background, a passion for solving complex problems, and experience in building scalable and...

  • Senior Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India AABM Cloud Data Solution Full time

    Senior Data Engineer (Remote, India)10+ Years Data Engineer Experience RequiredSQL, Python, Snowflake - 8+ years Mandatory hands-on experienceExperience with any ETL Tools (SSIS Preferred), Cloud Environments, and DB2/DBT is a plus.

  • GCP Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India CustomerLabs ❤️ 1P Data OPs Full time ₹ 12,00,000 - ₹ 30,00,000 per year

    Position Overview Yesterday is history, tomorrow is a mystery, but today is a gift. That's why we call it the present. - Master OogwayJoin CustomerLabs' dynamic data team as a Data Engineer and play a pivotal role in transforming raw marketing data into actionable insights that power our digital marketing platform. As a key member of our data infrastructure...

  • Data Centre Remote Technical Support Engineer

    6 days ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Data Centre...

  • Data & Business Insights Associate

    1 week ago

    Chennai, Tamil Nadu, India NTT DATA Full time ₹ 4,00,000 - ₹ 8,00,000 per year

    Works under supervision with internal and external teams to understand customer requirementsAbility to elaborate on the details of a project Collaborate with Automation Engineers, Machine Learning Engineers, Data Analysts and Business Intelligence Analyst to execute deliverables.Must possess technical writing, decision making, and problem-solving skills.Must...

  • Data Activity Monitoring

    2 weeks ago

    Chennai, Tamil Nadu, India NTT DATA, Inc. Full time ₹ 60,000 - ₹ 1,20,000 per year

    Role: Data Activity Monitoring (DAM) Engineers – L1 & L2Location:Chennai, IndiaEducation:BE / B.Tech / MCAinElectronics, Electrical, or Computer SciencePosition 1: DAM Engineer – L1Experience:2 yearsBudget:₹5.85 LPAKey Responsibilities:Support day-to-day operations of Data Activity Monitoring tools .Monitor database and application activity to...

  • Senior Engineer

    1 week ago

    Chennai, Tamil Nadu, India NTT DATA Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAAs a Security...

  • Testing Engineering Specialist

    6 days ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Req ID: 349941NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Testing Engineering Specialist to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). API Testing, Java,...

  • Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Req ID: 346251NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Engineer to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). Who We Are: NTT DATA America strives to hire...

  • Senior Data Consultant

    2 days ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Full time

    We are looking for a high-energy individual for the role of Senior Data Consultant at Eucloid. The candidate will collaborate with client stakeholders on multiple projects in the eCommerce team.The candidate is expected to lead following work-streams:• Drive client meetings with data-driven insights and observations on key website/app metricsfor Upper...