Data Engineer

1 week ago

Chennai, Tamil Nadu, India O2F Info Solutions Pvt. Ltd. Full time

Job Summary : We are seeking a highly skilled Senior Data Engineer with 4 to 8 years of experience in building robust data pipelines and working extensively with PySpark to join our data engineering team. Key Responsibilities : Data Pipeline Development : - Design, build, and maintain scalable data pipelines using PySpark to process large datasets and support data-driven applications and analytics. ETL Process Automation : - Develop and automate ETL (Extract, Transform, Load) processes using PySpark, ensuring efficient data processing, transformation, and loading from diverse sources into data lakes, warehouses, or databases. Distributed Computing with PySpark : - Leverage Apache Spark and PySpark to process large-scale data in a distributed computing environment, optimizing for performance and scalability. Cloud Data Solutions : - Develop and deploy data pipelines and processing frameworks on cloud platforms (AWS, Azure, GCP) using native tools like AWS Glue, Azure Databricks, or Google Dataproc. Data Integration & Transformation : - Integrate data from various internal and external sources, ensuring data consistency, quality, and reliability throughout the pipeline. Performance Optimization : - Optimize PySpark jobs and pipelines for faster data processing, handling large volumes of data efficiently with minimal latency. - Proven experience as a Data Engineer or similar role, with a strong background in database development, ETL processes, and software development. - Proficiency in SQL and scripting languages such as Python, with experience working with relational databases. - Proficiency in dataProc (PySpark), Pandas or other data processing libraries - Experience with data modeling, schema design, and optimization techniques for scalability. - Strong analytical and problem-solving skills, with the ability to troubleshoot complex data issues and optimize data processing pipelines for scale Required Qualifications : Experience : - 4-8 years of experience in data engineering, with a strong focus on PySpark and large-scale data processing. Technical Skills : - Expertise in PySpark for distributed data processing, data transformation, and job optimization. - Strong proficiency in Python and SQL for data manipulation and pipeline creation. - Hands-on experience with Apache Spark and its ecosystem, including Spark SQL, Spark Streaming, and PySpark MLlib. - Solid experience working with ETL tools and frameworks, such as Apache Airflow or similar orchestration tools. (ref:hirist.tech)

  • Data Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Job Description Req ID: 303790 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).Job Duties: As a data...

  • Data Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Services Full time

    Req ID: 303790 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).Job Duties: As a data engineer you will...

  • Data Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India Clustrex Data Private Limited Full time

    We're Hiring: Data Engineer Position: Data Engineer Experience: 2-4 years Location: Madipakkam, Chennai - work from office Skills & Requirements: Strong proficiency in Python with hands-on experience in Pandas and NumPy Expertise in PostgreSQL, including query optimization & performance analysis Solid understanding of statistical analysis and data...

  • Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India Clustrex Data Private Limited Full time

    We're Hiring: Data EngineerPosition: Data EngineerExperience: 2-4 yearsLocation: Madipakkam, Chennai - work from officeSkills & Requirements: Strong proficiency in Python with hands-on experience in Pandas and NumPy Expertise in PostgreSQL, including query optimization & performance analysis Solid understanding of statistical analysis and data transformation...

  • Engineer (L1) - Titus Data Classification

    4 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT...

  • 15h Left: Data Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India Clustrex Data Private Limited Full time

    We're Hiring: Data EngineerPosition: Data EngineerExperience: 2-4 yearsLocation: Madipakkam, Chennai - work from officeSkills & Requirements:Strong proficiency in Python with hands-on experience in Pandas and NumPyExpertise in PostgreSQL, including query optimization & performance analysisSolid understanding of statistical analysis and data transformation...

  • Django Full Stack Engineer with Data Insights

    3 days ago

    Chennai, Tamil Nadu, India Lapis Data Analytics Full time

    About Lapis Data AnalyticsLapis Data Analytics is a cutting-edge data engineering company founded in Chennai. We specialize in designing and building robust data pipelines to convert raw data into actionable insights. With a seasoned team of experts and leading-edge technology, we deliver customized data solutions to drive business growth and success.Role...

  • Data Engineer for Large Scale Data Analytics

    1 week ago

    Chennai, Tamil Nadu, India Vue Data Technologies Private Limited Full time

    About Vue Data Technologies Private Limited">We are a leading technology firm that provides innovative solutions to businesses. Our team of experts is dedicated to delivering high-quality services that drive growth and success.">Job Summary:">We are seeking an experienced Data Analytics Engineer to join our team. The ideal candidate will have strong...

  • Data Pipeline Specialist

    2 days ago

    Chennai, Tamil Nadu, India Lapis Data Analytics Full time

    Join Our TeamLapis Data Analytics is a dynamic and growing company looking for talented professionals to join our team.We are committed to providing a positive and inclusive work environment that fosters creativity, innovation, and collaboration.About the JobThis is a full-time on-site Java Software Engineer role located in Chennai.The Software Engineer will...

  • Digital Engineering Lead Engineer

    6 days ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Req ID 292803NTT DATA Services strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Digital Engineering Lead Engineer to join our team in Chennai Tamil N xc4 x81du IN-TN India IN Flutter...

  • Senior Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT...

  • Senior Data Development Specialist

    3 days ago

    Chennai, Tamil Nadu, India Lapis Data Analytics Full time

    Company Overview:Lapis Data Analytics is a data engineering company founded on the principles of quality, innovation, and reliability. Our team specializes in designing, building, and maintaining robust data pipelines that drive business growth and success.Job Summary:We are seeking an experienced Java Software Engineer to join our team in Chennai. As a...

  • Engineering Innovation Manager

    5 days ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Engineering Innovation at NTT DATAAt NTT DATA, we believe in harnessing the power of innovation to drive business growth and success. Our Digital Engineering Lead Engineer will play a key role in shaping our engineering vision and strategy, collaborating with cross-functional teams to develop cutting-edge software solutions that meet the evolving needs of...

  • Java software engineer

    2 days ago

    Chennai, Tamil Nadu, India Lapis Data Analytics Full time

    Company DescriptionLapis Data Analytics is a data engineering company founded in 2020, specializing in designing, building, and maintaining robust data pipelines. With a focus on data quality, security, and scalability, we deliver customized data solutions to drive business growth and success. Committed to quality, innovation, and reliability, we are your...

  • Java Software Engineer

    3 days ago

    Chennai, Tamil Nadu, India Lapis Data Analytics Full time

    Company DescriptionLapis Data Analytics is a data engineering company founded in 2020, specializing in designing, building, and maintaining robust data pipelines. With a focus on data quality, security, and scalability, we deliver customized data solutions to drive business growth and success. Committed to quality, innovation, and reliability, we are your...

  • Java Software Engineer

    1 week ago

    Chennai, Tamil Nadu, India Lapis Data Analytics Full time

    Job DescriptionCompany DescriptionLapis Data Analytics is a data engineering company founded in 2020, specializing in designing, building, and maintaining robust data pipelines. With a focus on data quality, security, and scalability, we deliver customized data solutions to drive business growth and success. Committed to quality, innovation, and reliability,...

  • TS Implementation Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT...

  • Network Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT...

  • Java Software Engineer Role

    2 days ago

    Chennai, Tamil Nadu, India Lapis Data Analytics Full time

    Data Engineering CompanyLapis Data Analytics is a leading provider of data engineering services, specializing in the design and implementation of robust data pipelines.Our team of experts utilizes the Spring Framework and Java to develop innovative data solutions that drive business growth and success.About the PositionWe are seeking an experienced Java...

  • Microsoft Security Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India IC Data Full time

    Security Engineer: ( Microsoft Subject Matter Expert) : Fulltime Remote About the Role:Do you want to be on the frontline fighting for safe use within the digital frontier? Does describing your job to your family and friends as being a cyberwarrior or threat hunter sound awesome or awe inspiring? Then join us as part of our Security Operations Center...