Data Engineer

4 weeks ago

Chennai, Tamil Nadu, India BSRI Solutions Full time

About the Role :


We are seeking a highly proficient and motivated GCP Data Engineer to design, build, and maintain robust and scalable data pipelines on the Google Cloud Platform. In this role, you will be instrumental in preparing, automating, and optimizing data ingestion and transformation processes, with a keen eye towards supporting advanced analytics and AI/ML initiatives. If you have a passion for leveraging cutting-edge cloud technologies and open-source tools to solve complex data challenges, we encourage you to Responsibilities :

- Design and Development : Architect, develop, and maintain efficient, scalable, and reliable data pipelines and ETL/ELT processes on Google Cloud Platform.

- Data Automation & Ingestion : Utilize and integrate AI tools to automate data preparation, ingestion, and transformation workflows, ensuring data quality and availability.

- Distributed Data Processing : Implement and optimize data processing jobs using Apache Spark, with a strong emphasis on MLlib for machine learning workflows.

- Python for Data : Write clean, efficient, and well-documented code in Python for data manipulation, scripting, and pipeline development.

- Big Data Frameworks : Leverage PySpark and Dask for distributed data processing, handling large datasets effectively.

- Data Wrangling : Perform local data wrangling and analysis efficiently using libraries like Pandas and NumPy.

- Orchestration : Design, schedule, and monitor complex ETL/ELT workflows using Apache Airflow.

- GCP Ecosystem : Work extensively with key Google Cloud services, particularly BigQuery for data warehousing and Vertex AI for AI/ML model development and deployment.

- Optimization & Performance : Continuously monitor, troubleshoot, and optimize data pipeline performance and cost efficiency.

- Collaboration : Collaborate closely with data scientists, machine learning engineers, and other stakeholders to understand data requirements and deliver robust solutions.

- Documentation : Maintain comprehensive documentation for data pipelines, architectures, and Skills & Qualifications :


- Experience : Proven experience as a Data Engineer, with a strong portfolio of projects involving large-scale data processing.

- GCP Expertise : In-depth hands-on experience with Google Cloud Platform, specifically with BigQuery and Vertex AI.

- Big Data Processing : Strong proficiency with Apache Spark, including practical experience with MLlib.

- Python Prowess : Expert-level programming skills in Python, especially within the context of data engineering and AI/ML tasks.

- Distributed Computing : Hands-on experience with PySpark and Dask for distributed data processing.

- Data Wrangling Libraries : Solid understanding and practical application of Pandas and NumPy for data manipulation.

- Workflow Orchestration : Demonstrated experience in designing, implementing, and managing data workflows using Apache Airflow.

- AI Tools for Data : Familiarity and practical experience with AI tools used to prepare and automate data pipelines and ingestion.

- Problem-Solving : Excellent analytical and problem-solving skills with a strong attention to detail.

- Communication : Ability to communicate complex technical concepts effectively to both technical and non-technical Qualifications :

- Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related quantitative field.

- GCP Certifications (e.g., Google Cloud Professional Data Engineer).

- Experience with other components of the Google Cloud data ecosystem.

Why Join This Opportunity :

This is an excellent opportunity to work with a forward-thinking organization leveraging cutting-edge AI and cloud technologies. You will have the chance to contribute to significant data initiatives, work with a talented team, and further develop your expertise in the rapidly evolving field of GCP data engineering. (ref:hirist.tech)

  • Data Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India Quantrail Data Full time

    Location: Chennai, India (On-site)Duration: 3 monthsStipend: UnpaidEligibility: 2025 graduates onlyConversion: Opportunity for a full-time role based on performance Note: Please apply only if you are serious about joining us. This is a learning-focused internship, and we expect interns to be committed to upskilling themselves and contributing to real-world...

  • Data Engineer

    7 days ago

    Chennai, Tamil Nadu, India NTT DATA North America Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN)."Job Duties: Key Responsibilities:•  ...

  • Data Analyst

    4 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Req ID 324657 NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Data Analyst to join our team in Chennai Tamil Nxc4x81du IN-TN India IN Key Responsibilities ...

  • Eucloid - Senior/Lead Data Engineer - Python

    2 weeks ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Pvt Ltd Full time

    Job Description : Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of...

  • Data & Business Insights Analyst

    2 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Key Responsibilities Designing data models using Power BI Desktop and DAX Data Analysis Expressions Cleaning and transforming data using Power Query Editor Integrating data from multiple sources SQL Excel cloud services APIs Creating interactive dashboards and reports tailored to business needs Implementing visualizations that clearly communicate...

  • Data Engineer

    7 hours ago

    Chennai, Tamil Nadu, India Seven Consultancy Full time

    JOB DETAILS 1.Experience in Big Data technologies and working exposure in at least one cloud platform on related data services (AWS / Azure / GCP)2.Hands-on experience with the Hadoop stack – HDFS, sqoop, kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, hive, oozie, airflow and other components required in building end to end data...

  • Data Engineer

    2 days ago

    Chennai, Tamil Nadu, India xponent Full time

    xponent.ai is on a mission to empower aspiring enterprises to be their best and achieve exceptional business outcomes by unlocking the innovation and transformative power of data, analytics, and AI.We are building a Data and AI first company which will deliver innovative and impactful, sustainable Data and AI led solutions for its clients. We are open,...

  • Devops Lead Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Req ID 304648NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a DevOps Lead Engineer to join our team in Chennai Tamil Nxc4x81du IN-TN India IN Once You Are Here You Will Be...

  • Data Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India NatWest Group Full time

    Join us as a Data Engineer This is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless digital first customer experiences Youll be simplifying the bank by developing innovative data driven solutions using insight to be commercially successful and keeping our customers and the banks data safe and...

  • Systems Engineering Specialist

    2 weeks ago

    Chennai, Tamil Nadu, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Systems Engineering Specialist to join our team in Chennai, Tamil Ndu (IN-TN), India (IN).- Work closely with the...