Senior Data Engineer

3 weeks ago


tamil nadu, India Altimetrik Full time
Primary Skill :Data Engineer
Secondary Skill : GCP, Big Query
Exp : 5 to 10 yrs
Job Description:
We are seeking a skilled Data Engineer with expertise in Google Cloud Platform (GCP), particularly in BigQuery and Pub/Sub, coupled with advanced proficiency in Python. As a Data Engineer, you will play a pivotal role in designing, implementing, and optimizing data pipelines, ensuring the smooth flow of data within our organization.
Responsibilities:
1. Architect and Develop Data Pipelines: Design, develop, and maintain scalable and efficient data pipelines using GCP services such as Pub/Sub for real-time data ingestion and BigQuery for storage and analysis.
2. Data Transformation and Processing: Implement data transformation processes to cleanse, enrich, and aggregate raw data from various sources, ensuring data quality and consistency.
3. Optimize Performance: Fine-tune data pipelines and queries to optimize performance and reduce latency, ensuring timely access to data for stakeholders.
4. Monitoring and Maintenance: Implement monitoring solutions to track pipeline performance and proactively address issues. Perform regular maintenance tasks to ensure the reliability and availability of data infrastructure.
5. Collaboration: Collaborate with cross-functional teams including Data Scientists, Software Engineers, and Business Analysts to understand data requirements and provide technical solutions to address business needs.
6. Documentation: Document data pipelines, processes, and best practices to ensure knowledge sharing and maintain a comprehensive understanding of data architecture.
Requirements:
1. Proficiency in GCP Services: Extensive hands-on experience with Google Cloud Platform services, particularly BigQuery for data storage and analysis, and Pub/Sub for real-time data streaming.
2. Python Programming: Strong programming skills in Python for data manipulation, scripting, and automation tasks. Experience with libraries such as Pandas, NumPy, and TensorFlow is highly desirable.
3. Data Modeling: Solid understanding of data modeling concepts and experience in designing efficient data models for analytics and reporting purposes.
4. SQL Skills: Proficiency in writing complex SQL queries for data extraction, transformation, and analysis within BigQuery.
5. Experience with Data Warehousing: Familiarity with data warehousing concepts and experience in implementing data warehouse solutions using GCP BigQuery or similar technologies.
6. Problem-solving Skills: Strong analytical and problem-solving skills with the ability to troubleshoot and resolve complex data-related issues.
7. Communication Skills: Excellent communication skills with the ability to effectively collaborate with cross-functional teams and articulate technical concepts to non-technical stakeholders.
8. Bachelor’s Degree: Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field. Advanced degree or relevant certifications in Data Engineering or GCP are a plus.

  • tamil nadu, India Wavicle Data Solutions Full time

    Job description Wavicle Data Solutions is seeking an experienced and visionary Director of Data Engineering to lead and manage our growing Data Engineering team. As a Director, you will be responsible for driving the technical strategy, overseeing complex data projects, and delivering high-quality solutions that support our clients' business goals. This...


  • tamil nadu, India Wavicle Data Solutions Full time

    Job descriptionWavicle Data Solutions is seeking an experienced and visionary Director of Data Engineering to lead and manage our growing Data Engineering team.As a Director, you will be responsible for driving the technical strategy, overseeing complex data projects, and delivering high-quality solutions that support our clients' business goals. This role...


  • tamil nadu, India Sriven Full time

    Job Title: Senior Big Data Engineer Experience Required: 7+ years Educational Qualification: Bachelor's degree from an accredited college or university Job Description: We are looking for a highly skilled Senior Big Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in designing and developing complex Big Data...


  • tamil nadu, India Sriven Full time

    Job Title: Senior Big Data EngineerExperience Required: 7+ yearsEducational Qualification: Bachelor's degree from an accredited college or universityJob Description:We are looking for a highly skilled Senior Big Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in designing and developing complex Big Data pipelines...

  • Senior Engineer

    4 months ago


    tamil nadu, India Tiger Analytics Full time

    Role: Senior Engineer – Quality EngineeringWork Location – Chennai, Bangalore or HyderabadTiger Analytics is a global AI and analytics consulting firm. With data and technology at the core of our solutions, our 4000+ tribe is solving problems that eventually impact the lives of millions globally. Our culture is modeled around expertise and respect with a...


  • tamil nadu, India Altimetrik Full time

    Primary Skill :Data Engineer Secondary Skill : GCP, Big Query Exp : 5 to 10 yrs Job Description: We are seeking a skilled Data Engineer with expertise in Google Cloud Platform (GCP), particularly in BigQuery and Pub/Sub, coupled with advanced proficiency in Python. As a Data Engineer, you will play a pivotal role in designing, implementing, and optimizing...

  • Senior Data Engineer

    3 weeks ago


    tamil nadu, India Mastech InfoTrellis Full time

    Position: Azure Data Engineer Lead and DeveloperLocation : Chennai(Hybrid-3 days a week)/ If Non Local, need quarterly once, sit with team for a week or twoDuration: Full TimePositions: 7Notice Period: 15-40 days (Non- Negotiable)**Should have good Comms skills- Its client facing role**1st preference will be given to local candidatesCandidates from...


  • tamil nadu, India Deloitte Full time

    Hi, We are looking for Candidates who are having experience in Data Bricks Engineer Along with any cloud experience. Experience Required: 7- 14 Year sLocation: Coimbatore, Tamil Nadu, Indi a Responsibiliti esDesign, develop, and optimize data workflows and notebooks using Databricks to ingest, transform, and load data from various sources into the data...

  • Senior Data Engineer

    3 weeks ago


    tamil nadu, India Poshmark Full time

    Job Description:The Big Data team is a central player in the Poshmark organization. Our mission is to build a world-class big data platform to bring value out of data for us and for our customers. Our goal is to democratize data, support exploding business, provide reporting and analytics self-service tools, and fuel existing and new business critical...

  • Data Architect

    1 week ago


    Tamil Nadu, India Wavicle Data Solutions Full time

    Multiple years of hands-on experience working as a data architect on Azure or AWS or Snowflake cloud ecosystems. Designing & implementing the organization's data architecture, including data models, data flows, and data storage solutions.Creating and maintaining data dictionaries and data maps to document data sources, data types, and data...

  • Data Architect

    4 days ago


    tamil nadu, India Wavicle Data Solutions Full time

    Multiple years of hands-on experience working as a data architect on Azure or AWS or Snowflake cloud ecosystems. Designing & implementing the organization's data architecture, including data models, data flows, and data storage solutions. Creating and maintaining data dictionaries and data maps to document data sources, data types, and data relationships....


  • tamil nadu, India Blackstraw Full time

    Data Engineer Company Name: Blackstraw.ai Office Location: Chennai (Work from Office) Job Type: Full-time Experience: 6+ Years Job Description: We are looking for Data Engineers to join our team. You will use various methods to transform raw data into useful data systems. For example, you’ll create algorithms and conduct statistical analysis. Overall,...

  • Sr Data Engineer

    3 weeks ago


    tamil nadu, India Truelancer.com Full time

    Job Title: Sr Data EngineerExperience: 5 - 9 yrsLocation: Work from Office ,ChennaiMandatory Skills: AWS Data Services+Strong in SQL QueriesAs a Senior Data Engineer, you will play a crucial role in designing, developing, and maintaining data pipelines and analytics solutions. You will work with a range of AWS technologies and data tools to ensure our...

  • AWS Data Engineers

    3 weeks ago


    tamil nadu, India PTS Consulting Full time

    We are seeking Experienced AWS Data Engineers with extensive expertise in AWS (EMR/S3/GLUE/REDSHIFT etc) to join our Cloud practice delivering transformative solutions to leading clients.This is a GREEN FIELD Projec t and NOT A lift and Shift Project. We are seeking highly experienced individuals who have experience in a AWS Green field setting.At this...


  • tamil nadu, India Crayon Data Full time

    Role: MLops Engineer Experience level: 5-7years Location: Chennai (Hybrid) Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship platform, maya.ai, helps enterprises...

  • Engineering Manager

    3 days ago


    tamil nadu, India Crayon Data Full time

    Engineering Manager (10 to 12 yrs) Chennai, Tamil Nadu, India Who are we? ​ Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship platform, maya.ai, helps enterprises...


  • tamil nadu, India Crayon Data Full time

    Role: MLops EngineerExperience level: 5-7yearsLocation: Chennai (Hybrid)Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship platform, maya.ai, helps enterprises...


  • tamil nadu, India Intelliserve Konsulting Pvt Ltd Full time

    Organization Leading Insurance company (Client of Intelliserve)Role : Data Engineer ManagerLocation: ChennaiExperience 11+ YearsIndustry: General InsuranceWorking Mode Work from the officePurpose:This role will own the data ingestion, storage, and engineering processes and functions on the data platform. The Data Engineering Manager will ensure the timely...

  • Senior Data Analyst

    2 weeks ago


    tamil nadu, India Mastech InfoTrellis Full time

    Position: Senior Data AnalystLocation: Chennai, Tamil Nadu(3 day/week) / (Remote with travel 1 week/ 3 months)Duration: Full TimeCompany: Mastech InfoTrellis Senior Data Analyst will be responsible for performing data extraction, data gap analysis, mapping data to Azure-based data models, and ensuring data integrity and consistency across systems. This...


  • tamil nadu, India Value Health Inc Full time

    Role: Senior Gen AI EngineerJob Location: CoimbatoreJob Overview:We are seeking a highly skilled and experienced Senior Gen AI Engineer to join our dynamic team. The ideal candidate will have a strong background in machine learning, artificial intelligence, natural language processing & Generative AI with a proven track record of developing and deploying...