Senior Data Engineer

3 months ago

chennai, India Cortex Consulting Pvt. Ltd. Full time

Job Description :


We are seeking a highly experienced and accomplished Senior AWS Data Engineer to join our growing team in Chennai. You will play a pivotal role in designing, developing, and managing large-scale data pipelines on the AWS cloud platform, leveraging your expertise in Python and data engineering best practices.


You will be responsible for the entire data lifecycle, from data ingestion to transformation and delivery, ensuring efficient and reliable data flow for our data-driven initiatives.

Responsibilities :

AWS Cloud Expertise :

- Possess in-depth knowledge of AWS services for data management and processing, including S3, Redshift, DynamoDB, EMR, and Lambda functions.

- Design and implement scalable data architectures on AWS that are secure, cost-effective, and meet performance requirements.

- Utilize Infrastructure as Code (IaC) tools like Terraform to automate infrastructure provisioning and configuration management.

- Stay up-to-date with the latest advancements in AWS services and best practices.

Data Pipelines and Automation :

- Develop and maintain robust data pipelines using Python and data engineering frameworks like Apache Airflow.

- Automate data ingestion, transformation, loading (ETL) processes for seamless data movement within the AWS ecosystem.

- Implement data quality checks and monitoring mechanisms to ensure data integrity and consistency throughout the pipeline.

- Leverage Python libraries like Pandas and Spark for data manipulation and processing tasks.

Data Warehousing and Analytics :

- Experience with data warehousing concepts and technologies, including data modeling and schema design.

- Collaborate with data analysts and scientists to understand data requirements and translate them into technical solutions.

- Prepare and optimize data for analytics workloads on platforms like Redshift or Athena.

Collaboration and Communication :

- Work effectively with cross-functional teams (data analysts, scientists, developers) to ensure successful data integration and insights generation.

- Provide clear and concise technical documentation for designed data pipelines and architectures.

- Proactively communicate potential issues and propose solutions for efficient problem-solving.

Qualifications :

- Minimum of 15 years of experience in data engineering or a related field.

- Proven track record of designing, developing, and deploying large-scale data pipelines on the AWS cloud platform.

- Expertise in Python programming and data engineering frameworks (e., Airflow).

- In-depth knowledge of AWS services for data management, processing, and analytics.

- Familiarity with data warehousing concepts and technologies.

- Experience with data quality management and monitoring tools.

- Excellent communication, collaboration, and problem-solving skills.

- Ability to work independently and manage multiple projects simultaneously.

- Strong analytical and critical thinking skills.

- Passion for learning and staying up-to-date with the latest advancements in data engineering technologies

(ref:hirist.tech)

  • Senior Data Engineer

    4 months ago

    Chennai, India Saaki Argus & Averil Consulting Full time

    Hiring Senior Data Engineer for an IT Product Company Location : Chennai / Pune Role: Senior Data Engineer: Minimum 4+Yrs of Relevant Experience Mandatory Skills: Strong expertise in Python ( Class methods), OOPS in Python, SQL , Databricks , AWS or Azure cloud Master's degree in either software engineering, data engineering, data science, computer...

  • Senior Data Engineer

    1 month ago

    Chennai, India Zuci Systems Full time

    Job Title: Senior Data EngineerResponsibilities: As a Senior Data Engineer, you will be working with data architects to take data throughout its lifecycle - acquisition, exploration, data cleaning, integration, analysis, and visualization. You will be creating the pipeline for data processing, data visualization, and analytics products. You will be the go-to...

  • Senior Data Engineer

    1 month ago

    Chennai, India Zuci Systems Full time

    Job Title: Senior Data EngineerResponsibilities: As a Senior Data Engineer, you will be working with data architects to take data throughout its lifecycle - acquisition, exploration, data cleaning, integration, analysis, and visualization. You will be creating the pipeline for data processing, data visualization, and analytics products. You will be the go-to...

  • Senior Data Engineer

    1 month ago

    Chennai, India Zuci Systems Full time

    Job Title: Senior Data Engineer Responsibilities: As a Senior Data Engineer, you will be working with data architects to take data throughout its lifecycle - acquisition, exploration, data cleaning, integration, analysis, and visualization. You will be creating the pipeline for data processing, data visualization, and analytics products. You will be the...

  • Senior Data Engineer

    15 hours ago

    chennai, India Zuci Systems Full time

    Job Title: Senior Data EngineerResponsibilities: As a Senior Data Engineer, you will be working with data architects to take data throughout its lifecycle - acquisition, exploration, data cleaning, integration, analysis, and visualization. You will be creating the pipeline for data processing, data visualization, and analytics products. You will be the go-to...

  • Director of Data Engineering

    13 hours ago

    Chennai, India Wavicle Data Solutions Full time

    Job descriptionWavicle Data Solutions is seeking an experienced and visionaryDirector of Data Engineeringto lead and manage our growing Data Engineering team.As a Director, you will be responsible for driving the technical strategy, overseeing complex data projects, and delivering high-quality solutions that support our clients' business goals. This role...

  • Senior Data Engineer

    1 month ago

    Chennai, India KANINI Full time

    Senior Data EngineerJob Overview:We are seeking a highly skilled Senior Data Engineer with 6-10 years of experience to lead and manage our data-driven projects. The ideal candidate should have a strong background in SQL, PySpark, AWS, Databricks, and Scala, with proven expertise in migration projects and Jira. This role requires excellent communication...

  • Director of Data Engineering

    3 days ago

    Chennai, India Wavicle Data Solutions Full time

    Job descriptionWavicle Data Solutions is seeking an experienced and visionary Director of Data Engineering to lead and manage our growing Data Engineering team.As a Director, you will be responsible for driving the technical strategy, overseeing complex data projects, and delivering high-quality solutions that support our clients' business goals. This role...

  • Director of Data Engineering

    2 days ago

    chennai, India Wavicle Data Solutions Full time

    Job description Wavicle Data Solutions is seeking an experienced and visionary Director of Data Engineering to lead and manage our growing Data Engineering team. As a Director, you will be responsible for driving the technical strategy, overseeing complex data projects, and delivering high-quality solutions that support our clients' business goals. This...

  • Director of Data Engineering

    3 days ago

    Chennai, India Wavicle Data Solutions Full time

    Job descriptionWavicle Data Solutions is seeking an experienced and visionary Director of Data Engineering to lead and manage our growing Data Engineering team.As a Director, you will be responsible for driving the technical strategy, overseeing complex data projects, and delivering high-quality solutions that support our clients' business goals. This role...

  • Director of Data Engineering

    3 days ago

    chennai, India Wavicle Data Solutions Full time

    Job descriptionWavicle Data Solutions is seeking an experienced and visionary Director of Data Engineering to lead and manage our growing Data Engineering team.As a Director, you will be responsible for driving the technical strategy, overseeing complex data projects, and delivering high-quality solutions that support our clients' business goals. This role...

  • Senior Data Engineer

    4 months ago

    Chennai, India Epam Full time

    Description EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that...

  • Senior Data Engineer

    3 months ago

    chennai, India PPLABILITY (OPC) PRIVATE LIMITED Full time

    Job Title: Senior Data Engineer-Data Modeling. Location: Bengaluru, Chennai, about the role? What your typical day would look like? As a Senior Data Engineer, you will work to solve some of the organizational data management problems that would enable them as a data-driven organization; Seamlessly switch between roles of an Individual Contributor, team...

  • Senior Data Engineer

    2 months ago

    Chennai, India Ciklum Full time

    Description Ciklum  is looking for a Senior Data Engineer  to join our team full-time in India. We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and...

  • Senior Data Engineer

    3 weeks ago

    chennai, India Truelancer.com Full time

    Job Title: Sr Data EngineerExperience: 5 - 9 yrsLocation: Work from Office ,ChennaiMandatory Skills: AWS Data Services+Strong in SQL QueriesAs a Senior Data Engineer, you will play a crucial role in designing, developing, and maintaining data pipelines and analytics solutions. You will work with a range of AWS technologies and data tools to ensure our...

  • Senior Data Engineer

    2 weeks ago

    Chennai, India Programming.com Full time

    Job DescriptionJob Title: Senior Data EngineerLocation : ChennaiSalary : ₹13 LPAExperience Required : 6 YearsNotice Period : Immediate to 15 daysResponsibilities :♦ Design and optimize data processing systems.♦ Collaborate with teams to implement data models.♦ Ensure data architecture aligns with business needs.♦ Build and maintain data...

  • Senior Consultant

    4 months ago

    chennai, India AstraZeneca Full time

    Job Title: Senior Consultant - Data Engineer Career Level - D3 Introduction to role: Join our Commercial IT Data Analytics & AI (DAAI) team as a Senior Consultant - Data Engineer. In this role, you will be responsible for developing, testing, and maintaining a scalable data product. You will have the opportunity to work with Databricks, PySpark,...

  • Senior Consultant

    4 months ago

    Chennai, India AstraZeneca Full time

    Job Title: Senior Consultant - Data Engineer Career Level - D3 Introduction to role: Join our Commercial IT Data Analytics & AI (DAAI) team as a Senior Consultant - Data Engineer. In this role, you will be responsible for developing, testing, and maintaining a scalable data product. You will have the opportunity to work with Databricks, PySpark, and...

  • Senior Consultant

    3 days ago

    chennai, India AstraZeneca Full time

    Job Title: Senior Consultant - Data Engineer Career Level - D3 Introduction to role: Join our Commercial IT Data Analytics & AI (DAAI) team as a Senior Consultant - Data Engineer. In this role, you will be responsible for developing, testing, and maintaining a scalable data product. You will have the opportunity to work with Databricks, PySpark, and...

  • Senior Data Engineer

    1 month ago

    Chennai, India KANINI Full time

    Senior Data Engineer Job Overview:We are seeking a highly skilled Senior Data Engineer with 6-10 years of experience to lead and manage our data-driven projects. The ideal candidate should have a strong background in SQL, PySpark, AWS, Databricks, and Scala, with proven expertise in migration projects and Jira. This role requires excellent communication...