Senior Data Engineer

1 month ago

Chennai, India Agilisium Full time

Key Responsibilities :

Data Pipeline Development :

- Design, develop, and maintain robust data pipelines using tools like Airflow, Databricks, and Pyspark.

- Extract, transform, and load data from various sources, including databases, APIs, and cloud storage.

- Ensure data quality and integrity through rigorous testing and validation procedures.

Big Data Technologies :

- Leverage big data technologies like Hadoop, Spark, and Snowflake to process and analyze large datasets.

- Optimize data pipelines for performance and scalability.

- Implement data warehousing and data lake solutions.

Cloud Infrastructure :

- Utilize AWS services (i.e., S3, EMR, Redshift) to build and manage cloud-based data infrastructure.

- Automate infrastructure provisioning and management using tools like Terraform and CloudFormation.

API Integration :

- Integrate with external APIs (i.e., Facebook Graph API, GA, Segment, Google Ad Words, Pinterest, Snapchat) to extract valuable data.

- Handle API rate limits, authentication, and error handling.

Data Quality Assurance :

- Implement data quality checks and monitoring to ensure data accuracy and consistency.

- Identify and resolve data quality issues proactively.

Collaboration :

- Collaborate with data analysts, data scientists, and other stakeholders to understand their data needs and deliver solutions.

- Contribute to a culture of data-driven decision-making.

Required Qualifications :

- 6+ years of experience in data engineering and ETL pipeline development.

- Proficiency in SQL, Python, and Pyspark.

- Strong understanding of big data technologies (Hadoop, Spark, etc.

- Experience with cloud platforms, especially AWS.

- Proficiency in data pipeline orchestration tools like Airflow.

- Experience with data warehousing and data lake architectures.

- Knowledge of data modeling and data warehousing concepts.

- Experience with data quality and validation techniques.

- Strong problem-solving and analytical skills.

- Excellent communication and collaboration skills.

Preferred Qualifications :

- Experience with Kubernetes and Docker.

- Knowledge of machine learning and AI concepts.

- Experience with data visualization tools (i.e., Tableau, PowerBI).

- Certifications in AWS, Azure, or GCP

(ref:hirist.tech)

  • Senior Data Engineer

    2 months ago

    chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Senior Data Engineer

    1 month ago

    Chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Senior Data Engineer

    2 months ago

    Chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Senior Data Engineer

    2 months ago

    Chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Senior Data Engineer

    2 months ago

    Chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Senior Data Engineer

    2 months ago

    chennai, India Wavicle Data Solutions Full time

    We are seeking an experienced Senior AWS Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. You will work closely with data scientists, analysts, and cross-functional teams to optimize and manage our data infrastructure. This role requires a...

  • Senior Machine Learning Engineer

    2 days ago

    Chennai, Tamil Nadu, India Crayon Data Full time

    We are seeking a highly skilled Senior Machine Learning Engineer to join our team in Chennai. This is a fantastic opportunity to work on large scale data solutions using machine learning, operation research and pattern searching methods.About Crayon DataCrayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore...

  • Senior Data Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India Wavicle Data Solutions Full time

    We are seeking an Experienced AWS Data Architect with a strong background in designing, building, and maintaining scalable data pipelines and solutions on AWS. The ideal candidate will have a deep understanding of AWS services, data engineering best practices, and cloud architecture.Key Responsibilities:Design, develop, and maintain robust ETL pipelines on...

  • Crayon Data | Senior Software Engineer | chennai

    3 weeks ago

    chennai, India Crayon Data Full time

    Role: Senior Software Engineer – BackendExperience level: 5+ yearsLocation: HybridWho are we?​Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship platform,...

  • Crayon Data | Senior Software Engineer | chennai

    3 weeks ago

    chennai, India Crayon Data Full time

    Role: Senior Software Engineer – Backend Experience level: 5+ years Location: Hybrid Who are we? ​ Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship...

  • Crayon Data | Senior Software Engineer | chennai

    3 weeks ago

    chennai, India Crayon Data Full time

    Role: Senior Software Engineer – Backend Experience level: 5+ years Location: Hybrid Who are we? ​ Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship...

  • Senior bi engineer

    1 week ago

    Chennai, India Wavicle Data Solutions Full time

    Job Description:We are seeking a highly skilled and dynamic Senior Engineer in Business Intelligence (BI) to spearhead our BI initiatives. The ideal candidate will have expertise in AWS Quick Sight, Tableau and Power BI, along with strong cloud knowledge, communication skills, and a background in ETL/Data Warehousing.Key Responsibilities:Develop and lead the...

  • Data Engineer

    1 month ago

    Chennai, India NTT DATA Full time

    Job Description Req ID: 298625 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).Data EngineerJob SummaryWe...

  • Senior Software Engineer

    3 weeks ago

    Chennai, India Crayon Data Full time

    Role: Senior Software Engineer – BackendExperience level: 5+ yearsLocation: HybridWho are we?​Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship platform,...

  • Senior Software Engineer

    3 weeks ago

    Chennai, India Crayon Data Full time

    Role: Senior Software Engineer – Backend Experience level: 5+ yearsLocation: HybridWho are we?​Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship platform,...

  • Senior software engineer

    3 weeks ago

    Chennai, India Crayon Data Full time

    Role: Senior Software Engineer – BackendExperience level: 5+ yearsLocation: HybridWho are we?​Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship platform,...

  • Senior Software Engineer

    3 weeks ago

    Chennai, India Crayon Data Full time

    Role: Senior Software Engineer – BackendExperience level: 5+ yearsLocation: HybridWho are we?​Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship platform,...

  • Senior Software Engineer

    3 weeks ago

    Chennai, India Crayon Data Full time

    Role: Senior Software Engineer – BackendExperience level: 5+ yearsLocation: HybridWho are we?​Crayon Data is a leading provider of AI-led revenue acceleration solutions, headquartered in Singapore with a local presence in India and the UAE. The company was founded in 2012 with the vision of simplifying the world’s choices. Our flagship platform,...

  • Senior Data Engineer

    2 months ago

    Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time

    Job Title: Senior Data EngineerJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team at Cynosure Corporate Solutions. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing systems using big data technologies such as Azure, AWS, and GCP. Key...

  • Chief Data Science Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Job SummaryWe are seeking an experienced Chief Data Science Engineer to join our team at NTT DATA. As a key member of our organization, you will be responsible for designing and implementing machine learning systems, researching and developing ML algorithms and tools, and selecting appropriate data sets.About the RoleYou will work closely with...