AWS Data Engineer

3 days ago

Madurai, India TerraGiG Full time

Role AWS Data Engineer Experience 8+ years Location Remote Time Zone UK Duration 2 months (Extendable)Job Description Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR. Writing reusable, testable, and efficient code Integration of data storage solutions in spark – especially with AWS S3 object storage. Performance tuning of pySpark scripts. Need to ensure overall build delivery quality is good and on-time delivery is done at all times. Should be able to handle meetings with customers with ease. Need to have excellent communication skills to interact with the customer. Be a team player and willing to work in an onsite-offshore model, mentor other folks in the team (onsite as well as offshore)*5+ years of experience in programming with python. Strong proficiency in python *Familiarity with functional programming concepts *3+ years of hands-on experience in developing ETL data pipelines using pySpark on AWS EMR *Experience in building pipelines and data lake for large enterprises on AWS *Good understanding of Spark’s Dataframe and API *Experience in configuring EMR clusters on AWS *Experience in dealing with AWS S3 object storage from Spark. *Experience in troubleshooting spark jobs. Knowledge of monitoring spark jobs using Spark UI *Performance tuning of Spark jobs. *Understanding fundamental design principles behind business processesProcess Knowledge and Expertise: Demonstrated experience in change management processes, including understanding of governance frameworks and preparation of supporting artefacts required for approvals. Strong clarity on the path to production, with hands-on involvement in deployments, testing cycles, and obtaining business sign-offs. Proven track record in technical solution design, with the ability to provide architectural guidance and support implementation strategies.Databricks-Specific Skills: Experience in at least developing and delivering end-to-end Proof of Concept (POC) solutions covering the below: Basic proficiency in Databricks, including creating jobs and configuring clusters. Exposure to connecting external data sources (e.g., Amazon S3) to Databricks. Understanding of Unity Catalog and its role in data governance. Familiarity with notebook orchestration and implementing modular code structures to enhance scalability and maintainability.Important Pointers: Candidates must haveactual hands-on work experience , not just home projects or academic exercises. Profiles should clearly statehow much experience they have in each skill area , as this helps streamline the interview process. Candidates mustknow their CV/profile inside out , including all projects and responsibilities listed. Any ambiguity or lack of clarity on the candidate’s part can lead to immediate rejection, as we value accuracy and ownership. They should be able to confidently explain their past experience, challenges handled, and technical contributions.

  • AWS Data Engineer

    1 day ago

    Madurai, Tamil Nadu, India Techmango Technology Services Private Limited Full time ₹ 80,00,000 - ₹ 2,40,00,000 per year

    About TechMango:Founded in 2014, TechMango Technology Services is a leading software development company with a strong focus on emerging technologies. Our primary goal is to deliver strategic solutions that align with our business partners' technological needs.We specialize in providing custom software solutions using the best available technologies,...

  • Senior AWS Data Engineer

    1 week ago

    Madurai, Tamil Nadu, India Techmango Technology Services Private Limited Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Job Title : Data EngineerLocation: Madurai / ChennaiExperience : 8+ YrsThe OpportunityWe're looking for a seasoned Data Engineer to join our Engineering organization and take ownership of the systems that power our data-driven decisions. You'll architect and maintain high-performance data pipelines and databases across platforms like MSSQL, Redshift, and...

  • DevOps Engineer

    1 week ago

    Madurai, India OneData Software Solutions Full time

    About OneData Software Solutions OneData Software Solutions is an AWS Advanced Tier Partner specializing in Cloud, Data Analytics, AI/ML, and IoT solutions. With over 12 years of industry excellence, 150+ technical experts, and 400+ global clients, we deliver scalable and intelligent software systems that drive business transformation and digital innovation....

  • Data Modeler

    5 days ago

    Madurai, India HCLTech Full time

    Looking for Immediate joiners - From 5+ to 7 yrs :- Key Responsibilities: Design and implement conceptual, logical, and physical data models aligned with business and regulatory requirements. Develop and maintain ETL/ELT pipelines to support data ingestion, transformation, and integration across systems. Collaborate with DevOps, BI, and application teams to...

  • Data Modeler

    3 days ago

    madurai, India HCLTech Full time

    Looking for Immediate joiners - From 5+ to 7 yrs :- Key Responsibilities: Design and implement conceptual, logical, and physical data models aligned with business and regulatory requirements. Develop and maintain ETL/ELT pipelines to support data ingestion, transformation, and integration across systems. Collaborate with DevOps, BI, and application teams to...

  • Data Modeler

    2 days ago

    Madurai, India Whatjobs IN C2 Full time

    Looking for Immediate joiners - From 5+ to 7 yrs :- Key Responsibilities: Design and implement conceptual, logical, and physical data models aligned with business and regulatory requirements. Develop and maintain ETL/ELT pipelines to support data ingestion, transformation, and integration across systems. Collaborate with DevOps, BI, and application teams to...

  • Data Modeler

    3 days ago

    Madurai, India HCLTech Full time

    Looking for Immediate joiners - From 5+ to 7 yrs :-Key Responsibilities: Design and implementconceptual, logical, and physical data modelsaligned with business and regulatory requirements. Develop and maintainETL/ELT pipelinesto support data ingestion, transformation, and integration across systems. Collaborate with DevOps, BI, and application teams to...

  • Lead Data Architect

    2 weeks ago

    Madurai, India ACL Digital Full time

    Technology Skills and Project Experience: - 12 years of experience in modeling and business system designs. - 10 years hands on experience as Data Engineer/Architect with proven experience on Big Data and Data Lakes on AWS - 7 years of experience in below Technologies - Scripts/Programming language: SQL, Python, PROC, Windows Functions - GitHub - AWS: S3,...

  • Quantexa Data Engineer

    6 days ago

    Madurai, India Teamware Solutions Full time

    About the CompanyGreetings from Teamware Solutions a division of Quantum Leap Consulting Pvt. LtdAbout the RoleWe are hiring aQuantexa Data Engineer (Big Data | Spark | Hadoop | Scala)Location: Bengaluru Work Model: HybridExperience: 4+ Years Notice Period: Immediate - 15 daysDesignation ConsultantRole type Big Data EngineerJob Requirements Mandatory Skills...

  • AWS Consultant

    5 days ago

    Madurai, India Playerhub Full time

    Company Description Playerhub offers an ultimate mobile platform and super app for casinos, designed to enhance player experiences both on and off the property. We provide a personalized, omni-channel experience that empowers players with self-service options and real-time connectivity. Our digital wallet integration enables seamless transactions across the...