AWS Data engineering lead

3 weeks ago

delhi, India YASH Technologies Full time
Location- Pune/Indore/Hyderabad
Primary Skills :
• Masters / Bachelors Degree in Computer Science, Information Technology or other relevant fields.
• Minimum 7+ Years of Mandatory experience in Advance Data & Analytics Architecture, ETL, Data Engineering solutions using the following Skills, tools and technologies:
• AWS Data & Analytics Services: Athena, Glue, EMR, DynamoDB, Redshift, Kinesis, Lambda
• Databricks Lakehouse Platform
• PySpark, Spark SQL, Spark Streaming
• Experience with Apache Kafka / Nifi for use with streaming data / event-based data
• Experience with any NoSQL Database
• 3+ year of coding experience with modern programming or scripting language (Python).
• Expert-level skills in writing and optimizing SQL.
• Experience operating very large data warehouses or data lakes/ data platform.
• Show efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data.
• Experience of working in Agile delivery
• Sound knowledge of distributed systems and data architecture (lambda)- design and implement batch and stream data processing pipelines, knows how to optimize the distribution, partitioning, and MPP of high-level data structures
• Experience with full software development life cycle, including coding standards, code reviews, source control management, build processes, and testing.
• Excellent business and communication skills to work with business owners to understand data requirements.
Responsibilities :
• You will be responsible to understand the client requirement and architect robust data platform on AWS cloud technologies and Databricks.
• You will be responsible for identifying and creating reusable data solutions (accelerators, MVP)
• You will be responsible for creating reusable components for rapid development of data platform
• Work closely with the Product Owners and stake holders to design the Technical Architecture for data platform to meet the requirements of the proposed solution.
• Work with the leadership to set the standards for software engineering practices within the Data engineering team and support across other disciplines
• Play an active role in leading team meetings and workshops with clients.
• Delivering and presenting proofs of concept to of key technology components to project stakeholders.
• Educate clients on Cloud technologies and influence direction of the solution.
• Choose and use the right analytical libraries, programming languages, and frameworks for each task.
• Help us to shape the next generation of our products.
• Explore and learn the latest AWS Data & Analytics and Databricks Platform features /technologies to provide new capabilities and increase efficiency.
• Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Databricks, Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda,Glue, Athena.
• Analyze, re-architect and re-platform, migrate on-premise data warehouses to data platforms on AWS cloud using AWS or 3rd party services.
• Design and build production data pipelines from ingestion to consumption within a big data architecture, using Python,PySpark, Databricks.
• Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.
• Perform detail assessments of current state data platforms and create an appropriate transition path to AWS cloud & Databricks.
• Build and deliver high quality data architecture to support business analyst, data scientists, and customer reporting needs.

  • AWS Data Engineer

    2 weeks ago

    delhi, India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    3 weeks ago

    delhi, India ITI Data Full time

    Job DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...

  • AWS Data Engineer

    3 months ago

    Delhi, India ITI Data Full time

    Job DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...

  • Aws data engineer

    2 weeks ago

    Delhi, India ITI Data Full time

    Job DescriptionWe are looking for an AWS Data with primary skills on Py Spark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • Lead data engineer

    2 weeks ago

    Delhi, India Wavicle Data Solutions Full time

    Job Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, Py Spark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...

  • Data Engineer

    1 month ago

    Delhi, India NucleusTeq Full time

    Company DescriptionNucleusTeq is a global software solutions and product company headquartered in Phoenix, Arizona. Specializing in Big Data, Analytics, Cloud Computing, Blockchain, Enterprise Automation, and Mobility, we empower businesses through digital innovation. Our diverse portfolio of Fortune 100 clients and global partners showcases our ability to...

  • AWS Data Engineer

    20 hours ago

    Delhi, India Cigres Technologies Private Limited Full time

    -      Work with IB AWS lead on the design of the architectures for the new capabilities hosted in our AWS platform.-      Deploy Infrastructure using CloudFormation and/or Terraform.-       Build our Data Stores hosted in RDS, DynamoDB, Redshift ..etc-      Work closely with our data engineers to optimize the performance of our...

  • AWS Data Engineer

    2 months ago

    delhi, India FLYONIT Full time

    Job Summary: We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing...

  • AWS Data Engineer

    2 weeks ago

    delhi, India FLYONIT Full time

    Job Summary:We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing and...

  • AWS Data Engineer

    2 months ago

    delhi, India FLYONIT Full time

    Job Summary:We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing and...

  • AWS Data Engineer

    3 weeks ago

    delhi, India FLYONIT Full time

    Job Summary:We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing and...

  • Aws data engineer

    3 weeks ago

    Delhi, India FLYONIT Full time

    Job Summary:We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing and...

  • AWS Data Engineer

    3 weeks ago

    delhi, India FLYONIT Full time

    Job Summary:We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing and...

  • AWS Data Engineer

    21 hours ago

    Delhi, India Cigres Technologies Private Limited Full time

    -Work with IB AWS lead on the design of the architectures for the new capabilities hosted in our AWS platform.-Deploy Infrastructure using CloudFormation and/or Terraform.-Build our Data Stores hosted in RDS, DynamoDB, Redshift ..etc-Work closely with our data engineers to optimize the performance of our applications by improving the...

  • Data Engineer

    4 weeks ago

    Delhi, India Adastra Full time

    Adastra India is a part of Adastra Group. The company was established in 2024 as an extension of Adastra Global Delivery in the APAC region. The company is focused on attracting, upskilling and utilizing local talent for projects delivered by Adastra in Europe and North America. The company has a local office in Pune hiring consultants locally and remotely...

  • AWS Data Engineer

    2 weeks ago

    delhi, India Recro Full time

    Job Title: AWS Data Engineer Location - Remote Experience - 3+ years Note- Looking for candidates who have experience working from the scratch and are currently serving their notice period, with availability to join within 10-15 days. We are seeking a Data Engineer with strong hands-on experience in data engineering techniques to create data pipelines...

  • AWS Data Engineer

    20 hours ago

    Delhi, India Recro Full time

    Job Title: Data EngineerExperience : 5+ yearsLocation : RemoteJob Type : Full-timeRole Overview:We are seeking an experienced Data Engineer to develop and optimize data pipelines and manage cloud infrastructure. The ideal candidate has strong Python, Pyspark, and AWS skills, with a focus on performance optimization and clean, maintainable code.Key...

  • AWS Data Engineer

    2 months ago

    Delhi, India Programmers.io Full time

    AWS Data Engineer - MarketingKey Tasks:Analyze existing marketing data/tables and summarize data.Collaborating with existing DE team to ingest additional data sources for marketingDeveloping data workflows to process, clean & link / unify consumer data sourcesStrong in SQL and has experience in implementing data quality checksIngesting unified / clean data...

  • AWS Data Engineer

    2 weeks ago

    delhi, India Recro Full time

    Job Title: AWS Data Engineer Location - RemoteExperience - 3+ yearsNote- Looking for candidates who have experience working from the scratch and are currently serving their notice period, with availability to join within 10-15 days.We are seeking a Data Engineer with strong hands-on experience in data engineering techniques to create data pipelines and...

  • AWS Data Engineer

    2 weeks ago

    delhi, India Recro Full time

    Job Title: AWS Data EngineerLocation - RemoteExperience - 3+ yearsNote- Looking for candidates who have experience working from the scratch and are currently serving their notice period, with availability to join within 10-15 days.We are seeking a Data Engineer with strong hands-on experience in data engineering techniques to create data pipelines and build...