Aws and Big Data

2 weeks ago

Pune, India WUElev8 Innovation Services Full time

**Invite Your Friend**
- Invite

**Share on**

**Posted on**
- 07/25/2023, 03:38:06 AM
- **Applications**

**AWS and Big Data (Scala & Spark)**
- DataVerze
Full Time
15 days Notice period acceptable
5 vacancies
AWS
Big Data
Spark
scala
EC2
S3
EMR
Java
Python
12 to 45 LPA
4 to 12 Years
Pune, Maharashtra
- Job Description- Skills Required- Company DetailsWe are seeking a highly skilled and experienced Lead with expertise in AWS and Big Data technologies, specifically Scala and Spark. As a Lead, you will play a crucial role in designing, implementing, and optimizing large-scale data processing solutions.

**Responsibilities**:

- Lead the development and implementation of data processing solutions using AWS and Big Data technologies.
- Design, architect, and maintain scalable and high-performance data pipelines and ETL processes.
- Collaborate with cross-functional teams to gather and analyze business requirements and translate them into technical specifications.
- Mentor and provide technical guidance to a team of developers, ensuring best practices and code quality standards are followed.
- Perform code reviews and provide constructive feedback to optimize performance and ensure scalability.
- Identify and resolve issues and bottlenecks in data processing workflows to ensure smooth and efficient operations.
- Stay up-to-date with the latest advancements in AWS services, Big Data frameworks, and emerging technologies.
- Work closely with stakeholders to understand their data needs and provide effective data solutions.
- Collaborate with data scientists and analysts to enable efficient data analysis and reporting.
- Conduct performance tuning and optimization of Spark jobs and data pipelines to enhance overall system efficiency.

**Requirements**:

- Extensive experience working with **AWS services such as EC2, S3, EMR, Glue, and Lambda.**:

- Strong proficiency in **Big Data technologies, particularly Scala and Spark.**:

- Solid understanding of data processing frameworks and concepts, including **ETL, data modeling, and data warehousing.**:

- Proven track record in designing and developing large-scale data pipelines using **AWS and Big Data technologies.**:

- Hands-on experience with data ingestion, transformation, and storage.
- Worked with Data Quality Framework while processing large scale data.
- Proficient in programming languages such as **Scala, Java, or Python.**:

- Strong knowledge of database systems and SQL.
- Familiarity with agile development methodologies and DevOps practices.
- Excellent problem-solving skills and the ability to work in a fast-paced, dynamic environment.
- Exceptional communication and leadership abilities to effectively collaborate and guide team members.
- Experience with Databricks, Data Catalog & Data Governance use cases.

**Preferred Qualifications**:

- AWS certification in Big Data or related areas.
- Experience with other Big Data frameworks such as Hadoop, Scala & Spark
- Knowledge of data streaming technologies like Apache Kafka or AWS Kinesis.
- Familiarity with machine learning and data science concepts.

Join our team as a Lead - AWS and Big Data to spearhead the development and optimization of our data processing solutions. This is an excellent opportunity to leverage your expertise in AWS, Big Data, Scala, and Spark while leading a talented team and making a significant impact on our organization's data-driven initiatives.

  • AWS Data Engineer

    3 days ago

    Pune, India HNM Solutions Full time

    Note : Only Immediate Joiner Role : AWS Data Engineer Location : Pune, India Experience : 8 Years JD: Roles and responsibilities : - With over 8 yrs. of hands-on experience with Data Technologies - Requirement analysis and assess the technical feasibility of proposed solutions. - Act as the technical specialist in designing and recommending architecture for...

  • AWS Data Engineer

    4 days ago

    Pune, India HNM Solutions Full time

    Note : Only Immediate JoinerRole : AWS Data EngineerLocation : Pune, IndiaExperience : 8 YearsJD: Roles and responsibilities :- With over 8 yrs. of hands-on experience with Data Technologies - Requirement analysis and assess the technical feasibility of proposed solutions. - Act as the technical specialist in designing and recommending architecture for...

  • AWS Data Engineer

    4 days ago

    Pune, Maharashtra, India HNM Solutions Full time

    Note : Only Immediate JoinerRole : AWS Data EngineerLocation : Pune, IndiaExperience : 8 YearsJD: Roles and responsibilities :- With over 8 yrs. of hands-on experience with Data Technologies - Requirement analysis and assess the technical feasibility of proposed solutions. - Act as the technical specialist in designing and recommending architecture for...

  • AWS Data Architect

    1 month ago

    Bangalore,Chennai,Gurgaon,Gurugram,Hyderabad,Pune, India HuQuo Full time

    Role Description: Data Engineer - Big Data The ideal candidate is a hands-on technology developer with experience in developing scalable applications and platforms. They must be at ease working in an agile environment with little supervision. The person should be a self-motivated person with a passion for problem solving and continuous learning.  Role and ...

  • AWS Data Architect

    4 weeks ago

    Bangalore/Chennai/Gurgaon/Gurugram/Hyderabad/Pune, IN Huquo Full time

    Role Description: Data Engineer - Big Data The ideal candidate is a hands-on technology developer with experience in developing scalable applications and platforms. They must be at ease working in an agile environment with little supervision. The person should be a self-motivated person with a passion for problem solving and continuous learning. Role and...

  • Big Data

    4 weeks ago

    Pune, India Zensar Technologies Full time

    **BIG DATA ( ANALYTICS AND DATA SCIENCE ) - BIG DATA - SNOWFLAKE** **-** **(** **0094963** **)** **Description** Snowflake Job Description: - Snowflake SQL Writing SQL queries against Snowflake Developing scripts Unix, Python, etc. to do Extract, Load, and Transform data - Hands-on experience with Snowflake utilities such as SnowSQL, SnowPipe, Python,...

  • Senior Data Engineer

    3 days ago

    Pune, India Data Axle Full time

    Job DescriptionAbout Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and...

  • Senior Data Engineer

    2 weeks ago

    pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Engineer

    1 month ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Big Data Engineer

    1 month ago

    Pune, India Hexaware Technologies Full time

    Description : 4 or more years of experience working directly with enterprise data solutions Hands on experience working in a public cloud environment and on-prem infrastructure. Specialty on Columnar Databases like Redshift Spectrum and the AWS cloud infrastructure services (Redshift, S3, Lambda) Excellent SQL skills and Python coding is...

  • Lead Big Data Software Engineer

    3 weeks ago

    Pune, India Epam Full time

    DescriptionEPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver...

  • Data Engineer

    1 month ago

    Pune, India Huquo Full time

    Basic understanding of :- Scheduling and workflow management & working experience in either ADF, Informatica, Airflow, or Similar- Enterprise Data Modelling and Semantic Modelling & working experience in ERwin, ER/Studio, PowerDesigner, or Similar- Logical/Physical model on Big Data sets or modern data warehouse & working experience in ERwin, ER/Studio,...

  • AWS Data Engineer

    1 week ago

    Pune, India Virtusa Full time

    AWS Data Engineer - CREQ188160 Description Responsibilities:Create and maintain optimal data pipeline architecture by designing and implementing data ingestion solutions on AWS using AWS native services (such as GLUE, Lambda) or using data management technologies (such as Talend or Informatica)Design and optimize data models on AWS Cloud using AWS data...

  • AWS Data Engineer

    1 week ago

    pune, India Virtusa Full time

    AWS Data Engineer - CREQ188160 Description Responsibilities:Create and maintain optimal data pipeline architecture by designing and implementing data ingestion solutions on AWS using AWS native services (such as GLUE, Lambda) or using data management technologies (such as Talend or Informatica)Design and optimize data models on AWS Cloud using AWS data...

  • AWS Data Engineer

    2 weeks ago

    Pune, India Virtusa Full time

    AWS Data Engineer - CREQ188160 Description Responsibilities: Create and maintain optimal data pipeline architecture by designing and implementing data ingestion solutions on AWS using AWS native services (such as GLUE, Lambda) or using data management technologies (such as Talend or Informatica) Design and optimize data models on AWS Cloud using AWS data...

  • Data Engineer

    1 month ago

    Bangalore,Pune, India Pylon Management Consulting Full time

    Requirements:- Overall 5+ years of IT experience with 3+ years in data-related technologies.- Minimum 2.5 years of experience in Big Data technologies and working exposure in at least one cloud platform on related data services (AWS / Azure / GCP).- Hands-on experience with the Hadoop stack - HDFS, sqoop, Kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink,...

  • Data Engineer

    4 weeks ago

    Bangalore/Pune, IN Pylon Management Consulting Full time

    Requirements:- Overall 5+ years of IT experience with 3+ years in data-related technologies.- Minimum 2.5 years of experience in Big Data technologies and working exposure in at least one cloud platform on related data services (AWS / Azure / GCP).- Hands-on experience with the Hadoop stack - HDFS, sqoop, Kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink,...

  • Lead Data Engineer- AWS

    1 month ago

    pune, India YASH Technologies Full time

    We are looking for Data Professionals with 7 years of minimum experience who can Explore and learn the latest Data & Analytics and Databricks Platform features /technologies to provide new capabilities and increase efficiency.Build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services...

  • Big Data

    1 month ago

    Pune, India Zensar Technologies Full time

    **BIG DATA ( ANALYTICS AND DATA SCIENCE ) - BIG DATA - SNOWFLAKE** **-** **(** **0095838** **)** **Description** Experience in developing ETL pipelines in and out of data warehouse using Snowflakes Snow SQL Knowledge on writing SQL queries against Snowflake. Experience in developing scripts Unix, Python etc. to do Extract, Load and Transform data Should...

  • Lead Data Engineer- AWS

    1 month ago

    Pune, India YASH Technologies Full time

    We are looking for Data Professionals with 7 years of minimum experience who can Explore and learn the latest Data & Analytics and Databricks Platform features /technologies to provide new capabilities and increase efficiency. Build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services...