Pyspark Architect

3 weeks ago

Bengaluru, India Talent Worx Full time

Job Description: PySpark Architect

Location: Bangalore,Pune,Kolkata

Experience Required: 7 to 13 Years

Role Overview:

We are seeking a highly skilled and motivated PySpark Manager to lead our data

engineering initiatives. The ideal candidate will have deep expertise in PySpark,

Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be responsible for

designing, building, and managing large-scale data processing solutions while leading a

team of data engineers to deliver high-quality outcomes.

Key Responsibilities:

1. Leadership & Team Management:

• Lead and mentor a team of data engineers, ensuring high performance and

professional growth.

• Oversee project timelines, quality, and deliverables for multiple data initiatives.

2. Data Engineering & Development:

• Design and implement robust data pipelines and workflows using PySpark and

Hadoop.

• Optimize performance and scalability of big data solutions for complex data sets.

• Develop and integrate solutions with AWS services for cloud-based data processing.

3. Technical Expertise:

• Leverage expertise in PySpark, Hadoop, and Scala to solve complex technical

challenges.

• Work with Teradata for large-scale data warehousing solutions.

• Utilize data visualization tools (e.g., Tableau, Power BI) to provide insights and

reporting solutions.

4. Stakeholder Collaboration:

• Collaborate with cross-functional teams, including business stakeholders and product

managers, to understand requirements and deliver solutions.

• Provide thought leadership on best practices in big data technologies and architecture.

Required Skills & Qualifications:

• Technical Expertise:

• Proficiency in PySpark, Hadoop, AWS (S3, Glue, EMR, Redshift), and Teradata.

• Hands-on experience with Scala for data engineering tasks.

• Familiarity with data visualization tools like Tableau, Power BI, or similar platforms.

• Experience:

• 7–13 years of experience in data engineering, with a focus on big data technologies.

• Proven track record of managing end-to-end data engineering projects and leading

teams.

• Educational Background:

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a

related field.

• Soft Skills:

• Strong analytical and problem-solving skills.

• Excellent communication and stakeholder management abilities.

• Ability to work in a fast-paced, dynamic environment.

Preferred Skills:

• Certification in AWS or related big data tools.

• Experience in real-time data streaming platforms like Kafka.

• Knowledge of DevOps practices, including CI/CD pipelines and infrastructure as code.

Requirements

Talent Worx is a growing services & recruitment consulting firm, we are hiring for our client which is a Global and leading provider of financial intelligence, data analytics, and AI-driven solutions, empowering businesses worldwide with insights for confident decision-making. Join to work on cutting-edge technologies, drive digital transformation, and shape the future of global markets.

  • Pyspark Developer

    6 hours ago

    Bengaluru, Chennai, Pune, India VHR Solutions Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    Job Description:We are looking for a skilled PySpark Developer to join our data engineering team. The ideal candidate will have strong experience in building scalable data pipelines using Apache Spark (PySpark), integrating with big data platforms, and working with large datasets in distributed environments. You will work closely with data engineers,...

  • Pyspark Developer

    2 weeks ago

    Bengaluru, Chennai, Kolkata, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Description:5+ Years of experience with Developing/Fine tuning and implementing programs/applicationsUsing Python/PySpark/Scala on Big Data/Hadoop Platform.Roles and Responsibilities:a) Work with a Leading Banks Risk Management team on specific projects/requirements pertaining to risk Models inconsumer and wholesale bankingb) Enhance Machine Learning...

  • PySpark Developer

    7 days ago

    Bengaluru, India NPG Consultants Full time

    We are seeking a highly skilled PySpark Developer with 7+ years of experience and expertise in Reltio MDM to join our data engineering team. You will design and implement scalable data processing solutions, integrate enterprise systems with Reltio, and ensure high-quality data governance.Key Responsibilities :- Develop and maintain PySpark data pipelines on...

  • Data Engineer – Databricks

    7 days ago

    Bengaluru, India Capgemini Engineering Full time

    Data Engineer – Databricks & PySpark Location: Bangalore Experience: 6–9 years Choosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock...

  • Data Engineer – Databricks

    5 days ago

    Bengaluru, India Capgemini Engineering Full time

    Data Engineer – Databricks & PySpark Location: Bangalore Experience: 6–9 years Choosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock...

  • Data Engineer – Databricks

    5 days ago

    Bengaluru, India Capgemini Engineering Full time

    Data Engineer – Databricks & PySpark Location: Bangalore Experience: 6–9 years Choosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock...

  • Data Engineer – Databricks

    3 days ago

    Bengaluru, India Capgemini Engineering Full time

    Data Engineer – Databricks & PySpark Location: Bangalore Experience: 6–9 years Choosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock...

  • Data Engineer – Databricks

    7 days ago

    Bengaluru, India Capgemini Engineering Full time

    Data Engineer – Databricks & PySparkLocation: BangaloreExperience: 6–9 yearsChoosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock the...

  • Data Engineer – Databricks

    5 days ago

    Bengaluru, India Capgemini Engineering Full time

    Data Engineer – Databricks & PySparkLocation: BangaloreExperience: 6–9 yearsChoosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock the...

  • Data Engineer – Databricks

    5 days ago

    Bengaluru, India Capgemini Engineering Full time

    Data Engineer – Databricks & PySparkLocation: BangaloreExperience: 6–9 yearsChoosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock the...