Pyspark Architect

1 week ago

Pune, Maharashtra, India Talent Worx Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Description: PySpark Architect

Location: Bangalore,Pune,Kolkata

Experience Required: 7 to 13 Years

Role Overview:

We are seeking a highly skilled and motivated PySpark Manager to lead our data

engineering initiatives. The ideal candidate will have deep expertise in PySpark,

Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be responsible for

designing, building, and managing large-scale data processing solutions while leading a

team of data engineers to deliver high-quality outcomes.

Key Responsibilities:

1. Leadership & Team Management:


• Lead and mentor a team of data engineers, ensuring high performance and

professional growth.


• Oversee project timelines, quality, and deliverables for multiple data initiatives.

2. Data Engineering & Development:


• Design and implement robust data pipelines and workflows using PySpark and

Hadoop.


• Optimize performance and scalability of big data solutions for complex data sets.


• Develop and integrate solutions with AWS services for cloud-based data processing.

3. Technical Expertise:


• Leverage expertise in PySpark, Hadoop, and Scala to solve complex technical

challenges.


• Work with Teradata for large-scale data warehousing solutions.


• Utilize data visualization tools (e.g., Tableau, Power BI) to provide insights and

reporting solutions.

4. Stakeholder Collaboration:


• Collaborate with cross-functional teams, including business stakeholders and product

managers, to understand requirements and deliver solutions.


• Provide thought leadership on best practices in big data technologies and architecture.

Required Skills & Qualifications:


• Technical Expertise:


• Proficiency in PySpark, Hadoop, AWS (S3, Glue, EMR, Redshift), and Teradata.


• Hands-on experience with Scala for data engineering tasks.


• Familiarity with data visualization tools like Tableau, Power BI, or similar platforms.


• Experience:


• 7–13 years of experience in data engineering, with a focus on big data technologies.


• Proven track record of managing end-to-end data engineering projects and leading

teams.


• Educational Background:


• Bachelor's or Master's degree in Computer Science, Information Technology, or a

related field.


• Soft Skills:


• Strong analytical and problem-solving skills.


• Excellent communication and stakeholder management abilities.


• Ability to work in a fast-paced, dynamic environment.

Preferred Skills:


• Certification in AWS or related big data tools.


• Experience in real-time data streaming platforms like Kafka.


• Knowledge of DevOps practices, including CI/CD pipelines and infrastructure as code.

Requirements

Talent Worx is a growing services & recruitment consulting firm, we are hiring for our client which is a Global and leading provider of financial intelligence, data analytics, and AI-driven solutions, empowering businesses worldwide with insights for confident decision-making. Join to work on cutting-edge technologies, drive digital transformation, and shape the future of global markets.

  • pyspark developer

    1 week ago

    Pune, Maharashtra, India Capgemini Engineering Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world's most innovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and...

  • pyspark developer

    1 week ago

    Pune, Maharashtra, India Capgemini Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world's most innovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and...

  • Architect

    6 days ago

    Pune, Maharashtra, India Infosys Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Educational RequirementsMaster Of Science,Master Of Comp. Applications,Master Of Technology,Bachelor Of Computer Science,Bachelor Of Technology,Bachelor of EngineeringService LineData & Analytics UnitResponsibilitiesHigh degree of proven competency in design and delivering End to End (Data Ingestion to Consumption) enterprise scale BI and Analytics...

  • Architect - Positions

    6 days ago

    Pune, Maharashtra, India Infosys Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Description:In the role of Technology Architect Senior Technology Architect you will interface with key stakeholders and apply your technical proficiency across different stages of the Software Development Life Cycle including Requirements Elicitation Application Architecture definition and DesignYou will play an important role in creating the...

  • MarsDevs - AI/ML Engineer - ETL/PySpark

    6 days ago

    Pune, Maharashtra, India Marsdevs Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Company Description : MarsDevs is a remote-first, offshore digital product development company, committed to delivering high-quality code and long-lasting technical architectures. We specialize in building MVPs, mobile and web applications, and SaaS products for startups and SMBs. Our services include systematic project analysis, UI/UX design,...

  • MS Fabric Architect

    7 days ago

    Pune, Maharashtra, India Persistent Systems Full time ₹ 12,00,000 - ₹ 25,00,000 per year

    About Position:We are hiring for MS Fabric Architect with hands on experience on Architect, MS Fabric, SQL, Azure, Python, PysparkRole: MS Fabric ArchitectLocation: Pune, BangaloreExperience: 15 to 18 YearsJob Type: Full Time EmploymentWhat You'll Do:15+ years of total experience Should be Hands-on delivery person to lead the design and architecture of the...

  • Technical Architect-Databricks

    4 hours ago

    Pune, Maharashtra, India Atos Full time ₹ 8,00,000 - ₹ 20,00,000 per year

    Role Overview:The Technical Architect - Databricks designs and implements scalable data architectures and solutions. The jobholder has expertise in Databricks Lakehouse, data modeling, and cloud integration, ensuring high performance, security, and reliability.Responsibilities:Design and implement Databricks-based data architectures to meet business...

  • D-TechWorks - Associate Technical Architect - Azure Data Services

    7 days ago

    Pune, Maharashtra, India D-Techworks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Title :Associate Technical Architect - AzureLocation :PuneRole Type :Full-time, PermanentExperience Required :8-10 yearsNotice Period :Immediate Joiners or 15 days PreferredJob DescriptionWe are seeking a highly skilled and experienced Associate Technical Architect to lead the design and implementation of scalable data solutions on Microsoft Azure. The...

  • Data Engineer/Data Architect

    2 weeks ago

    Pune, Maharashtra, India Inorg Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Summary:We're seeking an experienced Data Architect / Senior Data Engineer with 5-12 years ofexpertise in data engineering across major data platforms. The ideal candidate will have astrong background in Python, SQL, ETL, and data modeling, with experience in tools likeTeradata, Informatica, Hadoop, Spark, PySpark, ADF, Snowflake, and Big Data....

  • Big Data Architect

    6 days ago

    Pune, Maharashtra, India Datametica Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: Big Data Architect GCPExperience: 10–12 YearsLocation: PuneRoleWe are looking for a Big Data Architect with deep GCP expertise to define, design, and lead cloud-native data platforms for enterprise clients. You will be responsible for architecting large-scale data lakes, warehouses, and pipelines on GCP, modernizing on-premise workloads, and...