Pyspark Architect

3 days ago

Bengaluru, Karnataka, India Talent Worx Full time ₹ 1,04,000 - ₹ 1,30,878 per year

Job Description: PySpark Architect

Location: Bangalore,Pune,Kolkata

Experience Required: 7 to 13 Years

Role Overview:

We are seeking a highly skilled and motivated PySpark Manager to lead our data

engineering initiatives. The ideal candidate will have deep expertise in PySpark,

Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be responsible for

designing, building, and managing large-scale data processing solutions while leading a

team of data engineers to deliver high-quality outcomes.

Key Responsibilities:

1. Leadership & Team Management:


• Lead and mentor a team of data engineers, ensuring high performance and

professional growth.


• Oversee project timelines, quality, and deliverables for multiple data initiatives.

2. Data Engineering & Development:


• Design and implement robust data pipelines and workflows using PySpark and

Hadoop.


• Optimize performance and scalability of big data solutions for complex data sets.


• Develop and integrate solutions with AWS services for cloud-based data processing.

3. Technical Expertise:


• Leverage expertise in PySpark, Hadoop, and Scala to solve complex technical

challenges.


• Work with Teradata for large-scale data warehousing solutions.


• Utilize data visualization tools (e.g., Tableau, Power BI) to provide insights and

reporting solutions.

4. Stakeholder Collaboration:


• Collaborate with cross-functional teams, including business stakeholders and product

managers, to understand requirements and deliver solutions.


• Provide thought leadership on best practices in big data technologies and architecture.

Required Skills & Qualifications:


• Technical Expertise:


• Proficiency in PySpark, Hadoop, AWS (S3, Glue, EMR, Redshift), and Teradata.


• Hands-on experience with Scala for data engineering tasks.


• Familiarity with data visualization tools like Tableau, Power BI, or similar platforms.


• Experience:


• 7–13 years of experience in data engineering, with a focus on big data technologies.


• Proven track record of managing end-to-end data engineering projects and leading

teams.


• Educational Background:


• Bachelor's or Master's degree in Computer Science, Information Technology, or a

related field.


• Soft Skills:


• Strong analytical and problem-solving skills.


• Excellent communication and stakeholder management abilities.


• Ability to work in a fast-paced, dynamic environment.

Preferred Skills:


• Certification in AWS or related big data tools.


• Experience in real-time data streaming platforms like Kafka.


• Knowledge of DevOps practices, including CI/CD pipelines and infrastructure as code.

Requirements

Talent Worx is a growing services & recruitment consulting firm, we are hiring for our client which is a Global and leading provider of financial intelligence, data analytics, and AI-driven solutions, empowering businesses worldwide with insights for confident decision-making. Join to work on cutting-edge technologies, drive digital transformation, and shape the future of global markets.

  • Software Engineer, PySPark

    12 hours ago

    Bengaluru, Karnataka, India NatWest Group Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Join us as a Software Engineer, PySparkThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWe're offering this role at senior analyst levelWhat you'll...

  • Software Engineer, PySPark

    2 weeks ago

    Bengaluru, Karnataka, India RBS Full time ₹ 1,50,000 - ₹ 28,00,000 per year

    Join us as a Software Engineer, PySparkThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWe're offering this role at senior analyst levelWhat you'll...

  • Software Engineer, PySpark, AVP

    3 days ago

    Bengaluru, Karnataka, India RBS Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Join us as a Software Engineer, PySparkThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWe're offering this role at associate vice president levelWhat...

  • Data Architect

    2 weeks ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time US$ 1,50,000 - US$ 2,00,000

    Job Title: Data EngineerRole Summary:We are seeking an experienced Data Architect to join our data engineering team. This is a challenging role that requires a strong background in big data ecosystems, cloud-native tools, and advanced data processing techniques.The ideal candidate has hands-on experience with data ingestion, transformation, and optimization...

  • Senior Data Architect

    1 week ago

    Bengaluru, Karnataka, India beBeeDataArchitect Full time ₹ 20,00,000 - ₹ 25,00,000

    Key ResponsibilitiesWe are seeking a seasoned data architect to design and implement large-scale data systems for feature extraction, transformation, and loading (ETL). You will be responsible for governing and optimizing these systems to ensure high reliability, efficiency, and data quality across on-premise and cloud environments.The ideal candidate will...

  • Cloud Data Architect

    2 weeks ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Cloud Data ArchitectRole DescriptionWe are seeking a highly skilled Cloud Data Architect to design and implement scalable data solutions in the cloud. As a key member of our team, you will drive data-driven decision-making by building robust ETL pipelines using PySpark and AWS Glue.Your primary responsibility will be to collaborate with analysts...

  • Sr. Data Architect

    3 weeks ago

    Bengaluru, Karnataka, India Emids Full time

    We at EMIDS , are hiring for Sr. Data architect role. Please find the details below and share your interest at Exp : 12+ yrs Location : Noida / Bangalore / Hyderabad (Hybrid) Skills : Data Engineering, Azure, ETL, Pyspark, Databricks, Job Summary: We are seeking an experienced and highly skilled Senior Data Architect to lead the design,...

  • Sr. Data Architect

    2 weeks ago

    Bengaluru, Karnataka, India Emids Full time

    We at EMIDS, are hiring for Sr. Data architect role. Please find the details below and share your interest at aarati.pardhi@emids.comExp : 12+ yrsLocation : Noida / Bangalore / Hyderabad (Hybrid)Skills : Data Engineering, Azure, ETL, Pyspark, Databricks, Job Summary:We are seeking an experienced and highly skilled Senior Data Architect to lead the design,...

  • Sr. Data Architect

    2 weeks ago

    Bengaluru, Karnataka, India Emids Full time

    We at EMIDS, are hiring for Sr. Data architect role. Please find the details below and share your interest at aarati.pardhi@emids.comExp : 12+ yrsLocation : Noida / Bangalore / Hyderabad (Hybrid)Skills : Data Engineering, Azure, ETL, Pyspark, Databricks, Job Summary:We are seeking an experienced and highly skilled Senior Data Architect to lead the design,...

  • Azure Data Architect

    3 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Tata Consultancy Service Hiring for Azure Data ArchitectExperience: 10 to 11 YearsLocation: Pune / Indore / Bengaluru / ChennaiSkill: AWS Data (Pyspark / Redshift / Glue)Job Description:Strong hands-on experience in Python programming and PySpark, Experience using AWS services (RedShift, Glue, EMR, S3 & Lambda) Experience working with Apache Spark and Hadoop...