Pyspark Architect

3 days ago

Bengaluru, Karnataka, India Talent Worx Full time ₹ 15,00,000 - ₹ 20,00,000 per year

Job Description: PySpark Architect

Location: Bangalore,Pune,Kolkata

Experience Required: 7 to 13 Years

Role Overview:

We are seeking a highly skilled and motivated PySpark Manager to lead our data

engineering initiatives. The ideal candidate will have deep expertise in PySpark,

Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be responsible for

designing, building, and managing large-scale data processing solutions while leading a

team of data engineers to deliver high-quality outcomes.

Key Responsibilities:

1. Leadership & Team Management:


• Lead and mentor a team of data engineers, ensuring high performance and

professional growth.


• Oversee project timelines, quality, and deliverables for multiple data initiatives.

2. Data Engineering & Development:


• Design and implement robust data pipelines and workflows using PySpark and

Hadoop.


• Optimize performance and scalability of big data solutions for complex data sets.


• Develop and integrate solutions with AWS services for cloud-based data processing.

3. Technical Expertise:


• Leverage expertise in PySpark, Hadoop, and Scala to solve complex technical

challenges.


• Work with Teradata for large-scale data warehousing solutions.


• Utilize data visualization tools (e.g., Tableau, Power BI) to provide insights and

reporting solutions.

4. Stakeholder Collaboration:


• Collaborate with cross-functional teams, including business stakeholders and product

managers, to understand requirements and deliver solutions.


• Provide thought leadership on best practices in big data technologies and architecture.

Required Skills & Qualifications:


• Technical Expertise:


• Proficiency in PySpark, Hadoop, AWS (S3, Glue, EMR, Redshift), and Teradata.


• Hands-on experience with Scala for data engineering tasks.


• Familiarity with data visualization tools like Tableau, Power BI, or similar platforms.


• Experience:


• 7–13 years of experience in data engineering, with a focus on big data technologies.


• Proven track record of managing end-to-end data engineering projects and leading

teams.


• Educational Background:


• Bachelor's or Master's degree in Computer Science, Information Technology, or a

related field.


• Soft Skills:


• Strong analytical and problem-solving skills.


• Excellent communication and stakeholder management abilities.


• Ability to work in a fast-paced, dynamic environment.

Preferred Skills:


• Certification in AWS or related big data tools.


• Experience in real-time data streaming platforms like Kafka.


• Knowledge of DevOps practices, including CI/CD pipelines and infrastructure as code.

Requirements

Talent Worx is a growing services & recruitment consulting firm, we are hiring for our client which is a Global and leading provider of financial intelligence, data analytics, and AI-driven solutions, empowering businesses worldwide with insights for confident decision-making. Join to work on cutting-edge technologies, drive digital transformation, and shape the future of global markets.

  • Software Engineer, PySpark, AVP

    3 days ago

    Bengaluru, Karnataka, India RBS Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Join us as a Software Engineer, PySparkThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWe're offering this role at associate vice president levelWhat...

  • Urgent) Azure, Spark,PySpark,Scala

    3 weeks ago

    Bengaluru, Karnataka, India Infosys Limited Full time

    Job DescriptionJob Description:- Azure Spark PySpark Scala- Azure Synapse- Azure knowledge mustKey Responsibilities:- Engage with Business Stakeholders with status update on progress of development and issue fix- Setup Standards related to Coding Pipelines and Documentation- Understand the requirements from the business and translate it into an appropriate...

  • Data Architect

    2 days ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Job TitleA Data Architect will lead the design and development of scalable data solutions.Key Responsibilities:Collaborate with cross-functional teams to drive project successDesign and develop robust data pipelines for data extraction, transformation, and loadingImplement data quality and integrity processes using data validation and cleansing...

  • Databricks architect

    2 weeks ago

    Bengaluru, Karnataka, India Teamware Solutions Full time

    Job DescriptionWe are hiring aDatabricks architect- Location: Bangalore- Work Model: Hybrid- Years of Experience: 7-10 years- Notice Period: Immediate to 15 Days- Open positions: 10Additional Information:- Mandatory Skills Databricks architecture, framework built- Nice to have skills Python, Pyspark- Interview Mode Virtual Interview

  • Sr. Data Architect

    4 weeks ago

    Bengaluru, Karnataka, India Emids Full time

    We at EMIDS , are hiring for Sr. Data architect role. Please find the details below and share your interest at Exp : 12+ yrs Location : Noida / Bangalore / Hyderabad (Hybrid) Skills : Data Engineering, Azure, ETL, Pyspark, Databricks, Job Summary: We are seeking an experienced and highly skilled Senior Data Architect to lead the design,...

  • Sr. Data Architect

    3 days ago

    Bengaluru, Karnataka, India Emids Full time

    We at EMIDS , are hiring for Sr. Data architect role. Please find the details below and share your interest at Exp : 12+ yrs Location : Noida / Bangalore / Hyderabad (Hybrid) Skills : Data Engineering, Azure, ETL, Pyspark, Databricks, Job Summary: We are seeking an experienced and highly skilled Senior Data Architect to lead the design,...

  • Senior Data Architect

    4 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Senior Data ArchitectWe are seeking a highly skilled and experienced Data Architect to lead the design and implementation of scalable data pipelines for feature extraction, transformation, and loading (ETL). This role requires strong expertise in Pyspark and Scala, experience with big data technologies, and proven leadership skills to guide a team...

  • Data Architect

    5 hours ago

    Bengaluru, Karnataka, India Bibha Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Job Summary:We are seeking an experienced and highly skilled Senior Data Architect to lead the design, implementation, and optimization of our modern data architecture. The ideal candidate will bring deep technical expertise in data engineering, cloud platforms (preferably Azure), and big data tools such as Databricks and PySpark, along with a strong...

  • Azure Data Architect

    4 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Tata Consultancy Service Hiring for Azure Data ArchitectExperience: 10 to 11 YearsLocation: Pune / Indore / Bengaluru / ChennaiSkill: AWS Data (Pyspark / Redshift / Glue)Job Description:- Strong hands-on experience in Python programming and PySpark, Experience using AWS services (RedShift, Glue, EMR, S3 & Lambda)- Experience working with Apache Spark and...

  • Azure Data Architect

    2 days ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Tata Consultancy Service Hiring for Azure Data ArchitectExperience: 10 to 11 YearsLocation: Pune / Indore / Bengaluru / ChennaiSkill: AWS Data (Pyspark / Redshift / Glue)Job Description:Strong hands-on experience in Python programming and PySpark, Experience using AWS services (RedShift, Glue, EMR, S3 & Lambda) Experience working with Apache Spark and Hadoop...