Pyspark Architect

2 days ago

Kolkata, West Bengal, India Talent Worx Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Job Description: PySpark Architect

Location: Bangalore,Pune,Kolkata

Experience Required: 7 to 13 Years

Role Overview:

We are seeking a highly skilled and motivated PySpark Manager to lead our data

engineering initiatives. The ideal candidate will have deep expertise in PySpark,

Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be responsible for

designing, building, and managing large-scale data processing solutions while leading a

team of data engineers to deliver high-quality outcomes.

Key Responsibilities:

  1. Leadership & Team Management:


• Lead and mentor a team of data engineers, ensuring high performance and

professional growth.


• Oversee project timelines, quality, and deliverables for multiple data initiatives.

  1. Data Engineering & Development:


• Design and implement robust data pipelines and workflows using PySpark and

Hadoop.


• Optimize performance and scalability of big data solutions for complex data sets.


• Develop and integrate solutions with AWS services for cloud-based data processing.

  1. Technical Expertise:


• Leverage expertise in PySpark, Hadoop, and Scala to solve complex technical

challenges.


• Work with Teradata for large-scale data warehousing solutions.


• Utilize data visualization tools (e.g., Tableau, Power BI) to provide insights and

reporting solutions.

  1. Stakeholder Collaboration:


• Collaborate with cross-functional teams, including business stakeholders and product

managers, to understand requirements and deliver solutions.


• Provide thought leadership on best practices in big data technologies and architecture.

Required Skills & Qualifications:


• Technical Expertise:


• Proficiency in PySpark, Hadoop, AWS (S3, Glue, EMR, Redshift), and Teradata.


• Hands-on experience with Scala for data engineering tasks.


• Familiarity with data visualization tools like Tableau, Power BI, or similar platforms.


• Experience:


• 7–13 years of experience in data engineering, with a focus on big data technologies.


• Proven track record of managing end-to-end data engineering projects and leading

teams.


• Educational Background:


• Bachelor's or Master's degree in Computer Science, Information Technology, or a

related field.


• Soft Skills:


• Strong analytical and problem-solving skills.


• Excellent communication and stakeholder management abilities.


• Ability to work in a fast-paced, dynamic environment.

Preferred Skills:


• Certification in AWS or related big data tools.


• Experience in real-time data streaming platforms like Kafka.


• Knowledge of DevOps practices, including CI/CD pipelines and infrastructure as code.

Requirements

Talent Worx is a growing services & recruitment consulting firm, we are hiring for our client which is a Global and leading provider of financial intelligence, data analytics, and AI-driven solutions, empowering businesses worldwide with insights for confident decision-making. Join to work on cutting-edge technologies, drive digital transformation, and shape the future of global markets.

  • AWS Glue

    1 week ago

    Kolkata, West Bengal, India Cognizant Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Skill: AWS GlueExperience: 6 to 9 yearsLocation: KolkataJob descriptionTechnical Skills :AWS Glue: 3+ years of hands-on experience in AWS Glue ETL developmentPython/PySpark: Strong programming skills in Python and PySpark for data transformationAWS Services: Proficiency in S3, Redshift, Athena, Lambda, and EMRData Formats: Experience with Parquet, Avro,...

  • AWS Data Engineering

    4 days ago

    Kolkata, West Bengal, India Careernet Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Key Skills: AWS, Data EngineerRoles and Responsibilities:Architect and develop data pipelines using AWS services.Design and optimize relational and non-relational databases.Implement ETL/ELT processes using AWS Glue, Lambda, and other tools.Manage data lakes and warehouses (e.g., S3, Redshift, Athena).Collaborate with data scientists, analysts, and business...

  • Snowflake Data Engineer

    1 week ago

    Kolkata, West Bengal, India Vidpro Consultancy Services Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Exp: YrsWork Mode: HybridLocation: Bangalore, Chennai, Kolkata, Pune and GurgaonPrimary Skills: Python, pyspark, Azure Data Factory, snowflake, snowpipe, snowsql, Snowsight, Snowpark, ETL, SQL, and Architect Designing.Snowpro certified is plusPrimary Roles and Responsibilities:Developing Modern Data Warehouse solutions using Snowflake, Databricks and...

  • Data Engineer

    7 days ago

    Kolkata, West Bengal, India, West Bengal Tata Consultancy Services Full time

    TCS has been a great pioneer in feeding the fire of young techies like you. We are global leaders in the technology arena and there's nothing that can stop us from growing together. TCS Hiring for skill "Data Engineer". Role: Databricks Senior Developer Required Technical Skill Set: Databricks, Spark and Data Migration experience Experience: 8+yearsWork...

  • Databricks Senior Developer

    7 days ago

    Kolkata, West Bengal, India, West Bengal Tata Consultancy Services Full time

    Role: Databricks Senior DeveloperLocation: KolkataExperience: 8 Plus YearsRole**Databricks Senior DeveloperRequired Technical Skill Set**Databricks, Spark and Data Migration experienceDesired Competencies (Technical/Behavioral Competency)Must-Have**1. 8+ years with strong Databricks, Spark and Data Migration experience2. Should have experience of end-to-end...

  • Pyspark Architect

    3 weeks ago

    Kolkata, India Talent Worx Full time

    Job Description: PySpark Architect Location: Bangalore,Pune,Kolkata Experience Required: 7 to 13 Years Role Overview: We are seeking a highly skilled and motivated PySpark Manager to lead our data engineering initiatives. The ideal candidate will have deep expertise in PySpark, Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be...

  • Snowflake Architect(Python

    1 week ago

    Hyderabad, Kolkata, Pune, India Alike Thoughts Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Mandatory Skills: Snowflake Architect, Python, PysparkExperience: 8 to 15 yrsLocation: PAN IndiaNotice Period: 60 MaxInterested candidates Please apply.

  • Data Architect

    1 week ago

    Kolkata, West Bengal, India Lexmark International, Inc. Full time

    Responsibilities: About Lexmark: Founded in 1991 and headquartered in Lexington, Kentucky, Lexmark is recognized as a global leader in print hardware, service, software solutions and security by many of the technology industry’s leading market analyst firms. Lexmark creates cloud-enabled imaging and IoT solutions that help customers in more than 170...

  • Developer

    2 days ago

    Bengaluru, Kolkata, Pune, India Artech Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: DeveloperWork Location: Pune -MH, KolkataSkill Required: Digital : Python~Digital : Databricks~Digital : PySpark~Azure Data Factory~MySQLExperience Range : 6-8 YearsJob Description:A minimum of 5 years experience with large SQL data marts. Expert relational database experience, Candidate should demonstrate ability to navigate through massive...

  • Principal Consultant

    3 weeks ago

    Kolkata, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people – we...

  • Principal Consultant

    3 weeks ago

    Kolkata, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people – we...

  • Aws Glue

    1 week ago

    Kolkata, West Bengal, India Cognizant Full time

    **Skill**: AWS Glue **Experience**: 6 to 9 years **Location**: Kolkata **Job description** **Technical Skills**: - AWS Glue: 3+ years of hands-on experience in AWS Glue ETL development - Python/PySpark: Strong programming skills in Python and PySpark for data transformation - AWS Services: Proficiency in S3, Redshift, Athena, Lambda, and EMR - Data...

  • Databricks Senior Developer

    5 days ago

    Kolkata, India Tata Consultancy Services Full time

    Role: Databricks Senior DeveloperLocation: KolkataExperience: 8 Plus YearsRole**Databricks Senior DeveloperRequired Technical Skill Set**Databricks, Spark and Data Migration experienceDesired Competencies (Technical/Behavioral Competency)Must-Have**1. 8+ years with strong Databricks, Spark and Data Migration experience2. Should have experience of end-to-end...

  • Databricks Senior Developer

    6 days ago

    Kolkata, India Tata Consultancy Services Full time

    Role: Databricks Senior DeveloperLocation: KolkataExperience: 8 Plus YearsRole**Databricks Senior DeveloperRequired Technical Skill Set**Databricks, Spark and Data Migration experienceDesired Competencies (Technical/Behavioral Competency)Must-Have**1. 8+ years with strong Databricks, Spark and Data Migration experience2. Should have experience of end-to-end...

  • Databricks Senior Developer

    6 days ago

    Kolkata, India Tata Consultancy Services Full time

    Role: Databricks Senior DeveloperLocation: KolkataExperience: 8 Plus YearsRole**Databricks Senior DeveloperRequired Technical Skill Set**Databricks, Spark and Data Migration experienceDesired Competencies (Technical/Behavioral Competency)Must-Have**1. 8+ years with strong Databricks, Spark and Data Migration experience2. Should have experience of end-to-end...