Pyspark

1 day ago

Bengaluru Karnataka, India Cognizant Full time

**Job Summary**

**Responsibilities**
- Lead the design and implementation of data processing pipelines using Databricks PySpark and Python
- Oversee the development and maintenance of scalable data solutions to support business needs
- Provide technical guidance and mentorship to team members to ensure high-quality deliverables
- Collaborate with cross-functional teams to understand data requirements and translate them into technical solutions
- Ensure the performance quality and responsiveness of data processing systems
- Identify and resolve bottlenecks and bugs in data processing workflows
- Conduct code reviews and enforce best practices for coding and data management
- Develop and maintain documentation for data processing systems and workflows
- Stay updated with the latest industry trends and technologies to drive innovation
- Participate in project planning and provide estimates for development tasks
- Communicate effectively with stakeholders to provide updates on project progress and challenges
- Ensure compliance with data security and privacy regulations
- Contribute to the continuous improvement of data processing practices and tools

**Qualifications**
- Demonstrate expertise in Databricks PySpark and Python with a proven track record of successful projects
- Possess strong problem-solving skills and the ability to troubleshoot complex data issues
- Show experience in leading and mentoring technical teams
- Exhibit excellent communication and collaboration skills
- Have a background in Cheminformatics or Bioinformatics is a nice to have
- Display knowledge of data security and privacy best practices
- Demonstrate the ability to work in a hybrid work model effectively
- Show a commitment to continuous learning and professional development
- Possess a strong understanding of data processing and ETL workflows
- Have experience with agile development methodologies
- Exhibit the ability to manage multiple tasks and projects simultaneously
- Show attention to detail and a commitment to delivering high-quality work
- Demonstrate the ability to work independently and as part of a team.

  • Pyspark

    1 week ago

    Bengaluru, Karnataka, India Hero Moto Corp Full time

    » **Date**:12 Nov 2024 **Location**: Bengaluru, KA, IN, 560038 **Company**:Hero Motocorp **Function** - Digital & Information Technologies**Pay Band** - E4 to M2**Role** - PySpark, Glue or Databricks developer proficient in cloud technologies who can become a member of HMCL Digital Connected core tech team. The role demands amazing ETL coding skills...

  • Python & Pyspark

    1 week ago

    Karnataka, India Teradata Full time

    Teradata’s Global Delivery Center (GDC) brings together data enthusiasts from all corners of the world to support our customers across the globe. We analyze their business needs and address them with the most sophisticated data solutions including Data Warehouses, Data Lakes, Operational Data Stores and much more. We deliver and manage complex hybrid...

  • Pyspark Developer

    2 weeks ago

    Bengaluru, Karnataka, India Synechron Technologies Pvt. Ltd. Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Day-to-Day Activities:Design, develop, and maintain ETL pipelines using PySpark on CDP.Implement and manage data ingestion processes from various sources. Process, cleanse, and transform large datasets using PySpark. Conduct performance tuning and optimization of ETL processes. Implement data quality checks and validation routines. Automate data workflows...

  • Pyspark Lead

    20 hours ago

    Bengaluru, Karnataka, India Krazy Mantra HR Solutions Pvt. Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for a skilled PySpark Lead with 5 to 8 years of experience, based in Bangalore. The ideal candidate will have expertise in PySpark, SQL, ETL/Data Engineering Pipeline, Cloud understanding, CI/CD, Data Modelling, Airflow, Kafka, EMR/Databricks, Snowflake, and Hive.Roles and ResponsibilityDesign and develop scalable data pipelines using PySpark...

  • Etl (Talend/pyspark)

    3 days ago

    Bengaluru, India DEzen Technology Solutions Pvt Ltd. Full time

    **Recruitment Company Name**: - DEzen Technology Solutions Pvt Ltd. **Requirements**: - 4 -7 Years - ETL (Talend/PySpark) **Skills**: - ETL (Talend/PySpark) - ETL Developer Talend / Pyspark Upload Resume **Job Details**: - **Role**: ETL (Talend/PySpark) - **Job Type**: Full Time - **Location**: Bangalore - **Offered Salary**: open - **Posted**:...

  • Pyspark Developer

    5 days ago

    Bengaluru, India Sigma Allied Services Full time

    Position - Pyspark Developer Location - Bangalore Experience - 6 to 9 yrs Required Skills -Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : PySparkMinimum 5 year(s) of experience is requiredEducational Qualification : 15 years full time educationSummary:As an...

  • Pyspark Architect

    3 days ago

    Bengaluru, Karnataka, India Talent Worx Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Job Description: PySpark ArchitectLocation: Bangalore,Pune,Kolkata Experience Required: 7 to 13 YearsRole Overview:We are seeking a highly skilled and motivated PySpark Manager to lead our dataengineering initiatives. The ideal candidate will have deep expertise in PySpark,Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be responsible...

  • Pyspark Developer

    5 days ago

    Bengaluru, India Nuvento Systems Full time

    We are looking for immediate joiner | Work from Home Key skills: Pyspark, Apache Spark, AWS, Databricks, Scala, Java

  • Python Pyspark

    2 weeks ago

    Bengaluru, Karnataka, India Virtusa Full time

    7 Years overall IT experience with minimum 4 years of work experience in below tech skills Tech Skills Proficient in Python scripting and PySpark for data processing task Strong SQL capabilities with hands on experience managing big data using ETL tools like Informatica Experience with the AWS cloud platform and its data services including S3 Redshift Lambda...

  • PySpark Developer

    1 week ago

    Hyderabad, Bengaluru, Chennai, India Coders Brain Technology Private Limited Full time

    Job Description ROLE RESPONSIBILITIES Data Engineering and Processing: Develop and manage data pipelines using PySpark on Databricks. Implement ETL/ELT processes to process structured and unstructured data at scale. Optimize data pipelines for performance, scalability, and cost-efficiency in Databricks. Databricks Platform Expertise: Experience in...