Databricks Pyspark

3 weeks ago


Hyderabad, India Fusion Plus Solutions Full time

Job Description - Develop and optimize data processing jobs using PySpark to handle complex data transformations and aggregations efficiently. - Design and implement robust data pipelines on the AWS platform, ensuring scalability and efficiency. - Leverage AWS services such as EC2, S3 for comprehensive data processing and storage solutions. - Manage SQL database schema design, query optimization, and performance tuning to support data transformation and loading processes. - Design and maintain scalable and performant data warehouses, employing best practices in data modeling and ETL processes. - Utilize modern data platforms for collaborative data science, integrating seamlessly with various data sources and types. - Ensure high data quality and accessibility by maintaining optimal performance of Databricks clusters and Spark jobs. - Develop and implement security measures, backup procedures, and disaster recovery plans using AWS best practices. - Manage source code and automate deployment using GitHub along with CI/CD practices tailored for data operations in cloud environments. - Provide expertise in troubleshooting and optimizing PySpark scripts, Databricks notebooks, SQL queries, and Airflow DAGs. - Stay updated on the latest developments in cloud data technologies and recommend adoption of new tools and practices. - Use Apache Airflow to orchestrate and automate data workflows, ensuring timely and reliable execution of data jobs. - Collaborate with data scientists and business analysts to design data models and pipelines that support advanced analytics and machine learning projects.


  • Pyspark + Databricks

    2 weeks ago


    Hyderabad, Telangana, India Cognizant Full time

    **Job Summary** **Responsibilities** - Develop and maintain scalable data pipelines using Databricks SQL and Databricks Workflows. - Implement and optimize PySpark code for data processing and analytics. - Collaborate with cross-functional teams to understand data requirements and deliver solutions. - Ensure data quality and integrity through rigorous...


  • Hyderabad, India Fusion Plus Solutions Full time

    Job Description Roles and Responsibilities - 5+ years of experience on IT industry in Data Engineering & Data Analyst role. - 5 years of development experience using tool Databricks and PySpark, Python, SQL - Proficient in writing SQL queries including writing of windows functions - Good communication skills with analytical abilities in doing problem solving...

  • ML Engineer

    2 days ago


    Hyderabad, India Tiger Analytics Full time

    Job Description ML Engineer (Databricks + PySpark) Locations: Chennai / Hyderabad / Bangalore Tiger Analytics is a global leader in AI and analytics, helping Fortune companies solve their toughest challenges. We are on a mission to push the boundaries of what AI and analytics can do to help enterprises navigate uncertainty and move forward decisively. Come...


  • Hyderabad, India Fusion Plus Solutions Full time

    Job Description - Total Yrs. of Experience6+Relevant Yrs. of experience5+Detailed JD (Roles and Responsibilities)5+ years of hands-on Python development experience with excellent programming skills, 3+ years of experience with cloud-based platform, ideally Microsoft Azure, working experience and skills on big data technologies such as PySpark, Databricks,...

  • PySpark Developer

    1 week ago


    Hyderabad, Telangana, India algoleap Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job SummaryWe are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.Key...

  • Pyspark Developer

    3 days ago


    Hyderabad, Telangana, India NTT DATA Business Solutions Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Job SummaryWe are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.Key...

  • Azure Databricks

    4 days ago


    Hyderabad, India Tata Consultancy Services Full time

    Azure Databricks Greetings from TCS!! ! TCS has been a great pioneer in feeding the fire of young Techies like you. We are a global leader in the technology arena and there’s nothing that can stop us from growing together. Your role is of key importance, as it lays down the foundation for the entire project. Make sure you have a valid EP number before...


  • Hyderabad, India ENTERPRISE SOFTLABS PRIVATE LINITED Full time

    Job Title : Pyspark DeveloperLocation : HyderabadExperience Required : 4- 8Keywords : AWS, Pyspark, Databricks Skills and experiences required :- 3- 6 years of hands-on development in PySpark.- Experience with Databricks and performance tuning using Spark UI.- Strong understanding of AWS services, Kafka, and distributed data processing.- Proficient in...

  • Azure Databricks

    4 days ago


    Hyderabad, India Tata Consultancy Services Full time

    Azure Databricks Greetings from TCS!!! TCS has been a great pioneer in feeding the fire of young Techies like you. We are a global leader in the technology arena and there's nothing that can stop us from growing together. Your role is of key importance, as it lays down the foundation for the entire project. Make sure you have a valid EP number before...

  • Data Engineer

    1 week ago


    Hyderabad, Telangana, India Golden Opportunities Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job DescriptionJob Title: Data Engineer (PySpark + Azure Databricks)**Location: Bangalore, Hyderabad, Chennai & MumbaiExperience: 6 � 10 yearsJob Summary**We are looking for a skilled Data Engineer with strong experience in PySpark and Azure Databricks to design, build, and optimize scalable data pipelines and analytics solutions. The ideal candidate will...