Databricks Pyspark

3 days ago

Hyderabad, India Fusion Plus Solutions Full time

Job Description - Develop and optimize data processing jobs using PySpark to handle complex data transformations and aggregations efficiently. - Design and implement robust data pipelines on the AWS platform, ensuring scalability and efficiency. - Leverage AWS services such as EC2, S3 for comprehensive data processing and storage solutions. - Manage SQL database schema design, query optimization, and performance tuning to support data transformation and loading processes. - Design and maintain scalable and performant data warehouses, employing best practices in data modeling and ETL processes. - Utilize modern data platforms for collaborative data science, integrating seamlessly with various data sources and types. - Ensure high data quality and accessibility by maintaining optimal performance of Databricks clusters and Spark jobs. - Develop and implement security measures, backup procedures, and disaster recovery plans using AWS best practices. - Manage source code and automate deployment using GitHub along with CI/CD practices tailored for data operations in cloud environments. - Provide expertise in troubleshooting and optimizing PySpark scripts, Databricks notebooks, SQL queries, and Airflow DAGs. - Stay updated on the latest developments in cloud data technologies and recommend adoption of new tools and practices. - Use Apache Airflow to orchestrate and automate data workflows, ensuring timely and reliable execution of data jobs. - Collaborate with data scientists and business analysts to design data models and pipelines that support advanced analytics and machine learning projects.

  • Databricks, SQL, PySpark, Python

    3 days ago

    Hyderabad, India Fusion Plus Solutions Full time

    Job Description Roles and Responsibilities - 5+ years of experience on IT industry in Data Engineering & Data Analyst role. - 5 years of development experience using tool Databricks and PySpark, Python, SQL - Proficient in writing SQL queries including writing of windows functions - Good communication skills with analytical abilities in doing problem solving...

  • Azure Databricks with Pyspark

    6 days ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time

    TCS Hiring !!! Role: Azure databricks with Pyspark Exp: 4-8 Yr Location: Hyd JD - Total 4to 7 years of IT development experience - Minimum 4 years of experience on data warehouse or ETL platforms(including data mapping, ETL, data load and transformation) - Minimum 2 years of experience working as Azure engineer on AzureCloud platform. - Good hands-on...

  • ML Engineer

    3 weeks ago

    Hyderabad, India Tiger Analytics Full time

    Job Description ML Engineer (Databricks + PySpark) Locations: Chennai / Hyderabad / Bangalore Tiger Analytics is a global leader in AI and analytics, helping Fortune companies solve their toughest challenges. We are on a mission to push the boundaries of what AI and analytics can do to help enterprises navigate uncertainty and move forward...

  • ▷ (Immediate Start) Python, Pyspark, DataBricks

    3 days ago

    Hyderabad, India Fusion Plus Solutions Full time

    Job Description - Total Yrs. of Experience6+Relevant Yrs. of experience5+Detailed JD (Roles and Responsibilities)5+ years of hands-on Python development experience with excellent programming skills, 3+ years of experience with cloud-based platform, ideally Microsoft Azure, working experience and skills on big data technologies such as PySpark, Databricks,...

  • Azure Databricks/PySpark Developer

    2 weeks ago

    Hyderabad, India JRD Systems Private Ltd Full time

    Key Responsibilities :- Design, develop, and maintain scalable data pipelines and ETL/ELT processes using PySpark, SQL, and Python.- Build and manage data workflows on Azure Data Lake, Azure Data Factory (ADF), and Databricks.- Collaborate with data scientists, analysts, and other stakeholders to understand data needs and ensure data quality and...

  • PySpark Developer

    2 weeks ago

    Hyderabad, Telangana, India algoleap Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job SummaryWe are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.Key...

  • PySpark Developer

    4 days ago

    Hyderabad, Telangana, India Algoleap Technologies Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    SUMMARY Job SummaryWe are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.Key...

  • Pyspark Developer

    6 days ago

    Hyderabad, Telangana, India KloudPortal Technology Solutions PVT. LTD Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job DescriptionJob Summary We are hiring a Senior PySpark Developer with 4-7 years of experience in building and optimising data pipelines using PySpark on Databricks, within AWS cloud environments. This role involves modernising legacy systems, integrating with Kafka, and collaborating across cross-functional teams.Key ResponsibilitiesDevelop and optimise...

  • eSoftLabs - PySpark Developer

    3 days ago

    Hyderabad, India ENTERPRISE SOFTLABS PRIVATE LINITED Full time

    Job Title : Pyspark DeveloperLocation : HyderabadExperience Required : 4- 8Keywords : AWS, Pyspark, Databricks Skills and experiences required :- 3- 6 years of hands-on development in PySpark.- Experience with Databricks and performance tuning using Spark UI.- Strong understanding of AWS services, Kafka, and distributed data processing.- Proficient in...

  • Databricks

    4 days ago

    Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job SummaryWe are seeking a highly skilled Sr. Developer with 8 to 12 years of experience to join our team. The ideal candidate will have extensive experience in Spark in Scala Delta Sharing Databricks Unity Catalog Admin Databricks CLI Delta Live Pipelines Structured Streaming Risk Management Apache Airflow Amazon S3 Amazon Redshift Python Databricks SQL...