Databricks with Python

5 days ago

Chennai, India Teamware Solutions Full time

Job Description Teamware Solutions is seeking a skilled Databricks with Python Developer to build, optimize, and manage our big data processing and analytics solutions. This role is crucial for working with relevant technologies, ensuring smooth data operations, and contributing significantly to business objectives through expert analysis, development, implementation, and troubleshooting within the Databricks with Python domain. Roles and Responsibilities: - Big Data Solution Development: Design, develop, and implement scalable data processing pipelines and analytics solutions using PySpark within the Databricks platform. - Data Ingestion & Transformation: Write efficient PySpark code to extract, transform, and load (ETL/ELT) large volumes of data from various sources into Databricks. - Data Lake/Warehouse Management: Work with data lake and data warehouse concepts, ensuring data quality, consistency, and efficient storage within Databricks (e.g., Delta Lake). - Performance Optimization: Optimize PySpark jobs and Databricks notebooks for performance, cost-efficiency, and scalability, addressing bottlenecks in data processing. - Analysis & Insights: Perform complex data analysis using PySpark to uncover insights, build data models, and support business intelligence and machine learning initiatives. - Troubleshooting: Perform in-depth troubleshooting, debugging, and issue resolution for PySpark jobs, Databricks environments, and data pipeline failures. - Collaboration: Work closely with data engineers, data scientists, business analysts, and other stakeholders to understand data requirements and deliver robust solutions. - Code Quality & Best Practices: Write clean, modular, and well-documented PySpark code. Participate in code reviews and adhere to best practices for big data development. - Automation: Implement automation for data pipelines, job scheduling, and monitoring within the Databricks ecosystem. Preferred Candidate Profile: - PySpark Expertise: Strong hands-on development experience with PySpark for big data processing and analytics. - Databricks Platform: Proven experience working with the Databricks platform, including Databricks notebooks, clusters, Delta Lake, and related services. - SQL Proficiency: Excellent proficiency in SQL for data manipulation and querying. - Python Programming: Strong programming skills in Python for data engineering tasks. - Big Data Concepts: Solid understanding of big data concepts, distributed computing, and data warehousing principles. - Cloud Platforms (Plus): Familiarity with cloud services from AWS, Azure, or GCP, particularly those related to data storage and processing, is a plus. - Problem-Solving: Excellent analytical and problem-solving skills with a methodical approach to complex data challenges. - Communication: Strong verbal and written communication skills to articulate technical solutions and collaborate effectively within a team. - Education: Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related technical field.

  • Python/aws/databricks

    3 days ago

    Chennai, India Resource Tree Full time

    Job Name : Data Engineers with AWS and Databricks experience Role : Data Engineer Min years of experience : 3+ Location : Chennai/WFH, Open Positions : 2 Technology Requirement **:Databricks, SQL, Python, AWS **(Glue, S3, Lambda, EC2, SageMaker) Responsibilities: - Develop and implement data lake solutions and analytics solution using AWS and...

  • Azure Databricks Python Developer

    3 weeks ago

    Hyderabad, Chennai, India hexaforce Full time

    Job Description Azure Databricks Python Developer Location: Chennai/Hyderabad Notice Period: Immediate to 15 days Contract to Hire Position Walk-in Drive, only 1 round interview Chennai Drive Date: 25th Sep 2026, 26th Sep 2025, 27th Sep 2025 Hyderabad Drive Date: 26th Sep 2025, 27th Sep 2025 Required: - Overall Experience should be 5+ Years - Relevant...

  • AWS Glue Databricks Python

    2 weeks ago

    Chennai, Tamil Nadu, India Changeleaders Consulting Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    The Candidate must have 4 Year of relevant of experience in Databricks AWS Python Hands on experience on AWS Cloud platform especially S3 Glue Lamda. Experience on spark scripting.

  • Databricks

    6 days ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job SummaryJoin our dynamic team as a Developer where you will leverage your expertise in Spark in Scala Databricks and Amazon technologies to drive innovative solutions. With a hybrid work model and day shifts you will collaborate with cross-functional teams to optimize data processes and enhance our data infrastructure. Your contributions will directly...

  • Data Engineer – Airflow | Python | Databricks

    4 weeks ago

    Chennai, India InternationalClients Full time

    Location: Chennai, India (Onsite initially, Remote flexibility later)Work Days: Monday to FridayExperience: 4-7 yearsClient: Leading Life Sciences Company – Specialized in the Pharmaceutical IndustryJob Description: We are seeking a skilled Data Engineer with expertise in Apache Airflow , Python , and cloud-based data platforms to support our...

  • Databricks Data Engineer

    2 weeks ago

    Chennai, India Tata Consultancy Services Full time

    About the RoleThe Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.Experience- 3 to 6 yearsLocation - Kolkata /Chennai/TrivandrumRequired Technical Skill Set ------ Databricks, AWS, Python & PySparkGood to have- Experience in AWS platform (S3, EC2,...

  • Databricks Data Engineer

    2 weeks ago

    Chennai, India Tata Consultancy Services Full time

    About the RoleThe Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.Experience- 3 to 6 yearsLocation - Kolkata /Chennai/TrivandrumRequired Technical Skill Set ------ Databricks, AWS, Python & PySparkGood to have- Experience in AWS platform (S3, EC2,...

  • Databricks Data Engineer

    2 weeks ago

    Chennai, India Tata Consultancy Services Full time

    About the RoleThe Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.Experience- 3 to 6 yearsLocation - Kolkata /Chennai/TrivandrumRequired Technical Skill Set ------ Databricks, AWS, Python & PySparkGood to have- Experience in AWS platform (S3, EC2,...

  • Databricks Data Engineer

    2 weeks ago

    Chennai, India Tata Consultancy Services Full time

    About the RoleThe Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.Experience- 3 to 6 yearsLocation - Kolkata /Chennai/TrivandrumRequired Technical Skill Set ------ Databricks, AWS, Python & PySparkGood to have- Experience in AWS platform (S3, EC2,...

  • Databricks Data Engineer

    2 weeks ago

    Chennai, India Tata Consultancy Services Full time

    About the RoleThe Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.Experience- 3 to 6 yearsLocation - Kolkata /Chennai/TrivandrumRequired Technical Skill Set ------ Databricks, AWS, Python & PySparkGood to have- Experience in AWS platform (S3, EC2,...