Databricks Data Engineer

3 weeks ago


Chennai Tamil Nadu India, Tamil Nadu Tata Consultancy Services Full time

About the Role


The Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.


Experience- 3 to 6 years


Location - Kolkata /Chennai/Trivandrum


Required Technical Skill Set ------ Databricks, AWS, Python & PySpark


Good to have- Experience in AWS platform (S3, EC2, EMR, MWAA, and Redshift).


Responsibilities


  • Collaborate with stakeholders to gather requirements and design solutions using the Databricks platform.
  • Migrate Python, Pyspark, MWAA DAG to Databricks Notebook and Workflow.
  • Work with BJs Computer operations team and repoint Talend/MWAA Autosys jobs to Databricks workflow.
  • Implement medallion architecture in Databricks Enterprise Data Platform with industry best practices and recommend the best framework for Redshift to Delta tables.
  • Provide technical expertise and guidance to the team members and stakeholders.
  • Contribute to the development of documentation, training materials, and user guides.
  • Participate in testing and quality assurance activities to ensure reliability and performance.
  • Optimize long-running Databricks workflows.
  • Development experience in Python, Pyspark.
  • Hands-on experience in writing complex SQLs and performance optimization of very complex SQL.

  • Databricks Admin

    3 weeks ago


    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Databricks AdminExp- 5 to 8 yearsLocation- Chennai/ KolkataRequired SkillsDatabricks AdministrationTerraformAWS cloud servicesMLflowWorkflowsDatabricks Asset BundlesAWS IAMVPCPrivate endpointsFirewallsS3SQL + PySparkdbx CLIResponsibilitiesImplement, and maintain the Databricks platform, including workspace setup, user and group management, access control,...


  • Chennai, Tamil Nadu, India Zillion Technologies Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Senior Data Engineer – DatabricksLocation: Hyderabad/ChennaiWork Schedule: Timings 2:30 PM to 11:30 PMRole Overview:We are seeking a highly skilled Senior Data Engineer with deep expertise in Databricks to design, build, and optimize scalable data pipelines and architectures. You will collaborate with cross-functional teams including data scientists,...

  • Principal Architect

    3 weeks ago


    Chennai, Tamil Nadu, India, Tamil Nadu Tiger Analytics Full time

    Title : Principal Architect – Databricks Location : Chennai, Bengaluru, Hyderabad Who we areTiger Analytics is a global leader in AI and analytics, helping Fortune 1000 companies solve their toughest challenges. We offer full-stack AI and analytics services & solutions to empower businesses to achieve real outcomes and value at scale. We are on a mission...

  • Data Engineer

    2 days ago


    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    HiRole - Azure Data EngineerLocation - Chennai, Gurugram (5days Office)Full timeShift timing 2pm to 11pmKey Responsibilities:· Develop, maintain, and optimize scalable ETL/ELT pipelines using PySpark and Databricks.· Collaborate with cross-functional teams to design and implement data models and data integration solutions.· Create and maintain robust SQL...

  • Databricks Admin

    3 weeks ago


    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    TCS present an excellent opportunity for Databricks AdminJob Location: Chennai , KolkataExperience required : 6-12 yrsInterview date: 17th Oct (Friday)Required Technical Skill Set**Databricks Administration, Terraform, AWS cloud services, MLflow, Workflows, Databricks Asset Bundles, AWS IAM, VPC, private endpoints, firewalls, S3, SQL + PySpark, dbx...

  • Databricks Admin

    3 weeks ago


    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    TCS is Hiring For Databricks AdminJob Location : Chennai, KolkataExperience : 6 - 12 yearsRequired Skillset : Databricks Administration, Terraform, AWS cloud services, MLflow, Workflows, Databricks Asset Bundles, AWS IAM, VPC, private endpoints, firewalls, S3, SQL + PySpark, DBX CLIMust have skills :Databricks Platform Administration including environment...

  • Python Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India, Tamil Nadu thinQ24 Full time

    Job descriptionNote: Only candidates who meet the eligibility criteria should apply. Others are kindly requested not to apply.Location: Onsite / HybridExperience: 4+ YearsEmployment Type: Contract/Full-TimeCompany DescriptionthinQ24 is a Software Development Company based in Chennai, TN, India, founded in 2019. We focus on going beyond limits to turn...


  • Chennai, Tamil Nadu, India InfoCepts Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position: Data Architect (Databricks, Snowflake)Location: Nagpur, Pune, Chennai, BangaloreType of Employment: Full time EmploymentPurpose of the Position:Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with business needs.Key...

  • Databrick

    2 days ago


    Chennai, Tamil Nadu, India Virtusa Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are seeking an experienced Databricks Developer with 5 to 7 years of expertise in big data engineering and analytics platforms. The ideal candidate will have deep experience in Apache Spark Databricks PySpark and cloud platforms such as Azure or AWS and a strong background in data pipeline development performance optimization and data governance.Key...

  • Azure Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Hi,Greetings from TCS Team!Role Azure Data EngineerRequired Technical Skill Set DatabricksDesired Experience Range 8-10 YearsLocation of Requirement TCS – ChennaiIf you’re interested in the role if the experience, location and skillset match your profile then please share your updated resume do share this opportunity among family and friends.10+...